Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

INTERNAL REVIEW: btfgen v4 #9

Closed
wants to merge 9 commits into from
Closed

INTERNAL REVIEW: btfgen v4 #9

wants to merge 9 commits into from

Conversation

mauriciovasquezbernal
Copy link
Member

Again, only for internal review and automatic tests purposes.

@mauriciovasquezbernal mauriciovasquezbernal force-pushed the mauricio/btfgen-v4 branch 2 times, most recently from 02c93d4 to c9940b3 Compare December 20, 2021 16:19
@mauriciovasquezbernal
Copy link
Member Author

/cc @rafaeldtinoco @fntlnz @leodido

@mauriciovasquezbernal mauriciovasquezbernal force-pushed the mauricio/btfgen-v4 branch 2 times, most recently from 3670505 to 38813a5 Compare January 7, 2022 14:24
fntlnz
fntlnz approved these changes Jan 8, 2022
View reviewed changes
Copy link

@fntlnz fntlnz left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
Logic looks good, code seems polished and couldn't find any memory leaks in my testing. This seems g2g to me.

Nobody and others added 3 commits January 11, 2022 16:09
@mauriciovasquezbernal
adding ci files
d108af3
@mauriciovasquezbernal
libbpf: split bpf_core_apply_relo()
d75b3f6 
BTFGen needs to run the core relocation logic in order to understand
what are the types in the target BTF that involved in a given
relocation.

Currently bpf_core_apply_relo() calculates and **applies** a relocation
to an instruction. Having both operations in the same function makes it
difficult to only calculate the relocation without patching the
instruction. This commit splits that logic in two different phases: (1)
calculate the relocation and (2) patch the instruction.

For the first phase bpf_core_apply_relo() is renamed to
bpf_core_calc_relo_res() who is now only on charge of calculating the
relocation, the second phase uses the already existing
bpf_core_patch_insn(). bpf_object__relocate_core() uses both of them and
the BTFGen will use only bpf_core_calc_relo_res().

Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
Signed-off-by: Rafael David Tinoco <rafael.tinoco@aquasec.com>
Signed-off-by: Lorenzo Fontana <lorenzo.fontana@elastic.co>
Signed-off-by: Leonardo Di Donato <leonardo.didonato@elastic.co>
@mauriciovasquezbernal
libbpf: Implement changes needed for BTFGen in bpftool
74c6f81 
This commit extends libbpf with the features that are needed to
implement BTFGen:

- Implement bpf_core_create_cand_cache() and bpf_core_free_cand_cache()
to handle candidates cache.
- Expose bpf_core_add_cands() and bpf_core_free_cands to handle
candidates list.
- Expose bpf_core_calc_relo_insn() to bpftool.

Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
Signed-off-by: Rafael David Tinoco <rafael.tinoco@aquasec.com>
Signed-off-by: Lorenzo Fontana <lorenzo.fontana@elastic.co>
Signed-off-by: Leonardo Di Donato <leonardo.didonato@elastic.co>
@mauriciovasquezbernal mauriciovasquezbernal force-pushed the mauricio/btfgen-v4 branch 2 times, most recently from a711c0b to 335675d Compare January 11, 2022 21:13
@mauriciovasquezbernal
bpftool: Add gen btf command
0e8c4c6 
This command is implemented under the "gen" command in bpftool and the
syntax is the following:

$ bpftool gen btf INPUT OUTPUT OBJECT(S)

INPUT can be either a single BTF file or a folder containing BTF files,
when it's a folder, a BTF file is generated for each BTF file contained
in this folder. OUTPUT is the file (or folder) where generated files are
stored and OBJECT(S) is the list of bpf objects we want to generate the
BTF file(s) for (each generated BTF file contains all the types needed
by all the objects).

Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
Signed-off-by: Rafael David Tinoco <rafael.tinoco@aquasec.com>
Signed-off-by: Lorenzo Fontana <lorenzo.fontana@elastic.co>
Signed-off-by: Leonardo Di Donato <leonardo.didonato@elastic.co>
@mauriciovasquezbernal
bpftool: Implement btf_save_raw()
12d81ac 
Helper function to save a BTF object to a file.

Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
Signed-off-by: Rafael David Tinoco <rafael.tinoco@aquasec.com>
Signed-off-by: Lorenzo Fontana <lorenzo.fontana@elastic.co>
Signed-off-by: Leonardo Di Donato <leonardo.didonato@elastic.co>
@mauriciovasquezbernal
bpftool: Add struct definitions and helpers for BTFGen
14a55f7 
Add some structs and helpers that will be used by BTFGen in the next
commits.

Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
Signed-off-by: Rafael David Tinoco <rafael.tinoco@aquasec.com>
Signed-off-by: Lorenzo Fontana <lorenzo.fontana@elastic.co>
Signed-off-by: Leonardo Di Donato <leonardo.didonato@elastic.co>
@mauriciovasquezbernal
bpftool: Implement btfgen()
0cc3043 
btfgen() receives the path of a source and destination BTF files and a
list of BPF objects. This function records the relocations for all
objects and then generates the BTF file by calling btfgen_get_btf()
(implemented in the following commits).

btfgen_record_obj() loads the BTF and BTF.ext sections of the BPF
objects and loops through all CO-RE relocations. It uses
bpf_core_calc_relo_insn() from libbpf and passes the target spec to
btfgen_record_reloc() that saves the types involved in such relocation.

Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
Signed-off-by: Rafael David Tinoco <rafael.tinoco@aquasec.com>
Signed-off-by: Lorenzo Fontana <lorenzo.fontana@elastic.co>
Signed-off-by: Leonardo Di Donato <leonardo.didonato@elastic.co>
@mauriciovasquezbernal
bpftool: Implement relocations recording for BTFGen
95b2f76 
This commit implements the logic to record the relocation information
for the different kind of relocations.

btfgen_record_field_relo() uses the target specification to save all the
types that are involved in a field-based CO-RE relocation. In this case
types resolved and added recursively (using btfgen_put_type()).
Only the struct and union members and their types) involved in the
relocation are added to optimize the size of the generated BTF file.

On the other hand, btfgen_record_type_relo() saves the types involved in
a type-based CO-RE relocation. In this case all the members for the
struct and union types are added. This is not strictly required since
libbpf doesn't use them while performing this kind of relocation,
however that logic could change on the future. Additionally, we expect
that the number of this kind of relocations in an BPF object to be very
low, hence the impact on the size of the generated BTF should be
negligible.

Finally, btfgen_record_enumval_relo() saves the whole enum type for
enum-based relocations.

Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
Signed-off-by: Rafael David Tinoco <rafael.tinoco@aquasec.com>
Signed-off-by: Lorenzo Fontana <lorenzo.fontana@elastic.co>
Signed-off-by: Leonardo Di Donato <leonardo.didonato@elastic.co>
@mauriciovasquezbernal
bpftool: Implement btfgen_get_btf()
71f1e21 
The last part of the BTFGen algorithm is to create a new BTF object with
all the types that were recorded in the previous steps.

This function performs two different steps:
1. Add the types to the new BTF object by using btf__add_type(). Some
special logic around struct and unions is implemented to only add the
members that are really used in the field-based relocations. The type
ID on the new and old BTF objects is stored on a map.
2. Fix all the type IDs on the new BTF object by using the IDs saved in
the previous step.

Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
Signed-off-by: Rafael David Tinoco <rafael.tinoco@aquasec.com>
Signed-off-by: Lorenzo Fontana <lorenzo.fontana@elastic.co>
Signed-off-by: Leonardo Di Donato <leonardo.didonato@elastic.co>
@mauriciovasquezbernal
Copy link
Member Author

mauriciovasquezbernal commented Jan 18, 2022 •
edited
Loading

Sent upstream at https://lore.kernel.org/bpf/20220112142709.102423-1-mauricio@kinvolk.io.

mauriciovasquezbernal pushed a commit that referenced this pull request Feb 14, 2022
@davem330
Merge branch 'mlxsw-RJ45'
3cade91 
Ido Schimmel says:

====================
mlxsw: Add RJ45 ports support

We are in the process of qualifying a new system that has RJ45 ports as
opposed to the transceiver modules (e.g., SFP, QSFP) present on all
existing systems.

This patchset adds support for these ports in mlxsw by adding a couple of
missing BaseT link modes and rejecting ethtool operations that are
specific to transceiver modules.

Patchset overview:

Patches #1-#3 are cleanups and preparations.

Patch #4 adds support for two new link modes.

Patches #5-#6 query and cache the port module's type (e.g., QSFP, RJ45)
during initialization.

Patches #7-#9 forbid ethtool operations that are invalid on RJ45 ports.
====================

Signed-off-by: David S. Miller <davem@davemloft.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fntlnz fntlnz fntlnz approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mauriciovasquezbernal @fntlnz