Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overlay FS: Add fields proc.is_exe_lower_layer, fd.is_upper_layer and fd.is_lower_layer #1936

Merged
merged 7 commits into from
Aug 28, 2024

Conversation

eddyduer-sysdig
Copy link
Contributor

@eddyduer-sysdig eddyduer-sysdig commented Jun 26, 2024

Overlay FS: Add fields proc.is_exe_lower_layer, fd.is_upper_layer and fd.is_lower_layer

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area API-version

/area build

/area CI

/area driver-kmod

/area driver-bpf

/area driver-modern-bpf

/area libscap-engine-bpf

/area libscap-engine-gvisor

/area libscap-engine-kmod

/area libscap-engine-modern-bpf

/area libscap-engine-nodriver

/area libscap-engine-noop

/area libscap-engine-source-plugin

/area libscap-engine-savefile

/area libscap

/area libpman

/area libsinsp

/area tests

/area proposals

Does this PR require a change in the driver versions?

/version driver-API-version-major

/version driver-API-version-minor

/version driver-API-version-patch

/version driver-SCHEMA-version-major

/version driver-SCHEMA-version-minor

/version driver-SCHEMA-version-patch

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

new(driver,userspace): add fields proc.is_exe_lower_layer, fd.is_upper_layer and fd.is_lower_layer for Overlay FS

Copy link

Please double check driver/API_VERSION file. See versioning.

/hold

@incertum
Copy link
Contributor

Very nice 🚀 @eddyduer-sysdig! @loresuso could you help with the review since you worked on the upper layer flag? Thanks!

@loresuso
Copy link
Member

Woah this is cool! I'm gonna review it soon 👀 🚀

Copy link

github-actions bot commented Jun 28, 2024

Perf diff from master - unit tests

     7.13%     +1.75%  [.] sinsp::next
     2.06%     +1.11%  [.] sinsp_thread_manager::find_thread
     5.95%     -0.97%  [.] next
     1.89%     -0.96%  [.] sinsp_evt::get_ts
     2.77%     +0.92%  [.] gzfile_read
     3.40%     -0.54%  [.] sinsp_thread_manager::get_thread_ref
     1.21%     -0.39%  [.] sinsp_utils::find_longest_matching_evt_param
     2.28%     -0.39%  [.] scap_event_decode_params
     1.22%     +0.38%  [.] libsinsp::sinsp_suppress::process_event
     0.87%     -0.36%  [.] sinsp_filter_check::parse_field_name

Heap diff from master - unit tests

peak heap memory consumption: -1.41K
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Heap diff from master - scap file

peak heap memory consumption: -586B
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Copy link
Member

@Andreagit97 Andreagit97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this! I left some comments mainly on the driver side!

Maybe we could add some tests both driver side and userspace side:

driver/modern_bpf/helpers/extract/extract_from_kernel.h Outdated Show resolved Hide resolved
driver/modern_bpf/helpers/extract/extract_from_kernel.h Outdated Show resolved Hide resolved
driver/bpf/fillers.h Outdated Show resolved Hide resolved
driver/bpf/filler_helpers.h Outdated Show resolved Hide resolved
driver/bpf/fillers.h Outdated Show resolved Hide resolved
driver/bpf/fillers.h Outdated Show resolved Hide resolved
driver/ppm_fillers.c Outdated Show resolved Hide resolved
driver/event_table.c Outdated Show resolved Hide resolved
driver/event_table.c Outdated Show resolved Hide resolved
driver/event_table.c Outdated Show resolved Hide resolved
Copy link

github-actions bot commented Aug 9, 2024

Perf diff from master - unit tests

     6.27%     -1.02%  [.] next
     1.34%     -0.90%  [.] scap_next
     1.39%     +0.84%  [.] std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release
     4.63%     -0.81%  [.] gzfile_read
     2.74%     +0.81%  [.] sinsp_thread_manager::get_thread_ref
     1.96%     +0.81%  [.] std::_Hashtable<long, std::pair<long const, std::shared_ptr<sinsp_threadinfo> >, std::allocator<std::pair<long const, std::shared_ptr<sinsp_threadinfo> > >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_find_before_node
     2.85%     -0.63%  [.] scap_event_decode_params
     6.93%     -0.63%  [.] sinsp::next
     1.11%     +0.60%  [.] scap_event_encode_params_v
     1.40%     -0.56%  [.] sinsp_evt::get_ts

Perf diff from master - scap file

    15.37%     -5.31%  [.] scap_event_decode_params
    19.29%     -3.53%  [.] sinsp_filter_check_event::extract_single
     4.89%     +3.14%  [.] sinsp_filter_check::tostring
     4.86%     +3.09%  [.] sinsp::next
     9.75%     -2.72%  [.] libsinsp::runc::match_one_container_id
     5.98%     +2.56%  [.] rawstring_check::extract_single
    11.13%     -1.56%  [.] sinsp_filter_check_thread::extract_single
     5.25%     +1.12%  [.] sinsp_filter_check::get_transformed_field_info
     4.98%     -0.51%  [.] scap_next
     4.89%     -0.47%  [.] sinsp_evt::get_param_as_str

Heap diff from master - unit tests

peak heap memory consumption: -586B
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Heap diff from master - scap file

peak heap memory consumption: -586B
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Copy link
Member

@Andreagit97 Andreagit97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! i will check why the CI on ARM64 is failing. Some side comments:

  • It would be great to add some drivers test for checking we are correctly getting these new params from the kernel
  • it would be great to unify the extraction approach between the 3 drivers as suggested in the below comments.
  • it would be great to add some tests in userspace as suggested by codecov

Here you can find some references for drivers_tests and sinsp tests #1936 (review)

driver/ppm_events_public.h Outdated Show resolved Hide resolved
userspace/libsinsp/fdinfo.h Show resolved Hide resolved
@@ -265,6 +265,7 @@ typedef struct scap_threadinfo
char exepath[SCAP_MAX_PATH_SIZE+1]; ///< full executable path
bool exe_writable; ///< true if the original executable is writable by the same user that spawned it.
bool exe_upper_layer; //< True if the original executable belongs to upper layer in overlayfs
bool exe_lower_layer; //< True if the original executable belongs to lower layer in overlayfs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For myself: I need to check if we missed something with scap-files

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we need to append the new value to captures here: https://github.com/falcosecurity/libs/pull/1936/files#diff-7b4e2af897ce4b8645f4f862762f68425d8394805c45c194255730e99078767dR701
instead of putting it in the middle and also when reading it: https://github.com/falcosecurity/libs/pull/1936/files#diff-f62df5aa6a14a8687e897847637228c047695a25049b0b095769c4f7579f618fR688 we should read it as last param otherwise we would break existing scap files.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems this is also causing an issue with sanitizer builds scap files related tests: https://github.com/falcosecurity/libs/actions/runs/10563275899/job/29263411886?pr=1936

driver/bpf/fillers.h Outdated Show resolved Hide resolved
driver/flags_table.c Outdated Show resolved Hide resolved
driver/event_table.c Outdated Show resolved Hide resolved
bpf_probe_read_kernel(&upper_dentry, sizeof(upper_dentry), (char *)vfs_inode + sizeof(struct inode));
if(!upper_dentry)
{
return PPM_OVERLAY_LOWER;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not 100% sure but I don't think that !upper_dentry -> PPM_OVERLAY_LOWER WDYT?

Suggested change
return PPM_OVERLAY_LOWER;
return PPM_NOT_OVERLAY_FS;

driver/ppm_fillers.c Outdated Show resolved Hide resolved
@Andreagit97
Copy link
Member

Hey @Molter73 I see that e2e tests are failing but the report is not uploaded in CI https://github.com/falcosecurity/libs/actions/runs/10307297656/job/28555325360?pr=1936#step:8:8
Any idea why is this happening?

@eddyduer-sysdig
Copy link
Contributor Author

Fixed comments from code review and added some tests

Copy link

codecov bot commented Aug 22, 2024

Codecov Report

Attention: Patch coverage is 95.71429% with 3 lines in your changes missing coverage. Please review.

Project coverage is 74.25%. Comparing base (c7d7530) to head (a332c03).
Report is 16 commits behind head on master.

Files Patch % Lines
userspace/libsinsp/sinsp_filtercheck_thread.cpp 0.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1936      +/-   ##
==========================================
+ Coverage   74.20%   74.25%   +0.04%     
==========================================
  Files         253      253              
  Lines       30832    30895      +63     
  Branches     5411     5410       -1     
==========================================
+ Hits        22880    22941      +61     
+ Misses       7952     7935      -17     
- Partials        0       19      +19     
Flag Coverage Δ
libsinsp 74.25% <95.71%> (+0.04%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@eddyduer-sysdig eddyduer-sysdig force-pushed the overlay_fs_additions branch 2 times, most recently from 27e124d to 7f0b654 Compare August 26, 2024 16:13
@FedeDP
Copy link
Contributor

FedeDP commented Aug 27, 2024

/milestone next-driver

@poiana poiana added this to the next-driver milestone Aug 27, 2024
eddyduer-sysdig and others added 5 commits August 27, 2024 11:32
… fd.is_lower_layer

Signed-off-by: Eddy Duer <eddy.duer@sysdig.com>
Signed-off-by: Eddy Duer <eddy.duer@sysdig.com>
… syscall family

Signed-off-by: Eddy Duer <eddy.duer@sysdig.com>
Signed-off-by: Eddy Duer <eddy.duer@sysdig.com>
Signed-off-by: Andrea Terzolo <andreaterzolo3@gmail.com>
@Andreagit97 Andreagit97 force-pushed the overlay_fs_additions branch from 7f0b654 to 7a6bcd4 Compare August 27, 2024 09:54
Signed-off-by: Andrea Terzolo <andreaterzolo3@gmail.com>
@Andreagit97 Andreagit97 force-pushed the overlay_fs_additions branch from 7a6bcd4 to a9546c9 Compare August 27, 2024 11:10
Signed-off-by: Andrea Terzolo <andreaterzolo3@gmail.com>
@Andreagit97 Andreagit97 force-pushed the overlay_fs_additions branch from 624fa1c to a332c03 Compare August 27, 2024 14:38
Copy link
Member

@Andreagit97 Andreagit97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks! I just pushed some commits to fix the scap-file handling and the e2e sinsp tests

@Andreagit97
Copy link
Member

Results of kernel tests: https://github.com/falcosecurity/libs/actions/runs/10575083303

x86

KERNEL CMAKE-CONFIGURE KMOD BUILD KMOD SCAP-OPEN BPF-PROBE BUILD BPF-PROBE SCAP-OPEN MODERN-BPF SCAP-OPEN
amazonlinux2-4.19 🟢 🟢 🟢 🟢 🟢 🟡
amazonlinux2-5.10 🟢 🟢 🟢 🟢 🟢 🟢
amazonlinux2-5.15 🟢 🟢 🟢 🟢 🟢 🟢
amazonlinux2-5.4 🟢 🟢 🟢 🟢 🟢 🟡
amazonlinux2022-5.15 🟢 🟢 🟢 🟢 🟢 🟢
amazonlinux2023-6.1 🟢 🟢 🟢 🟢 🟢 🟢
archlinux-6.0 🟢 🟢 🟢 🟢 🟢 🟢
archlinux-6.7 🟢 🟢 🟢 🟢 🟢 🟢
centos-3.10 🟢 🟢 🟢 🟡 🟡 🟡
centos-4.18 🟢 🟢 🟢 🟢 🟢 🟢
centos-5.14 🟢 🟢 🟢 🟢 🟢 🟢
fedora-5.17 🟢 🟢 🟢 🟢 🟢 🟢
fedora-5.8 🟢 🟢 🟢 🟢 🟢 🟢
fedora-6.2 🟢 🟢 🟢 🟢 🟢 🟢
oraclelinux-3.10 🟢 🟢 🟢 🟡 🟡 🟡
oraclelinux-4.14 🟢 🟢 🟢 🟢 🟢 🟡
oraclelinux-5.15 🟢 🟢 🟢 🟢 🟢 🟢
oraclelinux-5.4 🟢 🟢 🟢 🟢 🟢 🟡
ubuntu-4.15 🟢 🟢 🟢 🟢 🟢 🟡
ubuntu-5.8 🟢 🟢 🟢 🟢 🟢 🟡
ubuntu-6.5 🟢 🟢 🟢 🟢 🟢 🟢

arm64

KERNEL CMAKE-CONFIGURE KMOD BUILD KMOD SCAP-OPEN BPF-PROBE BUILD BPF-PROBE SCAP-OPEN MODERN-BPF SCAP-OPEN
amazonlinux2-5.4 🟢 🟢 🟢 🟢 🟢 🟡
amazonlinux2022-5.15 🟢 🟢 🟢 🟢 🟢 🟢
fedora-6.2 🟢 🟢 🟢 🟢 🟢 🟢
oraclelinux-4.14 🟢 🟢 🟢 🟡 🟡 🟡
oraclelinux-5.15 🟢 🟢 🟢 🟢 🟢 🟢
ubuntu-6.5 🟢 🟢 🟢 🟢 🟢 🟢

Copy link
Member

@Andreagit97 Andreagit97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@poiana
Copy link
Contributor

poiana commented Aug 28, 2024

LGTM label has been added.

Git tree hash: dbd6247baa518ba2d5473ef695fa5edc7c0a3215

Copy link
Contributor

@FedeDP FedeDP left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@poiana
Copy link
Contributor

poiana commented Aug 28, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Andreagit97, eddyduer-sysdig, FedeDP

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@FedeDP
Copy link
Contributor

FedeDP commented Aug 28, 2024

/unhold

@poiana poiana merged commit bf3c89b into falcosecurity:master Aug 28, 2024
54 of 59 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants