Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't store an empty file report for err-first-hit/miss findings #579

Merged
merged 4 commits into from
Nov 4, 2024

Conversation

egibs
Copy link
Member

@egibs egibs commented Nov 4, 2024

This should resolve #577 -- since errIfHitOrMiss is supposed to return an error depending on whether --err-first-hit or --err-first-miss is used, we need to store the original file report in the findings map.

With this change:

$ go run cmd/mal/mal.go --err-first-hit scan out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/linux/2024.Kaiji/
🔎 Scanning "out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/linux/2024.Kaiji/"
├─ 🛑 out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/linux/2024.Kaiji/eight-nebraska-autumn-illinois
│     • impact/remote_access/systemctl — systemctl botnet client:
│     .bash_history, Mozilla/5.0, SELINUX, crontab, daemon-reload, id_rsa, known_hosts
│     • malware/family/kaiji — Kaiji IOT Malware: audit2allow -M my-Systemimgconf, chaos_cve_make, systemctl start linux.service
│     • persist/cron/hidden_tab — persists via a hidden crontab entry: */1 * * * * root /.img

💡 For detailed analysis, try "mal analyze <path>"

--err-first-miss looks like it's functioning as expected, however:

$ go run cmd/mal/mal.go --err-first-miss --verbose scan out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean
...
time=2024-11-04T07:31:45.623-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/idna/setup.py file"
time=2024-11-04T07:31:45.623-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/google-auth-library-python/setup.py file"
time=2024-11-04T07:31:45.624-06:00 level=INFO source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:106 msg="skipping out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/setuptools/discovery.py.simple [<unknown>]: data file or empty" path=out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/setuptools/discovery.py.simple
time=2024-11-04T07:31:45.624-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/setuptools/discovery.py.simple file"
time=2024-11-04T07:31:45.624-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/ml_sdk/setup.py file"
time=2024-11-04T07:31:45.624-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/magic_trace/magic_trace.py file"
time=2024-11-04T07:31:45.624-06:00 level=INFO source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:106 msg="skipping out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/setuptools/easy_install.py.simple [<unknown>]: data file or empty" path=out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/setuptools/easy_install.py.simple
time=2024-11-04T07:31:45.624-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/setuptools/easy_install.py.simple file"
time=2024-11-04T07:31:45.624-06:00 level=INFO source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:106 msg="skipping out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/setuptools/package_index.py.simple [<unknown>]: data file or empty" path=out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/setuptools/package_index.py.simple
time=2024-11-04T07:31:45.624-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/setuptools/package_index.py.simple file"
time=2024-11-04T07:31:45.625-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/gevent/test__monkey.py file"
time=2024-11-04T07:31:45.625-06:00 level=INFO source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:106 msg="skipping out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/setuptools/sandbox.py.simple [<unknown>]: data file or empty" path=out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/setuptools/sandbox.py.simple
time=2024-11-04T07:31:45.625-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/setuptools/sandbox.py.simple file"
time=2024-11-04T07:31:45.625-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/fonttools/psLib.py file"
time=2024-11-04T07:31:45.625-06:00 level=INFO source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:106 msg="skipping out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/tensorflow_model_analysis/tfjs_predict_extractor_util.py.simple [<unknown>]: data file or empty" path=out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/tensorflow_model_analysis/tfjs_predict_extractor_util.py.simple
time=2024-11-04T07:31:45.625-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/tensorflow_model_analysis/tfjs_predict_extractor_util.py.simple file"
time=2024-11-04T07:31:45.625-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/requests/setup.py file"
time=2024-11-04T07:31:45.627-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/jaraco/__init__.py file"
time=2024-11-04T07:31:45.627-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/tensorflow_model_analysis/tfjs_predict_extractor_util.py file"
time=2024-11-04T07:31:45.627-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/pyparsing/sparser.py file"
time=2024-11-04T07:31:45.628-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/setuptools/discovery.py file"
time=2024-11-04T07:31:45.630-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/setuptools/sandbox.py file"
time=2024-11-04T07:31:45.638-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/setuptools/package_index.py file"
time=2024-11-04T07:31:45.646-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/numpy/misc_util.py file"
time=2024-11-04T07:31:45.651-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/setuptools/easy_install.py file"
time=2024-11-04T07:31:45.775-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/setuptools/build_meta.py file"

💡 For detailed analysis, try "mal analyze <path>"

Signed-off-by: egibs <20933572+egibs@users.noreply.github.com>
@egibs egibs requested a review from tstromberg November 4, 2024 13:33
@tstromberg
Copy link
Collaborator

tstromberg commented Nov 4, 2024

Change looks OK, but this still doesn't address the intended behavior of --err-first-miss:

% go run ./cmd/mal --err-first-miss scan /bin/*                                                        🔎 Scanning "/bin/["
🔎 Scanning "/bin/ab"
🔎 Scanning "/bin/aconnect"
🔎 Scanning "/bin/activate-global-python-argcomplete"
🔎 Scanning "/bin/addr2line"

It should have exited after the first file, as nothing matched.

@tstromberg
Copy link
Collaborator

tstromberg commented Nov 4, 2024

I was a little confused by this:

--err-first-miss looks like it's functioning as expected, however:

go run cmd/mal/mal.go --err-first-miss --verbose scan out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean
...
time=2024-11-04T07:31:45.623-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/idna/setup.py file"
time=2024-11-04T07:31:45.623-06:00 level=DEBUG source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:319 msg="match short circuit: no matching capabilities in out/samples-0ff28cbe99bc4610c58016faeb1a806a6e5cebbb/python/clean/google-auth-library-python/setup.py file"

Shouldn't it have exited after parsing `setup.py` if no capabilities matched?

@tstromberg tstromberg closed this Nov 4, 2024
@tstromberg tstromberg reopened this Nov 4, 2024
@tstromberg
Copy link
Collaborator

Whoops, clicked the wrong button!

@egibs
Copy link
Member Author

egibs commented Nov 4, 2024

Good point. I'll work on figuring that out.

Signed-off-by: egibs <20933572+egibs@users.noreply.github.com>
@egibs
Copy link
Member Author

egibs commented Nov 4, 2024

@tstromberg -- fixed the errIfMiss behavior in 4287903 (#579).

$ go run cmd/mal/mal.go --err-first-miss scan /usr/bin/*
🔎 Scanning "/usr/bin/aa"
time=2024-11-04T08:19:29.078-06:00 level=ERROR source=.../repos/chainguard-dev/malcontent/pkg/action/scan.go:339 msg="error with processing no matching capabilities in /usr/bin/aa file\n"
error: scan: no matching capabilities in /usr/bin/aa file
exit status 2

Signed-off-by: egibs <20933572+egibs@users.noreply.github.com>
@egibs egibs merged commit 5954d1a into chainguard-dev:main Nov 4, 2024
8 checks passed
@egibs egibs deleted the fix-err-flows branch November 14, 2024 00:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

err-first-miss broken: no longer exits after miss
2 participants