Scenario file scanner scans inside Kubernetes configmap mount dirs

[W1zzardTPU]

What happened?

drwxrwxrwx    3 root     root          4096 Jul 16 10:46 .
drwxr-xr-x    3 root     root          4096 Jul 16 10:47 ..
drwxr-xr-x    2 root     root          4096 Jul 16 10:46 ..2024_07_16_10_46_34.998354025
lrwxrwxrwx    1 root     root            31 Jul 16 10:46 ..data -> ..2024_07_16_10_46_34.998354025
lrwxrwxrwx    1 root     root            24 Jul 16 10:46 http-401.yaml -> ..data/http-401.yaml
lrwxrwxrwx    1 root     root            24 Jul 16 10:46 http-403.yaml -> ..data/http-403.yaml

When Kubernetes mounts a configmap as folder inside a pod, it uses these directories prefixed with "..".

Crowdsec will try to scan into this directory and fail with "failed to sync items: failed to scan /etc/crowdsec: unknown configuration type for file '/etc/crowdsec/scenarios/foo/..2024_07_16_10_46_34.998354025/http-401.yaml'

In itemVisit(), can we ignore files and directories with leading '.' ?

crowdsec/pkg/cwhub/sync.go

Line 177 in c4bfdf1

func (h *Hub) itemVisit(path string, f os.DirEntry, err error) error {

What did you expect to happen?

Crowdsec should start without error and load the scenarios

How can we reproduce it (as minimally and precisely as possible)?

Not sure how to repro without a working Kubernetes setup

Anything else we need to know?

No response

Crowdsec version

cscli version
version: v1.6.2-16bfab86
Codename: alphaga
BuildDate: 2024-06-05_14:25:55
GoVersion: 1.22.3
Platform: docker
libre2: C++
User-Agent: crowdsec/v1.6.2-16bfab86-docker
Constraint_parser: >= 1.0, <= 3.0
Constraint_scenario: >= 1.0, <= 3.0
Constraint_api: v1
Constraint_acquis: >= 1.0, < 2.0

OS version

# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here

# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output here

Enabled collections and parsers

$ cscli hub list -o raw
# paste output here

Acquisition config

```console # On Linux: $ cat /etc/crowdsec/acquis.yaml /etc/crowdsec/acquis.d/* # paste output here

On Windows:

C:> Get-Content C:\ProgramData\CrowdSec\config\acquis.yaml

paste output here

Config show

$ cscli config show
# paste output here

Prometheus metrics

$ cscli metrics
# paste output here

Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

[github-actions]

@W1zzardTPU: Thanks for opening an issue, it is currently awaiting triage.

In the meantime, you can:

1. Check Crowdsec Documentation to see if your issue can be self resolved.
2. You can also join our Discord.
3. Check Releases to make sure your agent is on the latest version.

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[mmetc]

Hi, thanks for reporting the issue.

Actually the failure comes from the files being in a subdirectory (..data), which is not part of the expected hierarchy. This can be replicated by creating scenarios/somedir/somefile.yaml

We can nerf the errors because these are valid configurations, and I'll leave them as warnings, even with commands like "cscli hub list". Ignoring altogether the directories starting with a dot can be a future improvement.

The issue should be fixed in #3150

but since we just released 1.6.3-rc2 today, you can test it by deploying the crowdsecurity/crowdsec:dev image which follows the master branch.

Let me know if it works and we can ship it in the final release.

Thanks again!

[W1zzardTPU]

Indeed, you are right.

The underlying problem is that when mounting a configmap with multiple entries I can't "merge" directories, so I mounted into a subdir.

Is there any way to change the directory parser to understand nested directories? Maybe just one additional level? Maybe mark as "author" like for hub?

If I remember correctly, the documentation doesn't mention that subdirectories are not supported for these files.

hmm .. then why even scan into subdirs?

[mmetc]

There are collections/somefile.yaml, scenarios/somefile.yaml, parsers/stage/somefile.yaml... no subdirectories are recognized except for parsers/postoverflows, but linking to them is fine, so your configuration should work. The file is read twice and the symlink is the one that should be used, you'll get a benign warning for the other one. I'll remove the warning later by ignoring directories starting with '..'

[W1zzardTPU]

but linking to them is fine, so your configuration should work

These symlinks are generated automatically by Kubernetes, I have no control over them.

When mounting a configmap with multiple entries, an empty directory gets created, the actual files are mounted into these "..*" directories and then symlinked to the target directory.

I can't mount multiple files into just /scenarios/ (unless I mount them one-by-one, which is impractical once you have a significant number of them

[mmetc]

Hi @W1zzardTPU

we have a work in progress to allow subdirectories and link nesting but we are too close to 1.6.3. Could you mount your files one-by-one (hence no hidden directories or links) until 1.6.4 is out?

Thanks

[W1zzardTPU]

That's what I'm doing while growing my set of rules, not running in prod yet anyway. Thanks!

[mmetc]

@W1zzardTPU this should do it. You mount a single configmap per type, in a directory (one for scenarios, one per parsers..)

/etc/crowdsec/scenarios/myconfig/foo.yaml

and the name of the scenario will be "myconfig/foo.yaml" instead of "foo.yaml". It will work with files or links to ..data directories, which are ignored if nothing links to them.