Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possibility to exclude paths / files within a project from a scan #5018

Open
MarcelBochtler opened this issue Feb 3, 2022 · 2 comments
Open
Labels
enhancement Issues that are considered to be enhancements scanner About the scanner tool

Comments

@MarcelBochtler
Copy link
Member

MarcelBochtler commented Feb 3, 2022

Scenario:
I do have a repository with some huge files as test resources. Sometimes even binary files.
These result in ScanCode timeouts, when it tries to scans them.

The --skip-excluded CLI parameter currently only provides the possibility to exclude Packages and whole Projects.
See:

// Determine the projects to scan as packages.
val consolidatedProjects = consolidateProjectPackagesByVcs(ortResult.getProjects(skipExcluded))
val projectPackages = consolidatedProjects.keys
val projectPackageIds = projectPackages.map { it.id }
val packages = ortResult.getPackages(skipExcluded)
.filter { it.pkg.id !in projectPackageIds }
.map { it.pkg }

Possible solution:
Use the --ignore option from ScanCode to ignore the files specified via a pathExclude in .ort.yml if --skip-excluded is used for the ORT scanner.

For FossID it is already possible to exclude files from the Project: #4976

@MarcelBochtler MarcelBochtler added enhancement Issues that are considered to be enhancements scanner About the scanner tool labels Feb 3, 2022
@mnonnenmacher
Copy link
Member

To implement this correctly it is required that the exclude configuration is reflected in the stored scan result, to make sure that the returned scan result matches the excludes if they are changed. This might be already solved by the scanner configuration compatibility check in the scan storage, if the excludes are passed as an option to ScanCode, but this needs to be verified.

@mnonnenmacher
Copy link
Member

Open questions:

  • Should this apply only to projects or to packages as well?
  • Should this use path excludes or a new separate configuration?
  • Should this be implemented scanner specific (e.g. the --ignore option for ScanCode) or in a generic way (e.g. deleting files before scanning)?

Bosch will make a concrete suggestion for an implementation and propose it to the community.

sschuberth added a commit that referenced this issue Feb 2, 2024
Enable this by default for performance reasons of showcasing ORT. For
simplicity, only use a single parameter for all tools as there is
barely a need to configure this flag differently per tool. However, the
meaning of this parameter can slightly differ per tool, as not all tools
support e.g. path excludes yet, see [1] or also [2] in that context.

[1]: #5018
[2]: #3537

Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>
sschuberth added a commit that referenced this issue Feb 2, 2024
Enable this by default for performance reasons of showcasing ORT. For
simplicity, only use a single parameter for all tools as there is
barely a need to configure this flag differently per tool. However, the
meaning of this parameter can slightly differ per tool, as not all tools
support e.g. path excludes yet, see [1] or also [2] in that context.

[1]: #5018
[2]: #3537

Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>
sschuberth added a commit that referenced this issue Feb 2, 2024
Enable this by default for performance reasons of showcasing ORT. For
simplicity, only use a single parameter for all tools as there is
barely a need to configure this flag differently per tool. However, the
meaning of this parameter can slightly differ per tool, as not all tools
support e.g. path excludes yet, see [1] or also [2] in that context.

[1]: #5018
[2]: #3537

Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Issues that are considered to be enhancements scanner About the scanner tool
Projects
None yet
Development

No branches or pull requests

2 participants