Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ignored git folders/file are scanned while it shouldn't #280

Closed
gmontard opened this issue Dec 16, 2022 · 4 comments · Fixed by #309
Closed

Ignored git folders/file are scanned while it shouldn't #280

gmontard opened this issue Dec 16, 2022 · 4 comments · Fixed by #309
Assignees
@spdawson
Labels
bug Something isn't working

Comments

@gmontard
Copy link
Contributor

Folders/file present /gitignore are scanned.

Description

Folders and files referenced in a .gitignore shouldn't be scanned.

Expected Behavior

Avoid scanning files and folder present in a .gitignore

Steps to Reproduce

  1. Add a folder/file folder_to_ignore/test.txt in a sample app and add folder_to_ignore folder in the app .gitignore
  2. Scan the sample app in debug mode .
  3. Notice that it scan the files present in the folder_to_ignore folder.

Context

It slows down the scan a lot.

Your Environment

  • Version used: 0.19.0
  • Target codebase stack (e.g. Ruby): Ruby
  • Operating System and version: Mac OS 13
  • Link to your project or code sample: n/a
@gmontard gmontard added the bug Something isn't working label Dec 16, 2022
@cfabianski
Copy link
Collaborator

They should already be ignored https://github.com/Bearer/curio/blob/6aa6f39169ed0a7684e86dfc3dd6249d5b7b0512/pkg/util/file/file.go#L75 with this fileInfo.isGitIgnored

https://github.com/Bearer/curio/blob/fa16564adaea69ab8b8f658dc83bce6f793be150/pkg/util/file/file.go#L213-L216

@spdawson
Copy link
Contributor

I haven't been able to reproduce this locally. @gmontard Could you share the exact entry that you are adding to the .gitignore file, and what you see in the output that indicates the scanning of files that ought to be ignored?

@gmontard
Copy link
Contributor Author

For instance I have in my .gitignore
/storage/*

And when I scan in debug mode you can see that it goes in each file of this directory still:

{"level":"debug","time":"2022-12-19T15:20:03Z","message":"ignored due to file type /Users/guillaume/Documents/woody3/storage/eg/89/eg89qvwge5q7w5fo8azmv3ax1e7l"}
{"level":"debug","time":"2022-12-19T15:20:03Z","message":"ignored due to file type /Users/guillaume/Documents/woody3/storage/ei/8a/ei8abryomzelyy7o5p5wyt0xy2us"}
{"level":"debug","time":"2022-12-19T15:20:03Z","message":"ignored due to file type /Users/guillaume/Documents/woody3/storage/eo/29/eo29uck46rdzqvoebyidyckqlfc7"}
{"level":"debug","time":"2022-12-19T15:20:03Z","message":"ignored due to file type /Users/guillaume/Documents/woody3/storage/eo/3b/eo3bvriwbq6e2szoi9klzfcw2zgl"}
{"level":"debug","time":"2022-12-19T15:20:03Z","message":"ignored due to file type /Users/guillaume/Documents/woody3/storage/eo/j1/eoj1pd5imqf38pbtupdl4cl77kgo"}
{"level":"debug","time":"2022-12-19T15:20:03Z","message":"ignored due to file type /Users/guillaume/Documents/woody3/storage/ep/ez/epezmfevghkmdxlsosv0epnr5joo"}
{"level":"debug","time":"2022-12-19T15:20:03Z","message":"ignored due to file type /Users/guillaume/Documents/woody3/storage/eq/0r/eq0r35t1d8f68ezkyxdfb67jbyqv"}

@spdawson spdawson self-assigned this Dec 20, 2022
spdawson added a commit that referenced this issue Dec 21, 2022
@spdawson
feat: Do not visit Git-ignored files
0054cae 
When possible, obtain the list of files to be scanned by
means of `git ls-files`. This will automatically exclude files
that are covered by the Git ignore file, so that we no longer
need even visit these as we walk over the scanned directory tree.

Closes #280
@spdawson spdawson mentioned this issue Dec 21, 2022
Merged
4 tasks
@cfabianski
Copy link
Collaborator

The content of the .git folder also gets scanned :-(

spdawson added a commit that referenced this issue Dec 22, 2022
@spdawson
feat: Do not visit Git-ignored files
5944999 
When possible, obtain the list of files to be scanned by
means of `git ls-files`. This will automatically exclude files
that are covered by the Git ignore file, so that we no longer
need even visit these as we walk over the scanned directory tree.

Closes #280
spdawson added a commit that referenced this issue Dec 22, 2022
@spdawson
feat: Do not visit Git-ignored files
7ce4065 
When possible, obtain the list of files to be scanned by
means of `git ls-files`. This will automatically exclude files
that are covered by the Git ignore file, so that we no longer
need even visit these as we walk over the scanned directory tree.

Closes #280
spdawson added a commit that referenced this issue Dec 22, 2022
@spdawson
feat: Do not visit Git-ignored files
09f1304 
When possible, obtain the list of files to be scanned by
means of `git ls-files`. This will automatically exclude files
that are covered by the Git ignore file, so that we no longer
need even visit these as we walk over the scanned directory tree.

Closes #280
spdawson added a commit that referenced this issue Dec 22, 2022
@spdawson @cfabianski
feat: Do not visit Git-ignored files (#309)
06558b3 
* feat: Do not visit Git-ignored files

When possible, obtain the list of files to be scanned by
means of `git ls-files`. This will automatically exclude files
that are covered by the Git ignore file, so that we no longer
need even visit these as we walk over the scanned directory tree.

Closes #280

* Update pkg/util/gitutil/gitutil.go

Co-authored-by: Cédric Fabianski <cedric@bearer.sh>

Co-authored-by: Cédric Fabianski <cedric@bearer.sh>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@spdawson spdawson

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: Do not visit Git-ignored files Bearer/bearer
3 participants
@cfabianski @spdawson @gmontard