Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fixup] - skip files in the archive handler #2195

Merged
merged 3 commits into from
Dec 9, 2023

Conversation

ahrav
Copy link
Collaborator

@ahrav ahrav commented Dec 9, 2023

Description:

We should skip binary files that are within archived pkgs.
So we can avoid: https://console.cloud.google.com/logs/query;query=resource.type%3D%22k8s_container%22%0Aresource.labels.project_id%3D%22thog-prod%22%0Aresource.labels.location%3D%22us-central1%22%0Aresource.labels.cluster_name%3D%22prod-c1%22%0Aresource.labels.namespace_name%3D%22capable-energetic-pearl%22%0Alabels.k8s-pod%2Fapp%3D%22scanner-api%22%20severity%3E%3DDEFAULT%0Aerror%0A--%20%22max%20archive%20depth%22%0A--%20%22Max%20archive%20size%20reached%22%0A--%20%22scanning%20repo%22%0A--%20%22Starting%20to%20scan%20repo%22%0A--%20%22Enumerated%20GitLab%20projects%22%0A--%20%22memory%20pressure%20detected,%20throttling%20concurrency%22%0A--%20error%0A--%20jsonPayload.log.source_name%3D%22MnS%20Confluence%22%0A--%20jsonPayload.log.source_name%3D%22MnS%20Sharepoint%22%0A--%20%22updating%20final%20job%20status%22%0A--%20%22falling%20back%20to%20basic%20auth%22%0A--%20error%0A--%20%22could%20not%20get%20content%22%0A--%20%22rolling%20back%22%0A--%20%22Completed%20scanning%20confluence%20space%22%0A--%20%22Restaurant%20Platform%20Engineering%22%0A--%20%22Teams%22%0A--%20%22Restaurant%22%0A--%20%22Chick%22%0A--%20%22encountered%20errors%20while%20scanning%20confluence%22%0A--%20%22TEAM%22%0A-jsonPayload.log.msg%3D%22notified%20for%20secret%22;pinnedLogId=2023-11-29T19:52:49.297222537Z%2F64w54alutkgev3d1;summaryFields=jsonPayload%252Flog%252Fsource_name:false:32:beginning;cursorTimestamp=2023-12-09T00:25:59.910110038Z;duration=PT1H?project=thog-prod

Checklist:

  • Tests passing (make test-community)?
  • Lint passing (make lint this requires golangci-lint)?

@ahrav ahrav marked this pull request as ready for review December 9, 2023 01:13
@ahrav ahrav requested a review from a team as a code owner December 9, 2023 01:13
@ahrav ahrav requested a review from a team December 9, 2023 01:15
@ahrav ahrav merged commit 331336d into main Dec 9, 2023
8 checks passed
@ahrav ahrav deleted the fixup-skip-files-archive-handler branch December 9, 2023 04:23
haraldh referenced this pull request in matter-labs/vault-auth-tee Dec 14, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[trufflesecurity/trufflehog](https://github.com/trufflesecurity/trufflehog)
| action | patch | `v3.63.2` -> `v3.63.3` |

---

### Release Notes

<details>
<summary>trufflesecurity/trufflehog
(trufflesecurity/trufflehog)</summary>

###
[`v3.63.3`](https://github.com/trufflesecurity/trufflehog/releases/tag/v3.63.3)

[Compare
Source](https://github.com/trufflesecurity/trufflehog/compare/v3.63.2...v3.63.3)

#### What's Changed

- Use forked sevenzip by
[@&#8203;bill-rich](https://github.com/bill-rich) in
[https://github.com/trufflesecurity/trufflehog/pull/2180](https://github.com/trufflesecurity/trufflehog/pull/2180)
- fixing how to rotate URL by
[@&#8203;dylanTruffle](https://github.com/dylanTruffle) in
[https://github.com/trufflesecurity/trufflehog/pull/2183](https://github.com/trufflesecurity/trufflehog/pull/2183)
- \[fixup] - Skip trying to determine MIME type for directories by
[@&#8203;ahrav](https://github.com/ahrav) in
[https://github.com/trufflesecurity/trufflehog/pull/2178](https://github.com/trufflesecurity/trufflehog/pull/2178)
- \[feat] - Remove go-git dependency by
[@&#8203;ahrav](https://github.com/ahrav) in
[https://github.com/trufflesecurity/trufflehog/pull/2174](https://github.com/trufflesecurity/trufflehog/pull/2174)
- remove unnecessary Git cmd check by
[@&#8203;ahrav](https://github.com/ahrav) in
[https://github.com/trufflesecurity/trufflehog/pull/2175](https://github.com/trufflesecurity/trufflehog/pull/2175)
- \[chore] - use https for verification endpoints by
[@&#8203;ahrav](https://github.com/ahrav) in
[https://github.com/trufflesecurity/trufflehog/pull/2185](https://github.com/trufflesecurity/trufflehog/pull/2185)
- allow targets for the source manager by
[@&#8203;ahrav](https://github.com/ahrav) in
[https://github.com/trufflesecurity/trufflehog/pull/2182](https://github.com/trufflesecurity/trufflehog/pull/2182)
- Deprecate some detectors by
[@&#8203;dustin-decker](https://github.com/dustin-decker) in
[https://github.com/trufflesecurity/trufflehog/pull/2186](https://github.com/trufflesecurity/trufflehog/pull/2186)
- \[chore] - update regex by [@&#8203;ahrav](https://github.com/ahrav)
in
[https://github.com/trufflesecurity/trufflehog/pull/2184](https://github.com/trufflesecurity/trufflehog/pull/2184)
- \[chore] - Compile regex once by
[@&#8203;ahrav](https://github.com/ahrav) in
[https://github.com/trufflesecurity/trufflehog/pull/2176](https://github.com/trufflesecurity/trufflehog/pull/2176)
- Remove Java archives from ignored extensions by
[@&#8203;rosecodym](https://github.com/rosecodym) in
[https://github.com/trufflesecurity/trufflehog/pull/2188](https://github.com/trufflesecurity/trufflehog/pull/2188)
- \[chore] - Refactor common code into a separate function by
[@&#8203;ahrav](https://github.com/ahrav) in
[https://github.com/trufflesecurity/trufflehog/pull/2179](https://github.com/trufflesecurity/trufflehog/pull/2179)
- \[feat] - add metrics for gitlab by
[@&#8203;ahrav](https://github.com/ahrav) in
[https://github.com/trufflesecurity/trufflehog/pull/2190](https://github.com/trufflesecurity/trufflehog/pull/2190)
- \[bug] - move logic to main Chunks method by
[@&#8203;ahrav](https://github.com/ahrav) in
[https://github.com/trufflesecurity/trufflehog/pull/2194](https://github.com/trufflesecurity/trufflehog/pull/2194)
- \[fixup] - skip files in the archive handler by
[@&#8203;ahrav](https://github.com/ahrav) in
[https://github.com/trufflesecurity/trufflehog/pull/2195](https://github.com/trufflesecurity/trufflehog/pull/2195)
- Check private keys concurrently by
[@&#8203;rgmz](https://github.com/rgmz) in
[https://github.com/trufflesecurity/trufflehog/pull/2139](https://github.com/trufflesecurity/trufflehog/pull/2139)
- Propagate TruffleHog context to handlers by
[@&#8203;rgmz](https://github.com/rgmz) in
[https://github.com/trufflesecurity/trufflehog/pull/2191](https://github.com/trufflesecurity/trufflehog/pull/2191)
- \[bug] - close file after reading by
[@&#8203;ahrav](https://github.com/ahrav) in
[https://github.com/trufflesecurity/trufflehog/pull/2203](https://github.com/trufflesecurity/trufflehog/pull/2203)
- Use bad json in slackwebhooks by
[@&#8203;rosecodym](https://github.com/rosecodym) in
[https://github.com/trufflesecurity/trufflehog/pull/2193](https://github.com/trufflesecurity/trufflehog/pull/2193)
- Add disk buffer tempfile cleanup by
[@&#8203;codevbus](https://github.com/codevbus) in
[https://github.com/trufflesecurity/trufflehog/pull/2130](https://github.com/trufflesecurity/trufflehog/pull/2130)
- \[chore] Remove omitempty tags on JobProgressMetrics and UnitMetrics
by [@&#8203;mcastorina](https://github.com/mcastorina) in
[https://github.com/trufflesecurity/trufflehog/pull/2204](https://github.com/trufflesecurity/trufflehog/pull/2204)
- Fix azurestorage detector by [@&#8203;0x1](https://github.com/0x1)
in
[https://github.com/trufflesecurity/trufflehog/pull/2207](https://github.com/trufflesecurity/trufflehog/pull/2207)
- fix and refactor browserstack detector by
[@&#8203;0x1](https://github.com/0x1) in
[https://github.com/trufflesecurity/trufflehog/pull/2208](https://github.com/trufflesecurity/trufflehog/pull/2208)
- \[chore] Remove unnecessary string conversion in tefter detector by
[@&#8203;mcastorina](https://github.com/mcastorina) in
[https://github.com/trufflesecurity/trufflehog/pull/2209](https://github.com/trufflesecurity/trufflehog/pull/2209)
- Update metabase verification to check for a valid JSON response by
[@&#8203;mcastorina](https://github.com/mcastorina) in
[https://github.com/trufflesecurity/trufflehog/pull/2210](https://github.com/trufflesecurity/trufflehog/pull/2210)

**Full Changelog**:
trufflesecurity/trufflehog@v3.63.2...v3.63.3

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/matter-labs/vault-auth-tee).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy44Ny4yIiwidXBkYXRlZEluVmVyIjoiMzcuODcuMiIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants