-
Notifications
You must be signed in to change notification settings - Fork 474
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🐛 Check OSS Fuzz build file for Fuzzing check #2719
Conversation
Signed-off-by: Spencer Schrock <sschrock@google.com>
Signed-off-by: Spencer Schrock <sschrock@google.com>
Signed-off-by: Spencer Schrock <sschrock@google.com>
Signed-off-by: Spencer Schrock <sschrock@google.com>
Signed-off-by: Spencer Schrock <sschrock@google.com>
…he other approach of looking for a substring match would lead to false positives. Signed-off-by: Spencer Schrock <sschrock@google.com>
Signed-off-by: Spencer Schrock <sschrock@google.com>
Signed-off-by: Spencer Schrock <sschrock@google.com>
Signed-off-by: Spencer Schrock <sschrock@google.com>
Signed-off-by: Spencer Schrock <sschrock@google.com>
Signed-off-by: Spencer Schrock <sschrock@google.com>
Signed-off-by: Spencer Schrock <sschrock@google.com>
Signed-off-by: Spencer Schrock <sschrock@google.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This cool! Thanks
Signed-off-by: Spencer Schrock <sschrock@google.com>
Hmm, the unit tests work locally for me when run with |
Signed-off-by: Spencer Schrock <sschrock@google.com>
Signed-off-by: Spencer Schrock <sschrock@google.com>
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #2719 +/- ##
==========================================
+ Coverage 41.20% 41.39% +0.19%
==========================================
Files 123 124 +1
Lines 10041 10164 +123
==========================================
+ Hits 4137 4207 +70
- Misses 5609 5659 +50
- Partials 295 298 +3 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we really need a OssFuzzClient at all? How bad would it be if we always made a HTTP call and parsed the JSON?
It's very simple to do it just with a HTTP call, the initial commit did just that (ignore the byte search instead of JSON parsing which is very much needed) scorecard/checks/raw/fuzzing.go Lines 173 to 185 in 5335f4b
But there are numerous problems with this.
In my opinion the |
LGTM. For context, I'm hoping to reduce the number of dependencies we inject into |
* Check OSS-Fuzz using project list Signed-off-by: Spencer Schrock <sschrock@google.com> * Use clients.RepoClient interface to perform the new OSS Fuzz check Signed-off-by: Spencer Schrock <sschrock@google.com> * wip: add eager client for better repeated lookup of projects Signed-off-by: Spencer Schrock <sschrock@google.com> * Split lazy and eager behavior into different implementations. Signed-off-by: Spencer Schrock <sschrock@google.com> * Add tests and benchmarks Signed-off-by: Spencer Schrock <sschrock@google.com> * Switch to always parsing JSON to determine if a project is present. The other approach of looking for a substring match would lead to false positives. Signed-off-by: Spencer Schrock <sschrock@google.com> * Add eager constructor to surface status file errors sooner. Signed-off-by: Spencer Schrock <sschrock@google.com> * Switch existing users to new OSS Fuzz client Signed-off-by: Spencer Schrock <sschrock@google.com> * Mark old method as deprecated in the godoc Signed-off-by: Spencer Schrock <sschrock@google.com> * remove unused comment. Signed-off-by: Spencer Schrock <sschrock@google.com> * Use new OSS Fuzz client in e2e test. Signed-off-by: Spencer Schrock <sschrock@google.com> * fix typo. Signed-off-by: Spencer Schrock <sschrock@google.com> * Fix potential path bug with test server. Signed-off-by: Spencer Schrock <sschrock@google.com> * Force include the two JSON files which were being ignored by .gitignore Signed-off-by: Spencer Schrock <sschrock@google.com> * trim the status json file Signed-off-by: Spencer Schrock <sschrock@google.com> --------- Signed-off-by: Spencer Schrock <sschrock@google.com> Signed-off-by: Shofiya2003 <shofiyabootwala@gmail.com>
* Check OSS-Fuzz using project list Signed-off-by: Spencer Schrock <sschrock@google.com> * Use clients.RepoClient interface to perform the new OSS Fuzz check Signed-off-by: Spencer Schrock <sschrock@google.com> * wip: add eager client for better repeated lookup of projects Signed-off-by: Spencer Schrock <sschrock@google.com> * Split lazy and eager behavior into different implementations. Signed-off-by: Spencer Schrock <sschrock@google.com> * Add tests and benchmarks Signed-off-by: Spencer Schrock <sschrock@google.com> * Switch to always parsing JSON to determine if a project is present. The other approach of looking for a substring match would lead to false positives. Signed-off-by: Spencer Schrock <sschrock@google.com> * Add eager constructor to surface status file errors sooner. Signed-off-by: Spencer Schrock <sschrock@google.com> * Switch existing users to new OSS Fuzz client Signed-off-by: Spencer Schrock <sschrock@google.com> * Mark old method as deprecated in the godoc Signed-off-by: Spencer Schrock <sschrock@google.com> * remove unused comment. Signed-off-by: Spencer Schrock <sschrock@google.com> * Use new OSS Fuzz client in e2e test. Signed-off-by: Spencer Schrock <sschrock@google.com> * fix typo. Signed-off-by: Spencer Schrock <sschrock@google.com> * Fix potential path bug with test server. Signed-off-by: Spencer Schrock <sschrock@google.com> * Force include the two JSON files which were being ignored by .gitignore Signed-off-by: Spencer Schrock <sschrock@google.com> * trim the status json file Signed-off-by: Spencer Schrock <sschrock@google.com> --------- Signed-off-by: Spencer Schrock <sschrock@google.com> Signed-off-by: Shofiya2003 <shofiyabootwala@gmail.com>
* Check OSS-Fuzz using project list Signed-off-by: Spencer Schrock <sschrock@google.com> * Use clients.RepoClient interface to perform the new OSS Fuzz check Signed-off-by: Spencer Schrock <sschrock@google.com> * wip: add eager client for better repeated lookup of projects Signed-off-by: Spencer Schrock <sschrock@google.com> * Split lazy and eager behavior into different implementations. Signed-off-by: Spencer Schrock <sschrock@google.com> * Add tests and benchmarks Signed-off-by: Spencer Schrock <sschrock@google.com> * Switch to always parsing JSON to determine if a project is present. The other approach of looking for a substring match would lead to false positives. Signed-off-by: Spencer Schrock <sschrock@google.com> * Add eager constructor to surface status file errors sooner. Signed-off-by: Spencer Schrock <sschrock@google.com> * Switch existing users to new OSS Fuzz client Signed-off-by: Spencer Schrock <sschrock@google.com> * Mark old method as deprecated in the godoc Signed-off-by: Spencer Schrock <sschrock@google.com> * remove unused comment. Signed-off-by: Spencer Schrock <sschrock@google.com> * Use new OSS Fuzz client in e2e test. Signed-off-by: Spencer Schrock <sschrock@google.com> * fix typo. Signed-off-by: Spencer Schrock <sschrock@google.com> * Fix potential path bug with test server. Signed-off-by: Spencer Schrock <sschrock@google.com> * Force include the two JSON files which were being ignored by .gitignore Signed-off-by: Spencer Schrock <sschrock@google.com> * trim the status json file Signed-off-by: Spencer Schrock <sschrock@google.com> --------- Signed-off-by: Spencer Schrock <sschrock@google.com>
* Add make targets and E2E test target for GitLab only Signed-off-by: Raghav Kaul <raghavkaul@google.com> * Add GitLab support to RepoClient Signed-off-by: Raghav Kaul <raghavkaul@google.com> * Build * Make target for e2e-gitlab-token * Only run Gitlab tests in CI that don't require a token Signed-off-by: Raghav Kaul <raghavkaul@google.com> * Add tests Signed-off-by: Raghav Kaul <raghavkaul@google.com> * Remove spurious printf Signed-off-by: Raghav Kaul <raghavkaul@google.com> * 🐛 Check OSS Fuzz build file for Fuzzing check (#2719) * Check OSS-Fuzz using project list Signed-off-by: Spencer Schrock <sschrock@google.com> * Use clients.RepoClient interface to perform the new OSS Fuzz check Signed-off-by: Spencer Schrock <sschrock@google.com> * wip: add eager client for better repeated lookup of projects Signed-off-by: Spencer Schrock <sschrock@google.com> * Split lazy and eager behavior into different implementations. Signed-off-by: Spencer Schrock <sschrock@google.com> * Add tests and benchmarks Signed-off-by: Spencer Schrock <sschrock@google.com> * Switch to always parsing JSON to determine if a project is present. The other approach of looking for a substring match would lead to false positives. Signed-off-by: Spencer Schrock <sschrock@google.com> * Add eager constructor to surface status file errors sooner. Signed-off-by: Spencer Schrock <sschrock@google.com> * Switch existing users to new OSS Fuzz client Signed-off-by: Spencer Schrock <sschrock@google.com> * Mark old method as deprecated in the godoc Signed-off-by: Spencer Schrock <sschrock@google.com> * remove unused comment. Signed-off-by: Spencer Schrock <sschrock@google.com> * Use new OSS Fuzz client in e2e test. Signed-off-by: Spencer Schrock <sschrock@google.com> * fix typo. Signed-off-by: Spencer Schrock <sschrock@google.com> * Fix potential path bug with test server. Signed-off-by: Spencer Schrock <sschrock@google.com> * Force include the two JSON files which were being ignored by .gitignore Signed-off-by: Spencer Schrock <sschrock@google.com> * trim the status json file Signed-off-by: Spencer Schrock <sschrock@google.com> --------- Signed-off-by: Spencer Schrock <sschrock@google.com> --------- Signed-off-by: Raghav Kaul <raghavkaul@google.com> Signed-off-by: Spencer Schrock <sschrock@google.com> Co-authored-by: Spencer Schrock <sschrock@google.com>
commit 00da7a8be965c09d044f978d6b9eafee1350bd30 Author: Azeem Shaikh <azeemshaikh38@gmail.com> Date: Tue Mar 14 23:07:19 2023 +0000 Pr comments commit 1127dd9 Merge: 274448f 23bd295 Author: Azeem Shaikh <azeemshaikh38@gmail.com> Date: Wed Mar 15 04:23:32 2023 +0530 Merge branch 'main' into go-git commit 274448f Author: Azeem Shaikh <azeemshaikh38@gmail.com> Date: Tue Mar 14 22:52:30 2023 +0000 Initial implementation of go-git client Signed-off-by: Azeem Shaikh <azeemshaikh38@gmail.com> commit 23bd295 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Mar 14 20:28:41 2023 +0000 :seedling: Bump github/codeql-action from 2.2.4 to 2.2.6 (ossf#2741) commit fc026ef Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Mar 14 17:04:31 2023 +0000 :seedling: Bump github.com/google/ko from 0.12.0 to 0.13.0 in /tools (ossf#2742) commit 2e04214 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Mar 14 14:02:34 2023 +0000 :seedling: Bump tj-actions/changed-files from 35.6.2 to 35.7.0 Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 35.6.2 to 35.7.0. - [Release notes](https://github.com/tj-actions/changed-files/releases) - [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md) - [Commits](tj-actions/changed-files@5ce975c...bd376fb) --- updated-dependencies: - dependency-name: tj-actions/changed-files dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> commit e36b590 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue Mar 14 08:59:20 2023 -0500 :seedling: Bump actions/cache from 3.3.0 to 3.3.1 (ossf#2740) Bumps [actions/cache](https://github.com/actions/cache) from 3.3.0 to 3.3.1. - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@940f3d7...88522ab) --- updated-dependencies: - dependency-name: actions/cache dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 6ff94eb Author: Gabriela Gutierrez <gabigutierrez@google.com> Date: Mon Mar 13 19:42:37 2023 +0000 :bug: Handle editable pip installs (ossf#2731) * fix: Handle editable pip install Editable pip installs (-e) should be considered secure if the package is installed from a local source or a remote source (VCS install) but pinned by commit hash. To keep the behaviour we have for normal pip installs, we need to guarantee the package dependencies are pinned by hash too. For normal pip installs, we verify that by using --require-hashes flag. Unfortunately, --require-hashes flag is not compatible with editable installs, so we use --no-deps flag to verify the dependencies are not installed since we can't verify if they are pinned. Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> * test: Editable pip install in GHA Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> * test: Editable pip install in Dockerfile Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> * test: Editable pip install in shell script Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> * fix: Code complexity increase Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> * fix: Simplify boolean return Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> * docs: Add pip editable install references in comments Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> * fix: Handle multiple packages in editable pip install Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> * test: Multi editable pip install in GHA Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> * test: Multi editable pip install in Dockerfile Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> * test: Multi editable pip install in shell script Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> --------- Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com> Co-authored-by: laurentsimon <64505099+laurentsimon@users.noreply.github.com> commit 110e352 Author: raghavkaul <8695110+raghavkaul@users.noreply.github.com> Date: Mon Mar 13 11:13:50 2023 -0400 ✨ Gitlab support: RepoClient (ossf#2655) * Add make targets and E2E test target for GitLab only Signed-off-by: Raghav Kaul <raghavkaul@google.com> * Add GitLab support to RepoClient Signed-off-by: Raghav Kaul <raghavkaul@google.com> * Build * Make target for e2e-gitlab-token * Only run Gitlab tests in CI that don't require a token Signed-off-by: Raghav Kaul <raghavkaul@google.com> * Add tests Signed-off-by: Raghav Kaul <raghavkaul@google.com> * Remove spurious printf Signed-off-by: Raghav Kaul <raghavkaul@google.com> * 🐛 Check OSS Fuzz build file for Fuzzing check (ossf#2719) * Check OSS-Fuzz using project list Signed-off-by: Spencer Schrock <sschrock@google.com> * Use clients.RepoClient interface to perform the new OSS Fuzz check Signed-off-by: Spencer Schrock <sschrock@google.com> * wip: add eager client for better repeated lookup of projects Signed-off-by: Spencer Schrock <sschrock@google.com> * Split lazy and eager behavior into different implementations. Signed-off-by: Spencer Schrock <sschrock@google.com> * Add tests and benchmarks Signed-off-by: Spencer Schrock <sschrock@google.com> * Switch to always parsing JSON to determine if a project is present. The other approach of looking for a substring match would lead to false positives. Signed-off-by: Spencer Schrock <sschrock@google.com> * Add eager constructor to surface status file errors sooner. Signed-off-by: Spencer Schrock <sschrock@google.com> * Switch existing users to new OSS Fuzz client Signed-off-by: Spencer Schrock <sschrock@google.com> * Mark old method as deprecated in the godoc Signed-off-by: Spencer Schrock <sschrock@google.com> * remove unused comment. Signed-off-by: Spencer Schrock <sschrock@google.com> * Use new OSS Fuzz client in e2e test. Signed-off-by: Spencer Schrock <sschrock@google.com> * fix typo. Signed-off-by: Spencer Schrock <sschrock@google.com> * Fix potential path bug with test server. Signed-off-by: Spencer Schrock <sschrock@google.com> * Force include the two JSON files which were being ignored by .gitignore Signed-off-by: Spencer Schrock <sschrock@google.com> * trim the status json file Signed-off-by: Spencer Schrock <sschrock@google.com> --------- Signed-off-by: Spencer Schrock <sschrock@google.com> --------- Signed-off-by: Raghav Kaul <raghavkaul@google.com> Signed-off-by: Spencer Schrock <sschrock@google.com> Co-authored-by: Spencer Schrock <sschrock@google.com> commit 5625dda Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Sat Mar 11 17:14:42 2023 +0000 :seedling: Bump github.com/onsi/ginkgo/v2 from 2.8.3 to 2.9.0 in /tools Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.8.3 to 2.9.0. - [Release notes](https://github.com/onsi/ginkgo/releases) - [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md) - [Commits](onsi/ginkgo@v2.8.3...v2.9.0) --- updated-dependencies: - dependency-name: github.com/onsi/ginkgo/v2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> commit d591e38 Author: Spencer Schrock <sschrock@google.com> Date: Fri Mar 10 16:02:05 2023 -0800 🌱 Add RepoClient re-use E2E tests. (ossf#2625) * Add e2e test for re-used repoclient. Signed-off-by: Spencer Schrock <sschrock@google.com> * Improve diff logging Signed-off-by: Spencer Schrock <sschrock@google.com> * Skip scorecard e2e test during unit tests. Signed-off-by: Spencer Schrock <sschrock@google.com> * Fix linter. Signed-off-by: Spencer Schrock <sschrock@google.com> --------- Signed-off-by: Spencer Schrock <sschrock@google.com> commit a7e81bb Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Mar 10 08:20:28 2023 -0600 :seedling: Bump actions/cache from 3.2.6 to 3.3.0 (ossf#2738) Bumps [actions/cache](https://github.com/actions/cache) from 3.2.6 to 3.3.0. - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@69d9d44...940f3d7) --- updated-dependencies: - dependency-name: actions/cache dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
What kind of change does this PR introduce?
bug fix
What is the current behavior?
The fuzzing check uses a GitHub repo client to search
google/oss-fuzz
for a project to determine if it's fuzzed.Searching the
google/oss-fuzz
hasn't been working for a while.What is the new behavior (if this is a feature change)?**
There's a new
ossfuzz
client which only implements theSearch
method.This method fetches the OSS Fuzz status file once and parses the projects.
Using a client involves stubbing all of the unneeded methods, but allows the change to be made in a backwards compatible manner for everyone that calls
pkg.RunScorecard
.Which issue(s) this PR fixes
Fixes #2670
Special notes for your reviewer
Does this PR introduce a user-facing change?
For user-facing changes, please add a concise, human-readable release note to
the
release-note
(In particular, describe what changes users might need to make in their
application as a result of this pull request.)