Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate OSS-Fuzz feed functionality to independent infra #2892

Open
oliverchang opened this issue Nov 25, 2024 · 1 comment
Open

Migrate OSS-Fuzz feed functionality to independent infra #2892

oliverchang opened this issue Nov 25, 2024 · 1 comment
Assignees
@oliverchang
Labels
cleanup Code hygiene and cleanup enhancement New feature or request infra infrastructure bugs/FRs

Comments

@oliverchang
Copy link
Collaborator

There are a number of issues with the current infrastructure around reliability and maintainability.

  1. [Reliability] The current communication protocol occasionally has issues. Messages get dropped (or never sent) from OSS-Fuzz, leading to inconsistent states in OSS-Fuzz-Vulns (such as missing fixes). This is because messages are only ever sent once per OSS-Fuzz crash with no ability to retry.
  2. [Reliability] The current infrastructure does not account for all cases where issues in OSS-Fuzz that started off being legitimate then gets marked as WontFix (Propagate WontFixed OSS-Fuzz vulns into withdrawn entries on oss-fuzz-vulns.  #966).
  3. [Maintainability] Domain-specific OSS-Fuzz behaviour is tied to OSV infrastructure, and nobody is familiar with it apart from @oliverchang . This also increases our maintenance burden as OSV has a dependency on gVisor and the ability to build OSS-Fuzz projects. Removing this will additionally simplify other parts of OSV (such as the need to support private Bug entities).

This will be addressed in 2 stages:

  1. Move triggering of OSS-Fuzz bisections into an independent cron job as part of OSS-Fuzz-Vulns infrastructure. This will address reliability issues, as this will be a continuously running cron job that will check both issue state and what’s already analyzed in OSV.
  2. (Some time in 2025. Details TBD). Move OSS-Fuzz specific worker functionality into independent infrastructure in OSS-Fuzz-Vulns.
@oliverchang oliverchang added cleanup Code hygiene and cleanup enhancement New feature or request labels Nov 25, 2024
@oliverchang oliverchang self-assigned this Nov 25, 2024
@oliverchang oliverchang added the infra infrastructure bugs/FRs label Nov 25, 2024
oliverchang pushed a commit to google/oss-fuzz-vulns that referenced this issue Nov 27, 2024
Add standalone script for managing OSS-Fuzz -> OSV.
408167c 
This adds the start of standalone infrastructure to manage syncing of
state between OSS-Fuzz issues and the OSS-Fuzz OSV feed.

This replaces existing functionality inside ClusterFuzz to send requests
to OSV, which had issues with reliability.

Instead, this script queries the OSS-Fuzz issue tracker as the source of
truth to generate requests to OSV.dev. In the future, we'll move the
bisection and OSS-Fuzz-Vulns record management functionality out of
OSV.dev.

This is part of google/osv.dev#2892.
@oliverchang oliverchang mentioned this issue Nov 27, 2024
Open
@oliverchang
Copy link
Collaborator Author

Step 1 here: google/oss-fuzz-vulns#39

oliverchang added a commit to google/oss-fuzz-vulns that referenced this issue Nov 27, 2024
@oliverchang
Add standalone script for managing OSS-Fuzz -> OSV.
b8d1d42 
This adds the start of standalone infrastructure to manage syncing of
state between OSS-Fuzz issues and the OSS-Fuzz OSV feed.

This replaces existing functionality inside ClusterFuzz to send requests
to OSV, which had issues with reliability.

Instead, this script queries the OSS-Fuzz issue tracker as the source of
truth to generate requests to OSV.dev. In the future, we'll move the
bisection and OSS-Fuzz-Vulns record management functionality out of
OSV.dev.

This is part of google/osv.dev#2892.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@oliverchang oliverchang

Labels
cleanup Code hygiene and cleanup enhancement New feature or request infra infrastructure bugs/FRs
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@oliverchang