Scan only images that were pulled within predefined time frame

[dkulchinsky]

Is your feature request related to a problem? Please describe.
Our Harbor registry has 7 projects and over 100,000 image tags across ~1,000 repos.

In the past we relied on the SCAN_ALL job to ensure that the vulnerability reports are up to date, however as our registry grew the SCAN_ALL job simply doesn't scale, it can take several days to scan all the images and also resulted in few incidents where it caused the registry to stall.

Describe the solution you'd like
We would like to see more controls on how the SCAN_ALL runs (or an additional SCAN job that provides more controls), specifically:

1. Scan only image tags that were pulled within a provided duration (e.g. last 30 days), avoid scanning stale/old images
2. Introduce filters to select which projects to scan (or skip)
3. Introduce filters to select which repos to scan (or skip)

The above expands what I proposed in #15652 but never received any meaningful feedback on.

Describe the main design/architecture of your solution
I think the above illustrates well what is needed.

Describe the development plan you've considered
Not sure how to answer that.

Additional context
Add any other context or screenshots about the feature request here.

[OrlinVasilev]

added to 2.7 requirement collection! #16775

[EdKingscote]

Being able to configure scanning rules as part of the project policies alongside retention and immutability would be awesome.

[qnetter]

I think it is a worthwhile addition but will not fit in 2.7 unless someone in the community volunteers to work on it.

[dkulchinsky]

I think it is a worthwhile addition but will not fit in 2.7 unless someone in the community volunteers to work on it.

that's a shame, according to @OrlinVasilev this was flagged for 2.7, what has chnaged?

[qnetter]

It was flagged as a 2.7 candidate, not as a commitment. The 2.7 development window is shorter than usual because of China and US holiday closures during this period between now and end of year.

[dkulchinsky]

It was flagged as a 2.7 candidate, not as a commitment. The 2.7 development window is shorter than usual because of China and US holiday closures during this period between now and end of year.

I see, thanks for clarifying.

Is there a list of enhancements that were committed to 2.7? I couldn't see that in the discussion issue

Also, looks like this specific item is one of the most voted one, so was hoping it would get prioritized accordingly.

[qnetter]

I should correct myself - it is still on the 2.7 list, but at low priority so may fall off if we run out of time.

I will update the visible list shortly.

[dkulchinsky]

@qnetter @OrlinVasilev This was originally flagged to be added to 2.7 roadmap, but looks like never made the cut.

Is this is still being considered?

[OrlinVasilev]

@dkulchinsky we are a bit short on hands to take all new requirements and features!
can you please add it in the discussion and if you feel like developing it :) there are few new proposal in the community repository about scanning you can check them as well ! :) THanks!