Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement a "delta-analysis" of only the changed dependencies in a given project #4347

Open
porsche-rishisaxena opened this issue Aug 9, 2021 · 17 comments
Labels
analyzer About the analyzer tool new feature Issues that are considered to be new features

Comments

@porsche-rishisaxena
Copy link

porsche-rishisaxena commented Aug 9, 2021

Is there any road-map designed for enabling "delta-scan" feature for Scan-Code from ORT standpoint?
If yes, please share some information.

Edit by @sschuberth: Please see this comment which clarifies that actually no delta-scan but a delta-analysis is desired. The above mention of "Scan-Code" (sic) is misleading.

@sschuberth

This comment was marked as off-topic.

@sschuberth sschuberth added question An issue that is actually a question scanner About the scanner tool labels Aug 9, 2021
@nikpete

This comment was marked as resolved.

@tsteenbe

This comment was marked as off-topic.

@sschuberth

This comment was marked as off-topic.

@fviernau

This comment was marked as off-topic.

@sschuberth

This comment was marked as off-topic.

@fviernau

This comment was marked as off-topic.

@rbieniek
Copy link

rbieniek commented Aug 24, 2021

First of all, I would like to clarify what we at Porsche understand as a delta scan. Given our legal workflow, we want to build a full report for the first iteration of a project. For any subsequent release / revision, we want the lawyers to revisit only those dependencies which have changed. And by changed, we mean a dependency which has either been freshly introduced or updated with a more recent version.

To break it down into use cases, there are three different cases to be considered for a generic delta scan capability, given that you want to build a delta between two iterations (v1 and v2) of the same project:

  1. some new dependencies are introduced in the transition from v1 to v2
  2. the set of dependencies stays the same but updated versions of dependencies have been introduced
  3. dependencies are removed in the transition between v1 and v2

The third use case is, of course, trivial and IMHO there is nothing to be done for it.

For other use cases it is not that trivial. The afformentioned hash calculation can be a quick way to detect the need for a delta scan. But it doesn't solve the problem.

We built a simple test case to demonstrate our requirements and the expected result:

  • The base project is a simple spring boot project with a simple set of dependencies:
<dependencies>
        <dependency>
	        <groupId>org.springframework.boot</groupId>
	        <artifactId>spring-boot-starter-validation</artifactId>
        </dependency>
        <dependency>
	        <groupId>org.springframework.boot</groupId>
	        <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
	        <groupId>org.springframework.boot</groupId>
	        <artifactId>spring-boot-starter-test</artifactId>
	        <scope>test</scope>
        </dependency>
</dependencies>

This set of depenencies models our baseline project andn would be passed over to the lawyer for review

  • in the second iteration, we introduce just one additional dependency (spring data mongodb), thus leading to this set of dependencies:
<dependencies>
        <dependency>
	        <groupId>org.springframework.boot</groupId>
	        <artifactId>spring-boot-starter-validation</artifactId>
        </dependency>
        <dependency>
	        <groupId>org.springframework.boot</groupId>
	        <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-data-mongodb</artifactId>
        </dependency>
        <dependency>
	        <groupId>org.springframework.boot</groupId>
	        <artifactId>spring-boot-starter-test</artifactId>
	        <scope>test</scope>
        </dependency>
</dependencies>
  • Our expectation is that a delta scan results consists only of the spring boot data mongodb dependency and all the additional dependencies the spring data mongodb dependency relies on.

We have given it a try by using the orth subtract scan result function to build a difference, thus delta, between the two revisions but this didn't deliver a minimal set of add dependecies that we expected.

Implementationwise, I think ORT needs to be enhanced with a persistence capability to store the overall scan result per project and revision / timestamp of the scan, so that the difference between project iterations can be determined in the first place.
This persistence capabilty should support filesystem-based persistence as well as database- oder ReST-based persistence. And it should support an efficient way to determine the changeset between two project iterations.

@fviernau

This comment was marked as resolved.

@nikpete

This comment was marked as outdated.

@sschuberth

This comment was marked as resolved.

@nikpete
Copy link

nikpete commented Aug 26, 2021

We want to report findings from any newly added, changed and deleted dependencies.

Just to give you a simple example:

Version 1.0:
100 dependencies identified, curated and uploaded in the legal workflow tool => all good

now version 1.1 is coming around which might consist of 110 dependencies with the following structure:

  • 100 dependencies are the same as in version 1.0
  • 15 dependencies newly added
  • 5 dependencies have been deleted from version 1.0
  • 10 dependencies updated

Conclusion: we need to idetify the "delta", meaning we need to be able to report the 30 dependencies that have been either deleted, added or updated.

Do we have here the same udnerstanding?

I would be very happy if this can be already handled on the analyzer level?

@sschuberth

This comment was marked as resolved.

@sschuberth sschuberth changed the title Understanding on the Road-Map on Delta Scan Feature for Scan-Code Implement a "delta-analysis" of only the changed dependencies in a given project Aug 26, 2021
@sschuberth sschuberth added the on hold Pull requests that cannot currently be merged label Sep 16, 2021
@sschuberth

This comment was marked as outdated.

@arieltorti

This comment was marked as resolved.

@sschuberth

This comment was marked as resolved.

@arieltorti

This comment was marked as resolved.

@sschuberth sschuberth added the to triage Issues that need triaging label Jun 3, 2024
@sschuberth sschuberth added analyzer About the analyzer tool new feature Issues that are considered to be new features and removed question An issue that is actually a question scanner About the scanner tool on hold Pull requests that cannot currently be merged to triage Issues that need triaging labels Aug 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
analyzer About the analyzer tool new feature Issues that are considered to be new features
Projects
None yet
Development

No branches or pull requests

7 participants