Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add new pipeline for basic codebase scan #37

Closed
pombredanne opened this issue Oct 28, 2020 · 1 comment
Closed

Add new pipeline for basic codebase scan #37

pombredanne opened this issue Oct 28, 2020 · 1 comment
Labels
enhancement New feature or request high priority
Milestone
1.1

Comments

@pombredanne
Copy link
Member

pombredanne commented Oct 28, 2020 •
edited
Loading

The input to that pipeline would be code archive(s).
The pipeline would:

  1. extract the archives (not recursively at first)
  2. run the equivalent of scancode -clipeu scan on that code
  3. create one or more inventory analysis "workfile" report as CSV, JSON (or XLS TBD) listing all the captured resources and packages in a format TBD.

The reports would be created by the pipeline and stored on disk to be retrieved by the API
@pombredanne pombredanne added this to the 1.1 milestone Oct 28, 2020
@pombredanne pombredanne added enhancement New feature or request high priority labels Oct 28, 2020
tdruez added a commit that referenced this issue Nov 5, 2020
@tdruez
Add a outputs pipe module including csv export of results #37
28d2fc3 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Nov 5, 2020
@tdruez
Add a new ScanCodebase Pipeline #37
4e08e25 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Nov 5, 2020
@tdruez
Use explicit SUCCESS and FAILURE in place of S and F #37
a87b91b 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Nov 5, 2020
@tdruez
Fix failing test_scanpipe_pipes_outputs_as_csv #37
fc283e4 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Nov 5, 2020
@tdruez
Add a --run option in the create-project management command #37
8419b03 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Nov 6, 2020
@tdruez
Add --license-text to the scancode_options #37
0e90273 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Nov 6, 2020
@tdruez
Extract the JSON input logic from the get_virtual_codebase #37
c365d36 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Nov 6, 2020
@tdruez
Rename the scan_inventory pipeline to load_inventory #37
a8d75bf 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Nov 6, 2020
@tdruez
Move run_extractcode and run_scancode as scancode pipes #37
4deed90 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Nov 6, 2020
@tdruez
Add ability to provide default options for run_scancode #37
65fa330 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Nov 6, 2020
@tdruez
Rename as_csv to to_csv #37
dd3d300 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Nov 6, 2020
@tdruez
Add support for raising on scancode/extractcode errors #37
3923207 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Nov 6, 2020
@tdruez
Add documentation about the settings and new pipeline #37
2ce2a20 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Nov 6, 2020
@tdruez
Add missing project name in command #37
6cfcca8 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Nov 9, 2020
@tdruez
Add entry in the CHANGELOG.rst file #37
b4be068 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Nov 10, 2020
@tdruez
Merge main branch #37
e297c12 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
tdruez added a commit that referenced this issue Nov 10, 2020
@tdruez
Remove unused import #37
8a22e7f 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
pombredanne added a commit that referenced this issue Nov 10, 2020
@pombredanne
Merge pull request #39 from nexB/37-basic-scan
f03d04b 
Add new pipeline for codebase scan for #37

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@tdruez
Copy link
Contributor

tdruez commented Nov 10, 2020

New ScanCodebase Pipeline available at https://github.com/nexB/scancode.io/blob/main/scanpipe/pipelines/scan_codebase.py
Documented in https://scancodeio.readthedocs.io/en/latest/scanpipe-pipelines.html#scan-codebase and https://scancodeio.readthedocs.io/en/latest/scanpipe-tutorial-2.html

@tdruez tdruez closed this as completed Nov 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request high priority
Projects
None yet
Milestone
1.1
Development

No branches or pull requests
2 participants
@tdruez @pombredanne