Table of contents

• The problem
• The solution
• Project Layout
• Enabling CVE scanning in your project
  • NodeJS
  • Python
  • Maven
  • Gradle
  • Scala
  • Rust
  • .NET
  • Docker
  • Other languages and build platforms
• Dependency update tool
• Static code analysis
• License reporting and scanning
• Roadmap
• Contributing
• License

The problem

Given the wide range of platforms, languages and build systems used by FINOS projects, finding one solution that secures a codebase is not an easy task, especially considering the incredible amount of libraries available in public library repositories, which can be easily used, embedded, integrated and re-published; this proliferation of artifacts have dramatically influenced software development:

• On average, 95% of the code shipped in a software artifact is composed of upstream libraries (aka dependencies), built, released and managed by external teams, communities and companies that the consumer has no control/influence over.
• A developer has very little awareness of the codebase quality and software development process in the upstream dependencies of a project, unless going through code scrutiny, which is difficult and time consuming
• Every programming language and build tool has a different way of consuming dependencies, making security tool adoption harder and rarer; as a consequence, more security vulnerabilities are released into public library repositories, which leads to the exponential growth of vulnerabilities and risk for all consumers using these libraries

The solution

Let's first recap requirements, based on the considerations made above; a code scan should be:

• Proactive (triggered periodically, ie every day) and reactive (triggered on code changes)
• Compatible with all languages and build platforms adopted by FINOS hosted projects
• Easy to operate by project teams, git-based, without the need for external dashboards
• Integrated into FINOS project onboarding process and FINOS CVE Disclosure Policy
• Monitorable by FINOS Staff, allowing us to provide a proactive support to our projects

The proactive/reactive approach is crucial to enforce security, as it guarantees - granted that changes are always submitted via Pull Requests - that the code will always be free of CVEs (or at most for less than a day):

• The reactive setup will fail any Pull Request where code change introduces a new CVE from the (updated) dependency list
• The proactive setup will notify (ie on a daily schedule) the team if a new CVE - that affects a library in the dependency list - have been published

Based on the requirements discussed above, we tried to consolidate a list of technical specifications:

• 6 supported build platforms - Maven, Gradle, Python (and Poetry), Scala (with SBT), NodeJS and Rust
• CVE scanning
  • Can be configured to only scan runtime dependencies
  • Scans direct and transitive dependencies
  • Ability to ignore warnings/errors, using a git-hosted file
  • Ability to run as part of CI/CD (GitHub actions)
• Static code analysis
  • Ability to run as part of CI/CD (GitHub actions)
• Documentation that describes how to use the scanning and what to expect

It's worth emphasizing the importance of having a mechanism for ignoring warnings and errors, which may well be false positives. Without it, developers will eventually disable any security tool. And it’s important to store it in the codebase so that changes can be easily done by developers using the Git collaboration workflow.

Project Layout

In this codebase you'll find a folder for each of the build platforms listed below. Each folder includes a Hello World project, with a build descriptor that:

1. Pulls in a CVE
2. Configures a CVE scanning tool that is specific to the build tool
3. Defines a list of ignored warnings/errors caused by the CVE scanning

In the .github/workflows folder you'll find a GitHub Action for each of these projects, that you can simply copy/paste into your GitHub repository and edit to align to your project layout:

• Update the name of the GitHub Action file to cve-scanning.yml (and semgrep.yml), which allows FINOS to monitor which projects are/aren't adopting the tool; the file name is also reflected in the on: / push: / path: section of the action
• If the build files are located in the root project folder, remove all working-directory configurations
• Adapt runtime versions (ie Node, Python, JVM, etc) with the ones used in your projects

Enabling CVE scanning in your project

1. Identify the language(s) and build system(s) used in the repository you want to scan.
2. Checkout your repository locally.
3. Find - from the list below - which sections applies to you.
4. Follow the instructions to run the scan locally, make sure that the scan runs successfully and generates a list of CVEs.
5. Investigate CVEs, one by one; the majority of CVEs can be addressed by updating a given library to a newer version; in some cases, you'll find out that you're using a certain library in an unsecure way; in some other cases, you may stumble on false positives (that is, a CVE that doesn't apply to your codebase) and therefore you'd have to ignore the error by updating the ignore list file.
6. Copy the related GitHub Action (in .github/workflows) into your project; make sure to call them cve-scanning.yml, so that FINOS monitoring tools can find easily find it.
7. From the GitHub Actions tab, you can select the CVE Scanning action and Create status badge, which will allow you to copy Markdown code for your README.md file that shows a badge with the result of the last action run; this is quite useful for consumers to see that code is scanned and that no CVEs were spotted in the main codebase branch.
8. Push the changes to GitHub and checkout the Github Action run and output.

OWASP Dependency Check

The OWASP Dependency Check (or simply OWASP DC) is a code scanning tool that supports multiple languages, some of which are listed below; it is widely adopted and makes life easier, especially for multi-language projects, as it provides a standard way to define scanning configurations.

It also provides Docker images and GitHub Actions that are nightly built, including the latest and greatest CVE dictionaries that are used to scan project dependencies, allowing the scanning process to be self-contained yet fast (and less error-prone due to usage quota or connectivity issues).

We have used the OWASP Dependency Check Action to run the scanning continuosly, across the following build platforms:

• NodeJS
• Maven
• Gradle
• .NET

In these examples, the OWASP DC Action is also responsible to upload a report as build artifact, which you can access from the Github's Actions tab.

You can suppress false positives by creating an allow-list.xml file and adding CVEs. For more information, refer to the OWASP Dependency Check documentation.

You can exclude dependencies from the scan by adding --exclude in the args of your action see the OWASP Dependency Check documentation. There are various command line arguments available for your specific needs.

Supported languages

NodeJS

The NodeJS sample project uses AuditJS, a library built by Sonatype which provides a very good alternative to npm audit; you can read more about their comparison on https://blog.sonatype.com/compare-npm-audit-versus-auditjs .

The project descriptor pulls the chokidar 2.0.3 dependency, which contains some CVEs that are ignored into the list of ignored errors.

To run AuditJS locally:

1. Access the folder that contains the package.json file
2. Cleanup the codebase from previous runs - rm -rf node_modules
3. Install (only runtime) dependencies - npm ci --prod ; if using yarn, the command should be yarn install --production --frozen-lockfile
4. Run AuditJS - npx --yes auditjs ossi
5. If you want to ignore errors, create an allow-list.json file and append --whitelist allow-list.json to the command on step 4
6. You should see from this example 1 vulnerability - pkg:npm/is_js@0.9.0 - 1 vulnerability found!

The GitHub action can be copied from here into your repo under .github/workflows/cve-scanning.yml; make sure to adapt the code to your project layout.

Python

The Python sample project uses the safety library, which checks requirements.txt entries against the NVD database.

The python sample project defines a dependency on insecure-package, which pulls a CVE that is ignored in the safety-policy.yml, in order to demo how to manage false positives.

If you're using Poetry, you can simply export your libaries into a requirements.txt file and then follow the steps above, using:

poetry install
poetry export --without-hashes -f requirements.txt --output requirements.txt

To run Safety locally:

1. Access the folder containing the requirements.txt file
2. Make sure you're running Python 3.x using python --version, otherwise the version of safety that you're able to use would be quite outdated
3. Create a virtual environment using python -m venv .
4. Install safety with ./bin/pip install safety - we need to run this step since the scanning will run through all libraries available in the current Python environment
5. Run safety with ./bin/safety check --full-report -r requirements.txt
6. If you want to ignore errors, create a safety-policy.yml and append --policy-file safety-policy.yml to the command on step 4
7. You should see from this example 1 vulnerability - Vulnerability found in insecure-package version 0.1.0

The GitHub action can be copied from here into your repo under .github/workflows/cve-scanning-python.yml; make sure to adapt the code to your project layout.

Maven

The Maven sample project uses the OWASP Dependency Check plugin for Maven to scan runtime dependencies for known vulnerabilities.

To run the Maven Dependency Check Plugin locally:

1. Access the folder containing the pom.xml file (it supports multi-module builds)
2. Run mvn org.owasp:dependency-check-maven:check -DfailBuildOnCVSS=7
3. If you want to ignore errors, create an allow-list.xml and append -DsuppressionFile="allow-list.xml" to the command on step 2

The GitHub action can be copied from here into your repo under .github/workflows/cve-scanning.yml; make sure to adapt the code to your project layout.

Gradle

The Gradle sample project uses the OWASP Dependency Check plugin for Gradle. Sadly, Gradle doesn't allow to invoke plugins without altering the build manifest, namely build.gradle; follow instructions below to know how to add code scanning in your project.

To run the Gradle Dependency Check Plugin locally:

1. Access the folder containing the build.gradle file
2. Copy the allow-list.xml file into your project and remove all <suppress> items
3. Copy the dependencyCheck setup from build.gradle file into your build.gradle file
4. Run ./gradlew dependencyCheckAnalyze

The build.gradle file defines a (commented) dependency on struts2 version 2.3.8, which contains the CVE that led to the (famous) equifax hack. By uncommenting it, the build is expected to fail, assuming that CVEs are not suppressed by the allow-list.xml file, used to manage false positives.

The GitHub action can be copied from here into your repo under .github/workflows/cve-scanning.yml; make sure to adapt the code to your project layout.

Scala

The Scala sample project uses the OWASP Dependency Check plugin for SBT to scan runtime dependencies for known vulnerabilities.

To run the Scala Dependency Check Plugin locally:

1. Access the root project folder
2. Copy dependencyCheckFailBuildOnCVSS and dependencyCheckSuppressionFiles configurations from build.sbt file in your project
3. Copy the sbt-dependency-check plugin definition from plugins.sbt into your project
4. Run sbt dependencyCheck

The build.sbt file defines a (commented) dependency on struts2 version 2.3.8, which contains the CVE that led to the (famous) equifax hack. By uncommenting it, the build is expected to fail, assuming that CVEs are not suppressed by the allow-list.xml file, used to manage false positives.

If you want to test dependencyCheck:

1. Open the file build.sbt
2. Comment the line dependencyCheckSuppressionFiles ++= List(file("../allow-list.xml")),
3. Run sbt dependencyCheck The build should fail and show all CVEs pulled by struts2.

The GitHub action can be copied from here into your repo under .github/workflows/cve-scanning.yml; make sure to adapt the code to your project layout.

To keep your library dependencies, sbt plugins, and Scala and sbt versions up-to-date, checkout Scala Steward.

Rust

The Rust sample project uses Cargo audit to scan runtime dependencies for known vulnerabilities.

To run Cargo Audit locally:

1. Access the root project folder
2. Install Cargo audit with cargo install --force cargo-audit
3. Run the scan with cargo audit
4. Append --ignore RUSTSEC-2020-0071 to the command on step 3

The GitHub action can be copied from here into your repo under .github/workflows/cve-scanning.yml; make sure to adapt the code to your project layout.

For more information about Cargo audit configuration, visit https://docs.rs/cargo-audit/0.17.0/cargo_audit/config/index.html

.NET

The .NET sample project uses the dotnet CLI to scan runtime dependencies for known vulnerabilities.

To run dotnet locally:

1. Access the root project folder, where your .csproj file is defined
2. Install .NET CLI
3. Run dotnet build
4. Run the scan with dotnet list package --vulnerable --include-transitive
5. You should see this vulnerability Newtonsoft.Json 12.0.3 12.0.3 High https://github.com/advisories/GHSA-5crp-9r3c-p9vr

The GitHub action can be copied from here into your repo under .github/workflows/cve-scanning.yml; make sure to adapt the code to your project layout.

Unfortunately there is no way yet to ignore warnings and errors for dotnet, although it may be possible to add some bash logic into the GitHub Action to achieve it.

Docker

Docker scanning can be very useful to check if downstream Docker images are affected by vulnerabilities; it also scans for OS components and provides a solution for projects using C and C++ code.

There are many CLI tools that perform a docker image scanning; the easiest one is docker scan, as you'll probably have the docker command installed in your local environment, assuming you're already working with Docker.

For GitHub Actions, we are using [trivy][trivy.dev], wrapped into this GitHub Action.

To run locally, follow instructions on how to install trivy locally, then run:

1. docker build -f Dockerfile -t user/image-name:latest .
2. trivy image user/image-name:latest

The GitHub action can be copied from here into your repo under .github/workflows/cve-scanning.yml; make sure to adapt the code to your project layout.

Unfortunately there is no way yet to ignore warnings and errors, although it may be possible to add some bash logic into the GitHub Action to achieve it.

Other languages and build platforms

If your project is built using other languages or build platforms, checkout the list of analyzers offered by the OWASP Dependency Check plugin.

There is also a GitHub Dependency Check Action that uses a nightly build of the CVE database, along with the Dependency check plugin.

Dependency update tool

Keeping dependency versions up to date is a hard task, given the big amount of downstream libraries used by today's software projects and their frequent release cadence; adopting a tool to automate it can drastically save time for developers.

Github ships with Dependabot, which can be easily enabled on every repository, but we found Renovate to be easier and more powerful to use, you can read this comparison article, if you're interested.

Assuming that Renovate is running on your project, the CVE scanning tools mentioned above would generate way less alerts, making it easier to manage project's security; as a result, we strongly advise to enable Renovate first, then add CVE scanning tools.

In order to enable Renovate:

1. Email help@finos.org and request enabling the Github App on your FINOS repositories
2. Merge the Pull Request that gets generated after step 1

Renovate will create a GitHub issue (titled Renovate Dashboard) with the recap of the actions that it will take and a one Pull Request for each depdendency version update; please note that:

• The list of Pull Requests sent daily is limited to 10.
• In order to ignore an update, simply close its related Pull Request; Renovate won't ask for the update anymore, unless requested via the Renovate Dashboard issue.

Note that Renovate can be configured to group multiple updates together, using the groupName feature, which can save a lot of developers time, expecially on large codebases.

Static code analysis

To identify bugs in the hosted source code, that is, code that is written and hosted in your own repository, there are several tools out there; the one that proved to work well for us is https://semgrep.dev , and we designed a GitHub Action in .github/workflows/semgrep.yml that continuously scans the code upon every change.

Semgrep supports a long list of programming languages and defines a rich list of rulesets that tests the code against.

It also provides ways to ignore false positives by:

1. adding a // nosemgrep (or # nosemgrep) comment on top of the code block that causes the error
2. adding a .semgrepignore file with a list of file names that should be ignored during the scan

In order to use it, you need to

1. Sign up for free on https://semgrep.dev and generate a token
2. Create a GitHub Secret called SEMGREP_APP_TOKEN, with the token earlier created as value. If you want to enable scanning on a FINOS hosted repository, please email help@finos.org and they will take care of setting the SEMGREP_APP_TOKEN secret on the GitHub repository and enabling the Semgrep GitHub app.
3. Run semgrep scan --error --config auto

In order to test it locally, make sure to:

1. Install Semgrep
2. Signup to semgrep.dev
3. Generate a token, using the Settings menu option
4. (optional) export SEMGREP_APP_TOKEN=<your personal semgrep token> - to aggregate results into FINOS (private) dashboard
5. Run semgrep scan --error --config auto from the root folder, here the docs to install semgrep locally

License scanning and reporting

To enforce compliance of open source projects, it is crucial to validate that inbound libraries adopt a license that is "compatible" with the outbound one in terms of rights and obligations; for FINOS, the outbound license used is the Apache License v2.0.

There are hundreds of different open source licenses, some of which have conflicting clauses (you can learn more on tldrlegal.com), and it's sometimes hard to understand the consequences of adopting a library with a different license than the outbound one, especially without having some legal background or knowledge.

For this reason, we are working on automated tasks to continuously scan licenses being pulled within FINOS projects; such tools should be able to either:

• Run a scanning process that takes as input the list of allowed licenses and the packages to ignore (preferred)
• Build a report of licenses that can be manually reviewed and checked

Right now, we have managed to automate license scanning and reporting on:

• Maven
• Python
• Dotnet
• Rust
• Node.js
• Gradle - The allowed-licenses.json file allows to ignore specific libraries/licenses, whereas license-normalizer-bundle.json allows to define license aliases, such as Apache License Version 2.0 and Apache License, Version 2.0. Note that the build.gradle needs to be changed to make use of the Gradle-License-Report plugin

For more info about compliance requirements at FINOS, checkout our Contribution Compliance Requirements and License Categories pages.

Roadmap

1. Add documentation into community.finos.org
2. Publish post on FINOS blog - https://www.finos.org/blog/introducing-finos-security-scanning
3. Push for adoption across FINOS projects
4. Add license reporting and scanning features
5. Add support for C#
6. Add support for mill

Contributing

For any bug, question or enhancement request, please create a GitHub Issue

1. Fork it (https://github.com/finos/code-scanning/fork)
2. Create your feature branch (git checkout -b feature/fooBar)
3. Read our contribution guidelines and Community Code of Conduct
4. Commit your changes (git commit -am 'Add some fooBar')
5. Push to the branch (git push origin feature/fooBar)
6. Create a new Pull Request

NOTE: Commits and pull requests to FINOS repositories will only be accepted from those contributors with an active, executed Individual Contributor License Agreement (ICLA) with FINOS OR who are covered under an existing and active Corporate Contribution License Agreement (CCLA) executed with FINOS. Commits from individuals not covered under an ICLA or CCLA will be flagged and blocked by the FINOS Clabot tool (or EasyCLA). Please note that some CCLAs require individuals/employees to be explicitly named on the CCLA.

Need an ICLA? Unsure if you are covered under an existing CCLA? Email help@finos.org

License

Copyright 2022 FINOS

Distributed under the Apache License, Version 2.0.

SPDX-License-Identifier: Apache-2.0