Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for using API key to vuln checking script #43909

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 17 additions & 7 deletions tools/dep_checker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,29 @@ in Node's dependencies.

## How to use

In order to query the GitHub Advisory Database,
a [Personal Access Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
has to be created (no permissions need to be given to the token, since it's only used to query the public database).
### Database authentication

- In order to query the GitHub Advisory Database,
a [Personal Access Token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)
has to be created (no permissions need to be given to the token, since it's only used to query the public database).
- The NVD can be queried without authentication, but it will be rate limited to one query every six seconds. In order to
remove
that limitation [request an API key](https://nvd.nist.gov/developers/request-an-api-key) and pass it as a parameter.

### Running the script

Once acquired, the script can be run as follows:

```shell
cd node/tools/dep_checker/
pip install -r requirements.txt

# Python >= 3.9 required
python main.py --gh-token=$PERSONAL_ACCESS_TOKEN
python main.py --gh-token=$PERSONAL_ACCESS_TOKEN --nvd-key=$NVD_API_KEY

# or to skip querying the GitHub Advisory Database, simply run:
# The command can also be run without parameters
# This will skip querying the GitHub Advisory Database, and query the NVD
# using the anonymous (rate-limited) API
python main.py
```

Expand Down Expand Up @@ -51,8 +61,8 @@ non-affected version.
- The queries can return false positives (
see [this](https://github.com/nodejs/security-wg/issues/802#issuecomment-1144207417) comment for an example). These
can be ignored by adding the vulnerability to the `ignore_list` in `dependencies.py`
- The script takes a while to finish (~2 min) because queries to the NVD
are [rate-limited](https://nvd.nist.gov/developers)
- If no NVD API key is provided, the script will take a while to finish (~2 min) because queries to the NVD
are [rate-limited](https://nvd.nist.gov/developers/start-here)
- If any vulnerabilities are found, the script returns 1 and prints out a list with the ID and a link to a description
of
the vulnerability. This is the case except when the ID matches one in the ignore-list (inside `dependencies.py`) in
Expand Down
18 changes: 15 additions & 3 deletions tools/dep_checker/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
from gql.transport.aiohttp import AIOHTTPTransport
from nvdlib import searchCVE # type: ignore
from packaging.specifiers import SpecifierSet
from typing import Optional


class Vulnerability:
Expand Down Expand Up @@ -105,7 +106,7 @@ def query_ghad(gh_token: str) -> dict[str, list[Vulnerability]]:
return found_vulnerabilities


def query_nvd() -> dict[str, list[Vulnerability]]:
def query_nvd(api_key: Optional[str]) -> dict[str, list[Vulnerability]]:
"""Queries the National Vulnerability Database for vulnerabilities reported for Node's dependencies.
The database supports querying by CPE (Common Platform Enumeration) or by a keyword present in the CVE's
Expand All @@ -121,7 +122,9 @@ def query_nvd() -> dict[str, list[Vulnerability]]:
for name, dep in deps_in_nvd.items():
query_results = [
cve
for cve in searchCVE(cpeMatchString=dep.get_cpe(), keyword=dep.keyword)
for cve in searchCVE(
cpeMatchString=dep.get_cpe(), keyword=dep.keyword, key=api_key
)
if cve.id not in ignore_list
]
if query_results:
Expand All @@ -140,15 +143,24 @@ def main():
"--gh-token",
help="the GitHub authentication token for querying the GH Advisory Database",
)
parser.add_argument(
"--nvd-key",
help="the NVD API key for querying the National Vulnerability Database",
)
gh_token = parser.parse_args().gh_token
nvd_key = parser.parse_args().nvd_key
if gh_token is None:
print(
"Warning: GitHub authentication token not provided, skipping GitHub Advisory Database queries"
)
if nvd_key is None:
print(
"Warning: NVD API key not provided, queries will be slower due to rate limiting"
)
ghad_vulnerabilities: dict[str, list[Vulnerability]] = (
{} if gh_token is None else query_ghad(gh_token)
)
nvd_vulnerabilities = query_nvd()
nvd_vulnerabilities: dict[str, list[Vulnerability]] = query_nvd(nvd_key)

if not ghad_vulnerabilities and not nvd_vulnerabilities:
print(f"No new vulnerabilities found ({len(ignore_list)} ignored)")
Expand Down
2 changes: 1 addition & 1 deletion tools/dep_checker/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
gql[aiohttp]
nvdlib
nvdlib==0.5.8
packaging