Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing to analyze - with Cargo failed to resolve dependencies for path 'rust-client/Cargo.toml': NoSuchElementException: Collection contains no element matching the predicate. #8480

Closed
avifenesh opened this issue Apr 3, 2024 · 5 comments
Assignees
Labels
analyzer About the analyzer tool bug Issues that are considered to be bugs

Comments

@avifenesh
Copy link

When running analyze on our project with githubactions we get 07:49:06.333 [DefaultDispatcher-worker-2] ERROR org.ossreviewtoolkit.analyzer.PackageManager - Cargo failed to resolve dependencies for path 'rust-client/Cargo.toml': NoSuchElementException: Collection contains no element matching the predicate. Error.

The error doesn't indicate which dependency exactly it fails on, but even when trying to remove each dependency at a time and also all of the dependencies the error keep occurring, even with an empty file.
Iv'e tried to both use ORT from source and both using gradlew, both are failing the same.
The issue started about a week+ ago without any changes to the relevant files from our side.
Our project: glide-for-redis
Cargo.toml:

[package]
name = "glide-rs"
version = "0.1.0"
edition = "2021"
license = "Apache-2.0"
authors = ["Amazon Web Services"]

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[lib]
crate-type = ["cdylib"]

[dependencies]
redis = { path = "../../submodules/redis-rs/redis", features = ["aio", "tokio-comp", "tokio-rustls-comp"] }
glide-core = { path = "../../glide-core", features = ["socket-layer"] }
tokio = { version = "1", features = ["rt", "macros", "rt-multi-thread", "time"] }
napi = {version = "2.14", features = ["napi4", "napi6"] }
napi-derive = "2.14"
logger_core = {path = "../../logger_core"}
byteorder = "1.4.3"
num-traits = "0.2.17"
num-bigint = { version = "0.4.4", optional = true }
[target.'cfg(not(target_env = "msvc"))'.dependencies]
tikv-jemallocator = {version = "0.5.4", features = ["disable_initial_exec_tls"] }

[build-dependencies]
napi-build = "2.0.1"

[profile.release]
lto = true
debug = true

[features]
testing_utilities = ["num-bigint"]

The actions:

name: The OSS Review Toolkit (ORT)

on:
    schedule:
      - cron: "0 0 * * *"
    pull_request:
      paths:
        - .github/workflows/ort.yml
        - .github/workflows/run-ort-tools/action.yml
        - utils/get_licenses_from_ort.py
    workflow_dispatch:
      inputs:
        branch:
          description: 'The branch to run against the ORT tool'     
          required: true
        version:
          description: 'The release version of GLIDE'
          required: true
jobs:
    run-ort:
        if: github.repository_owner == 'aws'
        name: Create attribution files
        runs-on: ubuntu-latest
        strategy:
          fail-fast: false
        env: 
          PYTHON_ATTRIBUTIONS: "python/THIRD_PARTY_LICENSES_PYTHON"
          NODE_ATTRIBUTIONS: "node/THIRD_PARTY_LICENSES_NODE"
          RUST_ATTRIBUTIONS: "glide-core/THIRD_PARTY_LICENSES_RUST"
        steps:
            - name: Set the release version
              shell: bash
              run: |
                  export version=`if [ "$EVENT_NAME" == 'schedule' ] || [ "$EVENT_NAME" == 'pull_request' ]; then echo '255.255.255'; else echo "$INPUT_VERSION"; fi`
                  echo "RELEASE_VERSION=${version}" >> $GITHUB_ENV
              env:
                EVENT_NAME: ${{ github.event_name }}
                INPUT_VERSION: ${{ github.event.inputs.version }}
              
            - name: Set the base branch
              run: |
                export BASE_BRANCH=`if [ "$EVENT_NAME" == 'schedule' ]; then echo 'main'; elif [ "$EVENT_NAME" == 'workflow_dispatch' ]; then echo "$INPUT_BRANCH"; else echo ""; fi`
                echo "Base branch is: ${BASE_BRANCH}"
                echo "BASE_BRANCH=${BASE_BRANCH}" >> $GITHUB_ENV
              env:
                EVENT_NAME: ${{ github.event_name }}
                INPUT_BRANCH: ${{ github.event.inputs.branch }}

            - name: Checkout
              uses: actions/checkout@v4
              with:
                  submodules: "true"
                  ref: ${{ env.BASE_BRANCH }}

            - name: Set up JDK 11 for the ORT package
              uses: actions/setup-java@v4
              with:
                  distribution: "temurin"
                  java-version: 11

            - name: Cache ORT and Gradle packages
              uses: actions/cache@v4
              id: cache-ort
              with:
                path: |
                  ./ort
                  ~/.gradle/caches
                  ~/.gradle/wrapper
                key: ${{ runner.os }}-ort

            - name: Checkout ORT Repository
              if: steps.cache-ort.outputs.cache-hit != 'true'
              uses: actions/checkout@v4
              with: 
                  repository: "oss-review-toolkit/ort"
                  path: "./ort"
                  ref: main
                  submodules: recursive

            - name: Checkout ORT latest release tag
              if: steps.cache-ort.outputs.cache-hit != 'true'
              working-directory: ./ort/
              run: |
                # Get new tags from remote
                git fetch --tags
                # Get latest tag name
                LATEST_TAG=$(git describe --tags "$(git rev-list --tags --max-count=1)")
                # Checkout latest tag
                git checkout $LATEST_TAG

            - name: Install ORT
              if: steps.cache-ort.outputs.cache-hit != 'true'
              working-directory: ./ort/
              run: |
                export JAVA_OPTS="$JAVA_OPTS -Xmx8g"
                ./gradlew installDist

            - name: Create ORT config file
              run: |
                mkdir -p ~/.ort/config
                cat << EOF > ~/.ort/config/config.yml
                ort:
                  analyzer:
                    allowDynamicVersions: true
                    enabledPackageManagers: [Cargo, NPM, PIP]
                EOF
                cat ~/.ort/config/config.yml

          ### NodeJS ###

            - name: Set up Node.js 16.x
              uses: actions/setup-node@v4
              with:
                  node-version: 16.x

            - name: Create package.json file for the Node wrapper 
              uses: ./.github/workflows/node-create-package-file
              with:
                release_version: ${{ env.RELEASE_VERSION }}
                os: "ubuntu-latest"

            - name: Fix Node base NPM package.json file for ORT
              working-directory: ./node/npm/glide
              run: |
                # Remove the glide-rs dependency to avoid duplication 
                sed -i '/ "glide-rs":/d' ../../package.json
                export pkg_name=glide-for-redis-base
                export package_version="${{ env.RELEASE_VERSION }}"
                export scope=`if [ "$NPM_SCOPE" != ''  ]; then echo "$NPM_SCOPE/"; fi`
                mv package.json package.json.tmpl
                envsubst < package.json.tmpl > "package.json"
                cat package.json
            
            - name: Run ORT tools for Node
              uses: ./.github/workflows/run-ort-tools
              with:
                folder_path: "${{ github.workspace }}/node"
         And:
name: Run the OSS review tool

inputs:
    folder_path:
        description: "The root folder to run the ORT tool from"
        required: true
        type: string

runs:
    using: "composite"
    steps:
        - name: Run ORT tools
          working-directory: ./ort/
          shell: bash
          run: |
            echo "Running ORT tools for ${{ inputs.folder_path }}"
            FOLDER=${{ inputs.folder_path }}
            mkdir $FOLDER/ort_results
            # Analyzer (analyzer-result.json)
            ./gradlew cli:run --args="analyze -i $FOLDER -o $FOLDER/ort_results -f JSON"
            
            # NOTICE DEFAULT
            ./gradlew cli:run --args="report -i $FOLDER/ort_results/analyzer-result.json -o $FOLDER/ort_results/ -f PlainTextTemplate"

Iv'e tried:
Make sure that locale lang is fine.
Generating cargo.lock before analyzing.
Removing any comments from cargo file.
Sync submuodoles.
Deleting each dependencies at a time, and all of them together.
And more..

Any help or hint will be great.
Thanks!

@sschuberth sschuberth added bug Issues that are considered to be bugs analyzer About the analyzer tool labels Apr 3, 2024
@sschuberth sschuberth self-assigned this Apr 3, 2024
@sschuberth
Copy link
Member

Thanks for the report! I believe to know what the problem is. But just to double-check, which version of Cargo are you using?

@avifenesh
Copy link
Author

avifenesh commented Apr 3, 2024

Which one i use generally for development or which one is in use by this action?
In the action i believe it takes it from the tool. In development i use what come out of the box using rustup, i believe it is the last stable version.

@sschuberth
Copy link
Member

Nevermind, I've found out that the issue is caused by a change of package IDs in Cargo 1.77. I'll work on a fix. For now, downgrading to Cargo 1.76 works around the issue.

@avifenesh
Copy link
Author

avifenesh commented Apr 3, 2024

Thanks a lot. Will be happy to know when this issue is fixed.

avifenesh added a commit to valkey-io/valkey-glide that referenced this issue Apr 4, 2024
sschuberth added a commit that referenced this issue Apr 8, 2024
Instead of determining projects based on Cargo's internal package ID,
use the `source` propery which is `null` for local code.

Fixes #8480.

Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>
@sschuberth
Copy link
Member

How can i tell the tool which cargo version to use? Should i install it separately?

ORT will use whatever tool version is available in the environment.

In any case, #8497 is supposed to fix this specific issue, but as a follow-up I'm running into redis-rs/redis-rs#1143 and a variant of the issue that #6533 tries to address.

sschuberth added a commit that referenced this issue Apr 9, 2024
Instead of determining projects based on Cargo's internal package ID,
use the `source` property which is `null` for local code.

Fixes #8480.

Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>
sschuberth added a commit that referenced this issue Apr 9, 2024
Instead of determining projects based on Cargo's internal package ID,
use the `source` property which is `null` for local code.

Fixes #8480.

Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>
sschuberth added a commit that referenced this issue Apr 9, 2024
Instead of determining projects based on Cargo's internal package ID,
use the `source` property which is `null` for local code.

Fixes #8480.

Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
analyzer About the analyzer tool bug Issues that are considered to be bugs
Projects
None yet
Development

No branches or pull requests

2 participants