Artifacts are not visible for the test-reporter plugin

[inBrackets]

Describe the bug

Cannot find any artifacts if the junit parser is applied.

To Reproduce

Steps to reproduce the behavior:

Currently I have a gh worflow as the following:
https://github.com/inBrackets/JavaAlgorithms/blob/master/.github/workflows/maven.yml

name: Java CI with Maven

on:
  push:
    branches:
      - master
  pull_request:
    branches:
      - master
  workflow_dispatch:

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3

    - name: Set up JDK 17
      uses: actions/setup-java@v4
      with:
        java-version: '17'
        distribution: 'adopt'

    - name: Build with Maven
      run: mvn -B clean install

    - name: Run tests
      run: mvn test -Dsurefire.reports.directory=$PWD/target/surefire-reports

    - name: Archive test results
      uses: actions/upload-artifact@v2
      with:
        name: test-results
        path: target/surefire-reports/

  report:
    runs-on: ubuntu-latest
    needs: build
    steps:
      - name: Download test results artifact
        uses: actions/download-artifact@v2
        with:
          name: test-results
          path: /tmp/test-results

      - name: List downloaded artifacts
        run: ls -R /tmp/test-results

      - name: Generate test report
        uses: dorny/test-reporter@v1
        with:
          artifact: test-results
          name: 'Test report $1'
          path: /'/**/*.xml'/
          reporter: java-junit
          fail-on-empty: 'false'

As an output I get a warning that no artifacts were found despite they were listed explicitly in the previous step (List downloaded artifacts)

Expected behavior

The artifacts should be visible

Screenshots

Additional context

The "report" job passes only because the fail-on-empty is set to false

[JojOatXGME]

Why have you surrounded the path with slashes (/), and why is the Apostrophe (') within the slashes?

path: /'/**/*.xml'/

If you want to quote the value in YAML, the quotes would have to surround everything (path: '//**/*.xml/'). The surrounding slashes are also unnecessary for the path. You only have to use them if you want to use a regex in the name.

path: '/**/*.xml'

However, this glob expression would still be a bad idea, as it would effectively scan your entire disk. So you should probably rather use the following:

path: '/tmp/test-results/**/*.xml'

Besides, I have no idea if the action supports specifying an absolute directory. You may otherwise try to download the artifact to the current working directory by using relative paths, instead of /tmp.

[inBrackets]

About surrounding the path property, you were right. I tried to apply the path in many ways. Even Applying the regex that should be applied just for the name

I applied the changes for the path, and also provided the full absolute directory using the ${{ github.workspace }} environment variable. Unfortunately with no success. The files from the directory are printed out but not detected by the dorny/test-reporter@v1 plugin. (run #27)


I also downloaded all files to the base directory (run #28), but with the same result as above. The path has been changed to '*.xml'

My current yml file looks currently the following way:

name: Java CI with Maven

on:
  push:
    branches:
      - master
  pull_request:
    branches:
      - master
  workflow_dispatch:

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3

    - name: Set up JDK 17
      uses: actions/setup-java@v4
      with:
        java-version: '17'
        distribution: 'adopt'

    - name: Build with Maven
      run: mvn -B clean install

    - name: Run tests
      run: mvn test -Dsurefire.reports.directory=$PWD/target/surefire-reports

    - name: Archive test results
      uses: actions/upload-artifact@v2
      with:
        name: test-results
        path: target/surefire-reports/

  report:
    runs-on: ubuntu-latest
    needs: build
    steps:
      - name: Download test results artifact
        uses: actions/download-artifact@v2
        with:
          name: test-results
          path: ${{ github.workspace }}

      - name: List downloaded artifacts
        run: ls -R ${{ github.workspace }}

      - name: Generate test report
        uses: dorny/test-reporter@v1
        with:
          artifact: test-results
          name: 'Test report $1'
          path: '*.xml'
          reporter: java-junit
          fail-on-empty: 'false'

[JojOatXGME]

Mhh, I noticed few more potential problems:

1. You specified the artifact option. This means the action does not scan the directory on the machine, but tries to scan all the artifacts matching the given name. However, the plugin uses the API for downloading the artifacts. Since you are using an earlier version of actions/upload-artifact (older than v4), uploaded artifacts are not available in the API until the end of the workflow. Since there are no Artifacts available in the API, this action has no reports it can scan. You should remove the artifact option form your workflow to fix this.

2. When using actions/upload-artifact@v4, you should be able to use the artifact option of the action, and instead remove the download step above. See GitHub's block post about v4. However, dorny/test-reporter is currently broken for Artifacts uploaded with actions/upload-artifact@v4. See Test Reporter fails on workflow_run #343. You would have to wait for this issue being fixed before you can apply this alternative solution.

3. When you don't specify the artifact option, you need to clone the repository first. See fatal: Not a git repository #131.

Here is a version where I applied the changes. For the path, I would just use a relative path, instead of an absolute one. I changed this as well.

Show adjusted workflow file

name: Java CI with Maven

on:
  push:
    branches:
      - master
  pull_request:
    branches:
      - master
  workflow_dispatch:

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3

    - name: Set up JDK 17
      uses: actions/setup-java@v4
      with:
        java-version: '17'
        distribution: 'adopt'

    - name: Build with Maven
      run: mvn -B clean install

    - name: Run tests
      run: mvn test -Dsurefire.reports.directory=$PWD/target/surefire-reports

    - name: Archive test results
      uses: actions/upload-artifact@v2
      with:
        name: test-results
        path: target/surefire-reports/

  report:
    runs-on: ubuntu-latest
    needs: build
    steps:

      # We have to clone the repo as the test-reporter otherwise runs into an error.
      # See https://github.com/dorny/test-reporter/issues/131#issuecomment-2093880211
      - uses: actions/checkout@v4

      # We have to manually download the artifact because dorny/test-reporter@v1
      # does not see the artifacts uploaded by actions/upload-artifact@v3 and below before the end of the workflow.
      # The action dorny/test-reporter@v1 does also not support artifacts uploaded by actions/upload-artifact@v4.
      # See https://github.com/dorny/test-reporter/issues/343
      - name: Download test results artifact
        uses: actions/download-artifact@v2
        with:
          name: test-results
          path: target/surefire-reports

      - name: List downloaded artifacts
        run: ls -R target/surefire-reports

      - name: Generate test report
        uses: dorny/test-reporter@v1
        with:
          name: 'Test report $1'
          path: target/surefire-reports/**.xml
          reporter: java-junit
          fail-on-empty: 'false'

[jozefizso]

I am closing this issue as this is an error in the setup of the workflow itself.

Thanks @JojOatXGME for you valuable troubleshooting steps to help resolve this issue.