Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SCANDOCKER-27 Move sonar.working.directory outside of the project basedir #249

Merged
merged 1 commit into from
May 23, 2024
Merged

SCANDOCKER-27 Move sonar.working.directory outside of the project basedir #249

merged 1 commit into from
May 23, 2024

Conversation

henryju
Copy link
Member

@henryju henryju commented May 23, 2024

No description provided.

csaba-feher-sonarsource
csaba-feher-sonarsource approved these changes May 23, 2024
View reviewed changes
Copy link
Contributor

@csaba-feher-sonarsource csaba-feher-sonarsource left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

We have tested together. With the change, we could mount the user's folder and execute analysis without the scanner-cli user having write access to the user-mounted folder.

@henryju henryju force-pushed the jh/SCANDOCKER-27_move_scanner_workdir branch from cd851c3 to d70d9bb Compare May 23, 2024 15:21
@sonarqube-next SonarQube-Next
Copy link

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

@henryju henryju merged commit 851a18c into master May 23, 2024
6 checks passed
@henryju henryju deleted the jh/SCANDOCKER-27_move_scanner_workdir branch May 23, 2024 15:27
@vladimirjaric
Copy link

Hi, we are getting the following error, which seems to be due to this change:

INFO: Scanner configuration file: /opt/sonar-scanner/conf/sonar-scanner.properties
INFO: Project root configuration file: /usr/src/sonar-project.properties
INFO: SonarScanner 5.0.1.3006
INFO: Java 17.0.11 Alpine (64-bit)
INFO: Linux 6.1.79-99.164.amzn2023.x86_64 amd64
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 0.618s
INFO: Final Memory: 3M/17M
INFO: ------------------------------------------------------------------------
ERROR: Error during SonarScanner execution
java.lang.IllegalStateException: Unable to create user cache: /opt/sonar-scanner/.sonar/cache
	at org.sonarsource.scanner.api.internal.cache.FileCache.createDir(FileCache.java:147)
	at org.sonarsource.scanner.api.internal.cache.FileCache.<init>(FileCache.java:46)
	at org.sonarsource.scanner.api.internal.cache.FileCache.create(FileCache.java:52)
	at org.sonarsource.scanner.api.internal.cache.FileCacheBuilder.build(FileCacheBuilder.java:48)
	at org.sonarsource.scanner.api.internal.JarDownloaderFactory.create(JarDownloaderFactory.java:42)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.createLauncher(IsolatedLauncherFactory.java:68)
	at org.sonarsource.scanner.api.EmbeddedScanner.doStart(EmbeddedScanner.java:185)
	at org.sonarsource.scanner.api.EmbeddedScanner.start(EmbeddedScanner.java:123)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:74)
	at org.sonarsource.scanner.cli.Main.main(Main.java:62)
Caused by: java.nio.file.AccessDeniedException: /opt/sonar-scanner/.sonar/cache
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:397)
	at java.base/java.nio.file.Files.createDirectory(Files.java:700)
	at java.base/java.nio.file.Files.createAndCheckIsDirectory(Files.java:807)
	at java.base/java.nio.file.Files.createDirectories(Files.java:793)
	at org.sonarsource.scanner.api.internal.cache.FileCache.createDir(FileCache.java:145)
	... 9 more
ERROR:
ERROR: Re-run SonarScanner using the -X switch to enable full debug logging.

Feel free to let me know if you need any other info.
Regards

@henryju
Copy link
Member Author

henryju commented May 27, 2024

Hi @vladimirjaric

The error you get is not about the working directory, but the cache directory. It has been discussed on our community forum.

If you don't find the solution in the thread, please add (in the forum) some details about your use case:

  • your CI
  • how do you use the image (are you somehow mapping the directory /opt/sonar-scanner/.sonar/cache to another volume)?

@Paul-Zensoftware
Copy link

Paul-Zensoftware commented May 27, 2024 •
edited

Hi,

We are having the same issue after the update. We are using Gitlab runners.

On the latest version, the pipeline fails to create cache files.

Using docker image sha256:380051b07dba0e5872ee9343d59479a82d2c6586b1e8e8aeac6aca2a476378c4 for sonarsource/sonar-scanner-cli:latest with digest sonarsource/sonar-scanner-cli@sha256:866f32d840223c8178778ed7a28556fbe3cac00dd74aac8290d7af93debcfcb2 ...
$ sonar-scanner
INFO: Scanner configuration file: /opt/sonar-scanner/conf/sonar-scanner.properties
INFO: Project root configuration file: /builds/zensoftwarenl/agile/agile-analytics-fe/sonar-project.properties
INFO: SonarScanner 5.0.1.[30](https://gitlab.com/zensoftwarenl/agile/agile-analytics-fe/-/jobs/6946045004#L30)06
INFO: Java 17.0.11 Alpine (64-bit)
INFO: Linux 5.15.154+ amd64
INFO: User cache: /builds/zensoftwarenl/agile/agile-analytics-fe/.sonar/cache
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 1.113s
INFO: Final Memory: 4M/20M
INFO: ------------------------------------------------------------------------
ERROR: Error during SonarScanner execution
org.sonarsource.scanner.api.internal.ScannerException: Unable to execute SonarScanner analysis
	at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.lambda$createLauncher$0(IsolatedLauncherFactory.java:85)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:[31](https://gitlab.com/zensoftwarenl/agile/agile-analytics-fe/-/jobs/6946045004#L31)8)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.createLauncher(IsolatedLauncherFactory.java:74)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.createLauncher(IsolatedLauncherFactory.java:70)
	at org.sonarsource.scanner.api.EmbeddedScanner.doStart(EmbeddedScanner.java:185)
	at org.sonarsource.scanner.api.EmbeddedScanner.start(EmbeddedScanner.java:123)
	at org.sonarsource.scanner.cli.Main.execute(Main.java:74)
	at org.sonarsource.scanner.cli.Main.main(Main.java:62)
Caused by: java.lang.IllegalStateException: Fail to create temp file in /builds/zensoftwarenl/agile/agile-analytics-fe/.sonar/cache/_tmp
	at org.sonarsource.scanner.api.internal.cache.FileCache.newTempFile(FileCache.java:138)
	at org.sonarsource.scanner.api.internal.cache.FileCache.get(FileCache.java:83)
	at org.sonarsource.scanner.api.internal.JarDownloader.lambda$getScannerEngineFiles$0(JarDownloader.java:60)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:2[34](https://gitlab.com/zensoftwarenl/agile/agile-analytics-fe/-/jobs/6946045004#L34))
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at org.sonarsource.scanner.api.internal.JarDownloader.getScannerEngineFiles(JarDownloader.java:61)
	at org.sonarsource.scanner.api.internal.JarDownloader.download(JarDownloader.java:53)
	at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.lambda$createLauncher$0(IsolatedLauncherFactory.java:76)
	... 7 more
Caused by: java.nio.file.AccessDeniedException: /builds/zensoftwarenl/agile/agile-analytics-fe/.sonar/cache/_tmp/fileCache3855905100970462097.tmp
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:218)
	at java.base/java.nio.file.Files.newByteChannel(Files.java:380)
	at java.base/java.nio.file.Files.createFile(Files.java:658)
	at java.base/java.nio.file.TempFileHelper.create(TempFileHelper.java:1[36](https://gitlab.com/zensoftwarenl/agile/agile-analytics-fe/-/jobs/6946045004#L36))
	at java.base/java.nio.file.TempFileHelper.createTempFile(TempFileHelper.java:159)
	at java.base/java.nio.file.Files.createTempFile(Files.java:878)
	at org.sonarsource.scanner.api.internal.cache.FileCache.newTempFile(FileCache.java:136)
	... 19 more
ERROR: 
ERROR: Re-run SonarScanner using the -X switch to enable full debug logging.

Down grading form lastest to 5.0.1 seems to resolve the error.

Using docker image sha256:fd964ae82ee4d1d5d637a31e1f42fd00bef27035569565f66f84d1318d863900 for sonarsource/sonar-scanner-cli:5.0.1 with digest sonarsource/sonar-scanner-cli@sha256:02372948eaeeb10dfbe0cfd4174d44b8e405d0aeae431532b2bdb21d0347bf23 ...
$ sonar-scanner
......
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 2:05.952s
INFO: Final Memory: 150M/504M
INFO: ------------------------------------------------------------------------

We made no changes between updates, and only changing version number from "latest" to "5.0.1" resolved the issue.

If this is a config error from our side, could you please explain what needs to be changed so we can resolve the issues.

Thanks in advance

@vladimirjaric
Copy link

Hi @vladimirjaric

The error you get is not about the working directory, but the cache directory. It has been discussed on our community forum.

If you don't find the solution in the thread, please add (in the forum) some details about your use case:

  • your CI
  • how do you use the image (are you somehow mapping the directory /opt/sonar-scanner/.sonar/cache to another volume)?

Thank you @henryju.

FYI, we use Buildkite for CI. We don't map /opt/sonar-scanner/.sonar/cache to another volume:

sonar:
    image: sonarsource/sonar-scanner-cli:latest
    userns_mode: "host"
    env_file: .env
    working_dir: /usr/src
    volumes:
      - .:/usr/src:rw

As @Paul-Zensoftware mentioned above, the error started a few days ago with no changes on our end, apart from using the latest image.

@henryju
Copy link
Member Author

henryju commented May 27, 2024

@vladimirjaric I think the issue is coming from:

userns_mode: "host"

It was okay before because our image relied only on the root account. In the latest release, we changed to having a non-root user run the analysis. I can't find a lot of documentation on this parameter, but what happen if you remove it?

@henryju
Copy link
Member Author

henryju commented May 27, 2024

@Paul-Zensoftware

AccessDeniedException: /builds/zensoftwarenl/agile/agile-analytics-fe/.sonar/cache/_tmp/fileCache3855905100970462097.tmp

Please see the explanation in the forum. Basically, you have to clear the pipeline cache, since files cached by the previous version of the image cannot be read (due to the use of a non-root user for security reasons).

@Paul-Zensoftware
Copy link

@Paul-Zensoftware

AccessDeniedException: /builds/zensoftwarenl/agile/agile-analytics-fe/.sonar/cache/_tmp/fileCache3855905100970462097.tmp

Please see the explanation in the forum. Basically, you have to clear the pipeline cache, since files cached by the previous version of the image cannot be read (due to the use of a non-root user for security reasons).

Hi,

Thank you for the quick response!

Clearing the build cache resolved the issues I was having. I should have tried running without cache before asking for help.
My issue is now resolved.

Again, Thank you for the quick response and help resolving the issue!

@vladimirjaric
Copy link

@vladimirjaric I think the issue is coming from:

userns_mode: "host"

It was okay before because our image relied only on the root account. In the latest release, we changed to having a non-root user run the analysis. I can't find a lot of documentation on this parameter, but what happen if you remove it?

Many thanks @henryju. Removing userns_mode: "host" fixed our problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@csaba-feher-sonarsource csaba-feher-sonarsource csaba-feher-sonarsource approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@henryju @vladimirjaric @Paul-Zensoftware @csaba-feher-sonarsource