Unexpected EOF errors while analyzing image layers #3807

namandf · 2023-03-09T19:30:22Z

Description

Trivy image scans fail intermittently with unexpected EOF errors while analysing layers.

What did you expect to happen?

Scan should complete without errors.

What happened instead?

2023-03-10T00:53:13.273+0530	FATAL	image scan error:
    github.com/aquasecurity/trivy/pkg/commands/artifact.Run
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:427
  - scan error:
    github.com/aquasecurity/trivy/pkg/commands/artifact.(*runner).scanArtifact
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:266
  - scan failed:
    github.com/aquasecurity/trivy/pkg/commands/artifact.scan
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:669
  - failed analysis:
    github.com/aquasecurity/trivy/pkg/scanner.Scanner.ScanArtifact
        /home/runner/work/trivy/trivy/pkg/scanner/scan.go:146
  - analyze error:
    github.com/aquasecurity/trivy/pkg/fanal/artifact/image.Artifact.Inspect
        /home/runner/work/trivy/trivy/pkg/fanal/artifact/image/image.go:139
  - failed to analyze layer (sha256:2e9407335169dd770c900a232ae640b6fcd78d799788770f865ad867ccd5ca35):
    github.com/aquasecurity/trivy/pkg/fanal/artifact/image.Artifact.inspect.func1
        /home/runner/work/trivy/trivy/pkg/fanal/artifact/image/image.go:242
  - walk error:
    github.com/aquasecurity/trivy/pkg/fanal/artifact/image.Artifact.inspectLayer
        /home/runner/work/trivy/trivy/pkg/fanal/artifact/image/image.go:312
  - failed to extract the archive:
    github.com/aquasecurity/trivy/pkg/fanal/walker.LayerTar.Walk
        /home/runner/work/trivy/trivy/pkg/fanal/walker/tar.go:48
  - unexpected EOF
  -

Output of run with `-debug`:

trivy_0.38.2.txt

Output of `trivy -v`:

Version: 0.38.2
Vulnerability DB:
  Version: 2
  UpdatedAt: 2023-03-09 18:07:12.23041002 +0000 UTC
  NextUpdate: 2023-03-10 00:07:12.23040942 +0000 UTC
  DownloadedAt: 2023-03-09 18:26:20.593016 +0000 UTC

Additional details (base image name, container registry info...):

The text was updated successfully, but these errors were encountered:

DmitriyLewen · 2023-03-14T07:05:32Z

Hello @namandf
This is strange, because we save all jar files to close connection to avoid this error.

Do you try to useslow flag?

namandf · 2023-03-14T07:40:43Z

Hi @DmitriyLewen ,

Haven't tried out the slow flag. Let me give it a try.

But that would mean sequential processing of files within a layer. Is that correct?

I am assuming that will increase the overall scan time
How does it address the unexpected EOF issue?

DmitriyLewen · 2023-03-14T07:49:18Z

But that would mean sequential processing of files within a layer. Is that correct?

correct

I am assuming that will increase the overall scan time

right, but lets check, it can give us more information.
This error difficult to reproduce. e.g. i scanned your image few times, but didn't get error.

How does it address the unexpected EOF issue?

This error occurs when Trivy takes a long time to parse a file while connection is open. We currently save all jar files and open them from local PC.

namandf · 2023-03-17T06:09:46Z

Encountered a different error this time.

2023-03-17T10:31:06.153+0530	FATAL	image scan error:
    github.com/aquasecurity/trivy/pkg/commands/artifact.Run
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:427
  - scan error:
    github.com/aquasecurity/trivy/pkg/commands/artifact.(*runner).scanArtifact
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:266
  - scan failed:
    github.com/aquasecurity/trivy/pkg/commands/artifact.scan
        /home/runner/work/trivy/trivy/pkg/commands/artifact/run.go:669
  - failed analysis:
    github.com/aquasecurity/trivy/pkg/scanner.Scanner.ScanArtifact
        /home/runner/work/trivy/trivy/pkg/scanner/scan.go:146
  - analyze error:
    github.com/aquasecurity/trivy/pkg/fanal/artifact/image.Artifact.Inspect
        /home/runner/work/trivy/trivy/pkg/fanal/artifact/image/image.go:139
  - failed to analyze layer (sha256:337a26844ce7615f5e778cfd5edf523e5d0d90a054fbcdcc238538973e013ea1):
    github.com/aquasecurity/trivy/pkg/fanal/artifact/image.Artifact.inspect.func1
        /home/runner/work/trivy/trivy/pkg/fanal/artifact/image/image.go:242
  - post analysis error:
    github.com/aquasecurity/trivy/pkg/fanal/artifact/image.Artifact.inspectLayer
        /home/runner/work/trivy/trivy/pkg/fanal/artifact/image/image.go:320
  - post analysis error:
    github.com/aquasecurity/trivy/pkg/fanal/analyzer.AnalyzerGroup.PostAnalyze
        /home/runner/work/trivy/trivy/pkg/fanal/analyzer/analyzer.go:486
  - walk dir error:
    github.com/aquasecurity/trivy/pkg/fanal/analyzer/language/java/jar.(*javaLibraryAnalyzer).PostAnalyze
        /home/runner/work/trivy/trivy/pkg/fanal/analyzer/language/java/jar/jar.go:91
  - on file:
    github.com/aquasecurity/trivy/pkg/parallel.walk[...]
        /home/runner/work/trivy/trivy/pkg/parallel/walk.go:97
  - "usr/lib/jvm/graalvm11/languages/nfi/builder/svm-none.jar" parse error:
    github.com/aquasecurity/trivy/pkg/fanal/analyzer/language/java/jar.(*javaLibraryAnalyzer).PostAnalyze.func2
        /home/runner/work/trivy/trivy/pkg/fanal/analyzer/language/java/jar/jar.go:75
  - zip error:
    github.com/aquasecurity/go-dep-parser/pkg/java/jar.(*Parser).parseArtifact
        /home/runner/go/pkg/mod/github.com/aquasecurity/go-dep-parser@v0.0.0-20230302111817-e4068021315b/pkg/java/jar/parse.go:82
  - zip: not a valid zip file

DmitriyLewen · 2023-03-17T06:55:12Z

can you check usr/lib/jvm/graalvm11/languages/nfi/builder/svm-none.jar file?

Perhaps it is empty file. Same issue is #3818

namandf · 2023-03-17T07:00:23Z

Sure. Let me check. But isn't the fix for this issue already part of 0.38.2 #3832 ?

DmitriyLewen · 2023-03-17T07:08:52Z

But isn't the fix for this issue already part of 0.38.2 #3832 ?

No. Last release(v0.38.3) doesn't include these changes.

knqyf263 · 2023-04-03T12:08:17Z

v0.39.0 should fix the issue. Please let us know if it is not the case.

AguangMikeZhang · 2023-04-18T19:27:51Z

v0.39.0 should fix the issue. Please let us know if it is not the case.

I'm scanning a Drupal application and getting the same error. Composer.json file is 443.31 KB. I'm using the version 0.40.0.

DmitriyLewen · 2023-04-20T04:19:57Z

Hello @AguangMikeZhang
We talked about Drupal image and EOF error in #4003

Can you send me an image for investigation in #4003?

AguangMikeZhang · 2023-04-20T12:52:10Z

Hello @AguangMikeZhang We talked about Drupal image and EOF error in #4003

Can you send me an image for investigation in #4003?

Hi @DmitriyLewen , thank you for the reply. It turns out it was trying to scan a different composer.lock that was empty, which caused the issue. It's all good now. Thanks again!

janepelladinesh · 2023-04-24T09:03:44Z

@knqyf263 I am still seeing this issue in v0.39.0

[/jobservice/runner/redis.go:123]: Job 'IMAGE_SCAN:2f23113f5bd5e4c4a52162bb' exit with error: run error: unexpected EOF

also i am seeing new error for conversion

Failed to convert vulnerability data to new schema for report e64c20c0-2606-4490-9113-1df0a848ba6f, error Error when converting vulnerability report: unexpected EOF

++ @AyushCloud

janepelladinesh · 2023-05-02T12:06:02Z

@DmitriyLewen , could you please assist us

DmitriyLewen · 2023-05-03T06:21:12Z

Hello @janepelladinesh

Do you get this error for Trivy application?
Can you send log (with the debug flag enabled) to investigation?

janepelladinesh · 2023-05-03T08:36:07Z

Hello @janepelladinesh

Do you get this error for Trivy application?
yes
Can you send log (with the debug flag enabled) to investigation?

Hi @DmitriyLewen
I have shared all the logs in this ticket #3148 , errors were reduced

but still i see few images are failing with trivy v0.39.0

Apr 24 08:58:08 172.22.0.1 jobservice[12818]: 2023-04-24T08:58:08Z [INFO] [/pkg/scan/job.go:167]: Report mime types: [application/vnd.security.vulnerability.report; version=1.1]
Apr 24 08:58:08 172.22.0.1 jobservice[12818]: 2023-04-24T08:58:08Z [INFO] [/pkg/scan/job.go:224]: Get report for mime type: application/vnd.security.vulnerability.report; version=1.1
Apr 24 08:58:09 172.22.0.1 jobservice[12818]: 2023-04-24T08:58:09Z [INFO] [/pkg/scan/job.go:245]: Report with mime type application/vnd.security.vulnerability.report; version=1.1 is not ready yet, retry after 5 seconds
Apr 24 08:58:09 172.22.0.1 jobservice[12818]: 2023-04-24T08:58:09Z [ERROR] [/pkg/scan/job.go:302]: Failed to get report for artifact %s of mimetype %s, error %vsha256:bb68e4358c0bff4cb22dc5************b3fc4application/vnd.security.vulnerability.report; version=1.1unexpected EOF
Apr 24 08:58:09 172.22.0.1 jobservice[12818]: 2023-04-24T08:58:09Z [ERROR] [/jobservice/runner/redis.go:123]: Job 'IMAGE_SCAN:c998720d1d8ca6c59763d070' exit with error: run error: unexpected EOF

namandf added the kind/bug Categorizes issue or PR as related to a bug. label Mar 9, 2023

This was referenced Mar 13, 2023

Unable to open JAR files #3514

Closed

Failed to get report for artifact , Unexpected EOF Trivy scan issue #3148

Closed

knqyf263 closed this as completed Apr 3, 2023

knqyf263 mentioned this issue Apr 18, 2023

trivy v0.37.2 is not working with harbor v2.7.1 #4081

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unexpected EOF errors while analyzing image layers #3807

Unexpected EOF errors while analyzing image layers #3807

namandf commented Mar 9, 2023 •

edited

Loading

DmitriyLewen commented Mar 14, 2023

namandf commented Mar 14, 2023

DmitriyLewen commented Mar 14, 2023

namandf commented Mar 17, 2023

DmitriyLewen commented Mar 17, 2023

namandf commented Mar 17, 2023

DmitriyLewen commented Mar 17, 2023

knqyf263 commented Apr 3, 2023

AguangMikeZhang commented Apr 18, 2023

DmitriyLewen commented Apr 20, 2023

AguangMikeZhang commented Apr 20, 2023 •

edited

Loading

janepelladinesh commented Apr 24, 2023 •

edited

Loading

janepelladinesh commented May 2, 2023

DmitriyLewen commented May 3, 2023

janepelladinesh commented May 3, 2023 •

edited

Loading

Unexpected EOF errors while analyzing image layers #3807

Unexpected EOF errors while analyzing image layers #3807

Comments

namandf commented Mar 9, 2023 • edited Loading

Description

What did you expect to happen?

What happened instead?

Output of run with -debug:

Output of trivy -v:

Additional details (base image name, container registry info...):

DmitriyLewen commented Mar 14, 2023

namandf commented Mar 14, 2023

DmitriyLewen commented Mar 14, 2023

namandf commented Mar 17, 2023

DmitriyLewen commented Mar 17, 2023

namandf commented Mar 17, 2023

DmitriyLewen commented Mar 17, 2023

knqyf263 commented Apr 3, 2023

AguangMikeZhang commented Apr 18, 2023

DmitriyLewen commented Apr 20, 2023

AguangMikeZhang commented Apr 20, 2023 • edited Loading

janepelladinesh commented Apr 24, 2023 • edited Loading

janepelladinesh commented May 2, 2023

DmitriyLewen commented May 3, 2023

janepelladinesh commented May 3, 2023 • edited Loading

namandf commented Mar 9, 2023 •

edited

Loading

Output of run with `-debug`:

Output of `trivy -v`:

AguangMikeZhang commented Apr 20, 2023 •

edited

Loading

janepelladinesh commented Apr 24, 2023 •

edited

Loading

janepelladinesh commented May 3, 2023 •

edited

Loading