Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update and use upstream mimetype lib #307

Merged
merged 2 commits into from
Oct 14, 2024
Merged

Update and use upstream mimetype lib #307

merged 2 commits into from
Oct 14, 2024

Conversation

wagoodman
Copy link
Contributor

gabriel-vasile/mimetype#573 addresses many allocation concerns we were dealing with 🎉

@wagoodman
use upstream mimetype lib
2021711 
Signed-off-by: Alex Goodman <wagoodman@users.noreply.github.com>
@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 14, 2024 •
edited
Loading

Benchmark Test Results

Benchmark results from the latest changes vs base branch 
make .tool/task
make[1]: Entering directory '/home/runner/work/stereoscope/stereoscope'
make[1]: Leaving directory '/home/runner/work/stereoscope/stereoscope'
.tool/task show-benchstat
?   	github.com/anchore/stereoscope	[no test files]
?   	github.com/anchore/stereoscope/examples	[no test files]
PASS
ok  	github.com/anchore/stereoscope/internal	0.003s
?   	github.com/anchore/stereoscope/internal/bus	[no test files]
PASS
ok  	github.com/anchore/stereoscope/internal/containerd	0.009s
PASS
ok  	github.com/anchore/stereoscope/internal/docker	0.005s
?   	github.com/anchore/stereoscope/internal/log	[no test files]
PASS
ok  	github.com/anchore/stereoscope/internal/podman	0.005s
?   	github.com/anchore/stereoscope/pkg/event	[no test files]
?   	github.com/anchore/stereoscope/pkg/event/parsers	[no test files]
goos: linux
goarch: amd64
pkg: github.com/anchore/stereoscope/pkg/file
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkTarIndex-4   	   33700	     35906 ns/op	    5700 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   33469	     35767 ns/op	    5700 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   33480	     35783 ns/op	    5698 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   33489	     35791 ns/op	    5700 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   33492	     37099 ns/op	    5700 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   33158	     35775 ns/op	    5699 B/op	      93 allocs/op
BenchmarkTarIndex-4   	   33368	     35768 ns/op	    5701 B/op	      93 allocs/op
PASS
ok  	github.com/anchore/stereoscope/pkg/file	10.983s
PASS
ok  	github.com/anchore/stereoscope/pkg/filetree	0.005s
?   	github.com/anchore/stereoscope/pkg/filetree/filenode	[no test files]
PASS
ok  	github.com/anchore/stereoscope/pkg/image	0.006s
PASS
ok  	github.com/anchore/stereoscope/pkg/image/containerd	0.009s
PASS
ok  	github.com/anchore/stereoscope/pkg/image/docker	0.005s
PASS
ok  	github.com/anchore/stereoscope/pkg/image/oci	0.008s
PASS
ok  	github.com/anchore/stereoscope/pkg/image/oci/credhelpers	0.006s
?   	github.com/anchore/stereoscope/pkg/image/podman	[no test files]
PASS
ok  	github.com/anchore/stereoscope/pkg/image/sif	0.005s
?   	github.com/anchore/stereoscope/pkg/imagetest	[no test files]
PASS
ok  	github.com/anchore/stereoscope/pkg/tree	0.004s
PASS
ok  	github.com/anchore/stereoscope/pkg/tree/node	0.004s
goos: linux
goarch: amd64
pkg: github.com/anchore/stereoscope/test/integration
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkSimpleImage_GetImage/docker-archive-4 	    1044	   1137776 ns/op	  274990 B/op	    2326 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	    1066	   1132290 ns/op	  274424 B/op	    2325 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	    1051	   1131327 ns/op	  274420 B/op	    2325 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	    1059	   1129169 ns/op	  274184 B/op	    2324 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	    1053	   1237663 ns/op	  274118 B/op	    2324 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	    1056	   1216949 ns/op	  274027 B/op	    2324 allocs/op
BenchmarkSimpleImage_GetImage/docker-archive-4 	    1046	   1186925 ns/op	  274035 B/op	    2324 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      67	  18176128 ns/op	  401514 B/op	    2700 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      60	  17895184 ns/op	  401440 B/op	    2699 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      66	  18191148 ns/op	  401288 B/op	    2701 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      64	  21058477 ns/op	  401663 B/op	    2700 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      63	  17912875 ns/op	  402294 B/op	    2701 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      67	  18014333 ns/op	  401924 B/op	    2700 allocs/op
BenchmarkSimpleImage_GetImage/podman-4         	      60	  17940340 ns/op	  400893 B/op	    2699 allocs/op
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 345B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load build context
#3 transferring context: 209B done
#3 DONE 0.0s

#4 [1/3] ADD file-1.txt /somefile-1.txt
#4 CACHED

#5 [2/3] ADD file-2.txt /somefile-2.txt
#5 CACHED

#6 [3/3] ADD target /
#6 CACHED

#7 exporting to image
#7 exporting layers done
#7 writing image sha256:b78ad9356f8f3181be7a7c15d3e5931a77ea267a5d0853a4c3d069e516c9a46f done
#7 naming to docker.io/library/stereoscope-fixture-image-simple:04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7 done
#7 naming to docker.io/library/stereoscope-fixture-image-simple:latest done
#7 DONE 0.0s
ctr: failed to dial "/run/containerd/containerd.sock": connection error: desc = "transport: error while dialing: dial unix /run/containerd/containerd.sock: connect: permission denied"
--- FAIL: BenchmarkSimpleImage_GetImage
    image_fixtures.go:193: using existing image tar: 'test-fixtures/cache/stereoscope-fixture-image-simple-04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7.tar' (size: 22528, modified: 2024-10-14 19:35:12.047072743 +0000 UTC, mode: -rw-r--r--)
    image_fixtures.go:241: Build docker image: name="stereoscope-fixture-image-simple" tag="04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7"
    image_fixtures.go:291: saveImage running: docker image save stereoscope-fixture-image-simple:04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7
    image_fixtures.go:286: 
        	Error Trace:	/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:286
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:162
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:152
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:33
        	            				/home/runner/work/stereoscope/stereoscope/test/integration/fixture_image_simple_test.go:163
        	Error:      	Received unexpected error:
        	            	exit status 1
        	Test:       	BenchmarkSimpleImage_GetImage
        	Messages:   	could not import docker image to containerd (shell out)
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   53662	     22208 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   53625	     22149 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   53781	     22194 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   54073	     22283 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   53904	     22198 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   53755	     22183 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/docker-archive-4         	   54234	     22152 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   53358	     22192 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   53744	     22137 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   53520	     22410 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   53744	     22152 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   54306	     22225 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   54339	     22251 ns/op	    2712 B/op	      21 allocs/op
BenchmarkSimpleImage_FetchSquashedContents/podman-4                 	   53973	     22144 ns/op	    2712 B/op	      21 allocs/op
#0 building with "default" instance using docker driver

#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 345B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load build context
#3 transferring context: 209B done
#3 DONE 0.0s

#4 [1/3] ADD file-1.txt /somefile-1.txt
#4 CACHED

#5 [2/3] ADD file-2.txt /somefile-2.txt
#5 CACHED

#6 [3/3] ADD target /
#6 CACHED

#7 exporting to image
#7 exporting layers done
#7 writing image sha256:b78ad9356f8f3181be7a7c15d3e5931a77ea267a5d0853a4c3d069e516c9a46f done
#7 naming to docker.io/library/stereoscope-fixture-image-simple:04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7 done
#7 naming to docker.io/library/stereoscope-fixture-image-simple:latest done
#7 DONE 0.0s
ctr: failed to dial "/run/containerd/containerd.sock": connection error: desc = "transport: error while dialing: dial unix /run/containerd/containerd.sock: connect: permission denied"
--- FAIL: BenchmarkSimpleImage_FetchSquashedContents
    image_fixtures.go:193: using existing image tar: 'test-fixtures/cache/stereoscope-fixture-image-simple-04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7.tar' (size: 22528, modified: 2024-10-14 19:35:12.047072743 +0000 UTC, mode: -rw-r--r--)
    image_fixtures.go:241: Build docker image: name="stereoscope-fixture-image-simple" tag="04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7"
    image_fixtures.go:291: saveImage running: docker image save stereoscope-fixture-image-simple:04e16e44161c8888a1a963720fd0443cbf7eef8101434c431de8725cd98cc9f7
    image_fixtures.go:286: 
        	Error Trace:	/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:286
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:162
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:152
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:33
        	            				/home/runner/work/stereoscope/stereoscope/pkg/imagetest/image_fixtures.go:64
        	            				/home/runner/work/stereoscope/stereoscope/test/integration/fixture_image_simple_test.go:189
        	Error:      	Received unexpected error:
        	            	exit status 1
        	Test:       	BenchmarkSimpleImage_FetchSquashedContents
        	Messages:   	could not import docker image to containerd (shell out)
FAIL
exit status 1
FAIL	github.com/anchore/stereoscope/test/integration	39.044s
?   	github.com/anchore/stereoscope/test/integration/test-fixtures/registry	[no test files]
FAIL
goos: linux
goarch: amd64
pkg: github.com/anchore/stereoscope/pkg/file
cpu: AMD EPYC 7763 64-Core Processor                
ctr: 
           │ .tmp/benchmark-575f041.txt │
           │           sec/op           │
TarIndex-4                  35.78µ ± 4%

           │ .tmp/benchmark-575f041.txt │
           │            B/op            │
TarIndex-4                 5.566Ki ± 0%

           │ .tmp/benchmark-575f041.txt │
           │         allocs/op          │
TarIndex-4                   93.00 ± 0%

pkg: github.com/anchore/stereoscope/test/integration
                                      │ .tmp/benchmark-575f041.txt │
                                      │           sec/op           │
SimpleImage_GetImage/docker-archive-4                 1.138m ±  9%
SimpleImage_GetImage/podman-4                         18.01m ± 17%
geomean                                               4.527m

                                      │ .tmp/benchmark-575f041.txt │
                                      │            B/op            │
SimpleImage_GetImage/docker-archive-4                 267.8Ki ± 0%
SimpleImage_GetImage/podman-4                         392.1Ki ± 0%
geomean                                               324.0Ki

                                      │ .tmp/benchmark-575f041.txt │
                                      │         allocs/op          │
SimpleImage_GetImage/docker-archive-4                  2.324k ± 0%
SimpleImage_GetImage/podman-4                          2.700k ± 0%
geomean                                                2.505k

ctr: failed to dial "/run/containerd/containerd.sock": connection error: desc = "transport: error while dialing: dial unix /run/containerd/containerd.sock: connect: permission denied"
                                                   │ .tmp/benchmark-575f041.txt │
                                                   │           sec/op           │
SimpleImage_FetchSquashedContents/docker-archive-4                  22.19µ ± 0%
SimpleImage_FetchSquashedContents/podman-4                          22.19µ ± 1%
geomean                                                             22.19µ

                                                   │ .tmp/benchmark-575f041.txt │
                                                   │            B/op            │
SimpleImage_FetchSquashedContents/docker-archive-4                 2.648Ki ± 0%
SimpleImage_FetchSquashedContents/podman-4                         2.648Ki ± 0%
geomean                                                            2.648Ki

                                                   │ .tmp/benchmark-575f041.txt │
                                                   │         allocs/op          │
SimpleImage_FetchSquashedContents/docker-archive-4                   21.00 ± 0%
SimpleImage_FetchSquashedContents/podman-4                           21.00 ± 0%
geomean                                                              21.00
goos: linux
goarch: amd64
pkg: github.com/anchore/stereoscope/pkg/file
cpu: AMD EPYC 7763 64-Core Processor                
ctr: 
           │ .tmp/benchmark-575f041.txt │
           │           sec/op           │
TarIndex-4                  35.78µ ± 4%

           │ .tmp/benchmark-575f041.txt │
           │            B/op            │
TarIndex-4                 5.566Ki ± 0%

           │ .tmp/benchmark-575f041.txt │
           │         allocs/op          │
TarIndex-4                   93.00 ± 0%

pkg: github.com/anchore/stereoscope/test/integration
                                      │ .tmp/benchmark-575f041.txt │
                                      │           sec/op           │
SimpleImage_GetImage/docker-archive-4                 1.138m ±  9%
SimpleImage_GetImage/podman-4                         18.01m ± 17%
geomean                                               4.527m

                                      │ .tmp/benchmark-575f041.txt │
                                      │            B/op            │
SimpleImage_GetImage/docker-archive-4                 267.8Ki ± 0%
SimpleImage_GetImage/podman-4                         392.1Ki ± 0%
geomean                                               324.0Ki

                                      │ .tmp/benchmark-575f041.txt │
                                      │         allocs/op          │
SimpleImage_GetImage/docker-archive-4                  2.324k ± 0%
SimpleImage_GetImage/podman-4                          2.700k ± 0%
geomean                                                2.505k

ctr: failed to dial "/run/containerd/containerd.sock": connection error: desc = "transport: error while dialing: dial unix /run/containerd/containerd.sock: connect: permission denied"
                                                   │ .tmp/benchmark-575f041.txt │
                                                   │           sec/op           │
SimpleImage_FetchSquashedContents/docker-archive-4                  22.19µ ± 0%
SimpleImage_FetchSquashedContents/podman-4                          22.19µ ± 1%
geomean                                                             22.19µ

                                                   │ .tmp/benchmark-575f041.txt │
                                                   │            B/op            │
SimpleImage_FetchSquashedContents/docker-archive-4                 2.648Ki ± 0%
SimpleImage_FetchSquashedContents/podman-4                         2.648Ki ± 0%
geomean                                                            2.648Ki

                                                   │ .tmp/benchmark-575f041.txt │
                                                   │         allocs/op          │
SimpleImage_FetchSquashedContents/docker-archive-4                   21.00 ± 0%
SimpleImage_FetchSquashedContents/podman-4                           21.00 ± 0%
geomean                                                              21.00

@willmurphyscode
chore: remove noise from go.mod
848fcce 
Signed-off-by: Will Murphy <willmurphyscode@users.noreply.github.com>
@wagoodman wagoodman enabled auto-merge (squash) October 14, 2024 19:36
@wagoodman wagoodman mentioned this pull request Oct 14, 2024
Open
1 task
@wagoodman wagoodman merged commit 93f8a11 into main Oct 14, 2024
7 checks passed
@wagoodman wagoodman deleted the update-mimetype branch October 14, 2024 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@willmurphyscode willmurphyscode willmurphyscode approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@wagoodman @willmurphyscode