e2e podman private registry #17642

shoenig · 2023-06-21T19:17:00Z

note: I had to add -k to a few curl commands - I'm fairly sure the base Ubuntu image did a Bad Thing to its own trust store / curl package and we'll need to ignore certificate errors until that is fixed. Also replaced the use of the deprecated apt-key command with the gpg/tee dance recommended by Docker and Hashicorp packaging.

e2e: add a test for using private registry with podman driver

This PR adds e2e tests that stands up a private docker registry
and has a podman tasks run a container from an image in that private
registry.

Tests

user:password set in task config
auth_soft_fail works for public images when auth is set in driver
credentials helper is set in driver auth config
config auth.json file is set in driver auth config

Spot check against e2e:

nomad/e2e/podman on e2e-podman-private-registry
➜ go test -v
=== RUN   TestPodman
=== RUN   TestPodman/testRedis
=== RUN   TestPodman/testAuthBasic
=== RUN   TestPodman/testAuthFileStatic
=== RUN   TestPodman/testAuthHelper
    podman_test.go:109: registry 172.31.85.15 20633
--- PASS: TestPodman (76.38s)
    --- PASS: TestPodman/testRedis (19.60s)
    --- PASS: TestPodman/testAuthBasic (10.39s)
    --- PASS: TestPodman/testAuthFileStatic (9.32s)
    --- PASS: TestPodman/testAuthHelper (8.21s)
PASS
ok      github.com/hashicorp/nomad/e2e/podman   76.383s

This PR adds e2e tests that stands up a private docker registry and has a podman tasks run a container from an image in that private registry. Tests - user:password set in task config - auth_soft_fail works for public images when auth is set in driver - credentials helper is set in driver auth config - config auth.json file is set in driver auth config

gulducat

LGTM!

gulducat · 2023-07-19T18:16:49Z

e2e/podman/input/registry-auths.hcl

+# This job runs after the private registry is up and running, when we know
+# address and port provided by the bridge network. It is a sysbatch job
+# that writes these files on every linux client.
+#   - /usr/local/bin/docker-credential-test.sh


this is "docker-credential-" rather than "podman-credential-" because the registry itself is docker?

It's just because the "official" credentials helpers in the ecosystem are provided by docker and their documentation prefixes everything using "docker-credential-"

https://github.com/docker/docker-credential-helpers/tree/master

I suppose we could search for either prefix, but unless someone actually asks for it I think we should just keep it simple and working with the known tools.

gulducat · 2023-07-19T18:16:56Z

e2e/podman/input/registry-auths.hcl

+
+    # write out the test.sh file into var.helper_dir
+    task "create-helper-file" {
+      driver = "pledge"


fun use of pledge 🧼

gulducat · 2023-07-19T18:17:41Z

e2e/podman/input/registry-auths.hcl

+        destination = "local/auth.json"
+        data        = <<EOH
+{
+  "auths": {


This one's pretty trivial, but I like jsonencode to avoid possibly-invalid raw json.

gulducat · 2023-07-19T18:18:21Z

e2e/podman/input/registry.hcl

+# Copyright (c) HashiCorp, Inc.
+# SPDX-License-Identifier: MPL-2.0
+
+# This job stands up a private container registry for use in e2e tests.


idle pondering: I wonder if a general version of this might be nice for other things one day.

we should at least add similar tests for the docker driver, but that's for another PR

e2e/podman/input/registry.hcl

gulducat · 2023-07-19T18:21:06Z

e2e/podman/podman_test.go

+		jobs3.Var("user", "root"),
+		jobs3.Var("helper_dir", "/usr/local/bin"),
+		jobs3.Var("auth_dir", "/etc"),
+		jobs3.WaitComplete("create-files"),


took me a minute to determine that "create-files" here is a task group name?

oh good feedback, maybe this could be WaitGroupComplete or something

gulducat · 2023-07-19T18:21:23Z

e2e/terraform/packer/ubuntu-jammy-amd64/setup.sh

-sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
+wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
+echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
+sudo apt-get update && sudo apt-get install nomad


nomad is already being installed on line 65 down below with consul-enterprise. could save an apt update here too and let line 60 handle that.

Co-authored-by: Daniel Bennett <dbennett@hashicorp.com>

* e2e: add tests for using private registry with podman driver This PR adds e2e tests that stands up a private docker registry and has a podman tasks run a container from an image in that private registry. Tests - user:password set in task config - auth_soft_fail works for public images when auth is set in driver - credentials helper is set in driver auth config - config auth.json file is set in driver auth config * packer: use nomad-driver-podman v0.5.0 * e2e: eliminate unnecessary chmod Co-authored-by: Daniel Bennett <dbennett@hashicorp.com> * cr: no need to install nomad twice * cl: no need to install docker twice --------- Co-authored-by: Daniel Bennett <dbennett@hashicorp.com>

vercel bot deployed to Preview – nomad-storybook-and-ui June 21, 2023 19:18 View deployment

shoenig force-pushed the e2e-podman-private-registry branch from 842e44a to 3227041 Compare June 21, 2023 19:33

vercel bot deployed to Preview – nomad-storybook-and-ui June 21, 2023 19:35 View deployment

shoenig mentioned this pull request Jun 26, 2023

config: enable setting tlsVerify during image pull hashicorp/nomad-driver-podman#262

Merged

shoenig force-pushed the e2e-podman-private-registry branch from 3227041 to bebbc16 Compare June 26, 2023 13:29

vercel bot deployed to Preview – nomad-storybook-and-ui June 26, 2023 13:31 View deployment

shoenig force-pushed the e2e-podman-private-registry branch from 2ec4660 to bbe48bb Compare July 2, 2023 20:08

vercel bot deployed to Preview – nomad-storybook-and-ui July 2, 2023 20:13 View deployment

vercel bot deployed to Preview – nomad-storybook-and-ui July 2, 2023 20:59 View deployment

shoenig force-pushed the e2e-podman-private-registry branch from 946e613 to 026129e Compare July 5, 2023 17:16

vercel bot deployed to Preview – nomad-storybook-and-ui July 5, 2023 17:19 View deployment

vercel bot deployed to Preview – nomad-storybook-and-ui July 12, 2023 16:27 View deployment

shoenig force-pushed the e2e-podman-private-registry branch from 490cf77 to 3843526 Compare July 12, 2023 16:58

vercel bot deployed to Preview – nomad-storybook-and-ui July 12, 2023 17:05 View deployment

vercel bot deployed to Preview – nomad-storybook-and-ui July 12, 2023 22:10 View deployment

shoenig force-pushed the e2e-podman-private-registry branch from 9472a6b to 23d48ce Compare July 12, 2023 23:47

vercel bot deployed to Preview – nomad-storybook-and-ui July 12, 2023 23:50 View deployment

vercel bot deployed to Preview – nomad-storybook-and-ui July 13, 2023 01:08 View deployment

shoenig force-pushed the e2e-podman-private-registry branch from e5baa60 to 79844cb Compare July 13, 2023 01:29

vercel bot deployed to Preview – nomad-storybook-and-ui July 13, 2023 01:32 View deployment

vercel bot deployed to Preview – nomad-storybook-and-ui July 19, 2023 14:27 View deployment

shoenig force-pushed the e2e-podman-private-registry branch from 82416f7 to 3006a2c Compare July 19, 2023 14:42

vercel bot deployed to Preview – nomad-storybook-and-ui July 19, 2023 14:45 View deployment

shoenig force-pushed the e2e-podman-private-registry branch from 3006a2c to f32a57c Compare July 19, 2023 14:52

vercel bot deployed to Preview – nomad-storybook-and-ui July 19, 2023 14:54 View deployment

shoenig force-pushed the e2e-podman-private-registry branch from f32a57c to feea8b9 Compare July 19, 2023 15:02

vercel bot deployed to Preview – nomad-storybook-and-ui July 19, 2023 15:05 View deployment

shoenig force-pushed the e2e-podman-private-registry branch from feea8b9 to 68440d3 Compare July 19, 2023 15:07

vercel bot deployed to Preview – nomad-storybook-and-ui July 19, 2023 15:10 View deployment

packer: use nomad-driver-podman v0.5.0

abaa4fd

shoenig force-pushed the e2e-podman-private-registry branch from 68440d3 to abaa4fd Compare July 19, 2023 15:29

vercel bot deployed to Preview – nomad-storybook-and-ui July 19, 2023 15:32 View deployment

shoenig marked this pull request as ready for review July 19, 2023 16:27

shoenig requested review from lgfa29 and gulducat July 19, 2023 16:28

gulducat approved these changes Jul 19, 2023

View reviewed changes

shoenig and others added 3 commits July 19, 2023 15:51

e2e: eliminate unnecessary chmod

675ad74

Co-authored-by: Daniel Bennett <dbennett@hashicorp.com>

cr: no need to install nomad twice

5e2d6be

cl: no need to install docker twice

3777d6d

shoenig merged commit 8d28946 into main Jul 19, 2023
5 checks passed

shoenig deleted the e2e-podman-private-registry branch July 19, 2023 20:59

vercel bot deployed to Preview – nomad-storybook-and-ui July 19, 2023 21:03 View deployment

This pull request was closed.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

e2e podman private registry #17642

e2e podman private registry #17642

shoenig commented Jun 21, 2023 •

edited

Loading

gulducat left a comment

gulducat Jul 19, 2023

shoenig Jul 19, 2023

gulducat Jul 19, 2023

gulducat Jul 19, 2023

gulducat Jul 19, 2023

shoenig Jul 19, 2023

gulducat Jul 19, 2023

shoenig Jul 19, 2023

gulducat Jul 19, 2023

shoenig Jul 19, 2023

e2e podman private registry #17642

e2e podman private registry #17642

Conversation

shoenig commented Jun 21, 2023 • edited Loading

gulducat left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shoenig commented Jun 21, 2023 •

edited

Loading