Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: support for replacing images with custom substitutions #1719

Merged
merged 5 commits into from
Oct 20, 2023
Merged

feat: support for replacing images with custom substitutions #1719

merged 5 commits into from
Oct 20, 2023

Conversation

mdelapenya
Copy link
Member

@mdelapenya mdelapenya commented Oct 3, 2023 •
edited
Loading

What does this PR do?

This PR adds a mechanism for modifying the Docker image used in the container request, and for that, an slice of ImageSubstitutors can be configured at the container request level.

At the moment we resolve the tag to pull/use the Docker image, it will run the defined substitutors in order to calculate the final image.

To facilitate users to provide their own substitutors, we have exposed an interface, so users of tc-go can implement their own substitutors. Examples could be:

  • prepend a custom registry namespace: e.g. myregistry.com/docker.io/redis:latest
  • massage the tag: e.g. docker.io/redis:6-MYCUSTOMHASH

The PR includes documentation for the substitutor interface, also adding them to each module's functional options.

Why is it important?

In more locked down / secured environments, it can be problematic to pull images from Docker Hub and run them without additional precautions.

An image name substitutor converts a Docker image name, as may be specified in code, to an alternative name. This is intended to provide a way to override image names, for example to enforce pulling of images from a private registry.

Related issues

Follow-ups

We can support setting custom substitutions in the testcontainers.properties file. Please see https://java.testcontainers.org/features/image_name_substitution/#automatically-modifying-docker-hub-image-names to check requirements. cc/ @eddumelendez

@mdelapenya mdelapenya added feature New functionality or new behaviors on the existing one hacktoberfest Pull Requests accepted for Hacktoberfest. labels Oct 3, 2023
@mdelapenya mdelapenya self-assigned this Oct 3, 2023
@netlify
Copy link

netlify bot commented Oct 3, 2023 •
edited
Loading

✅ Deploy Preview for testcontainers-go ready!

Name Link
🔨 Latest commit 3fa2d1e
🔍 Latest deploy log https://app.netlify.com/sites/testcontainers-go/deploys/6528ecebbf1a9200085db3ac
😎 Deploy Preview https://deploy-preview-1719--testcontainers-go.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

vikstrous2
vikstrous2 reviewed Oct 3, 2023
View reviewed changes
docker.go Outdated Show resolved Hide resolved
generic.go Outdated Show resolved Hide resolved
@mdelapenya
Merge branch 'main' into support-for-image-resolvers
d0cfe0b 
* main:
  docs: Fix typo and mention the relevant function name in doc (testcontainers#1745)
  DOCKER_HOST var typo (testcontainers#1743)
  feat: Add Cassandra module (testcontainers#1726)
  K6 module (testcontainers#1721)
  Rancher Desktop instructions (testcontainers#1724)
  chore(deps): bump github.com/shirou/gopsutil/v3 from 3.23.8 to 3.23.9 (testcontainers#1720)
  chore(deps): bump urllib3 from 2.0.5 to 2.0.6 (testcontainers#1718)
  chore(deps): bump github.com/twmb/franz-go/pkg/kadm in /modules/redpanda (testcontainers#1714)
  chore(deps): bump github.com/couchbase/gocb/v2 in /modules/couchbase (testcontainers#1704)
  chore(deps): bump github.com/neo4j/neo4j-go-driver/v5 in /modules/neo4j (testcontainers#1713)
@mdelapenya
chore: support handling errors in image substitutor
bf56787
@mdelapenya
chore: add test for substitution errors
b29f602
@mdelapenya mdelapenya marked this pull request as ready for review October 13, 2023 07:03
@mdelapenya mdelapenya requested a review from a team as a code owner October 13, 2023 07:03
vikstrous2
vikstrous2 approved these changes Oct 16, 2023
View reviewed changes
Copy link

@vikstrous2 vikstrous2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love it! Thanks! I can't comment on if it works, but the API looks easy to use.

@mdelapenya
Copy link
Member Author

As discussed internally, and as a follow-up to this PR, we are going to provide a secondary way to substitute images at the properties level:

You can then configure Testcontainers to apply the prefix registry.mycompany.com/mirror/ to every image that it tries to pull from Docker Hub. This can be done in one of two ways:

  • Setting environment variables TESTCONTAINERS_HUB_IMAGE_NAME_PREFIX=registry.mycompany.com/mirror/
  • Via config file, setting hub.image.name.prefix in the ~/.testcontainers.properties file in your user home directory
    Testcontainers will automatically apply the prefix to every image that it pulls from Docker Hub - please verify that all the required images exist in your registry.

Testcontainers will not apply the prefix to:

  • non-Hub image names (e.g. where another registry is set)
  • Docker Hub image names where the hub registry is explicitly part of the name (i.e. anything with a docker.io or registry.hub.docker.com host part)

Please read more on this here: https://java.testcontainers.org/features/image_name_substitution/#automatically-modifying-docker-hub-image-names

@mdelapenya mdelapenya merged commit d49ecc2 into testcontainers:main Oct 20, 2023
114 checks passed
mdelapenya added a commit to mdelapenya/testcontainers-go that referenced this pull request Oct 24, 2023
@mdelapenya
Merge branch 'main' into ms/issue-946-v2
aec5067 
* main:
  chore(deps): combine and bump compose dependencies (testcontainers#1787)
  feat: support for replacing images with custom substitutions (testcontainers#1719)
mdelapenya added a commit that referenced this pull request Oct 26, 2023
@mdelapenya
Merge branch 'main' into dependabot/go_modules/golang.org/x/sys-0.13.0
5440de4 
* main: (27 commits)
  docs: remove OpenSSF scorecard (#1823)
  Auto-cleanup of k6 build cache (#1788)
  Add OpenSSF Scorecards GitHub Action (#1795)
  chore(deps): bump google.golang.org/grpc from 1.57.0 to 1.57.1 (#1822)
  chore: expose SessionID (#1793)
  chore: use HTTP calls to invoke the lambda from the tests (#1794)
  wait for log producer to really stop inside StopLogProducer func (#1701)
  chore(deps): bump github.com/nats-io/nats-server/v2 in /modules/nats (#1784)
  chore(deps): bump urllib3 from 2.0.6 to 2.0.7 (#1781)
  chore: add an example of using localstack alongside AWS lambdas (#1790)
  chore(deps): combine and bump compose dependencies (#1787)
  feat: support for replacing images with custom substitutions (#1719)
  fix: data race in docker client `Info()` (#1779)
  Use correct formatting directive for errors in lifecycle logs (#1780)
  chore(deps): bump golang.org/x/mod from 0.12.0 to 0.13.0 in /modules/{elasticsearch,kafka} and /modulegen (#1778)
  chore(deps): bump github.com/rabbitmq/amqp091-go in /modules/rabbitmq (#1728)
  chore(deps): bump github.com/ClickHouse/clickhouse-go/v2 (#1732)
  ignore patterns defined in dockerignore (#1725)
  Fix wrong module names (#1776)
  docs: add default options to k6 module (#1744)
  ...
mdelapenya added a commit that referenced this pull request Oct 26, 2023
@mdelapenya
Merge branch 'main' into dependabot/go_modules/github.com/opencontain…
88f3083 
…ers/image-spec-1.1.0-rc5

* main: (49 commits)
  docs: remove OpenSSF scorecard (#1823)
  Auto-cleanup of k6 build cache (#1788)
  Add OpenSSF Scorecards GitHub Action (#1795)
  chore(deps): bump google.golang.org/grpc from 1.57.0 to 1.57.1 (#1822)
  chore: expose SessionID (#1793)
  chore: use HTTP calls to invoke the lambda from the tests (#1794)
  wait for log producer to really stop inside StopLogProducer func (#1701)
  chore(deps): bump github.com/nats-io/nats-server/v2 in /modules/nats (#1784)
  chore(deps): bump urllib3 from 2.0.6 to 2.0.7 (#1781)
  chore: add an example of using localstack alongside AWS lambdas (#1790)
  chore(deps): combine and bump compose dependencies (#1787)
  feat: support for replacing images with custom substitutions (#1719)
  fix: data race in docker client `Info()` (#1779)
  Use correct formatting directive for errors in lifecycle logs (#1780)
  chore(deps): bump golang.org/x/mod from 0.12.0 to 0.13.0 in /modules/{elasticsearch,kafka} and /modulegen (#1778)
  chore(deps): bump github.com/rabbitmq/amqp091-go in /modules/rabbitmq (#1728)
  chore(deps): bump github.com/ClickHouse/clickhouse-go/v2 (#1732)
  ignore patterns defined in dockerignore (#1725)
  Fix wrong module names (#1776)
  docs: add default options to k6 module (#1744)
  ...
@mdelapenya mdelapenya mentioned this pull request Nov 3, 2023
Closed
@mdelapenya mdelapenya deleted the support-for-image-resolvers branch November 6, 2023 11:01
@mdelapenya mdelapenya mentioned this pull request Nov 28, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@eddumelendez eddumelendez eddumelendez left review comments

@vikstrous2 vikstrous2 vikstrous2 approved these changes

Assignees

@mdelapenya mdelapenya

Labels
feature New functionality or new behaviors on the existing one hacktoberfest Pull Requests accepted for Hacktoberfest.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Enhancement]: Provide a way to customize extra docker containers used
3 participants
@mdelapenya @eddumelendez @vikstrous2