Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Offer a Quarkus Dev Services compatible way to launch Kafka and PostgreSQL containers with API server #1188

Open
2 tasks done
nscuro opened this issue Apr 12, 2024 · 7 comments
Labels
component/api-server dx Issues related to developer experience (DX) enhancement New feature or request good first issue Good for newcomers p3 Nice-to-have features size/M Medium effort

Comments

@nscuro
Copy link
Member

nscuro commented Apr 12, 2024

Current Behavior

Currently, local development on the API server requires Kafka and PostgreSQL to be launched via Docker Compose.

While this works, it's a bit fiddly and not a good experience for new contributors.

Quarkus has Dev Services, which can automatically launch containers for infrastructure dependencies such as message buses and databases. Further, multiple services can discover and re-use existing containers started by another service.

The API server, being based on Alpine, does not offer a comparable capability. However, Dependency-Track v4.x already offers a dev-only feature to enable the H2 web console (DependencyTrack/dependency-track#2592), which can only be enabled through the h2-console Maven profile.

Proposed Behavior

Implement a new dev-only capability to automatically start Kafka and PostgreSQL containers.

Optimally, make it so that containers started by the API server can be reused by Quarkus services, and vice versa. This needs a bit of investigation as to how Quarkus performs discovery on existing containers. Based on my current understanding, container labels play a role.

Similar to the H2 console, leverage a Maven profile to ensure that production builds will not have this behavior.


Implementation notes:

  • Use docker-java to launch and discover containers
  • Execute this logic in a ServletContextListener so it's run on application startup
  • Register the listener as one of the first in web.xml so it executes before the database or Kafka clients are initialized

Checklist

@nscuro nscuro added enhancement New feature or request p3 Nice-to-have features size/M Medium effort component/api-server dx Issues related to developer experience (DX) good first issue Good for newcomers labels Apr 12, 2024
@withPrasheel
Copy link

withPrasheel commented Apr 13, 2024

Hey, I will replicate the issue and take it up. Can you assign this to me?

@nscuro
Copy link
Member Author

nscuro commented Apr 29, 2024

@withPrasheel Are you still planning to work on this?

@withPrasheel
Copy link

withPrasheel commented Apr 29, 2024

Hi @nscuro , I worked on the setup and understanding of the documentation. I have an exam due tomo 5pm right now, after which I will be available complete summer. So I can work on it after it. Is there a deadline you are expecting for this?

@nscuro
Copy link
Member Author

nscuro commented Apr 29, 2024

Thanks for reporting back @withPrasheel! No deadline at all, just checking in to see if we need to un-assign people from issues they don't have the capacity to work on. Otherwise they'd be blocked for other contributors interested to work on them. :)

Please let me know if you need help in any way!

@withPrasheel
Copy link

@nscuro Can you unassign this issue? It's taken too much time, and I am not able to find time. I will continue to work in my capacity, learning a few things first in case no one takes it up later.
Sorry for the inconvenience.

@nscuro
Copy link
Member Author

nscuro commented May 16, 2024

Sure, no worries!

@nscuro
Copy link
Member Author

nscuro commented May 16, 2024

Related quarkusio/quarkus#40376

nscuro added a commit to DependencyTrack/hyades-apiserver that referenced this issue Jun 14, 2024
Adds a new `dev-services` Maven profile. It is intended to be used in conjunction with the Jetty plugin. A respective IntelliJ run configuration is included.

When enabled, PostgreSQL and Redpanda containers will be launched automatically, and disposed when the application stops. Dependency-Track will be auto-configured to use the containers, no manual configuration required.

To avoid introducing runtime dependencies on test libraries for production builds, the logic uses reflection to interact with the `testcontainers` library.

Topics required by the API server will be created automatically. Database migrations are executed as usual.

The containers are not currently discoverable and re-usable by the Quarkus-based services. That capability is planned for a future iteration, see DependencyTrack/hyades#1188

With this functionality, it becomes easier to test features that do not rely on the other Hyades services (e.g. REST API related stuff), or frontend modifications. In those scenarios, using Docker Compose is no longer needed.

Signed-off-by: nscuro <nscuro@protonmail.com>
nscuro added a commit to DependencyTrack/hyades-apiserver that referenced this issue Jun 14, 2024
Adds a new `dev-services` Maven profile. It is intended to be used in conjunction with the Jetty plugin. A respective IntelliJ run configuration is included.

When enabled, PostgreSQL and Redpanda containers will be launched automatically, and disposed when the application stops. Dependency-Track will be auto-configured to use the containers, no manual configuration required.

To avoid introducing runtime dependencies on test libraries for production builds, the logic uses reflection to interact with the `testcontainers` library.

Topics required by the API server will be created automatically. Database migrations are executed as usual.

The containers are not currently discoverable and re-usable by the Quarkus-based services. That capability is planned for a future iteration, see DependencyTrack/hyades#1188

With this functionality, it becomes easier to test features that do not rely on the other Hyades services (e.g. REST API related stuff), or frontend modifications. In those scenarios, using Docker Compose is no longer needed.

Signed-off-by: nscuro <nscuro@protonmail.com>
nscuro added a commit to DependencyTrack/hyades-apiserver that referenced this issue Jun 14, 2024
Adds a new `dev-services` Maven profile. It is intended to be used in conjunction with the Jetty plugin. A respective IntelliJ run configuration is included.

When enabled, PostgreSQL and Redpanda containers will be launched automatically, and disposed when the application stops. Dependency-Track will be auto-configured to use the containers, no manual configuration required.

To avoid introducing runtime dependencies on test libraries for production builds, the logic uses reflection to interact with the `testcontainers` library.

Topics required by the API server will be created automatically. Database migrations are executed as usual.

The containers are not currently discoverable and re-usable by the Quarkus-based services. That capability is planned for a future iteration, see DependencyTrack/hyades#1188

With this functionality, it becomes easier to test features that do not rely on the other Hyades services (e.g. REST API related stuff), or frontend modifications. In those scenarios, using Docker Compose is no longer needed.

Signed-off-by: nscuro <nscuro@protonmail.com>
nscuro added a commit to DependencyTrack/hyades-apiserver that referenced this issue Jun 15, 2024
Adds a new `dev-services` Maven profile. It is intended to be used in conjunction with the Jetty plugin. A respective IntelliJ run configuration is included.

When enabled, PostgreSQL and Redpanda containers will be launched automatically, and disposed when the application stops. Dependency-Track will be auto-configured to use the containers, no manual configuration required.

To avoid introducing runtime dependencies on test libraries for production builds, the logic uses reflection to interact with the `testcontainers` library.

Topics required by the API server will be created automatically. Database migrations are executed as usual.

The containers are not currently discoverable and re-usable by the Quarkus-based services. That capability is planned for a future iteration, see DependencyTrack/hyades#1188

With this functionality, it becomes easier to test features that do not rely on the other Hyades services (e.g. REST API related stuff), or frontend modifications. In those scenarios, using Docker Compose is no longer needed.

Signed-off-by: nscuro <nscuro@protonmail.com>
nscuro added a commit to DependencyTrack/hyades-apiserver that referenced this issue Jun 15, 2024
Adds a new `dev-services` Maven profile. It is intended to be used in conjunction with the Jetty plugin. A respective IntelliJ run configuration is included.

When enabled, PostgreSQL and Redpanda containers will be launched automatically, and disposed when the application stops. Dependency-Track will be auto-configured to use the containers, no manual configuration required.

To avoid introducing runtime dependencies on test libraries for production builds, the logic uses reflection to interact with the `testcontainers` library.

Topics required by the API server will be created automatically. Database migrations are executed as usual.

The containers are not currently discoverable and re-usable by the Quarkus-based services. That capability is planned for a future iteration, see DependencyTrack/hyades#1188

With this functionality, it becomes easier to test features that do not rely on the other Hyades services (e.g. REST API related stuff), or frontend modifications. In those scenarios, using Docker Compose is no longer needed.

Signed-off-by: nscuro <nscuro@protonmail.com>
nscuro added a commit to DependencyTrack/hyades-apiserver that referenced this issue Jun 16, 2024
Adds a new `dev-services` Maven profile. It is intended to be used in conjunction with the Jetty plugin. A respective IntelliJ run configuration is included.

When enabled, PostgreSQL and Redpanda containers will be launched automatically, and disposed when the application stops. Dependency-Track will be auto-configured to use the containers, no manual configuration required.

To avoid introducing runtime dependencies on test libraries for production builds, the logic uses reflection to interact with the `testcontainers` library.

Topics required by the API server will be created automatically. Database migrations are executed as usual.

The containers are not currently discoverable and re-usable by the Quarkus-based services. That capability is planned for a future iteration, see DependencyTrack/hyades#1188

With this functionality, it becomes easier to test features that do not rely on the other Hyades services (e.g. REST API related stuff), or frontend modifications. In those scenarios, using Docker Compose is no longer needed.

Signed-off-by: nscuro <nscuro@protonmail.com>
nscuro added a commit to DependencyTrack/hyades-apiserver that referenced this issue Jun 20, 2024
Adds a new `dev-services` Maven profile. It is intended to be used in conjunction with the Jetty plugin. A respective IntelliJ run configuration is included.

When enabled, PostgreSQL and Redpanda containers will be launched automatically, and disposed when the application stops. Dependency-Track will be auto-configured to use the containers, no manual configuration required.

To avoid introducing runtime dependencies on test libraries for production builds, the logic uses reflection to interact with the `testcontainers` library.

Topics required by the API server will be created automatically. Database migrations are executed as usual.

The containers are not currently discoverable and re-usable by the Quarkus-based services. That capability is planned for a future iteration, see DependencyTrack/hyades#1188

With this functionality, it becomes easier to test features that do not rely on the other Hyades services (e.g. REST API related stuff), or frontend modifications. In those scenarios, using Docker Compose is no longer needed.

Signed-off-by: nscuro <nscuro@protonmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/api-server dx Issues related to developer experience (DX) enhancement New feature or request good first issue Good for newcomers p3 Nice-to-have features size/M Medium effort
Projects
None yet
Development

No branches or pull requests

2 participants