This is the application that manages the entities related to the controls service for the Tackle Application Inventory application.
This includes Business Services, Stakeholders, Stakeholder Groups, Tag Types and Tags.
This project uses Quarkus, the Supersonic Subatomic Java Framework.
If you want to learn more about Quarkus, please visit its website: https://quarkus.io/.
REST Data Panache (using the active record pattern) provides:
- sorting
- pagination
Quarkus Flyway extension provides:
- DB production management: SQL scripts must be placed in
src/main/resources/db/migration/
folder, following Quarkus Flyway default location - DB test data import: SQL scripts must be placed in
src/main/resources/db/test-data/
folder, leveraging thetest
profile property%test.quarkus.flyway.locations=db/migration,db/test-data
File names convention is based on Flyway "Versioned Migrations" docs and follows the pattern:
V<yyyymmdd>__<SQL action>_<resource>.sql
where:
yyyymmdd
is the date the file has been added (e.g.20200928
).
In case of multiple files added the same day, add a.X
suffix (e.g.20210104.1
)SQL action
is the "SQL action" executed inside the SQL script (e.g.create
,alter
,insert
)resource
is the resource managed by the SQL scripts (e.g.business-service
)
First start a PostreSQL container in Podman executing:
$ podman run -it --rm=true --memory-swappiness=0 \
--name postgres-controls -e POSTGRES_USER=controls \
-e POSTGRES_PASSWORD=controls -e POSTGRES_DB=controls_db \
-p 5432:5432 postgres:10.6
If running on Fedora 32 or below, remove the --memory-swappiness
switch, executing:
$ podman run -it --rm=true \
--name postgres-controls -e POSTGRES_USER=controls \
-e POSTGRES_PASSWORD=controls -e POSTGRES_DB=controls_db \
-p 5432:5432 postgres:10.6
It works the same with Docker just replacing podman
with docker
in the above command.
To connect to the running PostgreSQL instance, first retrieve the CONTAINER ID
value executing:
$ podman ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
36f92b030807 docker.io/library/postgres:10.6 postgres 22 minutes ago Up 22 minutes ago 0.0.0.0:5432->5432/tcp postgres-controls
In the example CONTAINER ID
is 36f92b030807
and use it in the next command:
$ podman exec -it <CONTAINER ID> bash
Once the container's terminal will be available, connect to the database running:
root@36f92b030807:/# psql -U controls -d controls_db
$ podman run -it --name keycloak --rm \
-e KEYCLOAK_USER=admin -e KEYCLOAK_PASSWORD=admin -e KEYCLOAK_IMPORT=/tmp/keycloak/quarkus-realm.json \
-e DB_VENDOR=h2 -p 8180:8080 -p 8543:8443 -v ./src/main/resources/keycloak:/tmp/keycloak:Z \
jboss/keycloak:12.0.4
You can run your application in dev mode that enables live coding using:
$ ./mvnw quarkus:dev
To do calls to application's endpoint while running it in dev mode, execute the following commands:
$ export access_token=$(\
curl -X POST http://localhost:8180/auth/realms/quarkus/protocol/openid-connect/token \
--user backend-service:secret \
-H 'content-type: application/x-www-form-urlencoded' \
-d 'username=alice&password=alice&grant_type=password' | jq --raw-output '.access_token' \
)
$ curl -X GET 'http://localhost:8080/controls/business-service?description=ser&sort=name' \
-H 'Accept: application/json' -H "Authorization: Bearer "$access_token |jq .
To help the development, there's a script that inserts three stakeholders and three business services.
To run it, execute the command:
$ ./src/main/resources/import-curl-locally.sh http://localhost:8080 http://localhost:8180
For creating the Foo
resource, follow these steps:
- add
Foo.java
bean class insrc/main/java/io/tackle/controls/entities/
adapting this template class:When adding fields to the entity, remember to add thepackage io.tackle.controls.entities; import io.tackle.commons.entities.AbstractEntity; import org.hibernate.annotations.ResultCheckStyle; import org.hibernate.annotations.SQLDelete; import org.hibernate.annotations.Where; import javax.persistence.Entity; import javax.persistence.Table; @Entity @Table(name = "foo") @SQLDelete(sql = "UPDATE foo SET deleted = true WHERE id = ?", check = ResultCheckStyle.COUNT) @Where(clause = "deleted = false") public class Foo extends AbstractEntity { <<add fields>> }
@Filterable
annotation (fromio.tackle.commons.annotations
package) to the fields accepted as entity filters in REST endpoint.
For example, if the entity's got adescription
field that we want to be able to filter by, add the annotation in this way:In case the field represents a relational association entity, then add the field@Filterable public String description;
filterName
to the@Filterable
annotation in order to specify, with dot notation, the related entity's field the filter will be applied to.
For example, theBusinessService
entity has an association withStakeholder
entity through theowner
field.
Since it has been decided theStakeholder
's field to filter by is thedisplayName
, theowner
field has been annotated in this way:so that a REST request to filter Business Service entities can use the@Filterable(filterName = "owner.displayName") public Stakeholder owner;
owner.displayName
as query parameter. - add
FooResource.java
resource class insrc/main/java/io/tackle/controls/resources/
adapting this template class:in this way the REST Data Panache extension will be used for providing all endpoints but the "list" one because we need filtering for lists which is not provided (yet?) from that Quarkus extension.package io.tackle.controls.resources; import io.quarkus.hibernate.orm.rest.data.panache.PanacheEntityResource; import io.quarkus.panache.common.Page; import io.quarkus.panache.common.Sort; import io.quarkus.rest.data.panache.MethodProperties; import io.quarkus.rest.data.panache.ResourceProperties; import io.tackle.controls.entities.Foo; import java.util.List; @ResourceProperties(hal = true) public interface FooResource extends PanacheEntityResource<Foo, Long> { @MethodProperties(exposed = false) List<Foo> list(Page page, Sort sort); }
- to add the "list" endpoints (both plain JSON and HAL-JSON), add
FooListFilteredResource.java
class insrc/main/java/io/tackle/controls/resources/
adapting this template class:package io.tackle.controls.resources; import io.tackle.commons.resources.ListFilteredResource; import io.tackle.controls.entities.Foo; import org.jboss.resteasy.links.LinkResource; import javax.ws.rs.DefaultValue; import javax.ws.rs.GET; import javax.ws.rs.Path; import javax.ws.rs.Produces; import javax.ws.rs.QueryParam; import javax.ws.rs.core.Context; import javax.ws.rs.core.Response; import javax.ws.rs.core.UriInfo; import java.util.List; @Path("foo") public class FooListFilteredResource implements ListFilteredResource<Foo> { @Override public Class<Foo> getPanacheEntityType() { return Foo.class; } @GET @Path("") @Produces({"application/json"}) @LinkResource( entityClassName = "io.tackle.controls.entities.Foo", rel = "list" ) public Response list(@QueryParam(QUERY_PARAM_SORT) @DefaultValue(DEFAULT_VALUE_SORT) List var1, @QueryParam(QUERY_PARAM_PAGE) @DefaultValue(DEFAULT_VALUE_PAGE) int var2, @QueryParam(QUERY_PARAM_SIZE) @DefaultValue(DEFAULT_VALUE_SIZE) int var3, @Context UriInfo var4) throws Exception { return ListFilteredResource.super.list(var1, var2, var3, var4, false); } @Path("") @GET @Produces({"application/hal+json"}) public Response listHal(@QueryParam(QUERY_PARAM_SORT) @DefaultValue(DEFAULT_VALUE_SORT) List var1, @QueryParam(QUERY_PARAM_PAGE) @DefaultValue(DEFAULT_VALUE_PAGE) int var2, @QueryParam(QUERY_PARAM_SIZE) @DefaultValue(DEFAULT_VALUE_SIZE) int var3, @Context UriInfo var4) throws Exception { return ListFilteredResource.super.list(var1, var2, var3, var4, true); } }
- start the application in dev mode following Running the application in dev mode
- open a browser to http://localhost:8080/controls/q/swagger-ui/ to trigger code reload
- check the application's log in terminal to retrieve Hibernate output about
Foo
entity's table, keys and indexes creation, something like:Hibernate: create table foo ( id int8 not null, << all the fields declared in Foo bean >> primary key (id) )
- copy all the SQL instructions needed to manage the new
Foo
resource - create a
Vyyyymmdd__create_foo.sql
file (e.g.V20200928__create_business-service.sql
) insrc/main/resources/db/migration/
folder (refer to Flyway paragraph) - paste inside the
Vyyyymmdd__create_foo.sql
file all the previously copied SQL DDL instructions - for tests execution, add test data into the database creating a
Vyyyymmdd__insert_foo.sql
file (e.g.V20201210__insert_business-service.sql
) insrc/main/resources/db/test-data/
folder (refer to Flyway paragraph) - create a new test
FooTest
class intosrc/test/java/io/tackle/controls/resources
folder and add test for the REST endpoints. A "reference" test class and methods isBusinessServiceTest
- create a new test
NativeFooIT
class intosrc/test/java/io/tackle/controls/resources
folder adapting this template class:package io.tackle.controls.resources; import io.quarkus.test.junit.NativeImageTest; @NativeImageTest public class NativeFooIT extends FooTest {}
When an entity's field is a natural key (e.g. the job function's role) that must be unique, then a PostgreSQL partial index can be defined in order to guarantee uniqueness and preserve the soft delete approach.
If the Foo
entity has a name
field that is a natural key and must be unique, then the partial index can be defined in a Flyway script in this way:
create unique INDEX UK<unique_id>
on foo (name)
where (deleted = false);
so that the value of the name
column in the foo
table has the unique constraint only if deleted = false
.
In this way it's guaranteed that a value, previously used in an already soft-deleted entity, can be used again.
- start the application in dev mode following Running the application in dev mode
- change
%dev.quarkus.hibernate-orm.database.generation = update
property - modify resource
- open a browser to http://localhost:8080/controls/swagger-ui/, this will trigger code reload
- create a
Vyyyymmdd__alter_foo.sql
file (e.g.V20210104.1__alter_business-service.sql
) insrc/main/resources/db/migration/
folder (refer to Flyway paragraph) - copy Hibernate SQL
Hibernate: alter table if exists foo add column updateTime timestamp Hibernate: alter table if exists foo add column updateUser varchar(255)
An example of how to rename/remove/add columns copying data from existing into new columns is provided in V20210128__alter_stakeholder.sql
file.
If you want to check the code generated by Quarkus extensions, both when running in dev mode or while executing tests, add the following properties:
-Dquarkus.debug.transformed-classes-dir=target/dump -Dquarkus.debug.generated-classes-dir=target/dump
to dump the generated code into the dump
folder.
More information available in Dump the Generated Classes to the File System guide.
The tests can be run without having to start any external components.
At the same time, there're situations, for example during development, when you run multiple times in a short time the tests and so it would be good to speed up the tests' execution.
If the TACKLE_KEYCLOAK_TEST_URL
environment variable is provided, the tests won't start the Keycloak testcontainer saving its startup time during tests execution.
Keycloak can be started following the instructions above in Keycloak paragraph.
Once Keycloak is started, set the TACKLE_KEYCLOAK_TEST_URL
environment variable executing:
$ export TACKLE_KEYCLOAK_TEST_URL=https://localhost:8543/auth/realms/quarkus
In this way the tests will use this provided Keycloak instance instead of starting a new one.
$ ./mvnw test
💡 In case of
DockerClientException: Could not pull image:
exception when using Podman, just try to re-run the test command.
$ ./mvnw verify -Pnative -Dquarkus-profile=test
where the quarkus-profile=test
property is mandatory to force the build of the native image to use the test
profile, otherwise the default prod
profile would be used.
If you want to just execute (again) the native tests without building again the native image, use the following command:
$ ./mvnw test-compile failsafe:integration-test@tackle-controls-IT -Pnative
Before starting locally the application, start the Required components.
The application can be packaged using the $ ./mvnw package -Dquarkus-profile=local
command.
It produces the quarkus-run.jar
file in the /target/quarkus-app
directory.
Be aware that it’s not an über-jar as the dependencies are copied into the target/quarkus-app/
subfolders.
The application is now runnable using java -jar target/quarkus-app/quarkus-run.jar
.
You can create a native executable using: $ ./mvnw package -Dquarkus-profile=local -Pnative -Dquarkus.native.container-build=true
If you have GraalVM installed locally, you can run the native executable build without the -Dquarkus.native.container-build=true
option.
You can then run the application in native mode executing: $ ./target/controls-*-runner
To get the report of test coverage of the application's code it's a matter of activating the coverage
Maven profile during verify
Maven goal execution like in the following command:
$ ./mvnw verify -Pjacoco
at the end the report will be available opening in a browser the target/site/jacoco-ut/index.html
file.
This is base on using Podman and CRI-O locally. Something very similar can be done using Docker.
$ minikube start --driver=podman --container-runtime=cri-o --feature-gates="LocalStorageCapacityIsolation=false"
Once started you can open its dashboard executing
$ minikube dashboard
First build the container image for the application executing:
$ ./mvnw package -Pcontainer-image # eventually with -Pnative to deploy the native application
If it's the first deployment then create (just once) a dedicated namespace with
$ kubectl create namespace tackle
then you can deploy applying the files generated during the build running the command:
$ kubectl apply -f target/kubernetes/minikube.yml -n tackle
If there are no changes to the resources in Kubernetes and you just need to have the latest image deployed, you can execute
$ kubectl rollout restart deployment controls -n tackle
$ export access_token=$(\
curl -X POST $(minikube service --url=true keycloak -n tackle)/auth/realms/quarkus/protocol/openid-connect/token \
--user backend-service:secret \
-H 'content-type: application/x-www-form-urlencoded' \
-d 'username=alice&password=alice&grant_type=password' | jq --raw-output '.access_token' \
)
$ curl -X GET "$(minikube service --url=true controls -n tackle)/controls/business-service?description=ser&sort=name" \
-H 'Accept: application/json' -H "Authorization: Bearer "$access_token |jq .
Execute the script:
$ ./src/main/resources/import-curl-remotely.sh http://192.169.17.2
To deploy the application you built locally into a Kubernetes instance, you can push your image to a container repository to make it available for deployment into the Kubernetes instance.
First create the container image (JVM or native mode, native in the example) providing the name of the repository you want to later push the image to: in this case I'm going to use quay.io:
$ ./mvnw package -Pnative -Pcontainer-image -Dquarkus.container-image.registry=quay.io
and then push the image to quay.io repository executing:
$ podman push quay.io/<your quay user>/controls:<version>-<type>
where
version
= the version of the current projecttype
= the package type (i.e.jar
ornative
)
TBD
$ ab -n 1000 -c 20 -H 'Accept: application/hal+json' 'http://<host>/controls/business-service?name=service&sort=name&size=1&page=1'
🚨 Backup and Restore instructions are provided for development purposes and NOT for production usage 🚨
This description is meant to support developers and contributors developing tackle-controls. A description how to backup/restore/move an operator-backed installation of Tackle (Application Assessment), consisting of tackle-application-inventory
, tackle-controls
, tackle-pathfinder
and others can be found here
- get the name of the PostgreSQL pod:
$ kubectl get pods -l app.kubernetes.io/name=controls-postgres -n tackle
- retrieve the database's user using pod's name (e.g.
controls-postgres-5b6cc47f66-fw48p
):$ kubectl exec controls-postgres-5b6cc47f66-fw48p -n tackle -- printenv POSTGRES_USER
- dump the database data only (excluding
flyway_schema_history
table) using pod's name and database's user (e.g.controls-postgres-5b6cc47f66-fw48p
andcontrols
):$ kubectl exec controls-postgres-5b6cc47f66-fw48p -n tackle -- /bin/bash -c "pg_dump -a -T flyway_schema_history -U controls controls_db" > $(date +%Y%m%d%H%M%S)_controls_db_data.sql
- get the name of the PostgreSQL pod:
$ kubectl get pods -l app.kubernetes.io/name=controls-postgres -n tackle
- retrieve the database's user using pod's name (e.g.
controls-postgres-5b6cc47f66-fw48p
):$ kubectl exec controls-postgres-5b6cc47f66-fw48p -n tackle -- printenv POSTGRES_USER
- insert the values in the database using pod's name and database's user (e.g.
controls-postgres-5b6cc47f66-fw48p
andcontrols
):$ cat 20210323195709_controls_db_data.sql | kubectl exec -i controls-postgres-5b6cc47f66-fw48p -n tackle -- psql -U controls -d controls_db