entrypoint.sh container script improvements

[jmontleon]

No description provided.

[jwmatthews]

Looks sane to me, but need to defer to @fabianvf for details

[jmontleon]

I've made some additional, changes to the entrypoint script so this now:

• adds support for using hub loader
• causes the kai container to wait for the postgres container to be ready so there is no pgsql error on start. Docker compose used to do this with the depends_on map. You could wait for start, health check to pass, etc. Apparently this is no longer the case. They have a --wait option so you can run docker compose up --wait, but podman compose up complains if I add this option.
• changes the load-data check a bit; it is a little uglier but prevents another possibly confusing message being printed.
• adds a message and a brief (5 second) pause to make users aware it will be a few minutes before the server starts the while data loads the first time, plus another message to let them know it is done.

[jmontleon]

Converted to a draft while I rework things for the hub importer

[jmontleon]

This will no longer try to run the hub importer if USE_HUB_IMPORTER is True.

I will follow on this PR with a compose change after investigating options, to run an additional hub importer container.

[jmontleon]

The hub importer will now be started in a separate pod if the use_hub_importer profile is used. The docs have been updated to explain how to do this.

@jwmatthews I have integrated your changes from #229

@JonahSussman @fabianvf can you help me understand if I am doing something wrong or we have a new issue? The importer is erroring on me:

[kai_hub_importer] | INFO - 2024-07-05 17:07:35,122 - [             psql.py:55   -             __init__()] - Connected to the PostgreSQL server.
[kai_hub_importer] | Traceback (most recent call last):
[kai_hub_importer] |   File "/kai/./kai/hub_importer.py", line 397, in <module>
[kai_hub_importer] |     main()
[kai_hub_importer] |   File "/kai/./kai/hub_importer.py", line 172, in main
[kai_hub_importer] |     poll_api(
[kai_hub_importer] |   File "/kai/./kai/hub_importer.py", line 205, in poll_api
[kai_hub_importer] |     new_last_analysis = import_from_api(
[kai_hub_importer] |                         ^^^^^^^^^^^^^^^^
[kai_hub_importer] |   File "/kai/./kai/hub_importer.py", line 231, in import_from_api
[kai_hub_importer] |     validated_analyses = [Analysis(**item) for item in analyses]
[kai_hub_importer] |                           ^^^^^^^^^^^^^^^^
[kai_hub_importer] |   File "/opt/app-root/lib64/python3.12/site-packages/pydantic/main.py", line 176, in __init__
[kai_hub_importer] |     self.__pydantic_validator__.validate_python(data, self_instance=self)
[kai_hub_importer] | pydantic_core._pydantic_core.ValidationError: 1 validation error for Analysis
[kai_hub_importer] | application
[kai_hub_importer] |   Field required [type=missing, input_value={'id': 1, 'createUser': '...66a5', 'archived': True}, input_type=dict]
[kai_hub_importer] |     For further information visit https://errors.pydantic.dev/2.7/v/missing

[jwmatthews]

@jmontleon I tested on MacOS and I needed to make 1 tweak.

$ podman compose up
>>>> Executing external compose provider "/usr/local/bin/docker-compose". Please refer to the documentation for details. <<<<

ERROR: Version in "./compose.yaml" is invalid - it should be a string.
Error: executing /usr/local/bin/docker-compose up: exit status 1

$ git diff
diff --git a/compose.yaml b/compose.yaml
index 2a4d1b7..eb13058 100644
--- a/compose.yaml
+++ b/compose.yaml
@@ -1,4 +1,4 @@
-version: 3
+version: "3"
 
 x-common-variables: &kai-variables
   DEMO_MODE: "False"

With above tweak confirmed working with seeding data from local (i.e. non hub):
https://gist.github.com/jwmatthews/45fe426f72e9312df05539cd5c8538aa

[jmontleon]

@jmontleon I tested on MacOS and I needed to make 1 tweak.

$ podman compose up
>>>> Executing external compose provider "/usr/local/bin/docker-compose". Please refer to the documentation for details. <<<<

ERROR: Version in "./compose.yaml" is invalid - it should be a string.
Error: executing /usr/local/bin/docker-compose up: exit status 1

$ git diff
diff --git a/compose.yaml b/compose.yaml
index 2a4d1b7..eb13058 100644
--- a/compose.yaml
+++ b/compose.yaml
@@ -1,4 +1,4 @@
-version: 3
+version: "3"
 
 x-common-variables: &kai-variables
   DEMO_MODE: "False"

With above tweak confirmed working with seeding data from local (i.e. non hub): https://gist.github.com/jwmatthews/45fe426f72e9312df05539cd5c8538aa

Thanks, updated!

[jmontleon]

The problems with hub seems to be that the application ref was accidentally dropped from the analyses api response during some cleanup. That is going to be fixed in the hub. As far as I am able to tell this PR is working as intended.