-
-
Notifications
You must be signed in to change notification settings - Fork 965
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ensure db
is ready to accept connections prior to migration
#1172
Merged
Naramsim
merged 1 commit into
PokeAPI:master
from
AlexandreCarlton:wait-for-postgres-to-be-ready-in-compose
Dec 13, 2024
Merged
Ensure db
is ready to accept connections prior to migration
#1172
Naramsim
merged 1 commit into
PokeAPI:master
from
AlexandreCarlton:wait-for-postgres-to-be-ready-in-compose
Dec 13, 2024
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When we run `make docker-setup`, we occasionally find the following: ``` Traceback (most recent call last): File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection self.connect() File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner return func(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 200, in connect self.connection = self.get_new_connection(conn_params) File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner return func(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/django/db/backends/postgresql/base.py", line 187, in get_new_connection connection = Database.connect(**conn_params) File "/usr/local/lib/python3.10/site-packages/psycopg2/__init__.py", line 122, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync) psycopg2.OperationalError: connection to server at "db" (172.18.0.2), port 5432 failed: Connection refused Is the server running on that host and accepting TCP/IP connections? The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/code/manage.py", line 10, in <module> execute_from_command_line(sys.argv) File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line utility.execute() File "/usr/local/lib/python3.10/site-packages/django/core/management/__init__.py", line 413, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 354, in run_from_argv self.execute(*args, **cmd_options) File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 398, in execute output = self.handle(*args, **options) File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 89, in wrapped res = handle_func(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/django/core/management/commands/migrate.py", line 75, in handle self.check(databases=[database]) File "/usr/local/lib/python3.10/site-packages/django/core/management/base.py", line 419, in check all_issues = checks.run_checks( File "/usr/local/lib/python3.10/site-packages/django/core/checks/registry.py", line 76, in run_checks new_errors = check(app_configs=app_configs, databases=databases) File "/usr/local/lib/python3.10/site-packages/django/core/checks/model_checks.py", line 34, in check_all_models errors.extend(model.check(**kwargs)) File "/usr/local/lib/python3.10/site-packages/django/db/models/base.py", line 1303, in check *cls._check_indexes(databases), File "/usr/local/lib/python3.10/site-packages/django/db/models/base.py", line 1695, in _check_indexes connection.features.supports_covering_indexes or File "/usr/local/lib/python3.10/site-packages/django/utils/functional.py", line 48, in __get__ res = instance.__dict__[self.name] = self.func(instance) File "/usr/local/lib/python3.10/site-packages/django/db/backends/postgresql/features.py", line 92, in is_postgresql_11 return self.connection.pg_version >= 110000 File "/usr/local/lib/python3.10/site-packages/django/utils/functional.py", line 48, in __get__ res = instance.__dict__[self.name] = self.func(instance) File "/usr/local/lib/python3.10/site-packages/django/db/backends/postgresql/base.py", line 329, in pg_version with self.temporary_connection(): File "/usr/local/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 603, in temporary_connection with self.cursor() as cursor: File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner return func(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 259, in cursor return self._cursor() File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 235, in _cursor self.ensure_connection() File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner return func(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 218, in ensure_connection with self.wrap_database_errors: File "/usr/local/lib/python3.10/site-packages/django/db/utils.py", line 90, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 219, in ensure_connection self.connect() File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner return func(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/django/db/backends/base/base.py", line 200, in connect self.connection = self.get_new_connection(conn_params) File "/usr/local/lib/python3.10/site-packages/django/utils/asyncio.py", line 33, in inner return func(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/django/db/backends/postgresql/base.py", line 187, in get_new_connection connection = Database.connect(**conn_params) File "/usr/local/lib/python3.10/site-packages/psycopg2/__init__.py", line 122, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync) django.db.utils.OperationalError: connection to server at "db" (172.18.0.2), port 5432 failed: Connection refused Is the server running on that host and accepting TCP/IP connections? make: *** [Makefile:52: docker-migrate] Error 1 ``` Here we have started `docker compose up -d` then immediately tried to run `docker compose exec -T app python manage.py migrate`, which operates on the `db` container (PostgreSQL). This is rejected as PostgreSQL is not ready to accept connections. To ameliorate this, we: - introduce a healthcheck to the `db` container in `docker-compose.yml` - have `app` wait for `db`'s healthcheck to pass before continuing. This allows `docker-setup` to work reliably.
Thanks so much for the improvement! It's really cool seeing the community helping us even with these CICD tasks! |
This Pull Request won't be deployed since the label |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When we run
make docker-setup
, we occasionally find the following:Here we have started
docker compose up -d
then immediately tried to rundocker compose exec -T app python manage.py migrate
, which operates on thedb
container (PostgreSQL). This is rejected as PostgreSQL is not ready to accept connections.To ameliorate this, we:
db
container indocker-compose.yml
that waits for PostgreSQL to be ready.app
wait fordb
's healthcheck to pass before continuing.This allows
docker-setup
to work reliably.(This should have the
no-deploy
label but I don't seem to be able to edit labels on this Pull Request.)