Skip to content

Commit

Permalink
Fix Postgres database importing/upgrading conflicts
Browse files Browse the repository at this point in the history
We were running into conflicts, because having initialized
the roles (users) and databases, trying to import leads to
errors (role XXX already exists, etc.).

We were previously ignoring the Synapse database (`homeserver`)
when upgrading/importing, because that one gets created by default
whenever the container starts.

For our additional databases, it's a similar situation now.
It's not created by default as soon as Postgres starts with an empty
database, but rather we create it as part of running the playbook.

So we either need to skip those role/database creation statements
while upgrading/importing, or to avoid creating the additional database
and rely on the import for that. I've gone for the former, because
it's already similar to what we were doing and it's simpler
(it lets `setup_postgres.yml` be the same in all scenarios).
  • Loading branch information
spantaleev committed Dec 14, 2020
1 parent 2a502db commit dd797ba
Show file tree
Hide file tree
Showing 4 changed files with 34 additions and 4 deletions.
14 changes: 14 additions & 0 deletions group_vars/matrix_servers
Original file line number Diff line number Diff line change
Expand Up @@ -1100,6 +1100,20 @@ matrix_postgres_additional_databases: |
}] if (matrix_dimension_enabled and matrix_dimension_database_engine == 'postgres' and matrix_dimension_database_hostname == 'matrix-postgres') else [])
}}

matrix_postgres_import_roles_to_ignore: |
{{
[matrix_postgres_connection_username]
+
matrix_postgres_additional_databases|map(attribute='username')
}}

matrix_postgres_import_databases_to_ignore: |
{{
[matrix_postgres_db_name]
+
matrix_postgres_additional_databases|map(attribute='name')
}}

######################################################################
#
# /matrix-postgres
Expand Down
16 changes: 16 additions & 0 deletions roles/matrix-postgres/defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,22 @@ matrix_postgres_container_postgres_bind_port: ""
# password: some_password
matrix_postgres_additional_databases: []

# A list of roles/users to avoid creating when importing (or upgrading) the database.
# If a dump file contains the roles and they've also been created beforehand (see `matrix_postgres_additional_databases`),
# importing would fail.
# We either need to not create them or to ignore the `CREATE ROLE` statements in the dump.
matrix_postgres_import_roles_to_ignore: [matrix_postgres_connection_username]

matrix_postgres_import_roles_ignore_regex: "^CREATE ROLE ({{ matrix_postgres_import_roles_to_ignore|join('|') }});"

# A list of databases to avoid creating when importing (or upgrading) the database.
# If a dump file contains the databases and they've also been created beforehand (see `matrix_postgres_additional_databases`),
# importing would fail.
# We either need to not create them or to ignore the `CREATE DATABASE` statements in the dump.
matrix_postgres_import_databases_to_ignore: [matrix_postgres_db_name]

matrix_postgres_import_databases_ignore_regex: "^CREATE DATABASE ({{ matrix_postgres_import_databases_to_ignore|join('|') }})\\s"

# The number of seconds to wait after starting `matrix-postgres.service`
# and before trying to run queries for creating additional databases/users against it.
#
Expand Down
4 changes: 2 additions & 2 deletions roles/matrix-postgres/tasks/import_postgres.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,8 @@
{{ matrix_postgres_docker_image_latest }}
-c "cat /{{ server_path_postgres_dump|basename }} |
{{ 'gunzip |' if server_path_postgres_dump.endswith('.gz') else '' }}
grep -vE '^CREATE ROLE {{ matrix_postgres_connection_username }}' |
grep -vE '^CREATE DATABASE {{ matrix_postgres_db_name }}' |
grep -vE '{{ matrix_postgres_import_roles_ignore_regex }}' |
grep -vE '{{ matrix_postgres_import_databases_ignore_regex }}' |
psql -v ON_ERROR_STOP=1 -h matrix-postgres"
# This is a hack.
Expand Down
4 changes: 2 additions & 2 deletions roles/matrix-postgres/tasks/upgrade_postgres.yml
Original file line number Diff line number Diff line change
Expand Up @@ -135,8 +135,8 @@
{{ matrix_postgres_docker_image_latest }}
-c "cat /in/{{ postgres_dump_name }} |
{{ 'gunzip |' if postgres_dump_name.endswith('.gz') else '' }}
grep -vE '^CREATE ROLE {{ matrix_postgres_connection_username }}' |
grep -vE '^CREATE DATABASE {{ matrix_postgres_db_name }}' |
grep -vE '{{ matrix_postgres_import_roles_ignore_regex }}' |
grep -vE '{{ matrix_postgres_import_databases_ignore_regex }}' |
psql -v ON_ERROR_STOP=1 -h matrix-postgres"
# This is a hack.
Expand Down

0 comments on commit dd797ba

Please sign in to comment.