Migrations not working in 24.6.0 -> DuplicateObject('constraint "sentry_externalactor_actor_id_a8478274_fk_sentry_actor_id" for relation "sentry_externalactor" already exists\n')

[thedevale]

Self-Hosted Version

24.6.0

CPU Architecture

x86_64

Docker Version

Docker version 26.1.4, build 5650f9b

Docker Compose Version

Docker Compose version v2.27.1

Steps to Reproduce

Take a normal 23.6.2 version, fully working, and try to upgrade to 24.6.0. When running migrations, the system crashes

Expected Result

System building correctly

Actual Result

DuplicateObject('constraint "sentry_externalactor_actor_id_a8478274_fk_sentry_actor_id" for relation "sentry_externalactor" already exists\n')

`Running migrations for default
Operations to perform:
Apply all migrations: auth, contenttypes, feedback, hybridcloud, nodestore, replays, sentry, ses
Running migrations:
Applying sentry.0551_drop_xactor_actor...Traceback (most recent call last):
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 91, in inner
return func(self, sql, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/db/postgres/base.py", line 85, in execute
return self.cursor.execute(sql)
^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.DuplicateObject: constraint "sentry_externalactor_actor_id_a8478274_fk_sentry_acto

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/.venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 103, in execute
return self.cursor.execute(sql)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 77, in inner
raise_the_exception(self.db, e)
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 75, in inner
return func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 18, in inner
return func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 93, in inner
raise type(e)(f"{e!r}\nSQL: {sql}").with_traceback(e.traceback)
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 91, in inner
return func(self, sql, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/db/postgres/base.py", line 85, in execute
return self.cursor.execute(sql)
^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.DuplicateObject: DuplicateObject('constraint "sentry_externalactor_actor_id_a84782
SQL: ALTER TABLE "sentry_externalactor" ADD CONSTRAINT "sentry_externalactor_actor_id_a8478274_fk

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/.venv/bin/sentry", line 4, in
raise SystemExit(main())
^^^^^^
File "/usr/src/sentry/src/sentry/runner/main.py", line 148, in main
func(**kwargs)
File "/.venv/lib/python3.11/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/runner/decorators.py", line 35, in inner
return ctx.invoke(f, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/runner/commands/upgrade.py", line 148, in upgrade
_upgrade(
File "/usr/src/sentry/src/sentry/runner/commands/upgrade.py", line 64, in _upgrade
dj_call_command(
File "/.venv/lib/python3.11/site-packages/django/core/management/init.py", line 194, in call
return command.execute(*args, **defaults)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/django/core/management/base.py", line 459, in execute
output = self.handle(*args, **options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/django/core/management/base.py", line 107, in wrapper
res = handle_func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/django/core/management/commands/migrate.py", line 356,
post_migrate_state = executor.migrate(
^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/django/db/migrations/executor.py", line 135, in migrat
state = self._migrate_all_forwards(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/django/db/migrations/executor.py", line 167, in migra
state = self.apply_migration(
^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/new_migrations/monkey/executor.py", line 141, in apply_migratio
return super().apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/django/db/migrations/executor.py", line 252, in apply
state = migration.apply(state, schema_editor)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/new_migrations/migrations.py", line 22, in apply
return super().apply(project_state, schema_editor, collect_sql)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/django/db/migrations/migration.py", line 132, in apply
operation.database_forwards(
File "/.venv/lib/python3.11/site-packages/django/db/migrations/operations/fields.py", line 235,
schema_editor.alter_field(from_model, from_field, to_field)
File "/usr/src/sentry/src/sentry/db/postgres/schema.py", line 52, in inner
func(self, *args, **kwargs)
File "/.venv/lib/python3.11/site-packages/django/db/backends/base/schema.py", line 893, in alter
self._alter_field(
File "/.venv/lib/python3.11/site-packages/django/db/backends/postgresql/schema.py", line 288, in
super()._alter_field(
File "/.venv/lib/python3.11/site-packages/django/db/backends/base/schema.py", line 1254, in _alt
self.execute(
File "/.venv/lib/python3.11/site-packages/django_zero_downtime_migrations/backends/postgres/sche
super().execute(statement, params)
File "/.venv/lib/python3.11/site-packages/django/db/backends/postgresql/schema.py", line 48, in
return super().execute(sql, None)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/django/db/backends/base/schema.py", line 202, in execu
cursor.execute(sql, params)
File "/.venv/lib/python3.11/site-packages/sentry_sdk/utils.py", line 1711, in runner
return sentry_patched_function(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/sentry_sdk/integrations/django/init.py", line 629,
result = real_execute(self, sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute
return self._execute_with_wrappers(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_wit
return executor(sql, params, many, context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/.venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute
with self.db.wrap_database_errors:
File "/.venv/lib/python3.11/site-packages/django/db/utils.py", line 91, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/.venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 103, in execute
return self.cursor.execute(sql)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 77, in inner
raise_the_exception(self.db, e)
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 75, in inner
return func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 18, in inner
return func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 93, in inner
raise type(e)(f"{e!r}\nSQL: {sql}").with_traceback(e.traceback)
File "/usr/src/sentry/src/sentry/db/postgres/decorators.py", line 91, in inner
return func(self, sql, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/sentry/src/sentry/db/postgres/base.py", line 85, in execute
return self.cursor.execute(sql)
^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.ProgrammingError: DuplicateObject('constraint "sentry_externalactor_actor_id_a8478
SQL: ALTER TABLE "sentry_externalactor" ADD CONSTRAINT "sentry_externalactor_actor_id_a8478274_fk
Error in install/set-up-and-migrate-database.sh:30.
'$dcr web upgrade --create-kafka-topics' exited with status 1
-> ./install.sh:main:38
--> install/set-up-and-migrate-database.sh:source:30

`

Event ID

No response

[msschl]

Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Usage: sentry upgrade [OPTIONS]
Try 'sentry upgrade --help' for help.

Error: No such option: --create-kafka-topics
Error in install/set-up-and-migrate-database.sh:30.
'$dcr web upgrade --create-kafka-topics' exited with status 2
-> ./install.sh:main:38
--> install/set-up-and-migrate-database.sh:source:30

Cleaning up...

[msschl]

Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
Usage: sentry upgrade [OPTIONS]
Try 'sentry upgrade --help' for help.

Error: No such option: --create-kafka-topics
Error in install/set-up-and-migrate-database.sh:30.
'$dcr web upgrade --create-kafka-topics' exited with status 2
-> ./install.sh:main:38
--> install/set-up-and-migrate-database.sh:source:30

Cleaning up...

I forgot to update the versions in my .env.custom file to 24.6.0

[azaslavsky]

It seems like it is attempting to run the migrations twice. What if you use ./sentry-admin.sh django migrate as a standalone command, then try again?

[xvilo]

I'm in the process of updating an instance that has been updated last in august 2023. I had to run:

ALTER TABLE sentry_externalactor
DROP CONSTRAINT sentry_externalactor_actor_id_a8478274_fk_sentry_actor_id;

[graeme44]

I think I find myself in a similar situation. I followed the hard-stop versions here when upgrading – but didn't upgrade docker or docker-compose at any point during this. I think this then caused some issues when I later did upgrade docker-compose to 2+ and saw that the migrations only ran after this (By then I was already on the latest sentry version) – so think I missed some migration steps.

[hubertdeng123]

We did bump the docker compose version in the last year or so. @thedevale were you able to figure out a solution here? Perhaps it may be helpful to upgrade to an intermediate version of Sentry before going to latest...we may have missed documenting some sort of hard stop in the upgrade process.

[peterdk]

I had similar issues, got a

Usage: sentry upgrade [OPTIONS]
Try 'sentry upgrade --help' for help.

Error: No such option: --create-kafka-topics
Error in install/set-up-and-migrate-database.sh:30.
'$dcr web upgrade --create-kafka-topics' exited with status 2
-> ./install.sh:main:38
--> install/set-up-and-migrate-database.sh:source:30

when running ./install.sh.

Problem was that copying 24.6 dir contents over my current 23.12 dir, did not copy .env file (cp -r ../24.6/* .)

After copying that file over, the install.sh did complete succesfully.