Upgrading from Release to Release Via Docker

[schwos]

Current Behavior

We have been attempting to update our Netbox docker container and have been going release by release. Though we continue to run into issues after importing the data from the previous versions. We have followed the database backup procedures and restore procedure and they run fine, but when we start the container up we get following errors as below:

Operations to perform:
Apply all migrations: admin, auth, circuits, contenttypes, dcim, django_rq, extras, ipam, sessions, social_django, taggit, tenancy, users, virtualization, wireless
Running migrations:
Traceback (most recent call last):
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
psycopg2.errors.DuplicateColumn: column "status" of relation "dcim_location" already exists
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/netbox/netbox/./manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/init.py", line 446, in execute_from_command_line
utility.execute()
File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/init.py", line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/base.py", line 414, in run_from_argv
self.execute(*args, **cmd_options)
File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/base.py", line 460, in execute
output = self.handle(*args, **options)
File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/base.py", line 98, in wrapped
res = handle_func(*args, **kwargs)
File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/commands/migrate.py", line 290, in handle
post_migrate_state = executor.migrate(
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 131, in migrate
state = self._migrate_all_forwards(
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 163, in _migrate_all_forwards
state = self.apply_migration(
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 248, in apply_migration
state = migration.apply(state, schema_editor)
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/migration.py", line 131, in apply
operation.database_forwards(
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/operations/fields.py", line 108, in database_forwards
schema_editor.add_field(
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py", line 641, in add_field
self.execute(sql, params)
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py", line 192, in execute
cursor.execute(sql, params)
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
with self.db.wrap_database_errors:
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column "status" of relation "dcim_location" already exists

Expected Behavior

Expect upon starting the NetBox container after importing the backup data that upgrade completes on the data without error. Hate to go through pgadmin to manually remote tables/columns to get past the errors.

Docker Compose Version

Docker Compose version v2.18.1

Docker Version

Client: Docker Engine - Community
 Version:           24.0.2
 API version:       1.43
 Go version:        go1.20.4
 Git commit:        cb74dfc
 Built:             Thu May 25 21:52:13 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.2
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.4
  Git commit:       659604f
  Built:            Thu May 25 21:52:13 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

The git Revision

git version 2.39.2

The git Status

On branch Main

Release 2.1.0 - Netbox Docker upgrading from Release 2.0.0

Startup Command

docker compose up -d

NetBox Logs

netbox-docker-netbox-1               | ⚙️ Applying database migrations
netbox-docker-netbox-1               | 🧬 loaded config '/etc/netbox/config/configuration.py'
netbox-docker-netbox-1               | 🧬 loaded config '/etc/netbox/config/extra.py'
netbox-docker-netbox-1               | 🧬 loaded config '/etc/netbox/config/logging.py'
netbox-docker-netbox-1               | 🧬 loaded config '/etc/netbox/config/plugins.py'
netbox-docker-netbox-1               | Operations to perform:
netbox-docker-netbox-1               |   Apply all migrations: admin, auth, circuits, contenttypes, dcim, django_rq, extras, ipam, sessions, social_django, taggit, tenancy, users, virtualization, wireless
netbox-docker-netbox-1               | Running migrations:
netbox-docker-postgres-1             | 2024-03-29 11:31:14.712 UTC [34] ERROR:  column "status" of relation "dcim_location" already exists
netbox-docker-postgres-1             | 2024-03-29 11:31:14.712 UTC [34] STATEMENT:  ALTER TABLE "dcim_location" ADD COLUMN "status" varchar(50) DEFAULT 'active' NOT NULL
netbox-docker-netbox-1               | Traceback (most recent call last):
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute
netbox-docker-netbox-1               |     return self.cursor.execute(sql, params)
netbox-docker-netbox-1               | psycopg2.errors.DuplicateColumn: column "status" of relation "dcim_location" already exists
netbox-docker-netbox-1               |
netbox-docker-netbox-1               |
netbox-docker-netbox-1               | The above exception was the direct cause of the following exception:
netbox-docker-netbox-1               |
netbox-docker-netbox-1               | Traceback (most recent call last):
netbox-docker-netbox-1               |   File "/opt/netbox/netbox/./manage.py", line 10, in <module>
netbox-docker-netbox-1               |     execute_from_command_line(sys.argv)
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
netbox-docker-netbox-1               |     utility.execute()
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 440, in execute
netbox-docker-netbox-1               |     self.fetch_command(subcommand).run_from_argv(self.argv)
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/base.py", line 414, in run_from_argv
netbox-docker-netbox-1               |     self.execute(*args, **cmd_options)
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/base.py", line 460, in execute
netbox-docker-netbox-1               |     output = self.handle(*args, **options)
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/base.py", line 98, in wrapped
netbox-docker-netbox-1               |     res = handle_func(*args, **kwargs)
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/core/management/commands/migrate.py", line 290, in handle
netbox-docker-netbox-1               |     post_migrate_state = executor.migrate(
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 131, in migrate
netbox-docker-netbox-1               |     state = self._migrate_all_forwards(
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 163, in _migrate_all_forwards
netbox-docker-netbox-1               |     state = self.apply_migration(
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 248, in apply_migration
netbox-docker-netbox-1               |     state = migration.apply(state, schema_editor)
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/migration.py", line 131, in apply
netbox-docker-netbox-1               |     operation.database_forwards(
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/migrations/operations/fields.py", line 108, in database_forwards
netbox-docker-netbox-1               |     schema_editor.add_field(
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py", line 641, in add_field
netbox-docker-netbox-1               |     self.execute(sql, params)
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/base/schema.py", line 192, in execute
netbox-docker-netbox-1               |     cursor.execute(sql, params)
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 67, in execute
netbox-docker-netbox-1               |     return self._execute_with_wrappers(
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
netbox-docker-netbox-1               |     return executor(sql, params, many, context)
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 84, in _execute
netbox-docker-netbox-1               |     with self.db.wrap_database_errors:
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__
netbox-docker-netbox-1               |     raise dj_exc_value.with_traceback(traceback) from exc_value
netbox-docker-netbox-1               |   File "/opt/netbox/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 89, in _execute
netbox-docker-netbox-1               |     return self.cursor.execute(sql, params)
netbox-docker-netbox-1               | django.db.utils.ProgrammingError: column "status" of relation "dcim_location" already exists
netbox-docker-netbox-1               |
netbox-docker-netbox-1 exited with code 0

Content of docker-compose.override.yml

version: '3.4'
services:
  netbox:
    ports:
      - 10.237.64.250:8000:8080
    restart: unless-stopped
    environment:
      NETBOX_DELETE_LEGACY_DATA: "1"

[schwos]

To add additional information; we were at 3.1.1 release moved to the next release then next but eventually we would run into similar error as about in which it appears the migrate.py upgrade startup script attempts to create tables that already exist causing it to loop until we stop it.

In reviewing the documentation of upgrading each release we read the release notes to include the specialist netbox 3.2 upgrade custom scripts which we executed. We didnt have any ASN date but the site information was corrected accordingly.

We would greatly appreciate any pointers and links to a process we may have overlooked in our hast to get to the latest release. Appreciate your time

[schwos]

There no thoughts, pointers, suggestions?