fix(migrations): correct parent_message_id for service-api records

[laipz8200]

Checklist:

Important

Please review the checklist below before submitting your pull request.

• Please open an issue before creating a PR or link to an existing issue
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

Description

• Set default parent_message_id to UUID_NIL when it's null for service-api entries post v0.9.0 release.
• Provide downgrade path to reset parent_message_id back to null in case of rollback.

This PR is a follow-up to #8965. We are updating the parent IDs of all historical messages from service-api to NIL to ensure they can be treated as compatible with older versions of the data.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update, included: Dify Document
• Improvement, including but not limited to code refactoring, performance optimization, and UI/UX improvement
• Dependency upgrade

Testing Instructions

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• Test A
• Test B

[laipz8200]

cc @xuzuodong

[xuzuodong]

LGTM

[kkailaasa]

When trying to update to 9.1-fix1 or 9.2, still facing this issue. Following are the api container logs:

psycopg2.errors.UndefinedColumn: column "parent_message_id" of relation "messages" does not exist

[laipz8200]

Did you deploy using the source code? Have you run flask db upgrade?

[kkailaasa]

Im deployed via docker

[laipz8200]

Im deployed via docker

If you upgraded to any beta version before, you may need to downgrade your database.

[kkailaasa]

Ive always checked out the main branch so don't think Ive upgraded to a beta version.... but even if this is the case, how would I downgrade the database?

[laipz8200]

If you're using Docker for deployment, here's what you need to do:

1. First, switch to the branch that contains the migration file of your database and start Dify.
2. Enter the container where the api is located.
3. Run flask db downgrade until the database version matches the one on the main branch.
4. Shut down the container, switch back to the main branch, and run the latest version again.

This process requires some knowledge and experience. Searching for Alembic and Flask-Migrate might be helpful for you.

Alternatively, you could delete the entire dify directory and redeploy, but keep in mind that this will result in the loss of all your data.

[kkailaasa]

my current dify version is 0.8.3
when I ran flask db downgrade, it took me back to 0.8.2 (verified with hash from hub.docker.com)

I upgraded back to 0.8.3 and consecutively tried upgrading to 0.9.0 and 0.9.1-fix1. Both times, got the following error logs:

db-1          | 2024-10-18 04:02:47.743 UTC [7847] ERROR:  duplicate key value violates unique constraint "message_pkey"
db-1          | 2024-10-18 04:02:47.743 UTC [7847] DETAIL:  Key (id)=(REDACTED_UUID) already exists.
db-1          | 2024-10-18 04:02:47.743 UTC [7847] STATEMENT:  UPDATE messages SET parent_message_id = uuid_nil() WHERE parent_message_id IS NULL

api-1         | ERROR [root] Database migration failed: (psycopg2.errors.UniqueViolation) duplicate key value violates unique constraint "message_pkey"
api-1         | DETAIL:  Key (id)=(REDACTED_UUID) already exists.

api-1         | [SQL: UPDATE messages SET parent_message_id = uuid_nil() WHERE parent_message_id IS NULL]

Another dify instance which I am running had no problems upgrading, that is where we run tests and develop workflows. The current instance where this problem is being faced is running in production and too much of data is on it for us to delete it completely.

Thanks a lot for your continued support, really hope we can find a solution to this

[kkailaasa]

@laipz8200 any suggestions on how to fix this?

[kkailaasa]

@laipz8200 have resolved this issue. The migration scripts were not able to run as there was a duplicate record in the messages table.
After manually removing the duplicate record, have been able to run the migration scripts.
Thank you again for your initial support. From there was able to find the exact issue

[glacierck]

How can I use the API to modify historical conversation messages without this parameter? @AllenWriter all
Please restore the use of 'parent_message_id'!