Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix tag normalization and migration not removing duplicate tags #11441

Merged
merged 1 commit into from
Jul 29, 2019

Conversation

Gargron
Copy link
Member

@Gargron Gargron commented Jul 29, 2019

Fix #11428

@Gargron Gargron added the bug Something isn't working label Jul 29, 2019
@mayaeh
Copy link
Contributor

mayaeh commented Jul 29, 2019

The following error occurred in my environment:
$ RAILS_ENV=production bundle exec rake db:migrate
Migrating to AddCaseInsensitiveIndexToTags (20190726175042)
== 20190726175042 AddCaseInsensitiveIndexToTags: migrating ====================
-- execute("CREATE UNIQUE INDEX CONCURRENTLY index_tags_on_name_lower ON tags (lower(name))")
rake aborted!
StandardError: An error has occurred, all later migrations canceled:

PG::UniqueViolation: ERROR:  could not create unique index "index_tags_on_name_lower"
DETAIL:  Key (lower(name::text))=(pawoo人増えたし自己紹介しようぜ) is duplicated.
: CREATE UNIQUE INDEX CONCURRENTLY index_tags_on_name_lower ON tags (lower(name))

However, this may be due to my trial and error...

EDIT:
I'm sorry, this was due to my lack of confirmation. I confirm one more time.

@mayaeh
Copy link
Contributor

mayaeh commented Jul 29, 2019

I applied this PR and ran db:migrate.
The following error occurred in my environment:

$ RAILS_ENV=production bundle exec rake db:migrate
Migrating to AddCaseInsensitiveIndexToTags (20190726175042)
== 20190726175042 AddCaseInsensitiveIndexToTags: migrating ====================
-- execute("UPDATE accounts_tags SET tag_id = 887 WHERE tag_id IN (2368)")
   -> 0.0035s
-- execute("UPDATE statuses_tags SET tag_id = 887 WHERE tag_id IN (2368)")
   -> 0.0078s
-- execute("UPDATE account_tag_stats SET tag_id = 887 WHERE tag_id IN (2368)")
   -> 0.0049s
-- execute("UPDATE featured_tags SET tag_id = 887 WHERE tag_id IN (2368)")
   -> 0.0008s
-- execute("UPDATE accounts_tags SET tag_id = 1814 WHERE tag_id IN (1813)")
   -> 0.0004s
-- execute("UPDATE statuses_tags SET tag_id = 1814 WHERE tag_id IN (1813)")
   -> 0.0032s
-- execute("UPDATE account_tag_stats SET tag_id = 1814 WHERE tag_id IN (1813)")
rake aborted!
StandardError: An error has occurred, all later migrations canceled:

PG::UniqueViolation: ERROR:  duplicate key value violates unique constraint "index_account_tag_stats_on_tag_id"
DETAIL:  Key (tag_id)=(1814) already exists.
: UPDATE account_tag_stats SET tag_id = 1814 WHERE tag_id IN (1813)

1813 and 1814 are as follows:

$ bundle exec rails console

Tag.where("name like 'pawoo人増えたし自己紹介しようぜ%'").select('id','name')
=> #<ActiveRecord::Relation [#<Tag id: 192, name: "pawoo人増えたし自己紹介しようぜ">, #<Tag id: 1687, name: "pawoo人増えたし自己紹介しようぜ">, #<Tag id: 1813, name: "pawoo人増えたし自己紹介しよう">, #<Tag id: 1814, name: "pawoo人増えたし自己紹介しよう">]>

Postscript: I'm sorry, I remembered that I was deleting id: 2368 when I was trial and error.
I ran Tag.find(2368).destroy with rails console.

@Gargron
Copy link
Member Author

Gargron commented Jul 29, 2019

I have updated the migration to omit updating rows where a row with the canonical tag already exists. Can you please try again @mayaeh?

@mayaeh
Copy link
Contributor

mayaeh commented Jul 29, 2019

Thank you! I will try now.

@mayaeh
Copy link
Contributor

mayaeh commented Jul 29, 2019

Errors still occur in my environment.

$ RAILS_ENV=production bundle exec rake db:migrate
Migrating to AddCaseInsensitiveIndexToTags (20190726175042)
== 20190726175042 AddCaseInsensitiveIndexToTags: migrating ====================
-- execute("UPDATE accounts_tags AS t0 SET tag_id = 1813 WHERE tag_id IN (1814) 
AND NOT EXISTS (SELECT t1.tag_id FROM accounts_tags AS t1 WHERE t1.tag_id = 1813
 AND t1.account_id = t0.account_id)")
   -> 0.0013s
-- execute("UPDATE statuses_tags AS t0 SET tag_id = 1813 WHERE tag_id IN (1814) 
AND NOT EXISTS (SELECT t1.tag_id FROM statuses_tags AS t1 WHERE t1.tag_id = 1813 
AND t1.status_id = t0.status_id)")
   -> 0.0011s
-- execute("UPDATE account_tag_stats AS t0 SET tag_id = 1813 WHERE tag_id IN 
(1814)  AND NOT EXISTS (SELECT t1.tag_id FROM account_tag_stats AS t1 WHERE t1.tag_
id = 1813 AND t1.account_id = t0.account_id)")
rake aborted!
StandardError: An error has occurred, all later migrations canceled:

PG::UndefinedColumn: ERROR:  column t1.account_id does not exist
LINE 1: ...ccount_tag_stats AS t1 WHERE t1.tag_id = 1813 AND t1.account...
                                                             ^
: UPDATE account_tag_stats AS t0 SET tag_id = 1813 WHERE tag_id IN (1814)  
AND NOT EXISTS (SELECT t1.tag_id FROM account_tag_stats AS t1 WHERE t1.tag_id = 1813 
AND t1.account_id = t0.account_id)

@Gargron
Copy link
Member Author

Gargron commented Jul 29, 2019

Sorry, I got confused about what account_tag_stats represents. There is need to update it. Fixed

@mayaeh
Copy link
Contributor

mayaeh commented Jul 29, 2019

Thank you! I will try now.

@mayaeh
Copy link
Contributor

mayaeh commented Jul 29, 2019

The following error still occurred.
Is this because I ran db: migrate after #11416 ?

rake aborted!
StandardError: An error has occurred, all later migrations canceled:

PG::DuplicateTable: ERROR:  relation "index_tags_on_name_lower" already exists
: CREATE UNIQUE INDEX CONCURRENTLY index_tags_on_name_lower ON tags (lower(name))

@Gargron
Copy link
Member Author

Gargron commented Jul 29, 2019

Yes, probably. Try to drop that index manually first.

@mayaeh
Copy link
Contributor

mayaeh commented Jul 29, 2019

I ran DROP INDEX index_tags_on_name_lower;.
But I got the following error:

$ RAILS_ENV=production bundle exec rake db:migrate
Migrating to AddCaseInsensitiveIndexToTags (20190726175042)
== 20190726175042 AddCaseInsensitiveIndexToTags: migrating ====================
-- execute("CREATE UNIQUE INDEX CONCURRENTLY index_tags_on_name_lower ON tags (lower(name))")
   -> 0.0542s
-- remove_index(:tags, {:name=>"index_tags_on_name"})
rake aborted!
StandardError: An error has occurred, all later migrations canceled:

PG::UndefinedObject: ERROR:  index "index_tags_on_name" does not exist
: DROP INDEX  "index_tags_on_name"

So I commented out this line (remove_index :tags, name: 'index_tags_on_name') and executed db:migrate.
Then db:migrate succeeded successfully.

Perhaps I think this error only happens to people running db:migrate after 11416

@Gargron Gargron merged commit e136112 into master Jul 29, 2019
@Gargron Gargron deleted the fix-11428 branch July 29, 2019 18:40
hiyuki2578 pushed a commit to ProjectMyosotis/mastodon that referenced this pull request Oct 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

db:migrate fails after #11416
2 participants