Migration 2019-04-28-201506_create_tantivy_index is failing to run

[rhaamo]

Updated to latest git from an old plume and the migration 2019-04-28-201506_create_tantivy_index doesn't run (postgresql).

plume-dashie@fedistodon:~/Plume$ plm migration run
Configuration read from /home/plume-dashie/Plume/.env
Running migration 20190428201506
thread 'main' panicked at 'Failed to run migrations: Db(DatabaseError(__Unknown, "column users.role does not exist"))', src/libcore/result.rs:997:5

this is probably due to the migration using the latest Database Model, but with an old database, because users.role is created in migration 2019-06-18-152700_moderator_role later.

[trinity-1686a]

I can't reproduce but I'm pretty sure I know what the issue is. Could you run the same command with RUST_BACKTRACE=1? I would expect Searcher::add_document to appear probably as the first non std call in the backtrace.

The following is my understanding from your report, but would require a bit more testing to make sure it's actually the issue.

Short story:
#555 introduced arbitrary rust in migration, that is totally broken.

Long story
At compile time, the schema used in rust structures is always the one of latest migration, and it is assumed that migrations will be run before trying to query, which works pretty much always. But in this case, some rust code is run before end of migrations (it is a migration), and so it might query the database (again, it is a migration), but will expect the model to be as it will be after the last migration will be run. This is a big design flaw, and I think the best way to handle it is to remove such migrations for now, and search for better ideas or some other way to do. While that seemed like a good idea at the time, I don't see any clean way of handling this very non rare situation in the future.

[rhaamo]

Here is the full backtrace:

plume-dashie@fedistodon:~/Plume$ RUST_BACKTRACE=1 plm migration run
Configuration read from /home/plume-dashie/Plume/.env
Running migration 20190428201506
thread 'main' panicked at 'Failed to run migrations: Db(DatabaseError(__Unknown, "column users.role does not exist"))', src/libcore/result.rs:997:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:71
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:59
             at src/libstd/panicking.rs:197
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:211
   4: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:474
   5: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:381
   6: rust_begin_unwind
             at src/libstd/panicking.rs:308
   7: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
   8: core::result::unwrap_failed
   9: plm::migration::run
  10: plm::main
  11: std::rt::lang_start::{{closure}}
  12: std::panicking::try::do_call
             at src/libstd/rt.rs:49
             at src/libstd/panicking.rs:293
  13: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:87
  14: std::rt::lang_start_internal
             at src/libstd/panicking.rs:272
             at src/libstd/panic.rs:388
             at src/libstd/rt.rs:48
  15: main
  16: __libc_start_main
  17: _start
plume-dashie@fedistodon:~/Plume$

Yeah I've thought of the same thing, unfortunately there is no magic solution for data migrations mixed with schema ones.

I had issues with data migrations in my schema ones on my projects and either I was able to use the ORM without binding a schema by querying directly the right things, or I ended up moving all the data migrations in separate CLI commands.

[trinity-1686a]

The goal of such migrations was to reduce such separate commands (the exact migration failing for you is strictly equivalent to plm search init, although running it won't mark the migration as run).
The fix will probably be to remove this migration, reintroduce it's command as an installation step, and remove one other, and try to rewrite it in sql