feat: database migrations #3457
Conversation
stuartwdouglas
force-pushed
the
stuartwdouglas/migrations
branch
2 times, most recently
from
3f5877b
to
ec9f01d
Compare
stuartwdouglas
force-pushed
the
stuartwdouglas/migrations
branch
9 times, most recently
from
d58d4a6
to
7aa111b
Compare
stuartwdouglas
requested review from
wesbillman
and removed request for
a team
stuartwdouglas
force-pushed
the
stuartwdouglas/migrations
branch
from
42eafe8
to
a3485a2
Compare
stuartwdouglas
requested review from
worstell
and removed request for
a team
stuartwdouglas
force-pushed
the
stuartwdouglas/migrations
branch
from
a3485a2
to
9434b31
Compare
stuartwdouglas
force-pushed
the
stuartwdouglas/migrations
branch
2 times, most recently
from
e88667f
to
d4f0282
Compare
| @@ -41,6 +42,12 @@ message MysqlResource { | |||
| MysqlResourceOutput output = 1; | |||
| } | |||
|
|
|||
| message MigrationResource { | |||
There was a problem hiding this comment.
Please prefix all these types/functions with "SQL", "migrations" is not specific enough.
There was a problem hiding this comment.
Just to clarify why I think this is important - it's very likely there will be other kinds of migrations in the future.
| Schema *schema.Schema | ||
| Dependencies []string | ||
| BuildEnv []string | ||
| AdditionalFiles []string |
There was a problem hiding this comment.
I'm 95% sure this should just be part of the ModuleConfig .proto rather than here (cc @matt2e) - this is exactly what it's designed for. Each language plugin then just returns the directory for any migrations, and the build engine does the bundling rather than each runtime having to have custom logic.
There was a problem hiding this comment.
I wasn't sure if we wanted the build engine to modify the schema after the language plugin was finished with it. I can definitely change it but I think it does kinda blur the lines about what is responsible for generating the schema.
I figured language plugins could just re-use the ExtractMigrations function and achieve re-use that way.
There was a problem hiding this comment.
It's not a requirement that language plugins be in Go, and in general we try to push as much as possible out of the plugins and into the engine for this reason.
The pattern is that the ModuleConfig tells the BuildEngine where to look for different parts of the build, and the BuildEngine does the work of collecting it into the schema+build. This fits that fairly well. It could be a glob, or a regex, or a map of db names to paths or something.
| targetDir := t.TempDir() | ||
|
|
||
| // Create dummy migration files | ||
| migrationSubDir := filepath.Join(migrationsDir, "testdb") |
There was a problem hiding this comment.
Didn't want to use testdata for the fixtures?
There was a problem hiding this comment.
I thought that was just for integration tests? Also it's possible Chad may have written this test...
| @@ -729,6 +731,7 @@ module todo { | |||
| secret secretValue String | |||
| // A database | |||
| database postgres testdb | |||
| +migration sha256:8cc04c75ab7967eb2ec82e11e886831e00b7cb00507e9a8ecf400bdc599eccfd | |||
| switch md := md.(type) { | ||
| case *MetadataMigration: | ||
| if found { | ||
| merr = append(merr, fmt.Errorf("database %q has multiple migration metadata", n.Name)) |
stuartwdouglas
force-pushed
the
stuartwdouglas/migrations
branch
3 times, most recently
from
8fb80c3
to
e03fdd8
Compare
stuartwdouglas
force-pushed
the
stuartwdouglas/migrations
branch
from
e03fdd8
to
c0f8696
Compare
|
I think I have addressed everything, except for the questions around if this should be handled by the build engine or language plugins. |
No description provided.