Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE]: Versioning OWL import metadata db for upgradability #203

Closed
mnamici opened this issue Jun 15, 2022 · 2 comments
Closed

[FEATURE]: Versioning OWL import metadata db for upgradability #203

mnamici opened this issue Jun 15, 2022 · 2 comments
Labels
enhancement New feature or request owl-import
Milestone

Comments

@mnamici
Copy link
Collaborator

mnamici commented Jun 15, 2022

Description

Design a versioning scheme for the ontology importer database, and provide an upgrade path from older versions.

Motivation

In future releases in may be the case that we change the schema in the ontology importer database. Since there is no way to restore it, and the user has no idea (and should not have) of this db existence and it's purpose, we would probably end up with bug reports related to the import process not working anymore.

Example Use Cases

The first thing that I can think of is to implement something like what is done for android or ios orm frameworks, probably a table with the latest db revision, and a set of SQL commands that perform the upgrade from one version to the next incrementally.

We could even consider introducing something like sqlalchemy if it helps, or write one with QtSql, or even ditch the orm entirely and use a different serialization format (but the upgrade problem still remains).

@mnamici mnamici added the enhancement New feature or request label Jun 15, 2022
@mnamici mnamici added this to the Eddy 3.4 milestone Jun 15, 2022
@mnamici
Copy link
Collaborator Author

mnamici commented Jul 15, 2022

Ok after giving it some thought I think for our purpose it is enough to implement a simple mechanism like the following:

  1. We assign a natural number to each schema refinement [1,+inf) and increment it at every change. We can store the current version using sqlite3 specific user_version pragma, designed specifically for this purpose (so no extra table needed).
  2. We store the version the plugin expects in a db-version property in the plugin spec;
  3. We manage in the plugin SQL migration statements between each incremental version;
  4. At startup time we query the user_version pragma value and for any integer between user_version and db-version, we run the corresponding migration script;
  5. Finally, we store db-version into user-version.

EDIT: I forgot to mention that this should already be implemented for v3.3 as we only need to set the initial user_version value for this release, a one-liner during schema creation. For later releases we will deal with the migration infrastructure.

mnamici added a commit to mnamici/eddy that referenced this issue Jul 20, 2022
Each db revision is simply an incremental number, which is
stored in the sqlite pragma `user_version`. The current version
of the plugin database is stored in the spec file.

Later releases will have to include the migration infrastructure,
as a set of SQL commands to change the schema and move data
from one version to the immediate successor.

See issue obdasystems#203 for reference.
mnamici added a commit to mnamici/eddy that referenced this issue Jul 20, 2022
Each db revision is simply an incremental number, which is
stored in the sqlite pragma `user_version`. The current version
of the plugin database is stored in the spec file.

Later releases will have to include the migration infrastructure,
as a set of SQL commands to change the schema and move data
from one version to the immediate successor.

See issue obdasystems#203 for reference.
@mnamici
Copy link
Collaborator Author

mnamici commented Dec 29, 2022

I'm closing this as we got a first implementation of the version upgrade logic merged in #247 for v3.4.

It is currently very basic, providing only support for executing arbitrary SQL queries over the database but we can always come back to this and implement a more refined abstraction in the future if needed.

@mnamici mnamici closed this as completed Dec 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request owl-import
Projects
None yet
Development

No branches or pull requests

2 participants