Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Version 3.0: Sqlite #347

Merged
merged 281 commits into from
Dec 2, 2024
Merged

Version 3.0: Sqlite #347

merged 281 commits into from
Dec 2, 2024

Conversation

SteffenBrinckmann
Copy link
Contributor

@SteffenBrinckmann SteffenBrinckmann commented Oct 7, 2024 •
edited by sourcery-ai bot
Loading

Summary by Sourcery

Migrate the backend from CouchDB to SQLite, introducing a new database schema and command-line tools for data management. Enhance the GUI with new features and themes, and update the build and CI systems to support the new architecture. Improve documentation and testing to ensure a smooth transition and robust functionality.

New Features:

  • Introduce a new SQLite-based backend for data storage, replacing the previous CouchDB implementation.
  • Add functionality to convert existing CouchDB data to the new SQLite format.
  • Implement a new command-line interface for managing the SQLite database, including data import and export.
  • Introduce a new GUI component for managing project groups and configurations.
  • Add support for creating and managing attachments to documents within the SQLite database.

Bug Fixes:

  • Fix issues with image handling in the GUI, ensuring proper display and management of image data.
  • Resolve inconsistencies in the handling of document metadata, ensuring accurate data representation.

Enhancements:

  • Refactor the codebase to improve modularity and maintainability, including the separation of string manipulation functions into a dedicated module.
  • Enhance the GUI with new themes and improved layout management for better user experience.
  • Improve the database schema to support more complex data structures and relationships.

Build:

  • Update the build system to include new dependencies required for the SQLite backend and GUI enhancements.

CI:

  • Update CI workflows to test against multiple Python versions and ensure compatibility with the new SQLite backend.

Deployment:

  • Modify deployment scripts to accommodate the new SQLite backend and updated project structure.

Documentation:

  • Update documentation to reflect changes in the backend architecture and new features introduced in the GUI.

Tests:

  • Add comprehensive tests for the new SQLite backend, including data import/export and conversion from CouchDB.
  • Introduce GUI tests to ensure the new components and features function as expected.

Chores:

  • Remove deprecated code and files related to the old CouchDB implementation.
  • Clean up the project directory structure to align with the new architecture.

@SteffenBrinckmann
Debug installation process
946a0af
@SteffenBrinckmann
Form is somewhat working
2b167cb
@SteffenBrinckmann
Fix HT's install issue
9cee58e
@SteffenBrinckmann
Eliminate all references to couchdb
4623b0f 
debug internal link to data
@SteffenBrinckmann
Add externalID to link to elabFTW
b97aa6a 
lots of work on form and sqlite
@SteffenBrinckmann
First draft translating couchdb to sqlite
49eae12
@SteffenBrinckmann
Finish couchdb translation
e0fa015
@SteffenBrinckmann
One place defines the config file name
0d1d0eb
@SteffenBrinckmann
Sort hierarchy to make hierarchy work
14c6ba9
@SteffenBrinckmann
Cleaning
52c3f16
@SteffenBrinckmann
Allow to run test
beeb264
@SteffenBrinckmann
Try pytest
5944ade
@SteffenBrinckmann
get tests working
4100182
@SteffenBrinckmann
Improve tests to include python 3.10, 3.11, 3.12
b7b881c
@SteffenBrinckmann
Try new tests
c58d6c7
@SteffenBrinckmann
try to find bug
57f9e87
@SteffenBrinckmann
Test
5b8d754
@SteffenBrinckmann
bug
97789a0
@SteffenBrinckmann
Install and test
1cc8dfc
@SteffenBrinckmann
Move content into subfolder
1fa2386
@SteffenBrinckmann
debug 1
f58a393
@SteffenBrinckmann
debug 2
2fc20c3
@SteffenBrinckmann
debug 3
65315fc
@SteffenBrinckmann
debug 5
0fbad80
@SteffenBrinckmann
debug 6
736456e
@SteffenBrinckmann
debug 8
26ce4d0
@SteffenBrinckmann
debug 9
c0c4f1c
@SteffenBrinckmann
debug10
7c45028
@SteffenBrinckmann
debug11
9152681
@SteffenBrinckmann
debug 12
226b7c1
SteffenBrinckmann and others added 14 commits November 25, 2024 10:01
@SteffenBrinckmann
mypy
53699e5
@SteffenBrinckmann
pylint, mypy, spelling
f678d0e
@jmurugan-fzj
- Added a new database abstraction for data hierarchy tool.
1d2a347 
- Moved some of the common models and adapter out of dataverse to pasta_eln/database folder.
@SteffenBrinckmann
Finish project group
d97fe2a
@SteffenBrinckmann
Finish project group
e665028
@jmurugan-fzj
Merge remote-tracking branch 'origin/sb_sqlite' into 353-adapt-datave…
9114d7c 
…rse-module-to-use-sqlite-backend
@SteffenBrinckmann
Finished projectGroups
95a5ac0
@SteffenBrinckmann
Clean menus
51204f6
@jmurugan-fzj
- Fixed pyLint error and failing tests for config_dialog.py.
110c75d
@SteffenBrinckmann
Cleaned up config
1bdd04d
@jmurugan-fzj
Merge pull request #368 from PASTA-ELN/353-adapt-dataverse-module-to-…
8fb0042 
…use-sqlite-backend

feat(dataverse): adapt data-verse module for new sqlite database
@SteffenBrinckmann
Clean up using vulture
678136d
@SteffenBrinckmann
Merge branch 'sb_sqlite' of github.com:PASTA-ELN/pasta-eln into sb_sq…
4cab2e2 
…lite
@SteffenBrinckmann
Clean close button in config
6a14b59
@SteffenBrinckmann SteffenBrinckmann marked this pull request as ready for review November 26, 2024 11:50
jmurugan-fzj and others added 10 commits November 27, 2024 11:59
@jmurugan-fzj
Merge remote-tracking branch 'origin/sb_sqlite' into 367-adapt-datahi…
2e6d24d 
…erarchy-to-use-sqlite-backend
@SteffenBrinckmann
clean window sizes: 1000 or 600
9bb261f
@SteffenBrinckmann
Merge branch 'main' into sb_sqlite
dc98af5
@jmurugan-fzj
- Implemented database abstraction layer for data hierarchy tool for …
3a98f0e 
…doing operations related to the data hierarchy types.

- Moved all the database models out of dataverse to a common location (pasta_eln/database folder). Added respective ORM models (data_hierarchy_definition_orm_model.py & data_hierarchy_definition_model.py) for the data hierarchy tool.
- Added a data_hierarchy_document_adapter.py which helps in back and forth conversion of data hierarchy document.
- Adapted and added tests and fixed the failing ones too.
- Other minor refactorings.
@jmurugan-fzj
- Minor refactoring and corrections for failing tests too.
7f60498
@jmurugan-fzj
- Fixed MyPy and PyLint errors.
3a0be56 
- Added missing tests.
@jmurugan-fzj
Merge remote-tracking branch 'origin/sb_sqlite' into 367-adapt-datahi…
0384090 
…erarchy-to-use-sqlite-backend
@jmurugan-fzj
- Corrected failing tests.
1bc4074
@jmurugan-fzj
- Corrected the message display to adapt to the size of the content b…
2d63fdd 
…een displayed.

- Adapted the tests too.
@jmurugan-fzj
Merge pull request #373 from PASTA-ELN/367-adapt-datahierarchy-to-use…
a4a18b0 
…-sqlite-backend

refactor(data_hierarchy): adapt datahierarchy tool to use sqlite backend
@jmurugan-fzj
Copy link
Contributor

@SteffenBrinckmann Do you need me to review this item?
Could you merge from main branch again? I still see some differences..!

@SteffenBrinckmann
Copy link
Contributor Author

No, I will clean and tidy everything and then yolo the branch into main

@SteffenBrinckmann SteffenBrinckmann merged commit 2af0193 into main Dec 2, 2024
22 checks passed
@SteffenBrinckmann SteffenBrinckmann deleted the sb_sqlite branch December 16, 2024 12:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sourcery-ai sourcery-ai[bot] sourcery-ai[bot] left review comments

Assignees

@SteffenBrinckmann SteffenBrinckmann

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@SteffenBrinckmann @jmurugan-fzj @enmar-alkhafagi