Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better test coverage for different database configurations #520

Open
jsmassa opened this issue Apr 26, 2022 · 2 comments
Open

Better test coverage for different database configurations #520

jsmassa opened this issue Apr 26, 2022 · 2 comments
Assignees
Labels
enhancement New feature or request testing

Comments

@jsmassa
Copy link
Collaborator

jsmassa commented Apr 26, 2022

Problem

In many of the tests only the default configuration for datahike is being tested.

Solution

There should be at least additionally proper testing for

  • {:attribute-refs? true}
  • {:index :datahike.index/persistent-set} (or in general non-default indices)

The best case scenario would be to enable testing of all config combinations:

  • :index [ :datahike.index/hitchhiker-tree :datahike.index/persistent-set]}
  • :backend [:mem :file]
  • :keep-history? [true false]
  • :attribute-refs? [true false]
  • :schema-flexibility [:read :write]

Consequences

  • The attribute-refs test folder in datahike.tests should be made obsolete and if at all replaced by a single attribute-refs test namespace testing only specific features

Difficulties

  1. It's a quite tedious task to port all the namespaces, so:
    • port namespaces one by one, step by step
    • start with the most important namespaces that test core functionalities
  2. Databases with {:schema-flexibility :write} yield more datoms than schema-on-read databases, so the entities of produced datoms will differ, plus they need a schema to be added to the database
    • a schema has to be thought of for tests not working with one yet
  3. Databases with {:attribute-refs? true} have even more datoms, they need a schema to work and their produced datoms look different than databases using attribute references directly
    • potentially use query/pull outputs for comparisons instead of raw datom listings
  4. Testing all configurations will potentially take a long time
    • kaocha can be used to configure the tests to run a fast subset of configurations

Tips

  • The attribute-refs test folder contains some ideas on
    • how to port schema-on-read test to schema-on-write tests
    • how to port tests to compare datoms with references, although there should be a better way
@jsmassa jsmassa added the enhancement New feature or request label Apr 26, 2022
@jsmassa
Copy link
Collaborator Author

jsmassa commented Apr 27, 2022

Blocked by #519

@jsmassa
Copy link
Collaborator Author

jsmassa commented May 30, 2022

PR #503 introduces a new way of running tests by setting the default configuration form the tests.edn test configuration file. This can be used to increase the test coverage for any valid configuration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request testing
Projects
Status: No status
Development

No branches or pull requests

1 participant