Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make integration tests independent of sample data #747

Open
1 task
dhruvkb opened this issue Aug 5, 2021 · 4 comments
Open
1 task

Make integration tests independent of sample data #747

dhruvkb opened this issue Aug 5, 2021 · 4 comments
Labels
🤖 aspect: dx Concerns developers' experience with the codebase 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🟩 priority: low Low priority and doesn't need to be rushed 🧱 stack: api Related to the Django API 🏁 status: ready for work Ready for work

Comments

@dhruvkb
Copy link
Member

dhruvkb commented Aug 5, 2021 •
edited
Loading

Problem

The API integration tests used by the CI workflow use the sample data. This makes them tightly coupled and causes problems when updating the sample data to match changes in models.

Description

The integration tests should use QA indices search-qa-images and search-qa-audio that are created programmatically.

Alternatives

  • The test cases need to be updated every time there is a change in the sample data.
  • The tests should do some computation on the sample data first and then adjust the requests accordingly.

Implementation

  • 🙋 I would be interested in implementing this feature.
@dhruvkb dhruvkb added 🟩 priority: low Low priority and doesn't need to be rushed 🚦 status: awaiting triage Has not been triaged & therefore, not ready for work ✨ goal: improvement Improvement to an existing user-facing feature 💻 aspect: code Concerns the software code in the repository 🏁 status: ready for work Ready for work 🟨 priority: medium Not blocking but should be addressed soon 🤖 aspect: dx Concerns developers' experience with the codebase and removed 💻 aspect: code Concerns the software code in the repository 🚦 status: awaiting triage Has not been triaged & therefore, not ready for work 🟩 priority: low Low priority and doesn't need to be rushed labels Aug 5, 2021
@dhruvkb dhruvkb mentioned this issue Aug 5, 2021
Merged
@dhruvkb
Copy link
Member Author

dhruvkb commented Aug 31, 2021

Can be related with WordPress/openverse-api#181 to make the examples work as test cases.

@dhruvkb dhruvkb mentioned this issue Sep 1, 2021
Merged
7 tasks
This was referenced Oct 11, 2021
Closed
Closed
@zackkrida zackkrida changed the title [Feature] Make integration tests independent of sample data Make integration tests independent of sample data Oct 25, 2021
@zackkrida zackkrida added 🟧 priority: high Stalls work on the project or its dependents and removed 🟨 priority: medium Not blocking but should be addressed soon labels Oct 25, 2021
@sarayourfriend
Copy link
Collaborator

@dhruvkb Is #725 a duplicate of this issue?

@zackkrida is this still high priority?

@zackkrida zackkrida added 🟩 priority: low Low priority and doesn't need to be rushed and removed 🟧 priority: high Stalls work on the project or its dependents labels May 3, 2022
@zackkrida
Copy link
Member

Priority has been downgraded to low.

@obulat obulat transferred this issue from WordPress/openverse-api Feb 22, 2023
@obulat obulat added 🧰 goal: internal improvement Improvement that benefits maintainers, not users and removed ✨ goal: improvement Improvement to an existing user-facing feature labels Feb 22, 2023
@github-project-automation github-project-automation bot moved this to 📋 Backlog in Openverse Backlog Feb 23, 2023
@obulat obulat added 🧱 stack: api Related to the Django API and removed 🧱 stack: backend labels Mar 20, 2023
dhruvkb pushed a commit that referenced this issue Apr 14, 2023
@AetherUnbound @sarayourfriend
Refactor Phylopic to use ProviderDataIngester (#747)
1506d6d 
* Add phylopic to provider module

* Initial phylopic refactor

* Finish provider refactor

* Only use endpoint conditionally, add should_continue & fix params

* Correct get_response_json logic

* Initial test changes

* Update tests

* Add provider data ingester to list

* Improve documentation

Co-authored-by: sarayourfriend <24264157+sarayourfriend@users.noreply.github.com>

* Use computed endpoint rather than get_response_json override

* Rename _uuid to uid

* Remove scratch code

Co-authored-by: sarayourfriend <24264157+sarayourfriend@users.noreply.github.com>

* Remove _get_meta_data function

* Make Phylopic daily

* Move start date to first day data is actually made available

Co-authored-by: sarayourfriend <24264157+sarayourfriend@users.noreply.github.com>
@obulat
Copy link
Contributor

obulat commented Mar 19, 2024

While working on #3931, I had to update the sample data to include the new audio provider with a different shape of the alt_files (some fields, such as bit_rate, were absent). I replaced the last 10 items, and the integration tests that, as it turned out, rely on them started failing. Updating sample data CSV is not easy, and it would be nice to create the public data programmatically. Provided that we know all of the data inconsistencies or have normalized all of them already.

I wonder if something like Schemathesis exists for synthetic data creation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
🤖 aspect: dx Concerns developers' experience with the codebase 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🟩 priority: low Low priority and doesn't need to be rushed 🧱 stack: api Related to the Django API 🏁 status: ready for work Ready for work
Projects
Openverse Backlog
Status: 📋 Backlog
Milestone
No milestone
Development

No branches or pull requests
4 participants
@zackkrida @obulat @dhruvkb @sarayourfriend