Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Add a Provider API Script template #1734

Closed
obulat opened this issue Jun 8, 2021 · 0 comments · Fixed by WordPress/openverse-catalog#93 or WordPress/openverse-catalog#128
Assignees

Comments

@obulat
Copy link
Contributor

obulat commented Jun 8, 2021

Problem Description

Previously, we identified writing the scripts that pull data from previously unknown APIs (Provider API Scripts) as the best entry points for new community contributors.

The Provider API Scripts are required to provide either a license url or a pair of license name and license version. If those were not valid, the ImageStore repository class would silently just not write the data for the image. We did not validate the licenses in the Provider API scripts because we felt that it would add complexity of learning about inner details of license naming for the new contributors.

During the discussion of moving license validation from the ImageStore repository to the individual provider API scripts, we probably found a way to both validate the licenses and clearly show if validation failed, as well as not to bog the new contributors down with details about licenses (there are more than 500 versions!)

Solution Description

We can create a cookiecutter-like template for creating a Provider API script:
Add a folder with a Provider API Script Template, a Provider API Script Test template, and a Provider API Workflow Template
Write a CLI script that fills in the templates with Provider name, and save the files in the appropriate locations.

To make sure that license validation is simplified for the new contributors, we can:

  1. provide clearer documentation about what we can accept as a valid license
  2. import the function that validates license_name and license_version pair into the Provider API Script template so that the user could use it during script writing and testing.
  3. have a clearer indication if the license_url or license_name and license_version pair are invalid so that the user can fix them if possible. Currently, if the license data is not valid, the scripts do not provide very clear feedback, failing quietly.

Alternatives

  • Move license validation to the Provider API scripts, and add more documentation about it.
    This would probably make the extraction and saving of license information easier. However, with a template, we can also get better-quality scripts by providing best practices and good examples within it.

Additional context

This template will also help when we are adding Audio sources in the short term, and other Open licenses in the long term.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
1 participant