Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some post_import messages should not be errors #4234

Merged
merged 8 commits into from
Jul 24, 2024
Merged

Conversation

paul-m
Copy link
Contributor

@paul-m paul-m commented Jul 22, 2024

The problem we're trying to solve:

The user might see a lot of errors during post-processing, for resources which don't have data dictionaries.

It's totally OK for resources to not have data dictionaries associated with them, so it should not be an error. However, it should show up in an import log and the import status dashboard because this still might be unexpected... The user might think the resource should have a data dictionary.

Our solution:

  • Introduces a new exception called ResourceDoesNotHaveDictionary. This is thrown when DictionaryEnforcer can't find a dictionary for the given resource. This allows us to handle this situation differently than a generic Exception.
  • PostImportResourceProcessor now handles the ResourceDoesNotHaveDictionary exception as a special case, so it can tell the user that it couldn't find a dictionary for that resource.
  • Modify tests to reflect these situations and prevent regressions.

QA:

  • Set the DKAN site to use 'Distribution reference' data dictionaries.
  • Import datasets with no associated data dictionaries.
  • Verify that no errors are thrown during the post_import phase, but instead an informative message is displayed.

Helpful script:

ddev dkan-site-install
ddev drush pm-enable sample_content
ddev drush dkan:sample-content:create
ddev drush config:set metastore.settings data_dictionary_mode reference
ddev drush cron
ddev drush cron

You should see messages like this:

 [notice] No data-dictionary found for resource with id "0310f6c2d28bbee5de1211e5b5bb0b1c" and version "1721843317".

You should then also see a message like this in the dataset import status dashboard:

done Resource 0310f6c2d28bbee5de1211e5b5bb0b1c does not have a data dictionary.

@paul-m paul-m marked this pull request as ready for review July 24, 2024 17:56
Copy link
Member

@dafeder dafeder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A lot of these tests were long overdue, thanks @paul-m!

@dafeder dafeder merged commit 8b3afec into 2.x Jul 24, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants