Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Format column empty results in opaque error message #22

Closed
netsensei opened this issue Aug 5, 2015 · 1 comment
Closed

Format column empty results in opaque error message #22

netsensei opened this issue Aug 5, 2015 · 1 comment

Comments

@netsensei
Copy link
Contributor

Problem:

If you try to import a CSV with a record that has an empty "format" property (null value), the resolver will break and return an opaque "Something went terribly wrong!" error message instead of a verbose message telling me what went wrong and why.

This makes it impossible for a data publisher to debug the CSV import file and correct the offending rows.

I could debug this by logging in via SSH into the server, opening up application.log and finding this error:

[ERROR] 2015-08-04 15:38:31,853 -- Incorrect data format
Traceback (most recent call last):
  File "/home/resolver/resolver/venv/local/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/resolver/resolver/venv/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/resolver/resolver/venv/local/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/resolver/resolver/venv/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/resolver/resolver/venv/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/resolver/resolver/resolver/controllers/user.py", line 18, in inner
    return func(*args, **kwargs)
  File "/home/resolver/resolver/resolver/controllers/csv.py", line 95, in admin_csv_import
    notes=record[6])
  File "<string>", line 4, in __init__
  File "/home/resolver/resolver/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/state.py", line 260, in _initialize_instance
    return manager.original_init(*mixed[1:], **kwargs)
  File "/home/resolver/resolver/resolver/model/data.py", line 27, in __init__
    raise Exception("Incorrect data format")
Exception: Incorrect data format

How to reproduce

  1. Create an import file with 1 entry
  2. Leave the format property of the record empty (no html, json, etc.)
  3. Notice how the import process breaks.

Cause:

The Data class in data.py does raise an exception with a message "Incorrect data format" (line 27), but the error is not reflected in the UI.

The exception also breaks the import process meaning that only a partial import is executed.

Resolution:

  1. Skip/Ignore the offending row and let the importer finish the rest of the import instead of breaking all together
  2. Add rows where required fields are empty or with offending data types to a (downloadable) error log that is presented to the data publisher via the UI after the import ends.
@nvgeele
Copy link
Collaborator

nvgeele commented Aug 5, 2015

I've made the necessary changes in 0fa9de0 to prevent incorrectly formatted data records from fudging up the import.

I'll look into making a better downloadable error log/csv with bad records soon.

@nvgeele nvgeele closed this as completed in a8f6694 Aug 6, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants