Feature/data uploads

[tsubik]

Bulk upload with keeping the history of uploads

[tsubik]

When we have import errors nothing is imported until all errors are resolved. Not sure if that is good behavior @simaob

[simaob]

When we have import errors nothing is imported until all errors are resolved. Not sure if that is good behavior @simaob

In Climate Watch we were inserting the data that got in, and showing the errors, maybe that's a better behaviour? But we'd need to ensure that no duplicates would go in.

Btw, I tried to import a non utf-8 file (by mistake actually) and the app crashed, but I just noticed that the PR is still WIP so I can wait until it's ready!

[tsubik]

Well, I'm not sure what could be better, if forcing the user to submit only valid ones wouldn't be better. Maybe we should ask. I will take a look at that UTF-8 hell stuff.

[tsubik]

It's already taking some time to import a couple of hundreds of rows with some associations. Maybe we would need a background job right away.

[simaob]

OK, maybe we won't be able to escape it.

[kowal]

PR looks great so far 👍

I have one comment regarding naming/architecture. Right now DataUpload calls uploader service - Upload::Litigations - I think that this service doesn't need to know about upload context -it is more like CsvImportService, which could be used both from DataUpload and from rake tasks import jobs. In other words, could we extract a service that works on CSV files (local or S3) and can be used from both user uploads and command line layers.

[tsubik]

True, I need to do that kind of refactoring here.

[tsubik]

Any refactoring to the data upload could be done in a separate PR I guess. This is the first version.