csv: implement file drag and drop #385

n-riesco · 2018-02-22T00:03:20Z

Accept and parse data URLs.
Added component <Filedrop>, so that it's possible to drop files into the URL box.
Added setting CSV_STORAGE_SIZE to limit the size of the CSV storage.
Added Datastores.disconnect(connection) to be invoked when a connection's tab is closed.
Implemented disconnect for CSV connectors, so that closing a connection's tab removes the CSV file from storage,
Converted CSV module to CommonJS, so that all the files that use
const CSV = require('./csv.js') share the same CSV storage.
Remove bad connection files at startup.

Closes #383

* Added function `setSize(bytes)` to limit the size of the CSV storage. * Converted CSV module to CommonJS, so that all the files that use `const CSV = require('.csv.js')` share the same CSV storage. * Added spec to test `setSize(bytes)`.

* Added setting CSV_STORAGE_SIZE to limit the size of the CSV storage.

* `connect` accepts and parses data URLs. * Added spec to test connection to a data URL. * Renamed `setSize` to `setStorageSize`. * Removed CSVError.

* Now it's possible to drag files and drop them on the URL box. * Removed packages `data-urls` and `whatwg-encoding`, because they would exhaust memory when parsing data URLs of 20MB. * Improved the generation of a connection label when using data URLs.

* Added method Datastores.disconnect(connection) that gets invoked whenever a connection is removed by deleteConnectionById(id). * Implemented disconnect for CSV connector, so that a disconnection removes the CSV file from the storage. * Added spec for disconnect.

n-riesco · 2018-02-23T09:43:47Z

@shannonlal This PR extends the CSV connector. Now, it's possible to drag files into the URL box. To do so, I've added a new component: <Filedrop>.

Another interesting thing in this PR is that I've extended the Datastores interface. Now, connectors can implement the method disconnect. disconnect is invoked when a connection tab is closed.

Would you like to review this PR when you find the time?

shannonlal

@n-riesco Really good job. Most are my comments are just minor documentation/questions. I think it is good to go. 💃

shannonlal · 2018-02-24T04:09:48Z

app/components/Settings/UserConnections/filedrop.jsx

+    }
+
+    /**
+     * Filedrop is an input component where users can type an URL or drop a file


General Comment. I like the style how you have documented the JSDocs and React class structure. I have noticed that there seems to be some inconsistency with the React Classes. Do we have a standard format? I have noticed some classes have PropTypes at the bottom. Is there a standard? Should we document this somewhere? If so where is a good place?

@shannonlal

I have noticed that there seems to be some inconsistency with the React Classes. Do we have a standard format? I have noticed some classes have PropTypes at the bottom. Is there a standard? Should we document this somewhere? If so where is a good place?

We don't have any guidelines. I'd keep them in a file: CODING_GUIDELINES.md.

I don't want the guidelines to become a burden (it's easier for us to tell contributors to use given files as models; e.g: we could chose one of the files in the project as a model for stateless components, another file for stateful components, and another for components that use redux). Also, by burden, I mean:

For example, the burden of prescribing that PropTypes go at the top is OK, because it's just a copy'n'paste.

But, for example, I don't think the burden of prescribing all React components should be class components is worth it (i.e. it's OK, and some would say better, that stateless components are written as functional components).

shannonlal · 2018-02-24T04:11:27Z

app/components/Settings/UserConnections/filedrop.jsx

+
+    /**
+     * Filedrop is an input component where users can type an URL or drop a file
+     * 


Code looks clean and easy to understand. Do you think this component is worth having a Jest test for or is this overkill?

I've just added the tests

shannonlal · 2018-02-24T04:15:07Z

backend/persistent/datastores/Datastores.js

@@ -70,13 +71,24 @@ export function query(queryObject, connection) {
 }

 /*
- * connect functions attempt to ping the connection and
- * return a promise that is empty
+ * connect attempts to ping the connection and


Is this JSDocs format? Should it be @params {object} connection

I've added a commit to format it as JSDoc

shannonlal · 2018-02-24T04:15:14Z

backend/persistent/datastores/Datastores.js

- * connect functions attempt to ping the connection and
- * return a promise that is empty
+ * connect attempts to ping the connection and
+ * returns a promise that resolves to the connection object


@returns {promise} an empty promise

This depends on the connector and it's likely to change once Falcon becomes smarter about the connections it creates ^{(at the moment, most of the connectors create a new connection for every request, which is OK if the DB driver implements a pool or the request is stateless)}.

I'll reword it, so that it only states that the promise resolves once the connection succeeds.

shannonlal · 2018-02-24T04:17:15Z

backend/persistent/datastores/csv.js

+}
+
+
+/**


It looks like you have two comment blocks for this function

The first block declares the type CSVConnection and the second block declares the function connect. See that CSVConnection is also used in the declarations of disconnect, query, tables and schemas.

n-riesco · 2018-02-26T17:35:21Z

@shannonlal I've added tests for <Filedrop>. I'll wait for you to review them before merging.

shannonlal · 2018-02-27T01:07:53Z

test/app/components/Settings/UserConnections/filedrop.test.jsx

@@ -0,0 +1,135 @@
+jest.unmock('../../../../../app/components/Settings/UserConnections/filedrop.jsx');


Really good job

shannonlal · 2018-02-27T01:09:22Z

@n-riesco Really good job. 💃 . Just curious. Your thoughts about Jest and enzyme as UI testing framework?

n-riesco · 2018-02-27T09:25:36Z

@shannonlal

Just curious. Your thoughts about Jest and enzyme as UI testing framework?

Testing the initial mount is convenient, but:

testing subsequent changes can be rather repetitive (this is the reason why I defined getCurrentInput())
and testing asynchronous changes can be rather hacky (I couldn't find any API to register a callback to be triggered when a component is updated)

n-riesco force-pushed the csv/limit-storage-size branch from 9c24a81 to 0599eb1 Compare February 22, 2018 00:16

n-riesco added 2 commits February 22, 2018 16:01

csv: limit storage size

041b286

* Added function `setSize(bytes)` to limit the size of the CSV storage. * Converted CSV module to CommonJS, so that all the files that use `const CSV = require('.csv.js')` share the same CSV storage. * Added spec to test `setSize(bytes)`.

settings: add CSV_STORAGE_SIZE

1553211

* Added setting CSV_STORAGE_SIZE to limit the size of the CSV storage.

n-riesco force-pushed the csv/limit-storage-size branch from 115b1db to 37b4110 Compare February 22, 2018 16:04

n-riesco changed the title ~~csv: limit storage size~~ csv: data URLS and limit storage size Feb 22, 2018

n-riesco changed the title ~~csv: data URLS and limit storage size~~ csv: data URLs and limit storage size Feb 22, 2018

n-riesco added 3 commits February 22, 2018 16:20

csv: accept data URLs

0722340

* `connect` accepts and parses data URLs. * Added spec to test connection to a data URL. * Renamed `setSize` to `setStorageSize`. * Removed CSVError.

init: remove bad connections at start up

f3df19f

n-riesco force-pushed the csv/limit-storage-size branch from 37b4110 to e7dc726 Compare February 23, 2018 09:04

n-riesco force-pushed the csv/limit-storage-size branch from e7dc726 to 30f4161 Compare February 23, 2018 09:19

n-riesco changed the title ~~csv: data URLs and limit storage size~~ csv: implement file drag and drop Feb 23, 2018

shannonlal reviewed Feb 24, 2018

View reviewed changes

n-riesco added 4 commits February 26, 2018 11:47

datastores: add JSDoc for SQL interface

e83bce4

ConnectButton: fix ESlint warnings

2d27534

eslint: lint also files with .jsx extension

caf6f2f

filedrop: fix eslint errors

1f58fdf

n-riesco mentioned this pull request Feb 26, 2018

Issue 362 options dropdown refactor #387

Merged

shannonlal reviewed Feb 27, 2018

View reviewed changes

n-riesco force-pushed the csv/limit-storage-size branch from efae52c to 918d39c Compare February 27, 2018 08:59

filedrop: add tests

f0effc2

n-riesco force-pushed the csv/limit-storage-size branch from 918d39c to f0effc2 Compare February 27, 2018 09:10

n-riesco merged commit 8d75c91 into plotly:master Feb 27, 2018

n-riesco mentioned this pull request Feb 27, 2018

CSV: let users upload a CSV file #383

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

csv: implement file drag and drop #385

csv: implement file drag and drop #385

n-riesco commented Feb 22, 2018 •

edited

Loading

n-riesco commented Feb 23, 2018

shannonlal left a comment

shannonlal Feb 24, 2018

n-riesco Feb 26, 2018

shannonlal Feb 24, 2018

n-riesco Feb 26, 2018

shannonlal Feb 24, 2018

n-riesco Feb 26, 2018

shannonlal Feb 24, 2018

n-riesco Feb 26, 2018

shannonlal Feb 24, 2018

n-riesco Feb 26, 2018

n-riesco commented Feb 26, 2018

shannonlal Feb 27, 2018

shannonlal commented Feb 27, 2018

n-riesco commented Feb 27, 2018 •

edited

Loading

		@@ -0,0 +1,135 @@
		jest.unmock('../../../../../app/components/Settings/UserConnections/filedrop.jsx');

csv: implement file drag and drop #385

csv: implement file drag and drop #385

Conversation

n-riesco commented Feb 22, 2018 • edited Loading

n-riesco commented Feb 23, 2018

shannonlal left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

n-riesco commented Feb 26, 2018

Choose a reason for hiding this comment

shannonlal commented Feb 27, 2018

n-riesco commented Feb 27, 2018 • edited Loading

n-riesco commented Feb 22, 2018 •

edited

Loading

n-riesco commented Feb 27, 2018 •

edited

Loading