Server-based parsing for continuous updates #86

jefffohl · 2016-03-15T23:40:29Z

@breznak I am making a pull request for this solution to the continuous updates issue. Adding this functionality required a dramatic rewrite, which includes:

Building a server for the reading of local (and in the future, remote) files.
Implemented socket.io as the primary method for exchanging data between the server and the client. This allows us to push data from the server to the client as new data comes in.

In this branch I also modified some things. Specifically:

The function used to highlight thresholds in the graph. It is now more accurate.
The notification system is improved.
The layout has been improved.

Please let me know your thoughts. I would appreciate your feedback before I merge, since there are such extensive changes.

…o the client.

…ld code in client.

…al files.

…em. Not working quite yet, but this is a good save point.

…and some would not. I am not sure why, but sending all of the lines together as a long string works, whereas sending individual lines to the parser one at a time only seemed to work sometimes.

…e browser. Though, I think the architecture is wrong. Right now, the client is handling the playing - it should be the server.

…e. Client just needs to signal to the server when to play, when to pause.

…e wave, and also improves the threshold highlighting

breznak · 2016-10-14T15:01:34Z

@jefffohl I'm going to test and merge this brange, as it's a lot of work.
I'd like to revive the project, eg. by using it in breznak/openDS-oss#28 and spreading more its usage.

jefffohl added 27 commits January 14, 2016 16:38

Initial server setup.

7cdcaab

Work in progress

6772448

Modified config

46faedd

File paths for server and field validity check in HTML

9d2bcb6

Local files must be relative to root

0c49ba5

Added socket.io library

57a9ad4

Partial work. So far, we have socket.io connected, and sending data t…

4205b0b

…o the client.

Basic functionality in place for server-side parsing. Commented out o…

18c72f9

…ld code in client.

Removed PapaParse dependency from client.

eac3ad9

Working proxy for handling local and remote files.

8d45ed8

Removed buffering on server. Changed no_timestamp.csv file.

b1013da

Working prototype for local and remote files.

8547733

Partial work on giving more direct control to the file stream for loc…

444e021

…al files.

An attempt at building an API for reading CSV files from the filesyst…

a864309

…em. Not working quite yet, but this is a good save point.

Udpdated package.json

4e7128f

This fixes a problem where some files would get parsed by csv-parse, …

62b3b12

…and some would not. I am not sure why, but sending all of the lines together as a long string works, whereas sending individual lines to the parser one at a time only seemed to work sometimes.

At this point we have the ability to read a file, and "play" it in th…

cb1bf6a

…e browser. Though, I think the architecture is wrong. Right now, the client is handling the playing - it should be the server.

New method for playing data. Relies on the server to handle play/paus…

adef486

…e. Client just needs to signal to the server when to play, when to pause.

Got streams worked out, now we have a good working prototype.

780fdd9

Functionality to allow files to be read as they are being written to.

316fb1f

New file used for updating

6dbaa63

This update includes a better update.sh script - which produces a sin…

50a8653

…e wave, and also improves the threshold highlighting

Removed commented out old highlighting code.

49d6df0

Added angular-toastr notifications module

015f196

UI cleanup

ad9219c

Removed generated data from update.csv

846f2b1

Fixed some server side error reporting

e776efa

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Server-based parsing for continuous updates #86

Server-based parsing for continuous updates #86

jefffohl commented Mar 15, 2016

breznak commented Oct 14, 2016

Server-based parsing for continuous updates #86

Are you sure you want to change the base?

Server-based parsing for continuous updates #86

Conversation

jefffohl commented Mar 15, 2016

breznak commented Oct 14, 2016