[Bug] fix file handler row parsing to support single geojson feature #1212
Conversation
Signed-off-by: Shan He <heshan0131@gmail.com>
| @@ -103,6 +103,7 @@ export async function* readBatch(asyncIterator, fileName) { | |||
| for await (const batch of asyncIterator) { | |||
| // Last batch will have this special type and will provide all the root | |||
| // properties of the parsed document. | |||
| // Only json parse will have `FINAL_RESULT` | |||
There was a problem hiding this comment.
We should probably not rely on this as a way of detecting json parsing, in the future some other loader might also emit a FINAL_RESULT batch.
But the check for jsonpath should be robust enough.
| // The streamed object is a ROW JSON-batch | ||
| } else if (batch.jsonpath && batch.jsonpath.length === 1) { | ||
| // The streamed object is a ROW JSON-batch (jsonpath = '$') | ||
| // row objects | ||
| result = batches; |
There was a problem hiding this comment.
It is a bit confusing to call this variable batches, because in my mind we are iterating over the batches and accumulating a data array/table.
I would just call this data (or if more elaborate perhaps streamedData or accumulatedData or table etc).
| @@ -112,8 +113,9 @@ export async function* readBatch(asyncIterator, fileName) { | |||
| if (batch.jsonpath && batch.jsonpath.length > 1) { | |||
There was a problem hiding this comment.
This is indeed a bit too messy. At minimum, loaders.gl should provide a helper function that takes the final batch and the accumulated data array and stich them together, handling all the corner cases. In preparation, I'd start by breaking out the logic here:
rebuildJSONObject(finalBatch, accumulatedData)
#1196
Signed-off-by: Shan He heshan0131@gmail.com