-
-
Notifications
You must be signed in to change notification settings - Fork 47
StreamArray
StreamArray
assumes that an input token stream represents an array of objects and streams out assembled JavaScript objects:
[1, "a", [], {}, true]
// StreamArray will produce an object stream:
{key: 0, value: 1}
{key: 1, value: 'a'}
{key: 2, value: []}
{key: 3, value: {}}
{key: 4, value: true}
As every streamer, it assumes that individual objects can fit in memory, but the whole file, or any other source, should be streamed.
const StreamArray = require('stream-json/streamers/StreamArray');
const fs = require('fs');
const pipeline = fs.createReadStream('sample.json')
.pipe(StreamArray.withParser());
pipeline.on('data', data => console.log(data));
Being based on StreamBase, StreamArray
has no special API.
make()
and streamArray()
are two aliases of the factory function. It takes options described above, and return a new instance of StreamArray
. streamArray()
helps to reduce a boilerplate when creating data processing pipelines:
const {chain} = require('stream-chain');
const {parser} = require('stream-json');
const {streamArray} = require('stream-json/streamers/StreamArray');
const fs = require('fs');
const pipeline = chain([
fs.createReadStream('sample.json'),
parser(),
streamArray()
]);
let objectCounter = 0;
pipeline.on('data', () => ++objectCounter);
pipeline.on('end', console.log(`Found ${objectCounter} objects.`));
Constructor property of make()
(and streamArray()
) is set to StreamArray
. It can be used for indirect creating of filters or metaprogramming if needed.
withParser()
takes one argument:
-
options
is an object described in Parser's options. It is used to initialize both streams (aParser
instance and a stream returned bymake()
).
It returns a stream produced by stream-chain, which wraps the pipeline. The most important utility of withParser()
is that it correctly sets object modes of the returned stream: object mode for the Readable part and text mode for the Writable part.
This static method is created using withParser() utility. It simplifies a case when a stream should be immediately preceded by a parser.
const StreamArray = require('stream-json/streamers/StreamArray');
const fs = require('fs');
const pipeline = fs.createReadStream('sample.json')
.pipe(StreamArray.withParser());
let objectCounter = 0;
pipeline.on('data', () => ++objectCounter);
pipeline.on('end', console.log(`Found ${objectCounter} objects.`));