createWriteStream writes whitespace instead of data

[ronnlixx]

I'm sure I'm doing something incorrectly, I'm new to steams. I'm trying to do some data filtering and write out a new file and I get white space instead of stringified json in updated-sample.json.

  const pipeline = chain([
    fs.createReadStream('./data/sample.json'),
    parser(),
    pick({filter: 'DataSet.Table'}),
    streamArray(),
    data => data.value.EmailAddress && data.value.CustomerStatus === 'Active' ? data.value : null,
    stringer(),
    fs.createWriteStream('./data/updated-sample.json')
  ])    

sample.json

{
	"DataSet": {
		"Table": [{
				"CustomerID": 18,
				"CustomerStatus": "Active",
				"EmailAddress": "eee@123.com"
			},
			{
				"CustomerID": 19,
				"CustomerStatus": "Active",
				"EmailAddress": "ddd@123.com"
			},
			{
				"CustomerID": 20,
				"CustomerStatus": "Terminated",
				"EmailAddress": "ccc@123.com"
			},
			{
				"CustomerID": 21,
				"CustomerStatus": "Active",
				"EmailAddress": "bbb@123.com"
			},
			{
				"CustomerID": 22,
				"CustomerStatus": "Terminated",
				"EmailAddress": "aaa@123.com"
			}
		]
	}
}

[uhop]

Your code fragment looks legit, but the devil is in the details. Let me annotate it with a type of stream produced by every stage of the pipeline:

const pipeline = chain([
  fs.createReadStream('./data/sample.json'), // TEXT
  parser(), // TOKEN
  pick({filter: 'DataSet.Table'}), // TOKEN
  streamArray(), // OBJECT
  data => // OBJECT
    data.value.EmailAddress && data.value.CustomerStatus === 'Active'
      ? data.value
      : null,
  stringer(), // TEXT
  fs.createWriteStream('./data/updated-sample.json')
]);

Now comes the question: what kind of stream is expected by stringer()? It is a trick question because the library provides two stringers:

• stringer() of stream-json/Stringer expects a TOKEN stream. It will ignore all non-tokens.
• stringer() of stream-json/jsonl/Stringer expects an OBJECT stream. It will produce a JSONL stream.

I suspect you used the former one because the pipeline fed it with the OBJECT stream and it produced nothing.

You can either switch to JSONL (just a change in the declaration) or add disssasembler() before stringer(), which will disassemble objects into tokens. Obviously, the JSONL route will be faster.