Summary of the results and exported output files are not generated

[BalanagarajanArumugam]

Hi Team,
After execution of the test suite, the summary of the test results are not generated.We got the following error message.Is it an issue from our side(the command which we used for execution)?

Result:
C:\Users\x087528\AppData\Roaming\npm\node_modules\newman\lib\reporters\json\index.js:13
content: JSON.stringify(_.omit(o.summary, 'exports'), 0, 2)
^
RangeError: Invalid string length
at join (native)
at Object.stringify (native)
at EventEmitter. (C:\Users\x087528\AppData\Roaming\npm\node_modules\newman\lib\reporters\json\index.js:13:27)
at EventEmitter.emit (C:\Users\x087528\AppData\Roaming\npm\node_modules\newman\node_modules\eventemitter3\index.js:138:33)
at .extend.done (C:\Users\x087528\AppData\Roaming\npm\node_modules\newman\lib\run\index.js:152:29)
at C:\Users\x087528\AppData\Roaming\npm\node_modules\newman\node_modules\postman-runtime\lib\backpack\index.js:56:34
at PostmanCollectionRun..extend._process (C:\Users\x087528\AppData\Roaming\npm\node_modules\newman\node_modules\postman-runtime\lib\runner\run.js:107:20)
at PostmanCollectionRun. (C:\Users\x087528\AppData\Roaming\npm\node_modules\newman\node_modules\postman-runtime\lib\runner\run.js:118:18)
at tryOnTimeout (timers.js:228:11)
at Timer.listOnTimeout (timers.js:202:5)

[kunagpal]

@BalanagarajanArumugam: Could you provide the command used please?

[BalanagarajanArumugam]

In bamboo, we configured the executable command as below:

-c "node/node lib/node_modules/newman/bin/newman run src/test/resources/collection.json --environment src/test/resources/env.json --reporters html,json,junit --reporter-json-export src/test/resources/output.json --reporter-junit-export src/test/resources/output.xml --reporter-html-export src/test/resources/output.html --timeout-request 30000"

[kunagpal]

@BalanagarajanArumugam We're looking into this, thanks for letting us know.

[shamasis]

@BalanagarajanArumugam Is it possible that you are running out of memory? Since a very large object will cause this type of error.

[shamasis]

If it is a memory issue then the following could be solutiuon:

1. increase max memory of node
2. we build a new "streaming" verbose reporter
3. we reduce the verbosity of the summary object (and make an option to increase it)

[kunagpal]

@BalanagarajanArumugam Apologies for the delay. Newman v3.1.1 has been released with memory leak fixes, please try it out and get back to us if your issues persist.

[BalanagarajanArumugam]

@kunagpal : Will check and update you.

[kunagpal]

@BalanagarajanArumugam Cool 👍

[BalanagarajanArumugam]

@kunagpal : It's working fine..

[mrtibs2000]

Still running into the same problem on version 3.9.3.

[Ramya-From-Postman]

@kunagpal, Below is the error that user is experiencing on the version 3.9.3 for your reference

./runs/certification_domestic_stc.postman_collection.json.html -k 
/usr/local/lib/node_modules/newman/lib/reporters/json/index.js:19 
content: JSON.stringify(_.omit(o.summary, 'exports'), 0, 2) 
^

RangeError: Invalid string length 
at join (native) 
at Object.stringify (native) 
at EventEmitter.<anonymous> (/usr/local/lib/node_modules/newman/lib/reporters/json/index.js:19:27) 
at EventEmitter.emit (/usr/local/lib/node_modules/newman/node_modules/eventemitter3/index.js:203:33) 
at _.assignIn.done (/usr/local/lib/node_modules/newman/lib/run/index.js:248:29) 
at /usr/local/lib/node_modules/newman/node_modules/postman-runtime/lib/backpack/index.js:56:34 
at PostmanCollectionRun._.assign._process (/usr/local/lib/node_modules/newman/node_modules/postman-runtime/lib/runner/run.js:161:13) 
at PostmanCollectionRun.<anonymous> (/usr/local/lib/node_modules/newman/node_modules/postman-runtime/lib/runner/run.js:166:76) 
at Timer.listOnTimeout (timers.js:92:15)

[kunagpal]

@mrtibs2000 @Ramyasri1212 This is happening because the resultant stringified summary is larger than the largest possible string in JavaScript, which is just under 2^28 for Node v6, and just under 2^30 for Node v8 and Node v9. Could you try updating to the latest v8.x release of Node (which is 8.11.3 at the time of writing) and check if the issue still persists?

Another alternative is to try the following reporter: https://www.npmjs.com/package/newman-reporter-json-light, which trims down JSON reports to their essentials.

[mrtibs2000]

I'm now getting a "FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory". Any recommendation there?

[kunagpal]

@mrtibs2000 Could you share the command/script used to run the collection?

[mrtibs2000]

newman run ./certification_international.postman_collection.json -g ./globals.postman_globals.json -r json --reporter-json-export ./runs/20180827031330/certification_international.postman_collection.json.html -k

Full dump:

<--- Last few GCs --->

[32726:0x34217f0]   192704 ms: Mark-sweep 749.9 (835.3) -> 749.5 (838.3) MB, 62.5 / 0.0 ms  allocation failure GC in old space requested
[32726:0x34217f0]   192770 ms: Mark-sweep 749.5 (838.3) -> 749.5 (801.8) MB, 66.0 / 0.0 ms  last resort GC in old space requested
[32726:0x34217f0]   192839 ms: Mark-sweep 749.5 (801.8) -> 749.5 (801.8) MB, 68.5 / 0.0 ms  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x4db76c25879 <JSObject>
    1: fromString(aka fromString) [buffer.js:314] [bytecode=0x243b62d620c9 offset=164](this=0x25ffb5d022d1 <undefined>,string=0x1e3961543361 <Very long string[359037679]>,encoding=0x4db76c34ba1 <String[4]: utf8>)
    2: from [buffer.js:177] [bytecode=0x243b62d61bd1 offset=11](this=0xafef6d36481 <JSFunction Buffer (sfi = 0x4db76c7b7e9)>,value=0x1e3961543361 <Very long string[359037679]>,encodingOr...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0x8c21ec [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [node]
 6: v8::internal::String::SlowFlatten(v8::internal::Handle<v8::internal::ConsString>, v8::internal::PretenureFlag) [node]
 7: v8::internal::String::Flatten(v8::internal::Handle<v8::internal::String>, v8::internal::PretenureFlag) [node]
 8: v8::String::WriteUtf8(char*, int, int*, int) const [node]
 9: node::StringBytes::Write(v8::Isolate*, char*, unsigned long, v8::Local<v8::Value>, node::encoding, int*) [node]
10: node::Buffer::New(v8::Isolate*, v8::Local<v8::String>, node::encoding) [node]
11: 0x8e7084 [node]
12: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) [node]
13: 0xad712c [node]
14: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) [node]
15: 0x7d80ed842fd

[kunagpal]

@mrtibs2000 See the workaround described in #1516 (comment)

[mrtibs2000]

Still the same error:

NODE_OPTIONS="--max-old-space-size=2048" newman run ./certification_international.postman_collection.json -g ./globals.postman_globals.json -r json --reporter-json-export ./runs/20180827062926/certification_international.postman_collection.json.html -k

<--- Last few GCs --->

[17650:0x2ef8820]   132243 ms: Mark-sweep 1079.0 (1190.8) -> 1078.5 (1196.3) MB, 60.5 / 0.0 ms  allocation failure GC in old space requested
[17650:0x2ef8820]   132312 ms: Mark-sweep 1078.5 (1196.3) -> 1078.5 (1157.8) MB, 69.1 / 0.0 ms  last resort GC in old space requested
[17650:0x2ef8820]   132377 ms: Mark-sweep 1078.5 (1157.8) -> 1078.5 (1155.8) MB, 63.8 / 0.0 ms  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x3ad2488a5879 <JSObject>
    1: fromString(aka fromString) [buffer.js:314] [bytecode=0x2625b03f4209 offset=164](this=0x8dba71022d1 <undefined>,string=0x29ba3cf0a331 <Very long string[531705243]>,encoding=0x3ad2488b4ba1 <String[4]: utf8>)
    2: from [buffer.js:177] [bytecode=0x2625b03f3d11 offset=11](this=0x1d268bab6481 <JSFunction Buffer (sfi = 0x3ad2488fb7e9)>,value=0x29ba3cf0a331 <Very long string[531705243]>,encodin...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0x8c21ec [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [node]
 6: v8::internal::String::SlowFlatten(v8::internal::Handle<v8::internal::ConsString>, v8::internal::PretenureFlag) [node]
 7: v8::internal::String::Flatten(v8::internal::Handle<v8::internal::String>, v8::internal::PretenureFlag) [node]
 8: v8::String::WriteUtf8(char*, int, int*, int) const [node]
 9: node::StringBytes::Write(v8::Isolate*, char*, unsigned long, v8::Local<v8::Value>, node::encoding, int*) [node]
10: node::Buffer::New(v8::Isolate*, v8::Local<v8::String>, node::encoding) [node]
11: 0x8e7084 [node]
12: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) [node]
13: 0xad712c [node]
14: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) [node]
15: 0x2ced3e2842fd
Aborted (core dumped)

[kunagpal]

@mrtibs2000 Looks like your specific run is consuming large amounts of memory. Could you increase the value to 4096 and re-try?

[mrtibs2000]

4096 did work. Any idea where things are heavy? Would it be my specific version of newman/postman/node? Or would it be the size of my Postman collection? Would it be my application server/docker/infrastructure?

[happysahota]

Just in case anyone is still facing issues like

content: JSON.stringify(_.omit(o.summary, 'exports'), 0, 2)
                          ^

RangeError: Invalid string length
    at JSON.stringify (<anonymous>)
     listOnTimeout (node:internal/timers:573:17)
     process.processTimers (node:internal/timers:514:7)

I have created a reported for my personal use case where I needed to save json results for long time and I had around 1000 APIs and json result file used to be of +400MB in size by shedding/dropping extra information I'm able to reduce size under 10MB.

https://www.npmjs.com/package/newman-reporter-json-extralight