Implement streaming json output

[kleinweby]

This takes another try at implementing "real-time" json output as requested in #444 and #923.

By specifying --json-stream on cli, instead of outputting a whole json document at the end multiple json "events" are emitted. These events are json objects that are unformatted (and therefore never contain a newline) and are then separated by a newline.

This can easily be parsed by simply reading until the next newline and then deserialise the read line with any json parser.

Example output from iperf3 -c 127.0.0.1 --json-stream:

{"event":"start","data":{"connected":[{"socket":5,"local_host":"127.0.0.1","local_port":62151,"remote_host":"127.0.0.1","remote_port":5201}],"version":"iperf 3.9+","system_info":"<snip>","timestamp":{"time":"Sun, 03 Jan 2021 10:00:43 UTC","timesecs":1609668043},"connecting_to":{"host":"127.0.0.1","port":5201},"cookie":"bm5yvcnf4lhitapzhq5cbcnzppdiahy4giue","tcp_mss_default":16332,"sock_bufsize":0,"sndbuf_actual":131072,"rcvbuf_actual":131072,"test_start":{"protocol":"TCP","num_streams":1,"blksize":131072,"omit":0,"duration":10,"bytes":0,"blocks":0,"reverse":0,"tos":0}}}
{"event":"interval","data":{"streams":[{"socket":5,"start":0,"end":1.000077,"seconds":1.00007700920105,"bytes":6359613440,"bits_per_second":50872989831.698044,"omitted":false,"sender":true}],"sum":{"start":0,"end":1.000077,"seconds":1.00007700920105,"bytes":6359613440,"bits_per_second":50872989831.698044,"omitted":false,"sender":true}}}
{"event":"interval","data":{"streams":[{"socket":5,"start":1.000077,"end":2.000101,"seconds":1.0000239610672,"bytes":6450053120,"bits_per_second":51599188588.374786,"omitted":false,"sender":true}],"sum":{"start":1.000077,"end":2.000101,"seconds":1.0000239610672,"bytes":6450053120,"bits_per_second":51599188588.374786,"omitted":false,"sender":true}}}
{"event":"interval","data":{"streams":[{"socket":5,"start":2.000101,"end":3.000209,"seconds":1.000108003616333,"bytes":6524764160,"bits_per_second":52192476303.813812,"omitted":false,"sender":true}],"sum":{"start":2.000101,"end":3.000209,"seconds":1.000108003616333,"bytes":6524764160,"bits_per_second":52192476303.813812,"omitted":false,"sender":true}}}
{"event":"interval","data":{"streams":[{"socket":5,"start":3.000209,"end":4.000212,"seconds":1.0000029802322388,"bytes":6264586240,"bits_per_second":50116540561.070122,"omitted":false,"sender":true}],"sum":{"start":3.000209,"end":4.000212,"seconds":1.0000029802322388,"bytes":6264586240,"bits_per_second":50116540561.070122,"omitted":false,"sender":true}}}
{"event":"interval","data":{"streams":[{"socket":5,"start":4.000212,"end":5.000141,"seconds":0.99992901086807251,"bytes":6309806080,"bits_per_second":50482032315.652023,"omitted":false,"sender":true}],"sum":{"start":4.000212,"end":5.000141,"seconds":0.99992901086807251,"bytes":6309806080,"bits_per_second":50482032315.652023,"omitted":false,"sender":true}}}
{"event":"interval","data":{"streams":[{"socket":5,"start":5.000141,"end":6.000152,"seconds":1.0000109672546387,"bytes":6586368000,"bits_per_second":52690366131.337631,"omitted":false,"sender":true}],"sum":{"start":5.000141,"end":6.000152,"seconds":1.0000109672546387,"bytes":6586368000,"bits_per_second":52690366131.337631,"omitted":false,"sender":true}}}
{"event":"interval","data":{"streams":[{"socket":5,"start":6.000152,"end":7.000045,"seconds":0.99989300966262817,"bytes":6550978560,"bits_per_second":52413436211.223053,"omitted":false,"sender":true}],"sum":{"start":6.000152,"end":7.000045,"seconds":0.99989300966262817,"bytes":6550978560,"bits_per_second":52413436211.223053,"omitted":false,"sender":true}}}
{"event":"interval","data":{"streams":[{"socket":5,"start":7.000045,"end":8.000421,"seconds":1.0003759860992432,"bytes":6384517120,"bits_per_second":51056940260.192276,"omitted":false,"sender":true}],"sum":{"start":7.000045,"end":8.000421,"seconds":1.0003759860992432,"bytes":6384517120,"bits_per_second":51056940260.192276,"omitted":false,"sender":true}}}
{"event":"interval","data":{"streams":[{"socket":5,"start":8.000421,"end":9.000131,"seconds":0.99971002340316772,"bytes":6305873920,"bits_per_second":50461624050.01265,"omitted":false,"sender":true}],"sum":{"start":8.000421,"end":9.000131,"seconds":0.99971002340316772,"bytes":6305873920,"bits_per_second":50461624050.01265,"omitted":false,"sender":true}}}
{"event":"interval","data":{"streams":[{"socket":5,"start":9.000131,"end":10.000412,"seconds":1.0002809762954712,"bytes":6415974400,"bits_per_second":51313377357.322021,"omitted":false,"sender":true}],"sum":{"start":9.000131,"end":10.000412,"seconds":1.0002809762954712,"bytes":6415974400,"bits_per_second":51313377357.322021,"omitted":false,"sender":true}}}
{"event":"end","data":{"streams":[{"sender":{"socket":5,"start":0,"end":10.000412,"seconds":10.000412,"bytes":64152535040,"bits_per_second":51319913651.557549,"sender":true},"receiver":{"socket":5,"start":0,"end":10.000478,"seconds":10.000412,"bytes":64152469504,"bits_per_second":51319522530.023071,"sender":true}}],"sum_sent":{"start":0,"end":10.000412,"seconds":10.000412,"bytes":64152535040,"bits_per_second":51319913651.557549,"sender":true},"sum_received":{"start":0,"end":10.000478,"seconds":10.000478,"bytes":64152469504,"bits_per_second":51319522530.023071,"sender":true},"cpu_utilization_percent":{"host_total":99.7494433685625,"host_user":2.09120457342586,"host_system":97.658188805784363,"remote_total":90.051635480188565,"remote_user":6.4717641779081179,"remote_system":83.579881301792469}}}

The events basically follow the top-level keys in the normal json output. The new option works in both server and client mode, however the server output will only be send to the client at the end the test. (As before).

[kleinweby]

Sorry I thought of something else and force pushed it:

Now when using the --json-stream option (and unless we're server side and the client requested get_server_output) we don't save the members of the intervals json property.

This avoids building up memory about informations we've already outputted. And as we don't output the full json_top anyway in that case, this should be safe to do.

[kleinweby]

I've rebased this pr onto the current master to resolve the merge conflicts.

@bmah888 Anything else I need to do in order to possible get this PR reviewed and merged?

[stevenj]

I just tested this PR against v3.10.1 and it works perfectly, and gives easy machine readable progressive output from an iperf3 test. I would like to see it merged upstream as well. Its a massive improvement to parsing the standard human readable text output.

[swlars]

Looks good! Thank you for the merge request!