node-fast could track client metrics #17

joshwilsdon · 2018-05-09T22:59:12Z

Similar to the work in #9, we'd like to also be able to track the requests on the client side.

Reviewed by: David Pacheco <dap@joyent.com> Reviewed by: Kody A Kantor <kody@kkantor.com> Approved by: David Pacheco <dap@joyent.com>

davepacheco · 2018-05-16T20:16:32Z

I went to publish v2.5.0, and I ran make prepush, but it failed:

$ make prepush
jsl --nologo --nosummary --conf=tools/jsl.node.conf bin/fastbench bin/fastcall bin/fastserve lib/fast_client_request.js lib/fast_protocol.js lib/subr.js lib/bench.js lib/fast.js lib/fast_client.js lib/demo_server.js lib/fast_server.js test/tst.client_request.js test/tst.allocator.js test/common.js test/tst.protocol_decoder.js test/tst.client_generic.js test/tst.protocol_encoder.js test/compat/legacy-server.js test/compat/manual-tst.client_compat.js test/compat/common.js test/compat/setup.js test/tst.server.js test/tst.socket_summarize.js test/common/client.js
/home/dap/node-fast/bin/fastbench
/home/dap/node-fast/bin/fastcall
/home/dap/node-fast/bin/fastserve
/home/dap/node-fast/lib/fast_client_request.js
/home/dap/node-fast/lib/fast_protocol.js
/home/dap/node-fast/lib/subr.js
/home/dap/node-fast/lib/bench.js
/home/dap/node-fast/lib/fast.js
/home/dap/node-fast/lib/fast_client.js
/home/dap/node-fast/lib/demo_server.js
/home/dap/node-fast/lib/fast_server.js
/home/dap/node-fast/test/tst.client_request.js
/home/dap/node-fast/test/tst.allocator.js
/home/dap/node-fast/test/common.js
/home/dap/node-fast/test/tst.protocol_decoder.js
/home/dap/node-fast/test/tst.client_generic.js
/home/dap/node-fast/test/tst.protocol_encoder.js
/home/dap/node-fast/test/compat/legacy-server.js
/home/dap/node-fast/test/compat/manual-tst.client_compat.js
/home/dap/node-fast/test/compat/common.js
/home/dap/node-fast/test/compat/setup.js
/home/dap/node-fast/test/tst.server.js
/home/dap/node-fast/test/tst.socket_summarize.js
/home/dap/node-fast/test/common/client.js
json --validate -f package.json
jsstyle  bin/fastbench bin/fastcall bin/fastserve lib/fast_client_request.js lib/fast_protocol.js lib/subr.js lib/bench.js lib/fast.js lib/fast_client.js lib/demo_server.js lib/fast_server.js test/tst.client_request.js test/tst.allocator.js test/common.js test/tst.protocol_decoder.js test/tst.client_generic.js test/tst.protocol_encoder.js test/compat/legacy-server.js test/compat/manual-tst.client_compat.js test/compat/common.js test/compat/setup.js test/tst.server.js test/tst.socket_summarize.js test/common/client.js
check ok
deps/catest/catest test/tst.client_request.js test/tst.allocator.js test/tst.protocol_decoder.js test/tst.client_generic.js test/tst.protocol_encoder.js test/tst.server.js test/tst.socket_summarize.js
Configuration:
    SRC:                         /home/dap/node-fast
    Output directory:            /var/tmp/catest.586
    Temp directory:              /var/tmp/catest.586_tmpfiles
    Keep successful test output: false
    Found 7 test(s) to run

===================================================

Executing test test/tst.client_request.js ... success.
Executing test test/tst.allocator.js ... success.
Executing test test/tst.protocol_decoder.js ... success.
Executing test test/tst.client_generic.js ... success.
Executing test test/tst.protocol_encoder.js ... success.
Executing test test/tst.server.js ... FAILED.
>>> failure details in /var/tmp/catest.586/failure.0

Executing test test/tst.socket_summarize.js ... success.

===================================================

Results:
        Tests passed:    6/ 7
        Tests failed:    1/ 7

===================================================
Cleaning up output from successful tests ... done.
make: *** [test] Error 1
dap@b44c74d6 node-fast $ cat /var/tmp/catest.586/failure.0/README 
/home/dap/node-fast/test/tst.server.js failed: test returned 8
dap@b44c74d6 node-fast $ cat /var/tmp/catest.586/failure.0/*.err
test case: basic RPC: no data
test case: basic RPC: 1 data item
test case: basic RPC: several data items
test case: basic RPC: 0 data items, plus error
test case: basic RPC: several data items, plus error
test case: RPC for non-existent method
test case: multiple RPCs for the same client run in parallel
test case: connection error with requests outstanding: protocol error
test case: connection error with requests outstanding: socket error
test case: connection error followed by server shutdown
test case: basic RPC with immediate end-of-stream
test case: RPC with large amount of data
test case: flow control from server to client
test case: 'onConnsDestroyed' callback test
test case: 'onConnsDestroyed' callback after successful rpc
test case: 'onConnsDestroyed' callback after protocol error
test case: 'onConnsDestroyed' multiple callbacks queued
test case: 'onConnsDestroyed' callback after many requests

/home/dap/node-fast/test/tst.server.js:1125
                var e = new EventEmitter();
                        ^
TypeError: object is not a function
    at /home/dap/node-fast/test/tst.server.js:1125:11
    at Socket.<anonymous> (/home/dap/node-fast/test/tst.server.js:89:3)
    at Socket.emit (events.js:117:20)
    at Object.afterConnect [as oncomplete] (net.js:887:10)

I forgot to verify the test plan before IA -- sorry about that. What tests were run before integration?

davepacheco · 2018-05-16T20:18:36Z

FYI, I filed #19 for the test failure.

joshwilsdon · 2018-05-16T21:16:07Z

I had run make test and it succeeded for me. I just ran again and it still works:

[root@build ~/node-fast]# make test
deps/catest/catest -k test/tst.protocol_encoder.js test/tst.socket_summarize.js test/tst.allocator.js test/tst.server.js test/tst.client_request.js test/tst.client_generic.js test/tst.protocol_decoder.js
Configuration:
    SRC:                         /root/node-fast
    Output directory:            /var/tmp/catest.972068
    Temp directory:              /var/tmp/catest.972068_tmpfiles
    Keep successful test output: true
    Found 7 test(s) to run

===================================================

Executing test test/tst.protocol_encoder.js ... success.
Executing test test/tst.socket_summarize.js ... success.
Executing test test/tst.allocator.js ... success.
Executing test test/tst.server.js ... success.
Executing test test/tst.client_request.js ... success.
Executing test test/tst.client_generic.js ... success.
Executing test test/tst.protocol_decoder.js ... success.

===================================================

Results:
        Tests passed:    7/ 7
        Tests failed:    0/ 7

===================================================
Note: Compatibility tests need to be run manually with "make test-compat".
[root@build ~/node-fast]#

to rule out something in that workspace I did a fresh clone and npm install and then make test and it works there too:

[root@build ~]# git clone git@github.com:joyent/node-fast node-fast-run-tests-again
Cloning into 'node-fast-run-tests-again'...
Warning: Permanently added the RSA host key for IP address '192.30.255.112' to the list of known hosts.
remote: Counting objects: 1105, done.
remote: Compressing objects: 100% (13/13), done.
remote: Total 1105 (delta 0), reused 14 (delta 0), pack-reused 1091
Receiving objects: 100% (1105/1105), 343.06 KiB | 0 bytes/s, done.
Resolving deltas: 100% (700/700), done.
Checking connectivity... done.
[root@build ~]# cd node-fast-run-tests-again/
[root@build ~/node-fast-run-tests-again]# npm install
npm WARN engine decompress-response@3.3.0: wanted: {"node":">=4"} (current: {"node":"0.12.2","npm":"2.7.4"})
npm WARN engine mimic-response@1.0.0: wanted: {"node":">=4"} (current: {"node":"0.12.2","npm":"2.7.4"})
npm WARN engine mime@1.6.0: wanted: {"node":">=4"} (current: {"node":"0.12.2","npm":"2.7.4"})

> microtime@2.1.6 install /root/node-fast-run-tests-again/node_modules/microtime
> prebuild-install || node-gyp rebuild

/root/node-fast-run-tests-again/node_modules/microtime/node_modules/prebuild-install/node_modules/which-pm-runs/index.js:11
  const pmSpec = userAgent.split(' ')[0]
  ^^^^^
SyntaxError: Use of const in strict mode.
    at exports.runInThisContext (vm.js:73:16)
    at Module._compile (module.js:443:25)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at Object.<anonymous> (/root/node-fast-run-tests-again/node_modules/microtime/node_modules/prebuild-install/bin.js:5:19)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
make: Entering directory '/root/node-fast-run-tests-again/node_modules/microtime/build'
  CXX(target) Release/obj.target/microtime/src/microtime.o
  SOLINK_MODULE(target) Release/obj.target/microtime.node
  SOLINK_MODULE(target) Release/obj.target/microtime.node: Finished
  COPY Release/microtime.node
make: Leaving directory '/root/node-fast-run-tests-again/node_modules/microtime/build'
npm WARN deprecated node-uuid@1.4.8: Use uuid module instead
npm WARN engine mime@1.6.0: wanted: {"node":">=4"} (current: {"node":"0.12.2","npm":"2.7.4"})
npm WARN engine mime@1.6.0: wanted: {"node":">=4"} (current: {"node":"0.12.2","npm":"2.7.4"})

> dtrace-provider@0.8.6 install /root/node-fast-run-tests-again/node_modules/dtrace-provider
> node-gyp rebuild || node suppress-error.js

make: Entering directory '/root/node-fast-run-tests-again/node_modules/dtrace-provider/build'
  ACTION binding_gyp_ndtp_target_build_ndtp .
  TOUCH Release/obj.target/ndtp.stamp
make: Leaving directory '/root/node-fast-run-tests-again/node_modules/dtrace-provider/build'

> dtrace-provider@0.6.0 install /root/node-fast-run-tests-again/node_modules/kang/node_modules/restify-clients/node_modules/dtrace-provider
> node scripts/install.js


> dtrace-provider@0.6.0 install /root/node-fast-run-tests-again/node_modules/kang/node_modules/restify/node_modules/dtrace-provider
> node scripts/install.js

assert-plus@1.0.0 node_modules/assert-plus

forkexec@1.1.0 node_modules/forkexec

crc@0.3.0 node_modules/crc

posix-getopt@1.2.0 node_modules/posix-getopt

extsprintf@1.4.0 node_modules/extsprintf

strsplit@1.0.0 node_modules/strsplit

verror@1.10.0 node_modules/verror
└── core-util-is@1.0.2

cmdutil@1.1.0 node_modules/cmdutil
├── assert-plus@0.1.5
└── extsprintf@1.3.0

lstream@0.0.4 node_modules/lstream
└── readable-stream@2.3.6 (process-nextick-args@2.0.0, inherits@2.0.3, util-deprecate@1.0.2, string_decoder@1.1.1, isarray@1.0.0, core-util-is@1.0.2, safe-buffer@5.1.2)

jsprim@1.4.1 node_modules/jsprim
├── extsprintf@1.3.0
└── json-schema@0.2.3

vasync@1.6.4 node_modules/vasync
└── verror@1.6.0 (extsprintf@1.2.0)

artedi@1.1.1 node_modules/artedi
└── jsprim@1.4.0 (extsprintf@1.0.2, verror@1.3.6, json-schema@0.2.3)

microtime@2.1.6 node_modules/microtime
├── bindings@1.2.1
├── nan@2.6.2
└── prebuild-install@2.5.3 (which-pm-runs@1.0.0, os-homedir@1.0.2, expand-template@1.1.1, noop-logger@0.1.1, detect-libc@1.0.3, github-from-package@0.0.0, minimist@1.2.0, tunnel-agent@0.6.0, rc@1.2.7, pump@2.0.1, simple-get@2.8.1, node-abi@2.4.1, mkdirp@0.5.1, npmlog@4.1.2, tar-fs@1.16.2)

bunyan@1.8.12 node_modules/bunyan
├── safe-json-stringify@1.1.0
├── mv@2.1.1 (ncp@2.0.0, mkdirp@0.5.1, rimraf@2.4.5)
└── moment@2.22.1

restify@5.2.0 node_modules/restify
├── escape-regexp-component@1.0.2
├── negotiator@0.6.1
├── once@1.4.0 (wrappy@1.0.2)
├── semver@5.5.0
├── clone-regexp@1.0.1 (is-regexp@1.0.0, is-supported-regexp-flag@1.0.1)
├── mime@1.6.0
├── lru-cache@4.1.3 (pseudomap@1.0.2, yallist@2.1.2)
├── uuid@3.2.1
├── restify-errors@4.3.0 (safe-json-stringify@1.1.0)
├── qs@6.5.2
├── formidable@1.2.1
├── csv@1.2.1 (csv-generate@1.1.2, stream-transform@0.2.2, csv-parse@1.3.3, csv-stringify@1.1.2)
├── spdy@3.4.7 (http-deceiver@1.2.7, handle-thing@1.2.5, select-hose@2.0.0, safe-buffer@5.1.2, debug@2.6.9, spdy-transport@2.1.0)
├── http-signature@1.2.0 (sshpk@1.14.1)
└── lodash@4.17.10

dtrace-provider@0.8.6 node_modules/dtrace-provider
└── nan@2.10.0

kang@1.2.0 node_modules/kang
├── restify-clients@1.1.1 (assert-plus@0.1.5, tunnel-agent@0.4.3, keep-alive-agent@0.0.1, once@1.4.0, lru-cache@2.7.3, semver@5.5.0, mime@1.6.0, node-uuid@1.4.8, restify-errors@3.1.0, backoff@2.5.0, lodash@3.10.1, dtrace-provider@0.6.0)
└── restify@4.1.1 (assert-plus@0.1.5, escape-regexp-component@1.0.2, tunnel-agent@0.4.3, keep-alive-agent@0.0.1, negotiator@0.6.1, once@1.4.0, lru-cache@4.1.3, mime@1.6.0, node-uuid@1.4.8, qs@3.1.0, backoff@2.5.0, formidable@1.2.1, semver@4.3.6, http-signature@0.11.0, vasync@1.6.3, csv@0.4.6, spdy@3.4.7, dtrace-provider@0.6.0)
[root@build ~/node-fast-run-tests-again]# make test
git submodule update --init deps/catest
Submodule 'deps/catest' (https://github.com/joyent/catest) registered for path 'deps/catest'
Cloning into 'deps/catest'...
remote: Counting objects: 29, done.
remote: Total 29 (delta 0), reused 0 (delta 0), pack-reused 29
Unpacking objects: 100% (29/29), done.
Checking connectivity... done.
Submodule path 'deps/catest': checked out 'ca138645cc9647d6976063c61fa9f28dd16c5023'
deps/catest/catest test/tst.client_request.js test/tst.client_generic.js test/tst.protocol_decoder.js test/tst.protocol_encoder.js test/tst.server.js test/tst.allocator.js test/tst.socket_summarize.js
Configuration:
    SRC:                         /root/node-fast-run-tests-again
    Output directory:            /var/tmp/catest.973225
    Temp directory:              /var/tmp/catest.973225_tmpfiles
    Keep successful test output: false
    Found 7 test(s) to run

===================================================

Executing test test/tst.client_request.js ... success.
Executing test test/tst.client_generic.js ... success.
Executing test test/tst.protocol_decoder.js ... success.
Executing test test/tst.protocol_encoder.js ... success.
Executing test test/tst.server.js ... success.
Executing test test/tst.allocator.js ... success.
Executing test test/tst.socket_summarize.js ... success.

===================================================

Results:
        Tests passed:    7/ 7
        Tests failed:    0/ 7

===================================================
Cleaning up output from successful tests ... done.
Note: Compatibility tests need to be run manually with "make test-compat".
[root@build ~/node-fast-run-tests-again]#

davepacheco · 2018-05-23T00:25:40Z

#19 has been fixed, and I've published v2.5.0 with these changes (plus an inadvertent local change to .gitmodules that should make no difference to anybody).

joshwilsdon mentioned this issue May 15, 2018

improve node-fast artedi tests #18

Open

joyent-automation pushed a commit that referenced this issue May 16, 2018

#17 node-fast could track client metrics

73dcb46

Reviewed by: David Pacheco <dap@joyent.com> Reviewed by: Kody A Kantor <kody@kkantor.com> Approved by: David Pacheco <dap@joyent.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

node-fast could track client metrics #17

node-fast could track client metrics #17

joshwilsdon commented May 9, 2018

davepacheco commented May 16, 2018

davepacheco commented May 16, 2018

joshwilsdon commented May 16, 2018

davepacheco commented May 23, 2018

node-fast could track client metrics #17

node-fast could track client metrics #17

Comments

joshwilsdon commented May 9, 2018

davepacheco commented May 16, 2018

davepacheco commented May 16, 2018

joshwilsdon commented May 16, 2018

davepacheco commented May 23, 2018