test.py tap13 output #9262

jbergstroem · 2016-10-24T19:42:12Z

Here's a work in progress that generates valid (ish -- haven't tested all kinds of stuff just yet) tap13 output from the test runner. I was initially thinking that we'd do a separate test runner (tap and tap13) but seeing how our current tap is invalid at best and this diff is easier to read I thought I'd post it in its current state for feedback/discussion.

Together with https://pypi.python.org/pypi/tap2junit we now generate junit that jenkins like.

Open issues/things to discuss:

~~cctest output needs to be changed -- invalid failures~
~~do we split up tap vs tap13 (i prefer not)~~
~~if not, is this a semver major or minor? hard to say with tooling; especially seeing how our tooling has been invalid for so long.~~

I'll rebase/clean up once we converge on what to do. Hopefully we can get this in as quick as possible -- the build groups general theory is that this will Make Jenkins Great Again.

bnoordhuis · 2016-10-24T20:03:58Z

do we split up tap vs tap13

Just change to tap13.

if not, is this a semver major or minor?

The test suite is semver-nothing.

jbergstroem · 2016-10-25T13:10:32Z

CI here: https://ci.nodejs.org/job/node-test-commit/5777/

This run is mostly about showing that the current jenkins tap parser accepts our tap13 output as well.

jbergstroem · 2016-10-25T15:26:25Z

@bnoordhuis not very happy with with change in gtest; "expected' also seems to be passed differently than through comment. Do you want to have a closer look or is this "good enough"?

jbergstroem · 2016-10-25T15:31:56Z

@mhdawson I recall you guys using this internally too, no? Do you have a different consumer than jenkins/tap? If so, can you try it?

bnoordhuis · 2016-10-25T15:32:50Z

deps/gtest/src/gtest.cc

-  for (int i = 0; i < result.total_part_count(); ++i) {
-    const TestPartResult& part = result.GetTestPartResult(i);
-    OutputTapComment(stream, part.message());
+  if (result.total_part_count() > 0) {


The if statement doesn't look like it's necessary.

I put it there to avoid writing "stack" unless used.

Unless I misread the diff that won't happen because it's inside the loop. If result.total_part_count() == 0 then i < result.total_part_count() with i == 0 is false so it's not reached.

yeah I messed up -- moved stuff around too quick.

perhaps should mention that i've fixed it since.

bnoordhuis · 2016-10-25T15:35:07Z

deps/gtest/src/gtest.cc

-    OutputTapComment(stream, part.message());
+  if (result.total_part_count() > 0) {
+    for (int i = 0; i < result.total_part_count(); ++i) {
+      *stream << "  stack: |-\n";


I don't understand what this is for. What does stack: signify?

It's how we handle tracebacks from failed tests in the other test suite. It comes from how the tap13 producer in mocha wrote it. There seems to be these "almost defined" best practises for how traces should look in yamlish. Not saying its the right thing, but it is also how test.py output looks.

jbergstroem · 2016-10-26T12:14:12Z

ping @nodejs/build, @bnoordhuis (updated commits)

gibfahn · 2016-10-26T15:49:11Z

@mhdawson I recall you guys using this internally too, no? Do you have a different consumer than jenkins/tap? If so, can you try it?

@jbergstroem We use Jenkins tap as well, but we do also have a different consumer, we'll give it a go.

@BethGriggs

gibfahn · 2016-10-26T16:40:12Z

So the new TAP output for a failing test seems to look like this:

not ok 257 parallel/test-dgram-send-multi-buffer-copy
  ---
  duration_ms: 12.902
  severity: fail
  stack: |-
    /home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2/test/parallel/test-dgram-send-multi-buffer-copy.js:10
      throw new Error('Timeout');
      ^

    Error: Timeout
        at Timeout._onTimeout (/home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2/test/parallel/test-dgram-send-multi-buffer-copy.js:10:9)
        at ontimeout (timers.js:365:14)
        at tryOnTimeout (timers.js:237:5)
        at Timer.listOnTimeout (timers.js:207:5)
  ...

Except for the first test, which looks like this:

Output of first test:

not ok 92 parallel/test-child-process-spawnsync-timeout
  ---
  duration_ms: 17.262
  severity: fail
  stack: |-
    { error: 
       { Error: spawnSync /home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2/out/Release/node ETIMEDOUT
           at exports._errnoException (util.js:1026:11)
           at spawnSync (child_process.js:466:20)
           at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2/test/parallel/test-child-process-spawnsync-timeout.js:19:15)
           at Module._compile (module.js:573:32)
           at Object.Module._extensions..js (module.js:582:10)
           at Module.load (module.js:490:32)
           at tryModuleLoad (module.js:449:12)
           at Function.Module._load (module.js:441:3)
           at Module.runMain (module.js:607:10)
           at run (bootstrap_node.js:382:7)
         code: 'ETIMEDOUT',
         errno: 'ETIMEDOUT',
         syscall: 'spawnSync /home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2/out/Release/node',
         path: '/home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2/out/Release/node',
         spawnargs: 
          [ '/home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2/test/parallel/test-child-process-spawnsync-timeout.js',
            'child' ] },
      status: 0,
      signal: 'SIGTERM',
      output: [ null, <Buffer >, <Buffer > ],
      pid: 66964,
      stdout: <Buffer >,
      stderr: <Buffer >,
      envPairs: 
       [ 'IGNORE_FLAKY_TESTS=false',
         'JENKINS_HOME=/var/lib/jenkins',
         'BUILD_CAUSE=UPSTREAMTRIGGER',
         'JOBS=48',
         'HUDSON_URL=https://ci.nodejs.org/',
         'BUILD_URL=https://ci.nodejs.org/job/node-test-commit-jbergstroem-alpine34/nodes=test-joyent-alpine34-x64-2/31/',
         'GIT_PREVIOUS_COMMIT=3d7145f3da4aaafff4b433a512b1ad1acf5cc57a',
         'BUILD_TAG=jenkins-node-test-commit-jbergstroem-alpine34-nodes=test-joyent-alpine34-x64-2-31',
         'HUDSON_SERVER_COOKIE=77c5c04cfa349d13',
         'JENKINS_URL=https://ci.nodejs.org/',
         'USER=iojs',
         'WORKSPACE=/home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2',
         'PATH=/usr/lib/ccache/bin:/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin',
         'EINFO_LASTCMD=ebegin',
         'NODE_COMMON_PIPE=/home/iojs/test.pipe',
         'JOB_NAME=node-test-commit-jbergstroem-alpine34/nodes=test-joyent-alpine34-x64-2',
         'FLAKY_TESTS=run',
         'GITHUB_ORG=jbergstroem',
         'TERM=xterm-256color',
         'SHLVL=2',
         'HUDSON_HOME=/var/lib/jenkins',
         'LD_LIBRARY_PATH=/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64',
         'NODE_LABELS=alpine34-x64 test-joyent-alpine34-x64-2',
         'JOB_URL=https://ci.nodejs.org/job/node-test-commit-jbergstroem-alpine34/nodes=test-joyent-alpine34-x64-2/',
         'ROOT_BUILD_CAUSE_MANUALTRIGGER=true',
         'MAKEFLAGS=w',
         'GIT_ORIGIN_SCHEME=https://github.com/',
         'nodes=test-joyent-alpine34-x64-2',
         'HOME=/home/iojs',
         'ROOT_BUILD_CAUSE=MANUALTRIGGER',
         'REPO_NAME=node',
         'BUILD_ID=31',
         'RC_SERVICE=/etc/init.d/jenkins',
         'BUILD_NUMBER=31',
         'HUDSON_COOKIE=f4e9dac6-089d-47dc-965f-8f785c83f320',
         'NODES_SUBSET=io.js',
         'POST_STATUS_TO_PR=false',
         'RC_SVCNAME=jenkins',
         'TEST_THREAD_ID=47',
         'GIT_URL=https://github.com/jbergstroem/node.git',
         'GIT_COMMIT=3d7145f3da4aaafff4b433a512b1ad1acf5cc57a',
         'NLSPATH=/usr/dt/lib/nls/msg/%L/%N.cat',
         'SVCNAME=jenkins',
         'GIT_REMOTE_REF=feature/testpy-tap13',
         'BUILD_CAUSE_UPSTREAMTRIGGER=true',
         'NODE_TEST_DIR=/home/iojs/tmp',
         'MFLAGS=-w',
         'EXECUTOR_NUMBER=0',
         'NODE_NAME=test-joyent-alpine34-x64-2',
         'MAKELEVEL=2',
         'PWD=/home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2',
         'EINFO_LOG=/etc/init.d/jenkins',
         'JENKINS_SERVER_COOKIE=77c5c04cfa349d13',
         'BUILD_DISPLAY_NAME=#31',
         'XFILESEARCHPATH=/usr/dt/app-defaults/%L/Dt',
         'GIT_BRANCH=origin/_jenkins_local_branch' ],
      options: 
       { timeout: 200,
         file: '/home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2/out/Release/node',
         args: 
          [ '/home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2/out/Release/node',
            '/home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2/test/parallel/test-child-process-spawnsync-timeout.js',
            'child' ],
         envPairs: 
          [ 'IGNORE_FLAKY_TESTS=false',
            'JENKINS_HOME=/var/lib/jenkins',
            'BUILD_CAUSE=UPSTREAMTRIGGER',
            'JOBS=48',
            'HUDSON_URL=https://ci.nodejs.org/',
            'BUILD_URL=https://ci.nodejs.org/job/node-test-commit-jbergstroem-alpine34/nodes=test-joyent-alpine34-x64-2/31/',
            'GIT_PREVIOUS_COMMIT=3d7145f3da4aaafff4b433a512b1ad1acf5cc57a',
            'BUILD_TAG=jenkins-node-test-commit-jbergstroem-alpine34-nodes=test-joyent-alpine34-x64-2-31',
            'HUDSON_SERVER_COOKIE=77c5c04cfa349d13',
            'JENKINS_URL=https://ci.nodejs.org/',
            'USER=iojs',
            'WORKSPACE=/home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2',
            'PATH=/usr/lib/ccache/bin:/bin:/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin',
            'EINFO_LASTCMD=ebegin',
            'NODE_COMMON_PIPE=/home/iojs/test.pipe',
            'JOB_NAME=node-test-commit-jbergstroem-alpine34/nodes=test-joyent-alpine34-x64-2',
            'FLAKY_TESTS=run',
            'GITHUB_ORG=jbergstroem',
            'TERM=xterm-256color',
            'SHLVL=2',
            'HUDSON_HOME=/var/lib/jenkins',
            'LD_LIBRARY_PATH=/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64',
            'NODE_LABELS=alpine34-x64 test-joyent-alpine34-x64-2',
            'JOB_URL=https://ci.nodejs.org/job/node-test-commit-jbergstroem-alpine34/nodes=test-joyent-alpine34-x64-2/',
            'ROOT_BUILD_CAUSE_MANUALTRIGGER=true',
            'MAKEFLAGS=w',
            'GIT_ORIGIN_SCHEME=https://github.com/',
            'nodes=test-joyent-alpine34-x64-2',
            'HOME=/home/iojs',
            'ROOT_BUILD_CAUSE=MANUALTRIGGER',
            'REPO_NAME=node',
            'BUILD_ID=31',
            'RC_SERVICE=/etc/init.d/jenkins',
            'BUILD_NUMBER=31',
            'HUDSON_COOKIE=f4e9dac6-089d-47dc-965f-8f785c83f320',
            'NODES_SUBSET=io.js',
            'POST_STATUS_TO_PR=false',
            'RC_SVCNAME=jenkins',
            'TEST_THREAD_ID=47',
            'GIT_URL=https://github.com/jbergstroem/node.git',
            'GIT_COMMIT=3d7145f3da4aaafff4b433a512b1ad1acf5cc57a',
            'NLSPATH=/usr/dt/lib/nls/msg/%L/%N.cat',
            'SVCNAME=jenkins',
            'GIT_REMOTE_REF=feature/testpy-tap13',
            'BUILD_CAUSE_UPSTREAMTRIGGER=true',
            'NODE_TEST_DIR=/home/iojs/tmp',
            'MFLAGS=-w',
            'EXECUTOR_NUMBER=0',
            'NODE_NAME=test-joyent-alpine34-x64-2',
            'MAKELEVEL=2',
            'PWD=/home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2',
            'EINFO_LOG=/etc/init.d/jenkins',
            'JENKINS_SERVER_COOKIE=77c5c04cfa349d13',
            'BUILD_DISPLAY_NAME=#31',
            'XFILESEARCHPATH=/usr/dt/app-defaults/%L/Dt',
            'GIT_BRANCH=origin/_jenkins_local_branch' ],
         stdio: [ [Object], [Object], [Object] ] },
      args: 
       [ '/home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2/out/Release/node',
         '/home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2/test/parallel/test-child-process-spawnsync-timeout.js',
         'child' ],
      file: '/home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2/out/Release/node' }

    assert.js:85
      throw new assert.AssertionError({
      ^
    AssertionError: false == true
        at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-jbergstroem-alpine34/nodes/test-joyent-alpine34-x64-2/test/parallel/test-child-process-spawnsync-timeout.js:24:5)
        at Module._compile (module.js:573:32)
        at Object.Module._extensions..js (module.js:582:10)
        at Module.load (module.js:490:32)
        at tryModuleLoad (module.js:449:12)
        at Function.Module._load (module.js:441:3)
        at Module.runMain (module.js:607:10)
        at run (bootstrap_node.js:382:7)
        at startup (bootstrap_node.js:137:9)
        at bootstrap_node.js:497:3
  ...

@jbergstroem Is this intended? It seems a bit odd to put all this extra info in the first failing test.

Looking at the TAP 13 spec it looks like lines starting with # are still supported, they're just not recommended for test output (as that's what the YAML is used for).

How does the format handle flaky tests?

jbergstroem · 2016-10-26T16:45:43Z

@gibfahn I pass both stdout and stderr, similar to what the old runner did. I'm happy to drop stdout if people agree that's what we want.

gibfahn · 2016-10-26T20:20:21Z

@jbergstroem I was talking about the envPairs stuff, I don't remember seeing that in any of the previous TAP output.

jbergstroem · 2016-11-04T15:19:24Z

@gibfahn: Did you pull that from console or the tapfile? Here's a new run; doesn't seem to have the same issue: https://ci.nodejs.org/job/node-test-commit-jbergstroem-alpine34/33/nodes=test-joyent-alpine34-x64-2/console

gibfahn · 2016-11-04T16:39:05Z

@jbergstroem It was from the console part of the tapfile, but I agree that it's not in the new run.

It might be useful to put some basic info in the tapfile, such as the machine name, Jenkins job and number, workspace, Node commit hash, etc. If that info is in the tapfiles, then they become useful by themselves. Looking at the TAP13 spec it seems that having them with comments at the start of the tapfile should be fine. Is that something that might make sense?

Background

We are using a simple Node dashboard that parses the tapfiles and stores the results in a database. This lets you do things like look at when a test has failed in the past and see what machines it tends to fail on. We've found it really helpful for testing, and we'd be interested to see whether it's something that might be worth contributing. The server gets machine and node version info from the tapfiles.

The dashboard was written by @BethGriggs.

cc/ @mhdawson

jbergstroem · 2016-11-04T17:32:50Z

@gibfahn junit natively supports things like hostname, so I can look at adding it from the tap2junit utility instead. If you're not planning on using junit stuff I guess it would make sense to add it as a comment.

bnoordhuis

The C++ changes LGTM but the python code looks like it still needs some work.

bnoordhuis · 2016-11-04T17:42:17Z

tools/test.py

+      self.severity = 'fail'
+      self.traceback = output.output.stdout + output.output.stderr
+
+      # suggestions welcome


What is supposed to go here?

Reference to old code. Removed.

bnoordhuis · 2016-11-04T17:42:21Z

tools/test.py


      if output.HasCrashed():
-        self._printDiagnostic(PrintCrashed(output.output.exit_code))
+        self.severity = 'crashed'
+        self.traceback = "oh no!\nexit code: " + PrintCrashed(output.output.exit_code)


bnoordhuis · 2016-11-04T17:42:37Z

tools/test.py

@@ -316,6 +331,10 @@ def HasRun(self, output):
    # It should read as "duration including ms" rather than "duration in ms"
    logger.info('  ---')
    logger.info('  duration_ms: %d.%d' % (total_seconds, duration.microseconds / 1000))
+    if self.severity is not 'ok' or self.traceback is not '':
+        if output.HasTimedOut():
+            self.traceback = 'timeout'


Indent error.

gibfahn · 2016-11-04T21:15:41Z

.gitignore

@@ -89,6 +89,7 @@ deps/npm/node_modules/.bin/
 tools/faketime
 icu_config.gypi
 test.tap
+cctest.tap


Should we just ignore *.tap?

gibfahn · 2016-11-04T21:47:11Z

@jbergstroem I was talking about adding it in the tap. Maybe if it was in the TAP then your junit converter could parse it into the native elements. It doesn't need to be in this PR though (obviously), we can raise it separately.

jbergstroem · 2016-11-04T21:47:51Z

@gibfahn: lets keep it out of this then.

jbergstroem · 2016-11-08T18:15:49Z

@bnoordhuis anything else?

bnoordhuis

Oh, you wanted me to do another review? LGTM.

jbergstroem · 2016-11-08T19:00:31Z

@bnoordhuis your latest review status was "request for change".

New CI here: https://ci.nodejs.org/job/node-test-commit/5956/

jbergstroem · 2016-11-08T19:31:17Z

@nodejs/build: does anyone else want to chip in? This is taking us from tap to tap13.

MylesBorins · 2016-11-08T19:59:37Z

Rubber stamp LGTM

Produce a tap13-compatible output which makes it simpler to parse. Output is still readable by the jenkins tap plugin. PR-URL: nodejs#9262 Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Myles Borins <myles.borins@gmail.com>

This makes yaml-ish parsers happy. Note: gtest still seems to output the expected/result slightly different making the full traceback less informational. PR-URL: nodejs#9262 Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Myles Borins <myles.borins@gmail.com>

We now have multiple tap producers; just ignore all files with the `.tap` extension. PR-URL: nodejs#9262 Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Myles Borins <myles.borins@gmail.com>

Produce a tap13-compatible output which makes it simpler to parse. Output is still readable by the jenkins tap plugin. PR-URL: nodejs#9262 Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Myles Borins <myles.borins@gmail.com>

This makes yaml-ish parsers happy. Note: gtest still seems to output the expected/result slightly different making the full traceback less informational. PR-URL: nodejs#9262 Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Myles Borins <myles.borins@gmail.com>

We now have multiple tap producers; just ignore all files with the `.tap` extension. PR-URL: nodejs#9262 Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Myles Borins <myles.borins@gmail.com>

Use yamlish instead of # to match community: nodejs/node#9262

Produce a tap13-compatible output which makes it simpler to parse. Output is still readable by the jenkins tap plugin. PR-URL: #9262 Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Myles Borins <myles.borins@gmail.com>

This makes yaml-ish parsers happy. Note: gtest still seems to output the expected/result slightly different making the full traceback less informational. PR-URL: #9262 Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Myles Borins <myles.borins@gmail.com>

We now have multiple tap producers; just ignore all files with the `.tap` extension. PR-URL: #9262 Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Myles Borins <myles.borins@gmail.com>

Use yamlish instead of # to match community: nodejs/node#9262

This LTS release comes with 144 commits. This includes 47 that are docs related, 46 that are test related, 15 which are build / tools related, and 9 commits which are updates to dependencies Notable Changes: * buffer: - coerce slice parameters consistently (Sakthipriyan Vairamani (thefourtheye)) #9101 * deps: - *npm*: - upgrade npm to 3.10.9 (Kat Marchán) #9286 - *V8*: - Various fixes to destructuring edge cases - cherry-pick 3c39bac from V8 upstream (Cristian Cavalli) #9138 - cherry pick 7166503 from upstream v8 (Cristian Cavalli) #9173 * gtest: - the test reporter now outputs tap comments as yamlish (Johan Bergström) #9262 * inspector: - inspector now prompts user to use 127.0.0.1 rather than localhost (Eugene Ostroukhov) #9451 * tls: - fix memory leak when writing data to TLSWrap instance during handshake (Fedor Indutny) #9586 PR-URL: #9735

This LTS release comes with 108 commits. This includes 30 which are doc related, 28 which are test related, 16 which are build / tool related, and 4 commits which are updates to dependencies. Notable Changes: The SEMVER-MINOR changes include: * build: - export openssl symbols on Windows making it possible to build addons linked against the bundled version of openssl (Alex Hultman) #7576 * debugger: - make listen address configurable in the debugger server (Ben Noordhuis) #3316 * dgram: - generalized send queue to handle close fixing a potential throw when dgram socket is closed in the listening event handler. (Matteo Collina) #7066 * http: - Introduce the 451 status code "Unavailable For Legal Reasons" (Max Barinov) #4377 * tls: - introduce `secureContext` for `tls.connect` which is useful for caching client certificates, key, and CA certificates. (Fedor Indutny) #4246 Notable SEMVER-PATCH changes include: * build: - introduce the configure --shared option for embedders (sxa555) #6994 * gtest: - the test reporter now outputs tap comments as yamlish (Johan Bergström) #9262 * src: - node no longer aborts when c-ares initialization fails (Ben Noordhuis) #8710 * tls: - fix memory leak when writing data to TLSWrap instance during handshake (Fedor Indutny) #9586 PR-URL: #9736

This LTS release comes with 144 commits. This includes 47 that are docs related, 46 that are test related, 15 which are build / tools related, and 9 commits which are updates to dependencies Notable Changes: * buffer: - coerce slice parameters consistently (Sakthipriyan Vairamani (thefourtheye)) #9101 * deps: - *npm*: - upgrade npm to 3.10.9 (Kat Marchán) #9286 - *V8*: - Various fixes to destructuring edge cases - cherry-pick 3c39bac from V8 upstream (Cristian Cavalli) #9138 - cherry pick 7166503 from upstream v8 (Cristian Cavalli) #9173 * gtest: - the test reporter now outputs tap comments as yamlish (Johan Bergström) #9262 * inspector: - inspector now prompts user to use 127.0.0.1 rather than localhost (Eugene Ostroukhov) #9451 * tls: - fix memory leak when writing data to TLSWrap instance during handshake (Fedor Indutny) #9586 PR-URL: #9735

This LTS release comes with 108 commits. This includes 30 which are doc related, 28 which are test related, 16 which are build / tool related, and 4 commits which are updates to dependencies. Notable Changes: The SEMVER-MINOR changes include: * build: - export openssl symbols on Windows making it possible to build addons linked against the bundled version of openssl (Alex Hultman) #7576 * debugger: - make listen address configurable in the debugger server (Ben Noordhuis) #3316 * dgram: - generalized send queue to handle close fixing a potential throw when dgram socket is closed in the listening event handler. (Matteo Collina) #7066 * http: - Introduce the 451 status code "Unavailable For Legal Reasons" (Max Barinov) #4377 * tls: - introduce `secureContext` for `tls.connect` which is useful for caching client certificates, key, and CA certificates. (Fedor Indutny) #4246 Notable SEMVER-PATCH changes include: * build: - introduce the configure --shared option for embedders (sxa555) #6994 * gtest: - the test reporter now outputs tap comments as yamlish (Johan Bergström) #9262 * src: - node no longer aborts when c-ares initialization fails (Ben Noordhuis) #8710 * tls: - fix memory leak when writing data to TLSWrap instance during handshake (Fedor Indutny) #9586 PR-URL: #9736

This LTS release comes with 144 commits. This includes 47 that are docs related, 46 that are test related, 15 which are build / tools related, and 9 commits which are updates to dependencies Notable Changes: * buffer: - coerce slice parameters consistently (Sakthipriyan Vairamani (thefourtheye)) #9101 * deps: - *npm*: - upgrade npm to 3.10.9 (Kat Marchán) #9286 - *V8*: - Various fixes to destructuring edge cases - cherry-pick 3c39bac from V8 upstream (Cristian Cavalli) #9138 - cherry pick 7166503 from upstream v8 (Cristian Cavalli) #9173 * gtest: - the test reporter now outputs tap comments as yamlish (Johan Bergström) #9262 * inspector: - inspector now prompts user to use 127.0.0.1 rather than localhost (Eugene Ostroukhov) #9451 * tls: - fix memory leak when writing data to TLSWrap instance during handshake (Fedor Indutny) #9586 PR-URL: #9735

This LTS release comes with 108 commits. This includes 30 which are doc related, 28 which are test related, 16 which are build / tool related, and 4 commits which are updates to dependencies. Notable Changes: The SEMVER-MINOR changes include: * build: - export openssl symbols on Windows making it possible to build addons linked against the bundled version of openssl (Alex Hultman) #7576 * debugger: - make listen address configurable in the debugger server (Ben Noordhuis) #3316 * dgram: - generalized send queue to handle close fixing a potential throw when dgram socket is closed in the listening event handler. (Matteo Collina) #7066 * http: - Introduce the 451 status code "Unavailable For Legal Reasons" (Max Barinov) #4377 * tls: - introduce `secureContext` for `tls.connect` which is useful for caching client certificates, key, and CA certificates. (Fedor Indutny) #4246 Notable SEMVER-PATCH changes include: * build: - introduce the configure --shared option for embedders (sxa555) #6994 * gtest: - the test reporter now outputs tap comments as yamlish (Johan Bergström) #9262 * src: - node no longer aborts when c-ares initialization fails (Ben Noordhuis) #8710 * tls: - fix memory leak when writing data to TLSWrap instance during handshake (Fedor Indutny) #9586 PR-URL: #9736

This LTS release comes with 108 commits. This includes 30 which are doc related, 28 which are test related, 16 which are build / tool related, and 4 commits which are updates to dependencies. Notable Changes: The SEMVER-MINOR changes include: * build: - export openssl symbols on Windows making it possible to build addons linked against the bundled version of openssl (Alex Hultman) nodejs/node#7576 * debugger: - make listen address configurable in the debugger server (Ben Noordhuis) nodejs/node#3316 * dgram: - generalized send queue to handle close fixing a potential throw when dgram socket is closed in the listening event handler. (Matteo Collina) nodejs/node#7066 * http: - Introduce the 451 status code "Unavailable For Legal Reasons" (Max Barinov) nodejs/node#4377 * tls: - introduce `secureContext` for `tls.connect` which is useful for caching client certificates, key, and CA certificates. (Fedor Indutny) nodejs/node#4246 Notable SEMVER-PATCH changes include: * build: - introduce the configure --shared option for embedders (sxa555) nodejs/node#6994 * gtest: - the test reporter now outputs tap comments as yamlish (Johan Bergstrom) nodejs/node#9262 * src: - node no longer aborts when c-ares initialization fails (Ben Noordhuis) nodejs/node#8710 * tls: - fix memory leak when writing data to TLSWrap instance during handshake (Fedor Indutny) nodejs/node#9586 PR-URL: nodejs/node#9736 Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>

This LTS release comes with 144 commits. This includes 47 that are docs related, 46 that are test related, 15 which are build / tools related, and 9 commits which are updates to dependencies Notable Changes: * buffer: - coerce slice parameters consistently (Sakthipriyan Vairamani (thefourtheye)) nodejs/node#9101 * deps: - *npm*: - upgrade npm to 3.10.9 (Kat Marchan) nodejs/node#9286 - *V8*: - Various fixes to destructuring edge cases - cherry-pick 3c39bac from V8 upstream (Cristian Cavalli) nodejs/node#9138 - cherry pick 7166503 from upstream v8 (Cristian Cavalli) nodejs/node#9173 * gtest: - the test reporter now outputs tap comments as yamlish (Johan Bergstrom) nodejs/node#9262 * inspector: - inspector now prompts user to use 127.0.0.1 rather than localhost (Eugene Ostroukhov) nodejs/node#9451 * tls: - fix memory leak when writing data to TLSWrap instance during handshake (Fedor Indutny) nodejs/node#9586 PR-URL: nodejs/node#9735 Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>

This LTS release comes with 108 commits. This includes 30 which are doc related, 28 which are test related, 16 which are build / tool related, and 4 commits which are updates to dependencies. Notable Changes: The SEMVER-MINOR changes include: * build: - export openssl symbols on Windows making it possible to build addons linked against the bundled version of openssl (Alex Hultman) nodejs/node#7576 * debugger: - make listen address configurable in the debugger server (Ben Noordhuis) nodejs/node#3316 * dgram: - generalized send queue to handle close fixing a potential throw when dgram socket is closed in the listening event handler. (Matteo Collina) nodejs/node#7066 * http: - Introduce the 451 status code "Unavailable For Legal Reasons" (Max Barinov) nodejs/node#4377 * tls: - introduce `secureContext` for `tls.connect` which is useful for caching client certificates, key, and CA certificates. (Fedor Indutny) nodejs/node#4246 Notable SEMVER-PATCH changes include: * build: - introduce the configure --shared option for embedders (sxa555) nodejs/node#6994 * gtest: - the test reporter now outputs tap comments as yamlish (Johan Bergstrom) nodejs/node#9262 * src: - node no longer aborts when c-ares initialization fails (Ben Noordhuis) nodejs/node#8710 * tls: - fix memory leak when writing data to TLSWrap instance during handshake (Fedor Indutny) nodejs/node#9586 PR-URL: nodejs/node#9736 Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>

This LTS release comes with 144 commits. This includes 47 that are docs related, 46 that are test related, 15 which are build / tools related, and 9 commits which are updates to dependencies Notable Changes: * buffer: - coerce slice parameters consistently (Sakthipriyan Vairamani (thefourtheye)) nodejs/node#9101 * deps: - *npm*: - upgrade npm to 3.10.9 (Kat Marchan) nodejs/node#9286 - *V8*: - Various fixes to destructuring edge cases - cherry-pick 3c39bac from V8 upstream (Cristian Cavalli) nodejs/node#9138 - cherry pick 7166503 from upstream v8 (Cristian Cavalli) nodejs/node#9173 * gtest: - the test reporter now outputs tap comments as yamlish (Johan Bergstrom) nodejs/node#9262 * inspector: - inspector now prompts user to use 127.0.0.1 rather than localhost (Eugene Ostroukhov) nodejs/node#9451 * tls: - fix memory leak when writing data to TLSWrap instance during handshake (Fedor Indutny) nodejs/node#9586 PR-URL: nodejs/node#9735 Signed-off-by: Ilkka Myller <ilkka.myller@nodefield.com>

nodejs-github-bot added the tools Issues and PRs related to the tools directory. label Oct 24, 2016

bnoordhuis requested changes Oct 25, 2016

View reviewed changes

jbergstroem force-pushed the feature/testpy-tap13 branch 2 times, most recently from 5e63d7e to fe70f13 Compare October 25, 2016 16:30

jbergstroem force-pushed the feature/testpy-tap13 branch from fe70f13 to 0a83232 Compare November 3, 2016 20:27

bnoordhuis requested changes Nov 4, 2016

View reviewed changes

gibfahn reviewed Nov 4, 2016

View reviewed changes

gibfahn approved these changes Nov 4, 2016

View reviewed changes

bnoordhuis approved these changes Nov 8, 2016

View reviewed changes

jbergstroem added 3 commits November 10, 2016 15:22

gitignore: ignore all tap files

fb05e31

We now have multiple tap producers; just ignore all files with the `.tap` extension. PR-URL: nodejs#9262 Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Myles Borins <myles.borins@gmail.com>

mscdex mentioned this pull request Nov 18, 2016

backport: cctest TAP output changes #9682

Closed

2 tasks

MylesBorins added dont-land-on-v4.x and removed lts-watch-v4.x labels Nov 18, 2016

gdams pushed a commit to gdams/citgm that referenced this pull request Nov 21, 2016

reporter: update to use TAP 13

53796bd

Use yamlish instead of # to match community: nodejs/node#9262

gdams pushed a commit to gdams/citgm that referenced this pull request Nov 21, 2016

reporter: update to use TAP 13

d864ce3

Use yamlish instead of # to match community: nodejs/node#9262

gdams pushed a commit to gdams/citgm that referenced this pull request Nov 21, 2016

reporter: update to use TAP 13

e1deaf4

Use yamlish instead of # to match community: nodejs/node#9262

gdams pushed a commit to gdams/citgm that referenced this pull request Nov 21, 2016

reporter: update to use TAP 13

d367831

Use yamlish instead of # to match community: nodejs/node#9262

gdams pushed a commit to gdams/citgm that referenced this pull request Nov 21, 2016

reporter: update to use TAP 13

50bed10

Use yamlish instead of # to match community: nodejs/node#9262

This was referenced Nov 22, 2016

v6.9.2 proposal #9735

Merged

v4.7.0 proposal #9736

Merged

gdams pushed a commit to gdams/citgm that referenced this pull request Nov 24, 2016

reporter: update to use TAP 13

5f4d033

Use yamlish instead of # to match community: nodejs/node#9262

gdams pushed a commit to gdams/citgm that referenced this pull request Nov 24, 2016

reporter: update to use TAP 13

2fff561

Use yamlish instead of # to match community: nodejs/node#9262

MylesBorins pushed a commit to nodejs/citgm that referenced this pull request Nov 24, 2016

reporter: update to use TAP 13

807ad62

Use yamlish instead of # to match community: nodejs/node#9262

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test.py tap13 output #9262

test.py tap13 output #9262

jbergstroem commented Oct 24, 2016 •

edited

Loading

bnoordhuis commented Oct 24, 2016

jbergstroem commented Oct 25, 2016

jbergstroem commented Oct 25, 2016

jbergstroem commented Oct 25, 2016

bnoordhuis Oct 25, 2016

jbergstroem Oct 25, 2016

bnoordhuis Oct 25, 2016

jbergstroem Oct 25, 2016

jbergstroem Nov 4, 2016

bnoordhuis Oct 25, 2016

jbergstroem Oct 25, 2016

jbergstroem commented Oct 26, 2016 •

edited

Loading

gibfahn commented Oct 26, 2016 •

edited

Loading

gibfahn commented Oct 26, 2016

jbergstroem commented Oct 26, 2016

gibfahn commented Oct 26, 2016

jbergstroem commented Nov 4, 2016

gibfahn commented Nov 4, 2016 •

edited

Loading

jbergstroem commented Nov 4, 2016

bnoordhuis left a comment

bnoordhuis Nov 4, 2016

jbergstroem Nov 4, 2016

bnoordhuis Nov 4, 2016

bnoordhuis Nov 4, 2016

gibfahn Nov 4, 2016

jbergstroem Nov 4, 2016

gibfahn commented Nov 4, 2016

jbergstroem commented Nov 4, 2016

jbergstroem commented Nov 8, 2016

bnoordhuis left a comment

jbergstroem commented Nov 8, 2016

jbergstroem commented Nov 8, 2016

MylesBorins commented Nov 8, 2016

test.py tap13 output #9262

test.py tap13 output #9262

Conversation

jbergstroem commented Oct 24, 2016 • edited Loading

bnoordhuis commented Oct 24, 2016

jbergstroem commented Oct 25, 2016

jbergstroem commented Oct 25, 2016

jbergstroem commented Oct 25, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbergstroem commented Oct 26, 2016 • edited Loading

gibfahn commented Oct 26, 2016 • edited Loading

gibfahn commented Oct 26, 2016

jbergstroem commented Oct 26, 2016

gibfahn commented Oct 26, 2016

jbergstroem commented Nov 4, 2016

gibfahn commented Nov 4, 2016 • edited Loading

Background

jbergstroem commented Nov 4, 2016

bnoordhuis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gibfahn commented Nov 4, 2016

jbergstroem commented Nov 4, 2016

jbergstroem commented Nov 8, 2016

bnoordhuis left a comment

Choose a reason for hiding this comment

jbergstroem commented Nov 8, 2016

jbergstroem commented Nov 8, 2016

MylesBorins commented Nov 8, 2016

jbergstroem commented Oct 24, 2016 •

edited

Loading

jbergstroem commented Oct 26, 2016 •

edited

Loading

gibfahn commented Oct 26, 2016 •

edited

Loading

gibfahn commented Nov 4, 2016 •

edited

Loading