Release proposal: v1.6.3

[rvagg]

I'd really like to see some work on timers get merged, specificially #1152 (@Fishrock123) and maybe also #1231 (@silverwind). Anecdotally I'm hearing challenges with io.js adoption because of "timer bugs".

A release tomorrow is doable.

• [7dd5e824be] - assert: simplify logic of testing buffer equality (Alex Yursha) #1171
• [a2ea16838f] - debugger: don't spawn child process in remote mode (Jackson Tian) #1282
• [2752da4b64] - deps: make node-gyp work with io.js (cjihrig) #990
• [f166cdecf1] - deps: upgrade npm to 2.7.4 (Forrest L Norvell)
• [318d9d8fd7] - deps: upgrade v8 to 4.1.0.27 (Ben Noordhuis) #1289
• [269e46be37] - deps: make node-gyp work with io.js (cjihrig) #990
• [b542fb94a4] - deps: upgrade npm to 2.7.3 (Forrest L Norvell) #1219
• [73de13511d] - doc: add WG links in WORKING_GROUPS.md & fix nits (Farrin Reid) #1113
• [19641b17be] - doc: decouple sidebar scrolling (Roman Reiss) #1274
• [dbccf8d3ed] - doc: fix spelling error in feature flags (Phillip Lamplugh) #1286
• [5e609e9324] - _Revert_ "doc: clarify real name requirement" (Jeremiah Senkpiel) #1276
• [45814216ee] - doc: fix format docs discrepancy (Brendan Ashworth) #1255
• [4e9bf93e9c] - doc: clarify real name requirement (Roman Reiss) #1250
• [e84dd5f651] - doc: document repl on-demand module loading (Roman Reiss) #1249
• [c9207f7fc2] - fs: fix corruption in writeFile and writeFileSync (Olov Lassus) #1063
• [2db758c562] - iojs: introduce internal modules (Vladimir Kurchatkin) #848
• [36f017afaf] - js2c: fix module id generation on windows (Ben Noordhuis) #1281
• [1832743e18] - lib: add missing new for errors lib/*.js (Mayhem) #1246
• [ea37ac04f4] - src: ignore ENOTCONN on shutdown race with child (Ben Noordhuis) #1214
• [f06b16f2e9] - src: fix minor memleak in preload-modules (Ali Ijaz Sheikh) #1265
• [2903410aa8] - src: don't lazy-load timer globals (Ben Noordhuis) #1280
• [2e5b87a147] - src: remove unnecessary environment lookups (Ben Noordhuis) #1238
• [7e88a9322c] - src: make accessors immune to context confusion (Ben Noordhuis) #1238
• [c8fa8ccdbc] - streams: use strict on _stream_wrap (Brendan Ashworth) #1279
• [8a945814dd] - string_decoder: optimize write() (Brian White) #1209
• [8d1c87ea0a] - test: fix race in parallel/test-vm-debug-context (Ben Noordhuis) #1294
• [955c1508da] - test: reduce sequential/test-fs-watch flakiness (Roman Reiss) #1275
• [77c2da10fd] - timers: make Timer.close idempotent (Petka Antonov) #1288
• [776b73b243] - timers: cleanup interval handling (Jeremiah Senkpiel) #1272
• [caf0b36de3] - timers: assure setTimeout callback only runs once (Roman Reiss) #1231
• [2ccc8f3970] - tls_wrap: fix this incredibly stupid leak (Fedor Indutny) #1244
• [e74b5d278c] - tls_wrap: fix BIO leak on SSL error (Fedor Indutny) #1244
• [ba93c583bc] - win,node-gyp: optionally allow node.exe/iojs.exe to be renamed (Bert Belder) #1266
• [08acf1352c] - win,node-gyp: make delay-load hook optional (Bert Belder) #1266
• [3d46fefe0c] - win,node-gyp: allow node.exe/iojs.exe to be renamed (Bert Belder) #1251

[piscisaureus]

I'd like to get #1251 in, because the .exe naming issue causes so many issues that it forced me to actually fix it myself :). But don't hold up the release for it.

[silverwind]

I'll try to finish #1231 today, just have to make sure it causes no side effects.

[alanpurple]

I'm wating for this, thanks again 👍 :)

[Fishrock123]

#1152 will likely not be ready for this.

[silverwind]

#1231 is ready to merge from my point. Edit: It has landed.

[Fishrock123]

#1075 (comment) -- we should get this out tomorrow #1244 is making a huge difference for the leak.

[Fishrock123]

Blocked by the entire windows test suite failing due to a bug in internal modules: #848 (comment)

[rvagg]

Ouch, we should revert that unless a fix gets in within the next day or so.

/cc @vkurchatkin

[Fishrock123]

New CI post 36f017a (internal module fix): https://jenkins-iojs.nodesource.com/view/iojs/job/iojs+any-pr+multi/

[Fishrock123]

7 specific tests have been timing out for the last several runs on win2012r2 only. Investigating.
Update: maybe just jenkins. I had an out of hdd/ram failure earlier.

https://gist.github.com/Fishrock123/be271ab4b9bbf393b23a

[Fishrock123]

#1152 is broken and should not go in. #1152 (comment)

@rvagg I think we should be mostly ok for a release if jenkins looks fine after a reboot.

Maybe just get #1285 (npm@2.7.4 in? I'll probably merge before the release if it's not in yet anyways.)

[rvagg]

https://jenkins-iojs.nodesource.com/job/iojs+any-pr+multi/395/ test run

[Fishrock123]

Same 6 (7 - 1) sort of timeouts we've been seeing on win2012r2 recently..

[rvagg]

https://jenkins-iojs.nodesource.com/job/iojs+any-pr+multi/396/ let's see if a reboot of those machines does anything, it has fixed windows timeouts in the past

[rvagg]

https://jenkins-iojs.nodesource.com/job/iojs+any-pr+multi/399/

I've cleaned out the workspaces on all the windows machines and rebooted them all .. I think this is going to have to be a regular thing with Windows slaves

[rvagg]

ugh, a mess, still getting timeouts

[rvagg]

I'm not confident pushing ahead on this with those Windows errors as they are, I'd like to know if they are persistent or related to this batch of commits. Does anyone have any data on this before I start getting messy on a Windows VM to understand?

[Fishrock123]

@rvagg best I can get off the CI is post 1.6.2, and likely pre- internal-modules.

[rvagg]

I'm compiling and running on one of the 2008 CI machines and can't reproduce, looks like the same running-within-Jenkins problems that have been solvable in the past with restarts and workspace cleans. Perhaps I'll push forward to release then.

[Fishrock123]

@rvagg Try on 2012? I only see the timeouts on 2012...

[jbergstroem]

@rvagg have you seen any trailing processes from running the tests on those windows machines? I had similar issues on old revisions on freebsd which messed up a lot of tests.

[rvagg]

2015-03-29, Version 1.6.3, @rvagg

Notable changes

• fs: corruption can be caused by fs.writeFileSync() and append-mode fs.writeFile() and fs.writeFileSync() under certain circumstances, reported in #1058, fixed in #1063 (Olov Lassus).
• iojs: an "internal modules" API has been introduced to allow core code to share JavaScript modules internally only without having to expose them as a public API, this feature is for core-only #848 (Vladimir Kurchatkin).
• timers: two minor problems with timers have been fixed:
  • Timer#close() is now properly idempotent #1288 (Petka Antonov).
  • setTimeout() will only run the callback once now after an unref() during the callback #1231 (Roman Reiss).
  • NOTE: there are still other unresolved concerns with the timers code, such as #1152.
• Windows: a "delay-load hook" has been added for compiled add-ons on Windows that should alleviate some of the problems that Windows users may be experiencing with add-ons in io.js #1251 (Bert Belder).
• V8: minor bug-fix upgrade for V8 to 4.1.0.27.
• npm: upgrade npm to 2.7.3. See npm CHANGELOG.md for details. Summary:
  • 1549106 #7641 Due to 448efd0, running npm shrinkwrap --dev caused production dependencies to no longer be included in npm-shrinkwrap.json. Whoopsie! (@othiym23)
  • fb0ac26 #7579 Only block removing files and links when we're sure npm isn't responsible for them. This change is hard to summarize, because if things are working correctly you should never see it, but if you want more context, just go read the commit message, which lays it all out. (@othiym23)
  • 051c473 #7552 bundledDependencies are now properly included in the installation context. This is another fantastically hard-to-summarize bug, and once again, I encourage you to read the commit message if you're curious about the details. The snappy takeaway is that this unbreaks many use cases for ember-cli. (@othiym23)
  • fe1bc38#7672 npm-registry-client@3.1.2: Fix client-side certificate handling by correcting property name. (@atamon)
  • 89ce829#7630 hosted-git-info@1.5.3: Part 3 of ensuring that GitHub shorthand is handled consistently. (@othiym23)
  • 63313eb#7630 realize-package-specifier@2.2.0: Part 2 of ensuring that GitHub shorthand is handled consistently. (@othiym23)
  • 3ed41bf#7630 npm-package-arg@3.1.1: Part 1 of ensuring that GitHub shorthand is handled consistently. (@othiym23)

Known issues

• Some problems exist with timers and unref() still to be resolved. See #1152.
• Possible small memory leak(s) may still exist but have yet to be properly identified, details at #1075.
• Surrogate pair in REPL can freeze terminal #690
• Not possible to build io.js as a static library #686
• process.send() is not synchronous as the docs suggest, a regression introduced in 1.0.2, see #760 and fix in #774
• Calling dns.setServers() while a DNS query is in progress can cause the process to crash on a failed assertion #894

Commits

[rvagg]

GitHub's DDoS situation is going to cause problems for this release, seems to be impacting them pretty hard right now and the build slaves are feeling it too

[silverwind]

setTimeout() will only run the callback once now after an unref() and re-ref()

please correct to:

setTimeout() will only run the callback once now after an unref() during the callback

[rvagg]

Got another test run going: https://jenkins-iojs.nodesource.com/job/iojs+any-pr+multi/401/ tho there's two git failures in there already.

Attempting a nightly: https://jenkins-iojs.nodesource.com/job/iojs+release+nightly/118/

[jbergstroem]

@rvagg thinking ddos-related?

[Fishrock123]

New CI, hopefully Github survives: https://jenkins-iojs.nodesource.com/job/iojs+any-pr+multi/412/

[silverwind]

ARMv6 and ARMv7 failures are mostly timeout related. I intend to look into these, but I fear we'd probably need platform-specific timeout values for slow platforms. ARMv8 on the other hand looks to be a whole new can of worms.

[bnoordhuis]

ARMv8 is currently compiled without openssl (i.e. crypto and tls) support pending an upgrade to 1.0.2 and not all tests handle that as graciously as they should. I also suspect that there are a few libuv bugs that need to be addressed in order to fix the failing child process tests.

[Fishrock123]

ARMv* and *-wheezy are currently not considered blockers, afaik.

[jbergstroem]

@bnoordhuis I can't reproduce any failing tests with --without-ssl.

[rvagg]

lets give this another go eh?

test: https://jenkins-iojs.nodesource.com/job/iojs+any-pr+multi/416/

nightly: https://jenkins-iojs.nodesource.com/job/iojs+release+nightly/120/ - https://iojs.org/download/nightly/v1.6.3-nightly20150331ba93c583bc/ (armv6 problems should be fixed)

[rvagg]

nightly again because I had armv6 instead of armv6l in ARCH

[rvagg]

This is new, on OSX:

=== release test-http-destroyed-socket-write2 ===
Path: parallel/test-http-destroyed-socket-write2
assert.js:88
  throw new assert.AssertionError({
        ^
AssertionError: false == true
    at test (/Users/iojs/build/workspace/iojs+any-pr+multi/nodes/osx1010/test/parallel/test-http-destroyed-socket-write2.js:89:5)
    at Immediate.write [as _onImmediate] (/Users/iojs/build/workspace/iojs+any-pr+multi/nodes/osx1010/test/parallel/test-http-destroyed-socket-write2.js:29:7)
    at processImmediate [as _immediateCallback] (timers.js:361:17)
Command: out/Release/iojs /Users/iojs/build/workspace/iojs+any-pr+multi/nodes/osx1010/test/parallel/test-http-destroyed-socket-write2.js

Running again to see if it's persistent: https://jenkins-iojs.nodesource.com/job/iojs+any-pr+multi/417/

[rvagg]

on OSX, second attempt:

=== release test-fs-readfilesync-pipe-large ===
Path: parallel/test-fs-readfilesync-pipe-large
assert.js:88
  throw new assert.AssertionError({
        ^
AssertionError: it reads the file and outputs it
    at /Users/iojs/build/workspace/iojs+any-pr+multi/nodes/osx1010/test/parallel/test-fs-readfilesync-pipe-large.js:31:3
    at ChildProcess.exithandler (child_process.js:707:7)
    at emitTwo (events.js:87:13)
    at ChildProcess.emit (events.js:169:7)
    at maybeClose (child_process.js:984:16)
    at Process.ChildProcess._handle.onexit (child_process.js:1057:5)
Command: out/Release/iojs /Users/iojs/build/workspace/iojs+any-pr+multi/nodes/osx1010/test/parallel/test-fs-readfilesync-pipe-large.js

Not great but different errors on both runs so I'm not going to let this hold up release.

[rvagg]

tagged and building @ https://jenkins-iojs.nodesource.com/job/iojs+release/51/

[rvagg]

done & landed @ https://iojs.org/dist/latest/

@iojs/website no need to act on this release, I hooked up the build-on-release thing we were testing so it's already rebuilt for you

@iojs/collaborators:

Thanks for the work in this release, unfortunately there's a bunch of intermittent failures in our CI that keep on popping up, and we have the strange TIMEOUTs on Windows that have yet to be figured out (happen when running in Jenkins). It'd be great if we could have a bit of focus on sorting out the tests across all platforms so the CI is more useful and informative than it is now. I know a lot of you don't have access to it yet and that's mostly my fault, I'll try and get emails out to everyone this week so you can start using it.

I'm personally a little concerned about the leak report (#1075) still not being sorted out, and the serious timers problems (#1151 and #1264) causing a lack of trust in our ability to deliver a stable and quality platform because I've heard feedback about both of these things from outsiders and users alike. We need to all try hard to make sure that detractors have no technical reasons to detract and those on the fence don't have technical reasons not to start deploying io.js.

[silverwind]

My RPi2 should arrive tomorrow, and then I'll look into adding platform-specific test timeout values for ARMv6 and ARMv7 so we get meaningful results on these CI runs.

[indutny]

@rvagg we resolved most of the aspects of the leak. It is only a matter of reproducing it/gathering more information until we'll fix it once and for all.

[Fishrock123]

Most of #1075 was fixed in 2db758c...2ccc8f3 (see: #1075 (comment))

Timer issues are probably just going to be solved by a re-write from @bnoordhuis. (all of which (and more) also exist in node 0.12, and some possibly worse now.)

[rvagg]

Timer issues are probably just going to be solved by a re-write from @bnoordhuis.

Yes, I've heard this but it makes me afraid that we're punting on fixing stuff with the dream of a rewrite, I know Ben's probably the most productive of us here but let's not defer simpler fixes with the hope that a rewrite will drop some day soon because we may end up waiting for a while because we have to add together coding + reviewing + testing which adds up for a large amount of code (think how long the Promises PR took to get in!)

(all of which (and more) also exist in node 0.12, and some possibly worse now.)

Yes, but primarily we're competing against 0.10 right now, it's where most significant deployments are stuck while there's a sense of limbo around 0.12 and io.js and 0.12 has a natural momentum in most people's minds (as strange as that may sound to us here!) so we just need to do 10x better.

[Fishrock123]

but let's not defer simpler fixes with the hope that a rewrite will drop some day soon

if only timers were that simple :P