buffer: move checkFloat from lib into src #3763

matthewloring · 2015-11-11T02:31:54Z

The type and range checks performed by this function can be done more
efficiently in native code.

thefourtheye · 2015-11-11T02:57:21Z

src/node_buffer.cc

+  Environment* env = Environment::GetCurrent(args);
+
+  bool noAssert = args[3]->BooleanValue();
+  if (!noAssert) {


You can avoid double negative stuff here, if you simply name it as should_assert or something like that.

It is named this way to maintain consistency with the existing JS api.

@matthewloring I think the convention in C++ is to use snake case for local variables and I am more interested in something like this

bool should_assert = !args[3]->BooleanValue(); ... if (should_assert) { ... }

It is convention to use snake_case in native code, and since I've been bitten in the butt by it before (in buffer code actually) I'll have to agree with @thefourtheye's observation of avoiding double negative is a better option.

Also, we'll need to do some benchmarking on BooleanValue(). Last time I used it (way way back in v0.10) the performance was bad enough that we actually prefixed the check with IsUndefined(). So may look something like this:

bool should_assert = !args[3]->IsUndefined() && !args[3]->BooleanValue();

Think that's right, but it's getting late.

I believe the condition should be args[3]->IsUndefined() || !args[3]->BooleanValue(). Alternatively, we could use !! on the JS side to ensure we only get booleans at this point the same way the other arguments are type coerced value = +value; offset = offset >>> 0;. Do you have a preference between these?

In the mean time, I've fixed the case and negation issues here.

These v8::Value::XValue methods are scheduled for deprecation in favor of the Maybe versions.

I've updated the value accesses in this function to use the new version. It looks like the non-Maybe versions are still used elsewhere throughout the file.

@matthewloring From my distant memory it was indeed faster to coerce noAssert in JS. Then you can do a simple IsTrue() check, and drop the other two.

@trevnorris I've moved the boolean coercion into JS.

trevnorris · 2015-11-11T07:31:42Z

src/node_buffer.cc

-  CHECK_LE(offset + sizeof(T), ts_obj_length);
+  if (!noAssert) {
+    CHECK_NOT_OOB(offset + sizeof(T) <= ts_obj_length);
+  }


Drop the CHECK_LE below the conditional check. We still want to abort to prevent writing to memory bits we don't have control over.

NM this comment. Logic will change w/ another upcoming PR.

trevnorris · 2015-11-11T07:40:14Z

@matthewloring Awesome job. Thanks much for taking care of this. Has been one of those changes in my backlog that never taken the time to get to. Since this is all about micro-optimization (we'll only be shaving off 20ns or so here) there may be some experimentation we'll need to do

trevnorris · 2015-11-11T07:50:19Z

@matthewloring There's going to be a collision with #3767. It only touches the location where you removed the CHECK_LE. Mind if I proceed with that one, the we rebase this one and continue to make things fast?

EDIT: Ref'd issue was this one. Has been fixed.

matthewloring · 2015-11-11T07:52:52Z

@trevnorris It should be an easy merge conflict to resolve. You're welcome to land #3767.

trevnorris · 2015-11-11T23:31:40Z

lib/buffer.js

-    checkFloat(this, val, offset, 4);
-  binding.writeFloatBE(this, val, offset);
+  noAssert = !!noAssert;
+  binding.writeFloatBE(this, val, offset, noAssert);


sorry, minor nit: do the boolean coercion in the call itself. e.g. writeFloatBE(this, val, offset, !!noAssert);

trevnorris · 2015-11-11T23:33:32Z

Looks great. As soon as I can land the previously mentioned patch, get this rebased and CI is happy we'll be good to land.

LGTM.

kkoopa · 2015-11-11T23:45:29Z

src/node_buffer.cc

-  T val = args[1]->NumberValue();
-  uint32_t offset = args[2]->Uint32Value();
-  CHECK_LE(offset + sizeof(T), ts_obj_length);
+  T val = args[1]->NumberValue(env->context()).FromMaybe(0);


Why does this function take the T parameter? T is used as the return type for v8::Value::NumberValue::FromMaybe, which returns a double.

val may either be a double or float.

jasnell · 2015-11-12T16:59:38Z

LGTM

jasnell · 2015-11-12T17:00:41Z

@trevnorris ... as an optimization, I'm inclined not to flag this for LTS but would like your input on that.

trevnorris · 2015-11-12T17:09:58Z

@jasnell Sounds reasonable. I think this is low risk, but also not critical.

jasnell · 2015-11-14T18:26:20Z

CI Failure: https://ci.nodejs.org/job/node-test-commit-linux/1198/nodes=centos5-32/console

matthewloring · 2015-11-15T02:45:08Z

Looks like this may be integer overflow on 32-bit architectures? I can verify and put together a fix.

trevnorris · 2015-11-16T23:10:56Z

@matthewloring ping me when you have a fix, or have a question, and will run CI again.

matthewloring · 2015-11-16T23:17:49Z

@trevnorris I've added the lower bounds check which I believe to fix the problem but do not have a centos 32 vm to reproduce the initial failure on. Are there existing overflow prevention checks elsewhere inside buffer that I should emulate or does the added check CHECK_NOT_OOB(offset + sizeof(T) >= sizeof(T)); at line 745 seem good? If so, it's ready for another CI.

trevnorris · 2015-11-16T23:30:19Z

Awesome. CI: https://ci.nodejs.org/job/node-test-pull-request/754/

matthewloring · 2015-11-17T03:26:19Z

The test-crypto-dh.js timeouts are happening on a few machines but it should be unrelated.

trevnorris · 2015-11-17T04:35:11Z

@matthewloring Just landed the other conflicting PR. Mind rebasing on latest master?

The other PR alerted me to the specific wording in the current documentation:

Set noAssert to true to skip validation of value and offset. This means
that value may be too large for the specific function and offset may be
beyond the end of the buffer leading to the values being silently dropped.

So I believe if noAssert is set and the offset is beyond the bound of the buffer we simply return early. Though I'd still leave in that specific CHECK just to make sure any future modifications don't accidentally allow writing beyond memory bounds.

matthewloring · 2015-11-17T06:38:16Z

@trevnorris I've rebased and reinserted the CHECK_LE just as a failsafe. Should be good to go.

trevnorris · 2015-11-17T07:21:08Z

Awesome. CI: https://ci.nodejs.org/job/node-test-pull-request/760/

matthewloring · 2015-11-17T15:24:17Z

Failures (mostly crypto timeouts) should be unrelated flakes.

trevnorris · 2015-11-18T22:12:20Z

@matthewloring We'll need to do some investigation. The call is slower than it is now. This is the fun part. :) I'll update if I find any details. Here's a simple script I'm using:

'use strict';
const ITER = 1e7;
var b = new Buffer(8);
var t = process.hrtime();
for (var i = 0; i < ITER; i++) {
  b.writeDoubleLE(0, 0, true);
}
t = process.hrtime(t);
console.log(((t[0] * 1e9 + t[1]) / ITER).toFixed(1) + ' ns/op');

(note: this is fine b/c the call into native isn't affected by the optimizing compiler like JS would be)

matthewloring · 2015-11-18T22:37:37Z

Hmm, I can look into this as well.

matthewloring · 2015-11-18T23:17:48Z

It looks like at least part of the slow down is caused by the new maybe versions of NumberValue and Uint32Value. I see ~53 ns/op with the maybe versions compared to ~46 ns/op with the old versions of NumberValue and Uint32Value and ~44 ns/op with the checks remaining in JS (at master).

trevnorris · 2015-11-18T23:18:23Z

@matthewloring Here's a diff gaining back the lost performance: https://gist.github.com/trevnorris/3bce2822893a98e9d971

Sorry, this is where writing performant code makes things look worse.

trevnorris · 2015-11-18T23:20:09Z

The new Maybe versions definitely aren't helping. Also getting the values out (e.g. NumberValue() are taking longer than I'd expect).

The type and range checks performed by this function can be done more efficiently in native code.

matthewloring · 2015-11-18T23:30:42Z

That change brings script down to ~42 for me. I've applied your change.

trevnorris · 2015-11-19T22:30:35Z

CI: https://ci.nodejs.org/job/node-test-pull-request/792/

matthewloring · 2015-11-19T22:59:35Z

Only failure is a child process fork connection reset, seems good to me.

trevnorris · 2015-11-20T19:24:06Z

Thanks much! Landed in 22478d3.

The type and range checks performed by this function can be done more efficiently in native code. PR-URL: #3763 Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

rvagg · 2016-01-15T05:41:49Z

Marking with dont-land-on-v4.x but noting that above discussion suggests that this could be a candidate if we changed our risk profile for LTS.

thefourtheye reviewed Nov 11, 2015
View reviewed changes

thefourtheye added the buffer Issues and PRs related to the buffer subsystem. label Nov 11, 2015

trevnorris reviewed Nov 11, 2015
View reviewed changes

kkoopa reviewed Nov 11, 2015
View reviewed changes

trevnorris self-assigned this Nov 12, 2015

trevnorris added the land-on-v5.x label Nov 12, 2015

buffer: move checkFloat from lib into src

358a440

The type and range checks performed by this function can be done more efficiently in native code.

trevnorris closed this Nov 20, 2015

Fishrock123 mentioned this pull request Nov 26, 2015

deps: upgrade npm to 3.5.0 #4032

Closed

This was referenced Dec 8, 2015

Release proposal: v5.2.0 #4180

Closed

Release proposal: v5.2.0 (Stable) #4181

Merged

rvagg mentioned this pull request Dec 17, 2015

Release 4.2.4 Planning #4321

Closed

MylesBorins mentioned this pull request Jan 14, 2016

Audit commits not found on v4.x-staging #4698

Closed

rvagg added the dont-land-on-v4.x label Jan 15, 2016

matthewloring deleted the buffer-check branch February 25, 2016 19:53

ChALkeR added the performance Issues and PRs related to the performance of Node.js. label Mar 8, 2016

buffer: move checkFloat from lib into src #3763

buffer: move checkFloat from lib into src #3763

Conversation

matthewloring commented Nov 11, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trevnorris commented Nov 11, 2015

trevnorris commented Nov 11, 2015

matthewloring commented Nov 11, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trevnorris commented Nov 11, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jasnell commented Nov 12, 2015

jasnell commented Nov 12, 2015

trevnorris commented Nov 12, 2015

jasnell commented Nov 14, 2015

matthewloring commented Nov 15, 2015

trevnorris commented Nov 16, 2015

matthewloring commented Nov 16, 2015

trevnorris commented Nov 16, 2015

matthewloring commented Nov 17, 2015

trevnorris commented Nov 17, 2015

matthewloring commented Nov 17, 2015

trevnorris commented Nov 17, 2015

matthewloring commented Nov 17, 2015

trevnorris commented Nov 18, 2015

matthewloring commented Nov 18, 2015

matthewloring commented Nov 18, 2015

trevnorris commented Nov 18, 2015

trevnorris commented Nov 18, 2015

matthewloring commented Nov 18, 2015

trevnorris commented Nov 19, 2015

matthewloring commented Nov 19, 2015

trevnorris commented Nov 20, 2015

rvagg commented Jan 15, 2016