Clarify when nextTickQueue is processed #1804

OmarDarwish · 2018-09-12T02:14:04Z

It was a little unclear for me when the nextTickQueue is processed (see this StackOverflow question). After some experimentation and more googling, I now understand the mechanics of this queue.

I'm hoping to make this a little more clear for the next reader!

ghost · 2018-09-12T09:22:54Z

/cc：@nodejs/documentation. Please have a careful check about this detailled describtions :)

a0viedo · 2018-09-12T17:12:04Z

I think this is a better description but I'm no event loop expert. pinging people who can shed some light @addaleax @mcollina @Fishrock123

mcollina

LGTM

sam-github · 2018-09-12T17:36:04Z

I think you may have found a problem, the current text may not be correct for all types of operations, but I am not sure the replacement is correct either. Perhaps not all operations are the same?

The original text used "operations" because not all phases contain events. Operations is a more generic term for what phases do:

While each phase is special in its own way, generally, when the event loop enters a given phase, it will perform any operations specific to that phase,

"events" in the sense of the event loop are read from the kernel only in the Poll phase, so the replacement shouldn't replace operations with events, they aren't the same.

I noticed what you noticed with your example code, too, when I wrote some exploratory code, but I think its particular to "immediates". In that case, it appears the entire immediate queue is executed sequentially, and then any ticks that were queued by the immediates are executed sequentially.

In contrast, when an I/O event causes a callback to be executed, and that callback calls nextTick(), my understanding is that the tick queue is guaranteed to be executed before any other I/O callback is executed. Normally on a lightly loaded system this happens because there is never more than one event processed per Poll phase, but even on a loaded system where libuv reads more than one event from the kernel with epoll() in a single shot, I believe this guarantee holds.

mcollina · 2018-09-12T17:40:53Z

An I/O event triggers some JavaScript to be executed. At the end of that execution, the functions in the nextTick queue are executed.

sam-github · 2018-09-12T17:49:49Z

@mcollina That is my understanding as well, so the change proposed in this PR is incorrect. It says that the nextTick queue won't be executed until the end of the entire phase.

@OmarDarwish I think that executing the entire queue of Immediates is a single "operation", from the point of view of clearing the nextTick queue, which isn't documented, and is what you noticed with your test code.

OmarDarwish · 2018-09-12T17:52:11Z

I think its particular to "immediates". In that case, it appears the entire immediate queue is executed sequentially, and then any ticks that were queued by the immediates are executed sequentially.

executing the entire queue of Immediates is a single "operation", from the point of view of clearing the nextTick queue

I'm up for any change in the documentation that makes this particular (edge?) case well known. @sam-github what do you think is a better way to make this easier to understand for the reader?

mcollina · 2018-09-12T17:55:37Z

@mcollina That is my understanding as well, so the change proposed in this PR is incorrect. It says that the nextTick queue won't be executed until the end of the entire phase.

I think we should clarify the phases, and reflect the fact that there is a single transition between C to JS for multiple immediates and timers. In fact, we are deduping them as a form of optimization and this is a side effect of that.

OmarDarwish · 2018-09-13T16:30:02Z

Took a stab at defining an operation, and giving an example of deduplication.

sam-github · 2018-09-13T16:34:52Z

locale/en/docs/guides/event-loop-timers-and-nexttick.md

@@ -190,8 +190,6 @@ _whose time thresholds have been reached_. If one or more timers are
 ready, the event loop will wrap back to the **timers** phase to execute
 those timers' callbacks.

-### check


Did you remove this by accident?

Yep, lemme fix real quick 😅

vsemozhetbyt · 2018-09-13T16:47:28Z

locale/en/docs/guides/event-loop-timers-and-nexttick.md

@@ -20,7 +20,7 @@ this in further detail later in this topic.
 ## Event Loop Explained

 When Node.js starts, it initializes the event loop, processes the
-provided input script (or drops into the [REPL][], which is not covered in
+provided input script (or drops into the [REPL](https://nodejs.org/api/repl.html), which is not covered in


This line exceeds 80 characters limit now.

It seems you can just fix the bottom reference if you prefer this URL.

vsemozhetbyt

Some nits.

vsemozhetbyt · 2018-09-13T17:41:54Z

locale/en/docs/guides/event-loop-timers-and-nexttick.md

@@ -303,6 +306,56 @@ situations because **it allows you to "starve" your I/O by making
 recursive `process.nextTick()` calls**, which prevents the event loop
 from reaching the **poll** phase.

+#### Deduplication
+
+For the `timers` and `check` phases, the there is a *single transition 


the there -> there

vsemozhetbyt · 2018-09-13T17:42:23Z

locale/en/docs/guides/event-loop-timers-and-nexttick.md

+
+For the `timers` and `check` phases, the there is a *single transition 
+between C to JS for multiple immediates and timers*. This deduplication
+is a form of optimzation, which may produce some unexpected side effects.


optimzation -> optimization

vsemozhetbyt · 2018-09-13T17:46:58Z

locale/en/docs/guides/event-loop-timers-and-nexttick.md

+const foo = [1, 2];
+const bar = ['a', 'b'];
+
+foo.forEach( num => {


Unneeded space before num.

vsemozhetbyt · 2018-09-13T17:57:36Z

locale/en/docs/guides/event-loop-timers-and-nexttick.md

+
+At this point, all previously added `setImmediate()` events have been processed.
+The `nextTickQueue` is now checked, and events are processed in FIFO order. When
+these `nextTickQueue` is emptied, the event loop is considers all operations to have 


these -> this
is considers -> considers
to have completed -> to have been completed?

OmarDarwish · 2018-09-14T15:55:30Z

@vsemozhetbyt Thanks for double checking. I forgot to spell check before pushing. Fixing now!

OmarDarwish · 2018-09-14T17:36:41Z

@sam-github @mcollina What do you think? If I understand the contributing policy, if/when you two approve we should be good to go to merge?

mcollina · 2018-09-17T13:24:37Z

I think responsibility for this repo is to @nodejs/website.

dmcghan · 2018-09-17T14:19:57Z

I just realized this doc doesn't discuss promises, which I believe have their own queue and processing timings which make them unique.

async function doWork(char) {
  console.log('promise', char);
}

// dedup.js
const foo = [1, 2];
const bar = ['a', 'b'];

foo.forEach(num => {
  setImmediate(() => {
    console.log('setImmediate', num);
    bar.forEach(async (char) => {
      process.nextTick(() => {
        console.log('process.nextTick', char);
      });
      await doWork(char);
    });
  });
});

setImmediate 1
promise a
promise b
setImmediate 2
promise a
promise b
process.nextTick a
process.nextTick b
process.nextTick a
process.nextTick b

Maybe this PR isn't the right place to address this...

davisjam · 2018-09-17T14:25:50Z

FWIW Here is an academic paper on this by @matthewloring and others.

amiller-gh · 2018-09-17T20:39:48Z

@mcollina, thanks for the ping.

This does fit in well with our redesign / getting started guide work, but I think its okay for the content to live in here for now. We're hoping to keep all getting started guides alongside the source code after we've figured out the docs site content ingestion pipeline, so it'll end up back here anyway. The new content here will be helpful in the meantime.

We'll definitely use this doc as a starting point for for the event loop page on the new site (likely take it wholesale) awesome job @OmarDarwish!

apapirovski · 2018-10-16T15:00:11Z

@dmcghan Your example isn't quite accurate. Promises would normally only resolve after the nextTicks. In your case, you have console.log executing in a sync manner. Meanwhile if you were to put the console.log after the await, the order would be more like:

setImmediate 1
setImmediate 2
process.nextTick a
process.nextTick b
process.nextTick a
process.nextTick b
promise a
promise b
promise a
promise b

Once #22842 lands though, it will be more like the browser:

setImmediate 1
process.nextTick a
process.nextTick b
promise a
promise b
setImmediate 2
process.nextTick a
process.nextTick b
promise a
promise b

Trott · 2018-10-16T16:54:39Z

locale/en/docs/guides/event-loop-timers-and-nexttick.md

-completes, regardless of the current phase of the event loop.
+the `nextTickQueue` will be processed after the current operation is
+completed, regardless of the current phase of the event loop. Here, 
+an **operation** is defined as a transition from the


Nit: Italics here, not bold.

Trott · 2018-10-16T16:55:10Z

locale/en/docs/guides/event-loop-timers-and-nexttick.md

+#### Deduplication
+
+For the `timers` and `check` phases, there is a *single transition 
+between C to JS for multiple immediates and timers*. This deduplication


Nit: No italics here. We overuse italics and bold for emphasis. Just don't do it. 😄

Trott · 2018-10-16T16:57:01Z

Once #22842 lands though, it will be more like the browser:

So in theory, the correct content of these guides is dependent on the version of Node.js...

Trott · 2018-10-16T16:57:21Z

(If this is an improvement over existing text, I'm totally OK with it landing with or without my nits addressed.)

dmcghan · 2018-10-16T18:24:49Z

@apapirovski Interesting. Looking back, I'm not sure I meant to add the await operator. I was trying to add something to the promise queue but I wasn't concerned with it resolving before continuing execution of the current stack.

Maybe this is a bit simpler:

const foo = [1, 2];
const bar = ['a', 'b'];

foo.forEach(num => {
  setImmediate(() => {
    console.log('setImmediate', num);
    bar.forEach(async (char) => {
      process.nextTick(() => {
        console.log('process.nextTick', char);
      });
      new Promise((resolve, reject) => {
        console.log('promise', char);
      });
    });
  });
});

You can reverse the order of the nextTick and Promise calls and the result is the same (expected):

setImmediate 1
promise a
promise b
setImmediate 2
promise a
promise b
process.nextTick a
process.nextTick b
process.nextTick a
process.nextTick b

This output surprises me. I'd have expected the output you showed:

setImmediate 1
setImmediate 2
process.nextTick a
process.nextTick b
process.nextTick a
process.nextTick b
promise a
promise b
promise a
promise b

I'm missing something. Does no

While I'd like to get a better understanding of this behavior, I was really just trying to point out that this level of detail isn't in the doc.

Do you have a link to 22842?

thefourtheye · 2018-10-16T20:37:55Z

locale/en/docs/guides/event-loop-timers-and-nexttick.md

@@ -303,6 +306,56 @@ situations because **it allows you to "starve" your I/O by making
 recursive `process.nextTick()` calls**, which prevents the event loop
 from reaching the **poll** phase.

+#### Deduplication


I am not sure if this is the correct place. The next section answers why the behaviour mentioned in the line above is allowed.

Hmm, you're right. Let me fix this.

locale/en/docs/guides/event-loop-timers-and-nexttick.md

ghost · 2018-12-16T23:43:46Z

@OmarDarwish：There're too many changes here, so can you merge them by using "rebase..." in your local PC and make a push submit?

Not sure whether this post can be merged? I think it's been a long time....

Define an operation. Give example of dedup. Remove empty section Fix trigger happy deletions Remove modififed REPL link Spellcheck Deduplication section Address Nits * Remove italics. * Change italics to bold Move Deduplication sectino down Update locale/en/docs/guides/event-loop-timers-and-nexttick.md Co-Authored-By: OmarDarwish <om.drwsh@gmail.com> Update locale/en/docs/guides/event-loop-timers-and-nexttick.md Co-Authored-By: OmarDarwish <om.drwsh@gmail.com>

OmarDarwish · 2018-12-17T18:29:08Z

@Maledong I've rebased and squashed

ghost · 2018-12-18T23:30:34Z

@OmarDarwish：Thanks!

@watson, @sam-github , @thefourtheye , @Trott ：Any other ideas or suggestions? Can this be merged?

ZYSzys · 2018-12-20T03:26:05Z

locale/en/docs/guides/event-loop-timers-and-nexttick.md

+process.nextTick b
+process.nextTick a
+process.nextTick b
+```


AFAIK, did the output incorrect ?
The output is always as below while my Node.js version is v11.4.0.

setImmediate 1 process.nextTick a process.nextTick b setImmediate 2 process.nextTick a process.nextTick b

So it's indeed as @Trott said:

So in theory, the correct content of these guides is dependent on the version of Node.js...

Maybe we also need to explain this there.

You're right. Looks like something changed for node 11:

bash-3.2$ pbpaste const foo = [1, 2]; const bar = ['a', 'b']; foo.forEach(num => { setImmediate(() => { console.log('setImmediate', num); bar.forEach(char => { process.nextTick(() => { console.log('process.nextTick', char); }); }); }); }); bash-3.2$ nvm use 10.15.0 Now using node v10.15.0 (npm v6.4.1) bash-3.2$ pbpaste | node setImmediate 1 setImmediate 2 process.nextTick a process.nextTick b process.nextTick a process.nextTick b bash-3.2$ nvm use 11.0.0 Now using node v11.0.0 (npm v6.4.1) bash-3.2$ pbpaste | node setImmediate 1 process.nextTick a process.nextTick b setImmediate 2 process.nextTick a process.nextTick b bash-3.2$ nvm use 11.6.0 Now using node v11.6.0 (npm v6.5.0-next.0) bash-3.2$ pbpaste | node setImmediate 1 process.nextTick a process.nextTick b setImmediate 2 process.nextTick a process.nextTick b

It looks like guides is not segregated by version. What would be the best way to indicate that this behavior is only applicable for Node 10 and below?

Also, what changed in node 11?

This is what changed nodejs/node#22842

ghost · 2019-02-01T12:01:55Z

Can this be merged? It's been for ages :)

a0viedo · 2019-02-01T22:45:17Z

I think all changes have been addressed I'm +1 on merging this.

Ref：#1804

mcollina approved these changes Sep 12, 2018

View reviewed changes

BridgeAR approved these changes Sep 12, 2018

View reviewed changes

sam-github reviewed Sep 13, 2018

View reviewed changes

vsemozhetbyt reviewed Sep 13, 2018

View reviewed changes

vsemozhetbyt approved these changes Sep 14, 2018

View reviewed changes

sam-github mentioned this pull request Oct 13, 2018

timers: run nextTicks after each immediate and timer nodejs/node#22842

Closed

3 tasks

ryzokuken approved these changes Oct 16, 2018

View reviewed changes

Trott reviewed Oct 16, 2018

View reviewed changes

thefourtheye reviewed Oct 16, 2018

View reviewed changes

watson reviewed Oct 27, 2018

View reviewed changes

locale/en/docs/guides/event-loop-timers-and-nexttick.md Outdated Show resolved Hide resolved

locale/en/docs/guides/event-loop-timers-and-nexttick.md Outdated Show resolved Hide resolved

OmarDarwish force-pushed the patch-1 branch from 53449b2 to 3317426 Compare December 17, 2018 18:27

sam-github approved these changes Dec 19, 2018

View reviewed changes

ZYSzys reviewed Dec 20, 2018

View reviewed changes

ghost deleted a comment from OmarDarwish Feb 1, 2019

ghost merged commit b440166 into nodejs:master Feb 2, 2019

ghost mentioned this pull request Feb 2, 2019

zh-CN：Trans for 'Clarify when nextTickQueue is processed' #2026

Merged

ghost pushed a commit that referenced this pull request Feb 3, 2019

zh-CN：Trans for 'Clarify when nextTickQueue is processed' (#2026)

c53bc56

Ref：#1804

jilvin mentioned this pull request Apr 14, 2020

Use of process.nextTick() at db/users.js passport/todos-express-password#23

Open

This pull request was closed.

Clarify when nextTickQueue is processed #1804

Clarify when nextTickQueue is processed #1804

Conversation

OmarDarwish commented Sep 12, 2018

ghost commented Sep 12, 2018

a0viedo commented Sep 12, 2018

mcollina left a comment

Choose a reason for hiding this comment

sam-github commented Sep 12, 2018

mcollina commented Sep 12, 2018

sam-github commented Sep 12, 2018

OmarDarwish commented Sep 12, 2018

mcollina commented Sep 12, 2018

OmarDarwish commented Sep 13, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vsemozhetbyt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vsemozhetbyt Sep 13, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

OmarDarwish commented Sep 14, 2018

OmarDarwish commented Sep 14, 2018

mcollina commented Sep 17, 2018

dmcghan commented Sep 17, 2018

davisjam commented Sep 17, 2018

amiller-gh commented Sep 17, 2018

apapirovski commented Oct 16, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Trott commented Oct 16, 2018

Trott commented Oct 16, 2018

dmcghan commented Oct 16, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ghost commented Dec 16, 2018

OmarDarwish commented Dec 17, 2018

ghost commented Dec 18, 2018

ZYSzys Dec 20, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ghost commented Feb 1, 2019

a0viedo commented Feb 1, 2019

vsemozhetbyt Sep 13, 2018 •

edited

Loading

ZYSzys Dec 20, 2018 •

edited

Loading