Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: describe when stdout/err is sync #10884

Merged
merged 1 commit into from
Feb 16, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 8 additions & 17 deletions doc/api/console.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,15 @@ The module exports two specific components:

* A `Console` class with methods such as `console.log()`, `console.error()` and
`console.warn()` that can be used to write to any Node.js stream.
* A global `console` instance configured to write to `stdout` and `stderr`.
Because this object is global, it can be used without calling
* A global `console` instance configured to write to [`process.stdout`][] and
[`process.stderr`][]. The global `console` can be used without calling
`require('console')`.

***Warning***: The global console object's methods are neither consistently
synchronous like the browser APIs they resemble, nor are they consistently
asynchronous like all other Node.js streams. See the [note on process I/O][] for
more information.

Example using the global `console`:

```js
Expand Down Expand Up @@ -47,21 +52,6 @@ myConsole.warn(`Danger ${name}! Danger!`);
// Prints: Danger Will Robinson! Danger!, to err
```

While the API for the `Console` class is designed fundamentally around the
browser `console` object, the `Console` in Node.js is *not* intended to
duplicate the browser's functionality exactly.

## Asynchronous vs Synchronous Consoles

The console functions are usually asynchronous unless the destination is a file.
Disks are fast and operating systems normally employ write-back caching;
it should be a very rare occurrence indeed that a write blocks, but it
is possible.

Additionally, console functions are blocking when outputting to TTYs
(terminals) on OS X as a workaround for the OS's very small, 1kb buffer size.
This is to prevent interleaving between `stdout` and `stderr`.

## Class: Console

<!--type=class-->
Expand Down Expand Up @@ -305,4 +295,5 @@ The `console.warn()` function is an alias for [`console.error()`][].
[`util.format()`]: util.html#util_util_format_format_args
[`util.inspect()`]: util.html#util_util_inspect_object_options
[customizing `util.inspect()` colors]: util.html#util_customizing_util_inspect_colors
[note on process I/O]: process.html#process_a_note_on_process_i_o
[web-api-assert]: https://developer.mozilla.org/en-US/docs/Web/API/console/assert
96 changes: 49 additions & 47 deletions doc/api/process.md
Original file line number Diff line number Diff line change
Expand Up @@ -859,10 +859,11 @@ added: v0.1.13

* `code` {Integer} The exit code. Defaults to `0`.

The `process.exit()` method instructs Node.js to terminate the process as
quickly as possible with the specified exit `code`. If the `code` is omitted,
exit uses either the 'success' code `0` or the value of `process.exitCode` if
specified.
The `process.exit()` method instructs Node.js to terminate the process
synchronously with an exit status of `code`. If `code` is omitted, exit uses
either the 'success' code `0` or the value of `process.exitCode` if it has been
set. Node.js will not terminate until all the [`'exit'`] event listeners are
called.

To exit with a 'failure' code:

Expand Down Expand Up @@ -895,7 +896,7 @@ if (someConditionNotMet()) {
```

The reason this is problematic is because writes to `process.stdout` in Node.js
are sometimes *non-blocking* and may occur over multiple ticks of the Node.js
are sometimes *asynchronous* and may occur over multiple ticks of the Node.js
event loop. Calling `process.exit()`, however, forces the process to exit
*before* those additional writes to `stdout` can be performed.

Expand Down Expand Up @@ -1488,23 +1489,11 @@ Android)

* {Stream}

The `process.stderr` property returns a [Writable][] stream equivalent to or
associated with `stderr` (fd `2`).
The `process.stderr` property returns a [Writable][] stream connected to
`stderr` (fd `2`).

Note: `process.stderr` and `process.stdout` differ from other Node.js streams
in several ways:
1. They cannot be closed ([`end()`][] will throw).
2. They never emit the [`'finish'`][] event.
3. Writes _can_ block when output is redirected to a file.
- Note that disks are fast and operating systems normally employ write-back
caching so this is very uncommon.
4. Writes on UNIX **will** block by default if output is going to a TTY
(a terminal).
5. Windows functionality differs. Writes block except when output is going to a
TTY.

To check if Node.js is being run in a TTY context, read the `isTTY` property
on `process.stderr`, `process.stdout`, or `process.stdin`:
Note: `process.stderr` differs from other Node.js streams in important ways,
see [note on process I/O][] for more information.

## process.stdin

Expand Down Expand Up @@ -1542,48 +1531,59 @@ must call `process.stdin.resume()` to read from it. Note also that calling

* {Stream}

The `process.stdout` property returns a [Writable][] stream equivalent to or
associated with `stdout` (fd `1`).
The `process.stdout` property returns a [Writable][] stream connected to
`stdout` (fd `2`).

For example:
For example, to copy process.stdin to process.stdout:

```js
console.log = (msg) => {
process.stdout.write(`${msg}\n`);
};
process.stdin.pipe(process.stdout);
```

Note: `process.stderr` and `process.stdout` differ from other Node.js streams
in several ways:
1. They cannot be closed ([`end()`][] will throw).
2. They never emit the [`'finish'`][] event.
3. Writes _can_ block when output is redirected to a file.
- Note that disks are fast and operating systems normally employ write-back
caching so this is very uncommon.
4. Writes on UNIX **will** block by default if output is going to a TTY
(a terminal).
5. Windows functionality differs. Writes block except when output is going to a
TTY.
Note: `process.stdout` differs from other Node.js streams in important ways,
see [note on process I/O][] for more information.

### A note on process I/O

To check if Node.js is being run in a TTY context, read the `isTTY` property
on `process.stderr`, `process.stdout`, or `process.stdin`:
`process.stdout` and `process.stderr` differ from other Node.js streams in
important ways:

### TTY Terminals and `process.stdout`
1. They are used internally by [`console.log()`][] and [`console.error()`][],
respectively.
2. They cannot be closed ([`end()`][] will throw).
3. They will never emit the [`'finish'`][] event.
4. Writes may be synchronous depending on the what the stream is connected to
and whether the system is Windows or Unix:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be good to mention that synchronous writes correspond to blocking behaviour?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we don't do this in any other place where we wrap system calls synchronously: https://nodejs.org/api/fs.html#fs_fs_writefilesync_file_data_options for example, not sure why its so necessary here, but I'm willing.

Whether a sync call blocks or not depends on the O/S, the write buffer sizes, whether whatever process on the other side of the pipe/pty/whatever is reading data, etc. We can't just say that sync calls block, because that isn't true.

How about

If the write blocks, the synchronous call won't return until it unblocks.

If not, perhaps you can provide some text?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@addaleax If you are commenting top-to-bottom, you wouldn't notice, but this statement exists already, just a couple lines down:

Warning: Synchronous writes block the event loop until the write has completed.

- Files: *synchronous* on Windows and Linux
- TTYs (Terminals): *asynchronous* on Windows, *synchronous* on Unix
- Pipes (and sockets): *synchronous* on Windows, *asynchronous* on Unix

The `process.stderr` and `process.stdout` streams are blocking when outputting
to TTYs (terminals) on OS X as a workaround for the operating system's small,
1kb buffer size. This is to prevent interleaving between `stdout` and `stderr`.
These behaviours are partly for historical reasons, as changing them would
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. They are the way they are because of factors outside the node process.

Ideally they would all be synchronous I think. That is entirely impossible to implement Windows consoles* and also has other problems with pipes. (But Windows pipes have to block regardless.)

*There is no end-of-message notification IIRC

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Fishrock123 Your statement above is exactly why the it is partly for historical reasons. In your opinion pipes would be sync on Unix, but they aren't. How would you care to explain this other than "for historical reasons... backwards compat"? Also, note that I covered your point of view in the sentence below the one you comment on: "expected by some users", and that I even elaborated on why its expected with a whole paragraph. Would you like to propose some other explaination of why unix and windows are the exact opposites of each other for consoles and pipes? If one is the right way, the other is the wrong way and we have to say something about why this is.

Copy link
Contributor

@Fishrock123 Fishrock123 Feb 6, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Pipes on both Windows and Unix are wrong but neither is entirely our fault.

Blocking in a pipe means the process may stall forever if the downstream consumer stalls. That is not good and why pipes are (should be) async. @piscisaureus or @bnoordhuis may have more info.

It is my impression that making pipes on windows does not work correctly. See nodejs/node-v0.x-archive#3584 and 20176a9, although after reading some of the comments I am less confident about that. Maybe we could make pipes async on windows... again?

Note of course, that being async in a pipe has most of the caveats that being async in a terminal does, so it's not really good either.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Fishrock123

How would you care to explain this other than "for historical reasons... backwards compat"?

Would you like to propose some other explaination of why unix and windows are the exact opposites of each other for consoles and pipes? If one is the right way, the other is the wrong way and we have to say something about why this is

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again:

Maybe we could make pipes async on windows... again?

I don't actually have an answer at the current time. I suppose the wording is fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a doc PR, I'm not about to change how the system works in it, but I'm 💯 on aligning Windows and *nix platforms.

create backwards incompatibility, but they are also expected by some users.

To check if Node.js is being run in a [TTY][] context, check the `isTTY`
property on `process.stderr`, `process.stdout`, or `process.stdin`.
Synchronous writes avoid problems such as output written with `console.log()` or
`console.write()` being unexpectedly interleaved, or not written at all if
`process.exit()` is called before an asynchronous write completes. See
[`process.exit()`][] for more information.

***Warning***: Synchronous writes block the event loop until the write has
completed. This can be near instantaneous in the case of output to a file, but
under high system load, pipes that are not being read at the receiving end, or
with slow terminals or file systems, its possible for the event loop to be
blocked often enough and long enough to have severe negative performance
impacts. This may not be a problem when writing to an interactive terminal
session, but consider this particularly careful when doing production logging to
the process output streams.

To check if a stream is connected to a [TTY][] context, check the `isTTY`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Link to isTTY?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried, but the TTY docs don't describe it as a property, its only mentioned in the text, so that's the most specific link until the TTY docs get reworked. https://nodejs.org/api/tty.html#tty_tty

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, ok

property.

For instance:
```console
$ node -p "Boolean(process.stdin.isTTY)"
true
$ echo "foo" | node -p "Boolean(process.stdin.isTTY)"
false

$ node -p "Boolean(process.stdout.isTTY)"
true
$ node -p "Boolean(process.stdout.isTTY)" | cat
Expand Down Expand Up @@ -1737,6 +1737,7 @@ cases:
the high-order bit, and then contain the value of the signal code.


[`'exit'`]: #process_event_exit
[`'finish'`]: stream.html#stream_event_finish
[`'message'`]: child_process.html#child_process_event_message
[`'rejectionHandled'`]: #process_event_rejectionhandled
Expand All @@ -1758,6 +1759,7 @@ cases:
[`promise.catch()`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/catch
[`require.main`]: modules.html#modules_accessing_the_main_module
[`setTimeout(fn, 0)`]: timers.html#timers_settimeout_callback_delay_args
[note on process I/O]: process.html#process_a_note_on_process_i_o
[process_emit_warning]: #process_process_emitwarning_warning_name_ctor
[process_warning]: #process_event_warning
[Signal Events]: #process_signal_events
Expand Down