Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking issue: stdio problems #6980

Closed
4 of 8 tasks
Fishrock123 opened this issue May 25, 2016 · 24 comments
Closed
4 of 8 tasks

Tracking issue: stdio problems #6980

Fishrock123 opened this issue May 25, 2016 · 24 comments
Labels
help wanted Issues that need assistance from volunteers or PRs that need help to proceed. libuv Issues and PRs related to the libuv dependency or the uv binding. process Issues and PRs related to the process subsystem. stalled Issues and PRs that are stalled. tty Issues and PRs related to the tty subsystem.

Comments

@Fishrock123
Copy link
Contributor

Fishrock123 commented May 25, 2016

This is a meta-issue for tracking multiple related but distinct stderr/stdout issues.

Below are checklists indicating the progress of these issues & pull requests.
A checkmark indicates it is resolved (or merged).

This issue encompasses the following issues:

And the following pull requests:

The current item in focus is:
n/a

The TTY (Terminal) issues have been resolved. Pipes still remain a problem.


This pr was a result of the discussion in #6968 to better track all the efforts going on here.

[EDIT by @Trott: Did some edits around the 6773 PR checklist item, but undid them]

@Fishrock123 Fishrock123 added the meta Issues and PRs related to the general management of the project. label May 25, 2016
@Fishrock123
Copy link
Contributor Author

Current plan is to make a release candidate (today/tomorrow) as discussed in #6934 with the OS X patch from #6895 to fix the largest issue (OS X stdio interleaving to TTYs) asap.

@kzc
Copy link

kzc commented May 25, 2016

You can add this one to the list:

#6867 Missing or truncated error message

It's a variation of the process.exit() stdio truncating problem but with uncaught exceptions.

Edit: reproduced on node 5.x on Linux. Same issue likely exists on node 4.x.

@kzc
Copy link

kzc commented May 25, 2016

This related merged PR for process.stdout._handle.fd should also be mentioned here for completeness:

unix,stream: fix getting the correct fd for a handle #6838

It ought to be back ported to node 5.x and 4.x.

@Fishrock123
Copy link
Contributor Author

No guarantees on v5.x since it's now in critical-maintinence-only mode, but we'll try to get in v4.x soon.

@mscdex mscdex added process Issues and PRs related to the process subsystem. tty Issues and PRs related to the tty subsystem. and removed meta Issues and PRs related to the general management of the project. labels May 26, 2016
@Fishrock123
Copy link
Contributor Author

Fishrock123 commented May 26, 2016

RC for #6895 building at https://ci-release.nodejs.org/job/iojs+release/921/ (Updated: new ci (2))

@Fishrock123
Copy link
Contributor Author

Fishrock123 commented May 27, 2016

I have an RC up at https://nodejs.org/download/rc/v6.2.1-rc.1/ that should fix the OS X problems.
Please test.

cc-ing the people from other threads: @isaacs, @nodejs/npm, @Qix-, @eljefedelrodeodeljefe, @bcoe, @mikemaccana, @mgol

@Qix-
Copy link

Qix- commented May 27, 2016

@Fishrock123

Qix- is missing the Overall/Read permission

@Fishrock123
Copy link
Contributor Author

Fishrock123 commented May 27, 2016

@Qix- Are you getting that attempting to access the directory, or the files themselves?

(or is node making that error?)

@Qix-
Copy link

Qix- commented May 27, 2016

@Fishrock123 Works now, not sure what changed. :) I'll give it a try.

saghul pushed a commit to saghul/node that referenced this issue Jul 11, 2016
OS X has a tiny 1kb hard-coded buffer size for stdout / stderr to
TTYs (terminals). Output larger than that causes chunking, which ends
up having some (very small but existent) delay past the first chunk.
That causes two problems:

1. When output is written to stdout and stderr at similar times, the
two can become mixed together (interleaved). This is especially
problematic when using control characters, such as \r. With
interleaving, chunked output will often have lines or characters erased
unintentionally, or in the wrong spots, leading to broken output.
CLI apps often extensively use such characters for things such as
progress bars.

2. Output can be lost if the process is exited before chunked writes
are finished flushing. This usually happens in applications that use
`process.exit()`, which isn't infrequent.

See nodejs#6980 for more info.

This became an issue as result of the Libuv 1.9.0 upgrade. A fix to
an unrelated issue broke a hack previously required for the OS X
implementation. This resulted in an unexpected behavior change in node.
The 1.9.0 upgrade was done in c3cec1e,
which was included in v6.0.0.
Full details of the Libuv issue that induced this are at
nodejs#6456 (comment)

Refs: nodejs#1771
Refs: nodejs#6456
Refs: nodejs#6773
Refs: nodejs#6816
PR-URL: nodejs#6895
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
saghul pushed a commit to saghul/node that referenced this issue Jul 11, 2016
Many thanks to thefourtheye and addaleax who helped make the python
bits of this possible.

See nodejs#6980 for more info regarding
the related TTY issues.

Refs: nodejs#6456
Refs: nodejs#6773
Refs: nodejs#6816
PR-URL: nodejs#6895
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
MylesBorins pushed a commit that referenced this issue Jul 11, 2016
OS X has a tiny 1kb hard-coded buffer size for stdout / stderr to
TTYs (terminals). Output larger than that causes chunking, which ends
up having some (very small but existent) delay past the first chunk.
That causes two problems:

1. When output is written to stdout and stderr at similar times, the
two can become mixed together (interleaved). This is especially
problematic when using control characters, such as \r. With
interleaving, chunked output will often have lines or characters erased
unintentionally, or in the wrong spots, leading to broken output.
CLI apps often extensively use such characters for things such as
progress bars.

2. Output can be lost if the process is exited before chunked writes
are finished flushing. This usually happens in applications that use
`process.exit()`, which isn't infrequent.

See #6980 for more info.

This became an issue as result of the Libuv 1.9.0 upgrade. A fix to
an unrelated issue broke a hack previously required for the OS X
implementation. This resulted in an unexpected behavior change in node.
The 1.9.0 upgrade was done in c3cec1e,
which was included in v6.0.0.
Full details of the Libuv issue that induced this are at
#6456 (comment)

Refs: #1771
Refs: #6456
Refs: #6773
Refs: #6816
PR-URL: #6895
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
MylesBorins pushed a commit that referenced this issue Jul 11, 2016
Many thanks to thefourtheye and addaleax who helped make the python
bits of this possible.

See #6980 for more info regarding
the related TTY issues.

Refs: #6456
Refs: #6773
Refs: #6816
PR-URL: #6895
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
MylesBorins pushed a commit that referenced this issue Jul 11, 2016
OS X has a tiny 1kb hard-coded buffer size for stdout / stderr to
TTYs (terminals). Output larger than that causes chunking, which ends
up having some (very small but existent) delay past the first chunk.
That causes two problems:

1. When output is written to stdout and stderr at similar times, the
two can become mixed together (interleaved). This is especially
problematic when using control characters, such as \r. With
interleaving, chunked output will often have lines or characters erased
unintentionally, or in the wrong spots, leading to broken output.
CLI apps often extensively use such characters for things such as
progress bars.

2. Output can be lost if the process is exited before chunked writes
are finished flushing. This usually happens in applications that use
`process.exit()`, which isn't infrequent.

See #6980 for more info.

This became an issue as result of the Libuv 1.9.0 upgrade. A fix to
an unrelated issue broke a hack previously required for the OS X
implementation. This resulted in an unexpected behavior change in node.
The 1.9.0 upgrade was done in c3cec1e,
which was included in v6.0.0.
Full details of the Libuv issue that induced this are at
#6456 (comment)

Refs: #1771
Refs: #6456
Refs: #6773
Refs: #6816
PR-URL: #6895
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
MylesBorins pushed a commit that referenced this issue Jul 11, 2016
Many thanks to thefourtheye and addaleax who helped make the python
bits of this possible.

See #6980 for more info regarding
the related TTY issues.

Refs: #6456
Refs: #6773
Refs: #6816
PR-URL: #6895
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
MylesBorins pushed a commit that referenced this issue Jul 12, 2016
OS X has a tiny 1kb hard-coded buffer size for stdout / stderr to
TTYs (terminals). Output larger than that causes chunking, which ends
up having some (very small but existent) delay past the first chunk.
That causes two problems:

1. When output is written to stdout and stderr at similar times, the
two can become mixed together (interleaved). This is especially
problematic when using control characters, such as \r. With
interleaving, chunked output will often have lines or characters erased
unintentionally, or in the wrong spots, leading to broken output.
CLI apps often extensively use such characters for things such as
progress bars.

2. Output can be lost if the process is exited before chunked writes
are finished flushing. This usually happens in applications that use
`process.exit()`, which isn't infrequent.

See #6980 for more info.

This became an issue as result of the Libuv 1.9.0 upgrade. A fix to
an unrelated issue broke a hack previously required for the OS X
implementation. This resulted in an unexpected behavior change in node.
The 1.9.0 upgrade was done in c3cec1e,
which was included in v6.0.0.
Full details of the Libuv issue that induced this are at
#6456 (comment)

Refs: #1771
Refs: #6456
Refs: #6773
Refs: #6816
PR-URL: #6895
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
MylesBorins pushed a commit that referenced this issue Jul 12, 2016
Many thanks to thefourtheye and addaleax who helped make the python
bits of this possible.

See #6980 for more info regarding
the related TTY issues.

Refs: #6456
Refs: #6773
Refs: #6816
PR-URL: #6895
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
MylesBorins pushed a commit that referenced this issue Jul 14, 2016
OS X has a tiny 1kb hard-coded buffer size for stdout / stderr to
TTYs (terminals). Output larger than that causes chunking, which ends
up having some (very small but existent) delay past the first chunk.
That causes two problems:

1. When output is written to stdout and stderr at similar times, the
two can become mixed together (interleaved). This is especially
problematic when using control characters, such as \r. With
interleaving, chunked output will often have lines or characters erased
unintentionally, or in the wrong spots, leading to broken output.
CLI apps often extensively use such characters for things such as
progress bars.

2. Output can be lost if the process is exited before chunked writes
are finished flushing. This usually happens in applications that use
`process.exit()`, which isn't infrequent.

See #6980 for more info.

This became an issue as result of the Libuv 1.9.0 upgrade. A fix to
an unrelated issue broke a hack previously required for the OS X
implementation. This resulted in an unexpected behavior change in node.
The 1.9.0 upgrade was done in c3cec1e,
which was included in v6.0.0.
Full details of the Libuv issue that induced this are at
#6456 (comment)

Refs: #1771
Refs: #6456
Refs: #6773
Refs: #6816
PR-URL: #6895
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
MylesBorins pushed a commit that referenced this issue Jul 14, 2016
Many thanks to thefourtheye and addaleax who helped make the python
bits of this possible.

See #6980 for more info regarding
the related TTY issues.

Refs: #6456
Refs: #6773
Refs: #6816
PR-URL: #6895
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
MylesBorins pushed a commit that referenced this issue Jul 14, 2016
OS X has a tiny 1kb hard-coded buffer size for stdout / stderr to
TTYs (terminals). Output larger than that causes chunking, which ends
up having some (very small but existent) delay past the first chunk.
That causes two problems:

1. When output is written to stdout and stderr at similar times, the
two can become mixed together (interleaved). This is especially
problematic when using control characters, such as \r. With
interleaving, chunked output will often have lines or characters erased
unintentionally, or in the wrong spots, leading to broken output.
CLI apps often extensively use such characters for things such as
progress bars.

2. Output can be lost if the process is exited before chunked writes
are finished flushing. This usually happens in applications that use
`process.exit()`, which isn't infrequent.

See #6980 for more info.

This became an issue as result of the Libuv 1.9.0 upgrade. A fix to
an unrelated issue broke a hack previously required for the OS X
implementation. This resulted in an unexpected behavior change in node.
The 1.9.0 upgrade was done in c3cec1e,
which was included in v6.0.0.
Full details of the Libuv issue that induced this are at
#6456 (comment)

Refs: #1771
Refs: #6456
Refs: #6773
Refs: #6816
PR-URL: #6895
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
MylesBorins pushed a commit that referenced this issue Jul 14, 2016
Many thanks to thefourtheye and addaleax who helped make the python
bits of this possible.

See #6980 for more info regarding
the related TTY issues.

Refs: #6456
Refs: #6773
Refs: #6816
PR-URL: #6895
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
@Fishrock123
Copy link
Contributor Author

#6816 was merged in ab3306a fixing the TTY (terminal) problems.

Pipes still remain an issue and this probably needs to be fixed in libuv, far beyond my expertise.

@Fishrock123
Copy link
Contributor Author

ref: #12970

@jasnell
Copy link
Member

jasnell commented Sep 15, 2017

Closing as it's not clear if at all if there's anything remaining to do here and there's been zero activity since may

@jasnell jasnell closed this as completed Sep 15, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Issues that need assistance from volunteers or PRs that need help to proceed. libuv Issues and PRs related to the libuv dependency or the uv binding. process Issues and PRs related to the process subsystem. stalled Issues and PRs that are stalled. tty Issues and PRs related to the tty subsystem.
Projects
None yet
Development

No branches or pull requests

9 participants