Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow redirecting subprocess stdout to our stderr etc. #88561

Closed
wants to merge 15 commits into from

Conversation

ijackson
Copy link
Contributor

@ijackson ijackson commented Sep 1, 2021

Currently there is no safe or portable way to redirect a Commands stdout to the invoking program's stderr, nor vice versa.

There a comment in the source that suggests that this was intended, and trying to handle the fd shuffling problem properly, but it can never be reached other than by crazy unsafe stunts.

Instead, provide appropriate From impls.

I have only tested the implementation for Unix because I know how to do that. There is now a Windows implementation which seems to pass the CI. This branch currently contains a DO NOT MERGE commit to enable the Windows CI build on this MR branch. That should be removed after code review and before merge.

Being trait impls, these are insta-stable. The question is just whether we want to support platforms that cannot do this, and if so, whether we want that to be a compile error or are happy for it to be detected by Command methods returning errors.

Closes #79731

@rust-highfive
Copy link
Collaborator

r? @kennytm

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 1, 2021
@rust-log-analyzer

This comment has been minimized.

@kennytm kennytm added the needs-fcp This change is insta-stable, so needs a completed FCP to proceed. label Sep 2, 2021
@the8472 the8472 added the T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. label Sep 8, 2021
@camelid camelid added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 8, 2021
library/std/src/process.rs Outdated Show resolved Hide resolved
@camelid camelid added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 5, 2021
@ijackson
Copy link
Contributor Author

ijackson commented Nov 8, 2021

I have rebased onto current head, given it its own feature tag (and updated the version) and provided what I thiink is an implementation on Windows. I don't have any Windows here so I may have to iterate through the CI, in which case sorry for the noise.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@ijackson
Copy link
Contributor Author

ijackson commented Nov 9, 2021

The failing test is that for #30490. The test case, and the code there, involves passing a static 0, 1 or 2 to a function which says it consumes ownership of the specified file descriptor, which is clearly UB.

I think I am going to have to break this MR up or work around this somehow. Nnnng.

@camelid
Copy link
Member

camelid commented Nov 9, 2021

Just FYI: fixup! commits are not automatically squashed upon merge.

@ijackson
Copy link
Contributor Author

ijackson commented Nov 9, 2021

Just FYI: fixup! commits are not automatically squashed upon merge.

Indeed. This MR needs more work and an FCP, so there's no danger of them ending up in tree before I've had a chance to tell my git to autosquash them. But thanks for the note.

@ijackson
Copy link
Contributor Author

Well, here's a version with Windows and which sidesteps whether FileDesc is allowed to contain 0/1/2. Let's see if the CI likes it. I have not tested it on Windows at all.

@rust-log-analyzer

This comment has been minimized.

@ijackson
Copy link
Contributor Author

Would someone be able to help with the Windows tests? Ideally, by adapting my new doctests so they work on Windows. (Breaking the Unix version while doing that would be fine; I can fix it up.)

Alternatively, if someone could suggest a command to run that will succeed and produce some nonempty stdlout output, and no stderr output, that will probably suffice and I can iterate it through the CI.

I am going to try setting the E-easy tag because I think this should be straightforward for someone familiar with Windows.

@rustbot modify labels +O-windows +E-help-wanted +E-easy

@rustbot rustbot added E-easy Call for participation: Easy difficulty. Experience needed to fix: Not much. Good first issue. E-help-wanted Call for participation: Help is requested to fix this issue. O-windows Operating system: Windows labels Nov 17, 2021
@camelid camelid added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 3, 2022
@rfcbot rfcbot added final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. and removed proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. labels Jul 25, 2023
@rfcbot
Copy link

rfcbot commented Jul 25, 2023

🔔 This is now entering its final comment period, as per the review above. 🔔

@ijackson
Copy link
Contributor Author

FTAOD, the actual MR branch here is not in a fit state to merge - it needs some squashing etc.

So, assuming the FCP completes with approval, don't merge the branch right away. I will tidy it and force push, first.

@workingjubilee workingjubilee added the A-process Area: `std::process` and `std::env` label Jul 30, 2023
library/std/src/process.rs Outdated Show resolved Hide resolved
library/std/src/process.rs Outdated Show resolved Hide resolved
@rfcbot rfcbot added finished-final-comment-period The final comment period is finished for this PR / Issue. to-announce Announce this issue on triage meeting and removed final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. labels Aug 4, 2023
@rfcbot
Copy link

rfcbot commented Aug 4, 2023

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

@ijackson
Copy link
Contributor Author

ijackson commented Aug 7, 2023

Thanks everyone for the approval, and the code review. I will update this branch soon, so that we can merge it.

Co-authored-by: Lonami <totufals@hotmail.com>
@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Aug 7, 2023
Co-authored-by: teor <teor@riseup.net>
@rust-log-analyzer

This comment has been minimized.

@ijackson
Copy link
Contributor Author

ijackson commented Aug 7, 2023

For convenience, I have made the squashed and tidied branch into a new MR: #114590

@pitaj
Copy link
Contributor

pitaj commented Aug 7, 2023

Opinions will obviously differ, but personally, what you've done feels less convenient.

@apiraino apiraino removed the to-announce Announce this issue on triage meeting label Aug 10, 2023
Copy link
Member

@dtolnay dtolnay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! I have approved your tidied PR in #114590 (review).

@dtolnay dtolnay closed this Sep 7, 2023
bors added a commit to rust-lang-ci/rust that referenced this pull request Sep 8, 2023
Allow redirecting subprocess stdout to our stderr etc. (redux)

This is the code from rust-lang#88561, tidied up, including review suggestions, and with the for-testing-only CI commit removed.  FCP for the API completed in rust-lang#88561.

I have made a new MR to facilitate review.  The discussion there is very cluttered and the branch is full of changes (in many cases as a result of changes to other Rust stdlib APIs since then).  Assuming this MR is approvedl we should close that one.

### Reviewer doing a de novo review

Just code review these four commits..  FCP discussion starts here: rust-lang#88561 (comment)

Portability tests: you can see that this branch works on Windows too by looking at the CI results in rust-lang#88561, which has the same code changes as this branch but with an additional "DO NOT MERGE" commit to make the Windows tests run.

### Reviewer doing an incremental review from some version of rust-lang#88561

Review the new commits since your last review.  I haven't force pushed the branch there.

git diff the two branches (eg `git diff 176886197d6..0842b69c219`).  You'll see that the only difference is in gitlab CI files.  You can also see that *this* MR doesn't touch those files.
bors added a commit to rust-lang-ci/rust that referenced this pull request Sep 9, 2023
Allow redirecting subprocess stdout to our stderr etc. (redux)

This is the code from rust-lang#88561, tidied up, including review suggestions, and with the for-testing-only CI commit removed.  FCP for the API completed in rust-lang#88561.

I have made a new MR to facilitate review.  The discussion there is very cluttered and the branch is full of changes (in many cases as a result of changes to other Rust stdlib APIs since then).  Assuming this MR is approvedl we should close that one.

### Reviewer doing a de novo review

Just code review these four commits..  FCP discussion starts here: rust-lang#88561 (comment)

Portability tests: you can see that this branch works on Windows too by looking at the CI results in rust-lang#88561, which has the same code changes as this branch but with an additional "DO NOT MERGE" commit to make the Windows tests run.

### Reviewer doing an incremental review from some version of rust-lang#88561

Review the new commits since your last review.  I haven't force pushed the branch there.

git diff the two branches (eg `git diff 176886197d6..0842b69c219`).  You'll see that the only difference is in gitlab CI files.  You can also see that *this* MR doesn't touch those files.
github-actions bot pushed a commit to rust-lang/miri that referenced this pull request Sep 12, 2023
Allow redirecting subprocess stdout to our stderr etc. (redux)

This is the code from #88561, tidied up, including review suggestions, and with the for-testing-only CI commit removed.  FCP for the API completed in #88561.

I have made a new MR to facilitate review.  The discussion there is very cluttered and the branch is full of changes (in many cases as a result of changes to other Rust stdlib APIs since then).  Assuming this MR is approvedl we should close that one.

### Reviewer doing a de novo review

Just code review these four commits..  FCP discussion starts here: rust-lang/rust#88561 (comment)

Portability tests: you can see that this branch works on Windows too by looking at the CI results in #88561, which has the same code changes as this branch but with an additional "DO NOT MERGE" commit to make the Windows tests run.

### Reviewer doing an incremental review from some version of #88561

Review the new commits since your last review.  I haven't force pushed the branch there.

git diff the two branches (eg `git diff 176886197d6..0842b69c219`).  You'll see that the only difference is in gitlab CI files.  You can also see that *this* MR doesn't touch those files.
lnicola pushed a commit to lnicola/rust-analyzer that referenced this pull request Apr 7, 2024
Allow redirecting subprocess stdout to our stderr etc. (redux)

This is the code from #88561, tidied up, including review suggestions, and with the for-testing-only CI commit removed.  FCP for the API completed in #88561.

I have made a new MR to facilitate review.  The discussion there is very cluttered and the branch is full of changes (in many cases as a result of changes to other Rust stdlib APIs since then).  Assuming this MR is approvedl we should close that one.

### Reviewer doing a de novo review

Just code review these four commits..  FCP discussion starts here: rust-lang/rust#88561 (comment)

Portability tests: you can see that this branch works on Windows too by looking at the CI results in #88561, which has the same code changes as this branch but with an additional "DO NOT MERGE" commit to make the Windows tests run.

### Reviewer doing an incremental review from some version of #88561

Review the new commits since your last review.  I haven't force pushed the branch there.

git diff the two branches (eg `git diff 176886197d6..0842b69c219`).  You'll see that the only difference is in gitlab CI files.  You can also see that *this* MR doesn't touch those files.
RalfJung pushed a commit to RalfJung/rust-analyzer that referenced this pull request Apr 27, 2024
Allow redirecting subprocess stdout to our stderr etc. (redux)

This is the code from #88561, tidied up, including review suggestions, and with the for-testing-only CI commit removed.  FCP for the API completed in #88561.

I have made a new MR to facilitate review.  The discussion there is very cluttered and the branch is full of changes (in many cases as a result of changes to other Rust stdlib APIs since then).  Assuming this MR is approvedl we should close that one.

### Reviewer doing a de novo review

Just code review these four commits..  FCP discussion starts here: rust-lang/rust#88561 (comment)

Portability tests: you can see that this branch works on Windows too by looking at the CI results in #88561, which has the same code changes as this branch but with an additional "DO NOT MERGE" commit to make the Windows tests run.

### Reviewer doing an incremental review from some version of #88561

Review the new commits since your last review.  I haven't force pushed the branch there.

git diff the two branches (eg `git diff 176886197d6..0842b69c219`).  You'll see that the only difference is in gitlab CI files.  You can also see that *this* MR doesn't touch those files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-process Area: `std::process` and `std::env` A-testsuite Area: The testsuite used to check the correctness of rustc disposition-merge This issue / PR is in PFCP or FCP with a disposition to merge it. finished-final-comment-period The final comment period is finished for this PR / Issue. needs-fcp This change is insta-stable, so needs a completed FCP to proceed. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Command cannot replumb stdout to stderr