Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rollup of 9 pull requests #49684

Merged
merged 29 commits into from
Apr 5, 2018
Merged

Rollup of 9 pull requests #49684

merged 29 commits into from
Apr 5, 2018

Conversation

kennytm
Copy link
Member

@kennytm kennytm commented Apr 5, 2018

Successful merges:

Failed merges:

chmanchester and others added 25 commits March 28, 2018 11:38
A problem caused by not doing so in Chrome has been reported here:
https://randomascii.wordpress.com/2018/02/25/compiler-bug-linker-bug-windows-kernel-bug/amp/

File::sync_all() calls FlushFileBuffers() down the line,
causing potentially unflushed buffers on high I/O-load systems to flush
and prevent nasty non-reproducible bugs.

The force-flush is only done on Windows and if the linker exited successfully

Closes rust-lang#48545
This adds a new method to all numeric `Atomic*` types with a
safe compare-and-set loop, so users will no longer need to write
their own, except in *very* strange circumstances.

This solves rust-lang#48384 with `x.fetch_max(_)`/`x.fetch_min(_)`. It
also relates to rust-lang#48655 (which I misuse as tracking issue for now).

*note* This *might* need a crater run because the functions could
clash with third party extension traits.
This should be enough and shouldn't require append(true) since we're not
explicitly writing anything so we're not flushing it so we've no risk of
overwriting it
the rust-std component only contains the core and compiler-builtins (+c +mem) crates

cc rust-lang#49382
vec![0; n], via implementations of SpecFromElem, has an optimization
that uses with_capacity_zeroed instead of with_capacity, which will use
calloc instead of malloc, and avoid an extra memset.

This adds the same optimization for vec![ptr::null(); n] and
vec![ptr::null_mut(); n], assuming their bit value is 0 (which is true
on all currently supported platforms).

This does so by adding an intermediate trait IsZero, which looks very
much like nonzero::Zeroable, but that one is on the way out, and doesn't
apply to pointers anyways.

Adding such a trait allows to avoid repeating the logic using
with_capacity_zeroed or with_capacity, or making the macro more complex
to support generics.
Similarly to vec![ptr::null{,_mut}(); n] in previous change, this adds
the optimization for vec!['\0'; n].
the goal is to build, in a single Cargo invocation, several no-std crates that we want to put in the
rust-std component of no-std targets. The nostd crate builds these crates:

- core
- compiler-builtin (with the "c" and "mem" features enabled)
- alloc
- std_unicode
This builder is starting to time out frequently causing PRs to bounce and
otherwise doesn't seem to be catching too many bugs, so this commit removes it
entirely. We've had a number of timeouts in the last few weeks related to this
builder:

* https://travis-ci.org/rust-lang/rust/jobs/360947582
* https://travis-ci.org/rust-lang/rust/jobs/360464190
* https://travis-ci.org/rust-lang/rust/jobs/359946975
* https://travis-ci.org/rust-lang/rust/jobs/361213241
* https://travis-ci.org/rust-lang/rust/jobs/362346279
* https://travis-ci.org/rust-lang/rust/jobs/362072331

On a good run this builder takes about 2h15m, which is already too long for
Travis and the variable build times end up pushing it beyond the 3h limit
occasionally.

The timeouts here are somewhat expected in that an incrementally compiled rustc
compiler isn't optimized like a normal rustc, disallowing inlining between
codegen units and losing lots of optimization opportunities.
Add a generic CAS loop to std::sync::Atomic*

This adds two new methods to both `AtomicIsize` and `AtomicUsize` with optimized safe compare-and-set loops, so users will no longer need to write their own, except in *very* strange circumstances.

`update_and_fetch` will apply the function and return its result, whereas `fetch_and_update` will apply the function and return the previous value.

This solves rust-lang#48384 with `x.update_and_fetch(|x| x.max(y))`. It also relates to rust-lang#48655 (which I misuse as tracking issue for now)..

*note* This *might* need a crater run because the functions could clash with third party extension traits.
…chton

Take the original extra-filename passed to a crate into account when resolving it as a dependency

resolving it as a dependency.

Fixes rust-lang#46816
RFC 2008: Finishing Touches

Part of rust-lang#44109.

r? @nikomatsakis
(not sure who was best for this PR).
…rister

Flush executables to disk after linkage

A problem caused by not doing so in Chrome has been reported [here](https://randomascii.wordpress.com/2018/02/25/compiler-bug-linker-bug-windows-kernel-bug/amp/).

`File::sync_all()` calls `FlushFileBuffers()` down the line, causing potentially unflushed buffers on high I/O-load systems to flush and preventing nasty non-reproducible bugs.

Closes rust-lang#48545
Add more vec![... ; n] optimizations

vec![0; n], via implementations of SpecFromElem, has an optimization that uses with_capacity_zeroed instead of with_capacity, which will use calloc instead of malloc, and avoid an extra memset.

This PR adds the same optimization for ptr::null, ptr::null_mut, and None, when their in-memory representation is zeroes.
@kennytm
Copy link
Member Author

kennytm commented Apr 5, 2018

@bors r+ p=9

@bors
Copy link
Contributor

bors commented Apr 5, 2018

📌 Commit 30c830f has been approved by kennytm

@bors bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Apr 5, 2018
@bors
Copy link
Contributor

bors commented Apr 5, 2018

⌛ Testing commit 30c830f050e9e84e9cf2c0a4e6fa71f522deb40a with merge 72a46c7af7b9053c4512ef9aa0ed76cfd12516f1...

@bors
Copy link
Contributor

bors commented Apr 5, 2018

💔 Test failed - status-travis

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Apr 5, 2018
@TimNN
Copy link
Contributor

TimNN commented Apr 5, 2018

Your PR failed on Travis. Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[00:26:50] error[E0463]: can't find crate for `compiler_builtins`
[00:26:50]
[00:26:50] error: aborting due to previous error
[00:26:50]
[00:26:50] For more information about this error, try `rustc --explain E0463`.
[00:26:50] [RUSTC-TIMING] profiler_builtins test:false 0.121
[00:26:50] The following warnings were emitted during compilation:
---
[00:26:50] error: Could not compile `profiler_builtins`.
[00:26:50]
[00:26:50] Caused by:
[00:26:50]   process didn't exit successfully: `/checkout/obj/build/bootstrap/debug/rustc --crate-name profiler_builtins libprofiler_builtins/lib.rs --color always --error-format json --crate-type lib --emit=dep-info,link -C opt-level=2 -C metadata=b56639074230e1d1 -C extra-filename=-b56639074230e1d1 --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps --target x86_64-unknown-linux-gnu -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/release/deps --extern core=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/libcore-6fe704c3bfbe80d2.rlib -L native=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/build/profiler_builtins-ac848cf491139595/out -l static=profiler-rt` (exit code: 101)
[00:26:50] warning: build failed, waiting for other jobs to finish...
[00:26:53] [RUSTC-TIMING] compiler_builtins test:false 2.936
[00:28:07] error: build failed
[00:28:07] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "x86_64-unknown-linux-gnu" "-j" "4" "--release" "--locked" "--color" "always" "--features" "panic-unwind jemalloc backtrace profiler" "--manifest-path" "/checkout/src/libstd/Cargo.toml" "--message-format" "json"
[00:28:07] expected success, got: exit code: 101
[00:28:07] thread 'main' panicked at 'cargo must succeed', bootstrap/compile.rs:1085:9
---
$ ls -lat $HOME/Library/Logs/DiagnosticReports/
ls: cannot access /home/travis/Library/Logs/DiagnosticReports/: No such file or directory
travis_time:end:11aa78f8:start=1522924131545412675,finish=1522924131553056385,duration=7643710
travis_fold:end:after_failure.2
travis_fold:start:after_failure.3
travis_time:start:02c3477a
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
find: `/home/travis/Library/Logs/DiagnosticReports': No such file or directory
travis_time:end:02c3477a:start=1522924131558347480,finish=1522924131564187228,duration=5839748
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:072af3ff
$ dmesg | grep -i kill
[   10.580310] init: failsafe main process (1093) killed by TERM signal

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN.

1 similar comment
@TimNN
Copy link
Contributor

TimNN commented Apr 5, 2018

Your PR failed on Travis. Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[00:26:50] error[E0463]: can't find crate for `compiler_builtins`
[00:26:50]
[00:26:50] error: aborting due to previous error
[00:26:50]
[00:26:50] For more information about this error, try `rustc --explain E0463`.
[00:26:50] [RUSTC-TIMING] profiler_builtins test:false 0.121
[00:26:50] The following warnings were emitted during compilation:
---
[00:26:50] error: Could not compile `profiler_builtins`.
[00:26:50]
[00:26:50] Caused by:
[00:26:50]   process didn't exit successfully: `/checkout/obj/build/bootstrap/debug/rustc --crate-name profiler_builtins libprofiler_builtins/lib.rs --color always --error-format json --crate-type lib --emit=dep-info,link -C opt-level=2 -C metadata=b56639074230e1d1 -C extra-filename=-b56639074230e1d1 --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps --target x86_64-unknown-linux-gnu -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/release/deps --extern core=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/deps/libcore-6fe704c3bfbe80d2.rlib -L native=/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu/release/build/profiler_builtins-ac848cf491139595/out -l static=profiler-rt` (exit code: 101)
[00:26:50] warning: build failed, waiting for other jobs to finish...
[00:26:53] [RUSTC-TIMING] compiler_builtins test:false 2.936
[00:28:07] error: build failed
[00:28:07] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "x86_64-unknown-linux-gnu" "-j" "4" "--release" "--locked" "--color" "always" "--features" "panic-unwind jemalloc backtrace profiler" "--manifest-path" "/checkout/src/libstd/Cargo.toml" "--message-format" "json"
[00:28:07] expected success, got: exit code: 101
[00:28:07] thread 'main' panicked at 'cargo must succeed', bootstrap/compile.rs:1085:9
---
$ ls -lat $HOME/Library/Logs/DiagnosticReports/
ls: cannot access /home/travis/Library/Logs/DiagnosticReports/: No such file or directory
travis_time:end:11aa78f8:start=1522924131545412675,finish=1522924131553056385,duration=7643710
travis_fold:end:after_failure.2
travis_fold:start:after_failure.3
travis_time:start:02c3477a
$ find $HOME/Library/Logs/DiagnosticReports -type f -name '*.crash' -not -name '*.stage2-*.crash' -not -name 'com.apple.CoreSimulator.CoreSimulatorService-*.crash' -exec printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" {} \; -exec head -750 {} \; -exec echo travis_fold":"end:crashlog \; || true
find: `/home/travis/Library/Logs/DiagnosticReports': No such file or directory
travis_time:end:02c3477a:start=1522924131558347480,finish=1522924131564187228,duration=5839748
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:072af3ff
$ dmesg | grep -i kill
[   10.580310] init: failsafe main process (1093) killed by TERM signal

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN.

add a dist builder to build rust-std components for the THUMB targets

the rust-std component only contains the core and compiler-builtins (+c +mem) crates

cc rust-lang#49382

- I'm not entirely sure if this PR alone will produce rust-std components installable by rustup or if something else needs to be changed
- I could have done the THUMB builds in an existing builder / image; I wasn't sure if that was a good idea so I added a new image
- I could build other crates like alloc into the rust-std component but, AFAICT, that would require calling Cargo a second time (one for alloc and one for compiler-builtins), or have alloc depend on compiler-builtins (rust-lang#49503 will perform that change) *and* have alloc resurface the "c" and "mem" Cargo features.

r? @alexcrichton
Host compiler documentation: Include private items

Fixes rust-lang#29893. Now that compiler documentation is being hosted, including private items seems sensible as these types are going to be being used by contributors working on the compiler.

However, including this means that doc comments that contain codeblocks with invalid Rust and can fail the documenting of a given crate (as evidenced by the changes in the second commit included in this PR). We'd need some way of ensuring that this cannot happen so that these failures don't cause documenting to fail. I'm unsure whether this change to documentation steps will cause this to happen already or if something new will be required.

r? @alexcrichton
…omatsakis

Add more features to rust_2018_preview

Doesn't handle lib features like never_type; we need to figure out a scheme for that.
…r=michaelwoerister

ci: Remove x86_64-gnu-incremental builder

This builder is starting to time out frequently causing PRs to bounce and
otherwise doesn't seem to be catching too many bugs, so this commit removes it
entirely. We've had a number of timeouts in the last few weeks related to this
builder:

* https://travis-ci.org/rust-lang/rust/jobs/360947582
* https://travis-ci.org/rust-lang/rust/jobs/360464190
* https://travis-ci.org/rust-lang/rust/jobs/359946975
* https://travis-ci.org/rust-lang/rust/jobs/361213241
* https://travis-ci.org/rust-lang/rust/jobs/362346279
* https://travis-ci.org/rust-lang/rust/jobs/362072331

On a good run this builder takes about 2h15m, which is already too long for
Travis and the variable build times end up pushing it beyond the 3h limit
occasionally.

The timeouts here are somewhat expected in that an incrementally compiled rustc
compiler isn't optimized like a normal rustc, disallowing inlining between
codegen units and losing lots of optimization opportunities.
@kennytm
Copy link
Member Author

kennytm commented Apr 5, 2018

@bors r+

Removed #49503 (test report indicates the failure is related to the compiler-builtin crate).

Added #49674.

@bors
Copy link
Contributor

bors commented Apr 5, 2018

📌 Commit f4511e2 has been approved by kennytm

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 5, 2018
@bors
Copy link
Contributor

bors commented Apr 5, 2018

⌛ Testing commit f4511e2 with merge 56714ac...

bors added a commit that referenced this pull request Apr 5, 2018
Rollup of 9 pull requests

Successful merges:

 - #48658 (Add a generic CAS loop to std::sync::Atomic*)
 - #49253 (Take the original extra-filename passed to a crate into account when resolving it as a dependency)
 - #49345 (RFC 2008: Finishing Touches)
 - #49432 (Flush executables to disk after linkage)
 - #49496 (Add more vec![... ; n] optimizations)
 - #49563 (add a dist builder to build rust-std components for the THUMB targets)
 - #49654 (Host compiler documentation: Include private items)
 - #49667 (Add more features to rust_2018_preview)
 - #49674 (ci: Remove x86_64-gnu-incremental builder)

Failed merges:
@bors
Copy link
Contributor

bors commented Apr 5, 2018

☀️ Test successful - status-appveyor, status-travis
Approved by: kennytm
Pushing 56714ac to master...

@bors bors merged commit f4511e2 into rust-lang:master Apr 5, 2018
@Centril Centril added the rollup A PR which is a rollup label Oct 2, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
rollup A PR which is a rollup S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.