Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Make usize overflow always have debug-assertions semantics #58475

Closed
wants to merge 3 commits into from

Conversation

alex
Copy link
Member

@alex alex commented Feb 15, 2019

This is a prototype for rust-lang/rfcs#2635 to enable us to collect some measurements.

@rust-highfive
Copy link
Collaborator

r? @oli-obk

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 15, 2019
@alex
Copy link
Member Author

alex commented Feb 15, 2019

(Ok, I tried to use Github's draft features, but it appears that it doesn't run CI, oh well.)

@alex alex marked this pull request as ready for review February 15, 2019 02:17
@alex alex closed this Feb 15, 2019
@alex alex reopened this Feb 15, 2019
@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:01ee0784:start=1550197315098556938,finish=1550197318591352780,duration=3492795842
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
Setting environment variables from .travis.yml
$ export IMAGE=x86_64-gnu-llvm-6.0
---
travis_time:start:test_debuginfo
Check compiletest suite=debuginfo mode=debuginfo-both (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:12:30] 
[01:12:30] running 119 tests
[01:12:55] .iiiii...i.....i..i...i..i.i..i.ii...i.....i..i....i..........iiii..........i...ii...i.......ii.i.i. 100/119
[01:12:59] i......iii.i.....ii
[01:12:59] 
[01:12:59]  finished in 29.695
[01:12:59] travis_fold:end:test_debuginfo

---
travis_time:start:test_run-make-fulldeps
Check compiletest suite=run-make-fulldeps mode=run-make (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
[01:39:19] 
[01:39:19] running 195 tests
[01:39:45] ..................i...i............................................F....................i........... 100/195
[01:40:31] thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:496:22
[01:40:31] ......F............................................i......................................i....
[01:40:31] 
[01:40:31] ---- [run-make] run-make-fulldeps/issue-14500 stdout ----
[01:40:31] 
[01:40:31] error: make failed
[01:40:31] error: make failed
[01:40:31] status: exit code: 2
[01:40:31] command: "make"
[01:40:31] stdout:
[01:40:31] ------------------------------------------
[01:40:31] make[1]: Entering directory '/checkout/src/test/run-make-fulldeps/issue-14500'
[01:40:31] LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/issue-14500/issue-14500:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/issue-14500/issue-14500 -L /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/issue-14500/issue-14500  foo.rs --crate-type=rlib
[01:40:31] LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/issue-14500/issue-14500:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/issue-14500/issue-14500 -L /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/issue-14500/issue-14500  bar.rs --crate-type=staticlib -C lto -L. -o /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/issue-14500/issue-14500/libbar.a
[01:40:31] cc -ffunction-sections -fdata-sections -fPIC -m64 foo.c /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/issue-14500/issue-14500/libbar.a -lm -lrt -ldl -lpthread -o /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/issue-14500/issue-14500/foo
[01:40:31] Makefile:14: recipe for target 'all' failed
[01:40:31] make[1]: Leaving directory '/checkout/src/test/run-make-fulldeps/issue-14500'
[01:40:31] ------------------------------------------
[01:40:31] stderr:
[01:40:31] ------------------------------------------
[01:40:31] ------------------------------------------
[01:40:31] warning: ignoring --out-dir flag due to -o flag
[01:40:31] 
[01:40:31] /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/issue-14500/issue-14500/libbar.a(compiler_builtins-ec024234a641382a.compiler_builtins.17giirf6-cgu.0.rcgu.o): In function `__clzsi2':
[01:40:31] compiler_builtins.17giirf6-cgu.0:(.text.__clzsi2+0x5d): undefined reference to `core::panicking::panic::h05784555a26cf2e4'
[01:40:31] compiler_builtins.17giirf6-cgu.0:(.text.__clzsi2+0x8a): undefined reference to `core::panicking::panic::h05784555a26cf2e4'
[01:40:31] compiler_builtins.17giirf6-cgu.0:(.text.__clzsi2+0xa9): undefined reference to `core::panicking::panic::h05784555a26cf2e4'
[01:40:31] compiler_builtins.17giirf6-cgu.0:(.text.__clzsi2+0xb8): undefined reference to `core::panicking::panic::h05784555a26cf2e4'
[01:40:31] collect2: error: ld returned 1 exit status
[01:40:31] make[1]: *** [all] Error 1
[01:40:31] ------------------------------------------
[01:40:31] 
[01:40:31] thread '[run-make] run-make-fulldeps/issue-14500' panicked at 'explicit panic', src/tools/compiletest/src/runtest.rs:3295:9
[01:40:31] note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
---
[01:40:31] status: exit code: 2
[01:40:31] command: "make"
[01:40:31] stdout:
[01:40:31] ------------------------------------------
[01:40:31] make[1]: Entering directory '/checkout/src/test/run-make-fulldeps/lto-smoke-c'
[01:40:31] LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/lto-smoke-c/lto-smoke-c:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib" '/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc' --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/lto-smoke-c/lto-smoke-c -L /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/lto-smoke-c/lto-smoke-c  foo.rs -C lto
[01:40:31] cc -ffunction-sections -fdata-sections -fPIC -m64 bar.c /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/lto-smoke-c/lto-smoke-c/libfoo.a \
[01:40:31]  -o /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/lto-smoke-c/lto-smoke-c/bar \
[01:40:31]  -lm -lrt -ldl -lpthread -lstdc++
[01:40:31] Makefile:7: recipe for target 'all' failed
[01:40:31] make[1]: Leaving directory '/checkout/src/test/run-make-fulldeps/lto-smoke-c'
[01:40:31] ------------------------------------------
[01:40:31] stderr:
[01:40:31] ------------------------------------------
[01:40:31] ------------------------------------------
[01:40:31] /checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps/lto-smoke-c/lto-smoke-c/libfoo.a(compiler_builtins-ec024234a641382a.compiler_builtins.17giirf6-cgu.0.rcgu.o): In function `__clzsi2':
[01:40:31] compiler_builtins.17giirf6-cgu.0:(.text.__clzsi2+0x5d): undefined reference to `core::panicking::panic::h05784555a26cf2e4'
[01:40:31] compiler_builtins.17giirf6-cgu.0:(.text.__clzsi2+0x8a): undefined reference to `core::panicking::panic::h05784555a26cf2e4'
[01:40:31] compiler_builtins.17giirf6-cgu.0:(.text.__clzsi2+0xa9): undefined reference to `core::panicking::panic::h05784555a26cf2e4'
[01:40:31] compiler_builtins.17giirf6-cgu.0:(.text.__clzsi2+0xb8): undefined reference to `core::panicking::panic::h05784555a26cf2e4'
[01:40:31] collect2: error: ld returned 1 exit status
[01:40:31] make[1]: *** [all] Error 1
[01:40:31] ------------------------------------------
[01:40:31] 
[01:40:31] thread '[run-make] run-make-fulldeps/lto-smoke-c' panicked at 'explicit panic', src/tools/compiletest/src/runtest.rs:3295:9
[01:40:31] 
---
[01:40:31] test result: FAILED. 188 passed; 2 failed; 5 ignored; 0 measured; 0 filtered out
[01:40:31] 
[01:40:31] 
[01:40:31] 
[01:40:31] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--rustdoc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" "--src-base" "/checkout/src/test/run-make-fulldeps" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make-fulldeps" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "run-make" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-6.0/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "6.0.0\n" "--system-llvm" "--cc" "cc" "--cxx" "c++" "--cflags" "-ffunction-sections -fdata-sections -fPIC -m64" "--llvm-components" "aarch64 aarch64asmparser aarch64asmprinter aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils all all-targets amdgpu amdgpuasmparser amdgpuasmprinter amdgpucodegen amdgpudesc amdgpudisassembler amdgpuinfo amdgpuutils analysis arm armasmparser armasmprinter armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter binaryformat bitreader bitwriter bpf bpfasmparser bpfasmprinter bpfcodegen bpfdesc bpfdisassembler bpfinfo codegen core coroutines coverage debuginfocodeview debuginfodwarf debuginfomsf debuginfopdb demangle dlltooldriver engine executionengine fuzzmutate globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interpreter ipo irreader lanai lanaiasmparser lanaiasmprinter lanaicodegen lanaidesc lanaidisassembler lanaiinfo libdriver lineeditor linker lto mc mcdisassembler mcjit mcparser mips mipsasmparser mipsasmprinter mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmprinter msp430codegen msp430desc msp430info native nativecodegen nvptx nvptxasmprinter nvptxcodegen nvptxdesc nvptxinfo objcarcopts object objectyaml option orcjit passes powerpc powerpcasmparser powerpcasmprinter powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata runtimedyld scalaropts selectiondag sparc sparcasmparser sparcasmprinter sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzasmprinter systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target transformutils vectorize windowsmanifest x86 x86asmparser x86asmprinter x86codegen x86desc x86disassembler x86info x86utils xcore xcoreasmprinter xcorecodegen xcoredesc xcoredisassembler xcoreinfo" "--llvm-cxxflags" "-I/usr/lib/llvm-6.0/include -std=c++0x -fuse-ld=gold -Wl,--no-keep-files-mapped -Wl,--no-map-whole-files -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wno-maybe-uninitialized -Wdelete-non-virtual-dtor -Wno-comment -ffunction-sections -fdata-sections -O2 -DNDEBUG  -fno-exceptions -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS" "--ar" "ar" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
[01:40:31] 
[01:40:31] 
[01:40:31] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[01:40:31] Build completed unsuccessfully in 0:39:45
[01:40:31] Build completed unsuccessfully in 0:39:45
[01:40:31] make: *** [check] Error 1
[01:40:31] Makefile:48: recipe for target 'check' failed
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:02f042fd
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Fri Feb 15 04:02:39 UTC 2019

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@alex
Copy link
Member Author

alex commented Feb 15, 2019

Test failures are examples of LTO + linking with C that die with undefined reference to core::panicking::panic::h05784555a26cf2e4. I'm going to to speculate that somewhere there's some usize arithmetic that we can now panic on it, but which for whatever reason doesn't cause panic to be properly linked in.

This is obviously a bug and needs to be fixed, but I don't think it's a blocker to doing any performance measurement on the impact of this.

@scottmcm
Copy link
Member

scottmcm commented Feb 15, 2019

Triggering perf needs a try build, which AFAIK does need to pass tests. Maybe just delete the tests that are failing for now, since we just need something that runs, not something ready to merge?

@oli-obk
Copy link
Contributor

oli-obk commented Feb 15, 2019

@bors try

@bors
Copy link
Contributor

bors commented Feb 15, 2019

⌛ Trying commit 5680ee2bdc557143d41e3746d19831b3dafd1110 with merge 7231393efd7d34ee85e8f21b5f969232c3df0c8e...

@bors
Copy link
Contributor

bors commented Feb 15, 2019

☀️ Test successful - checks-travis
State: approved= try=True

@oli-obk
Copy link
Contributor

oli-obk commented Feb 15, 2019

@rust-timer build 7231393efd7d34ee85e8f21b5f969232c3df0c8e

@rust-timer
Copy link
Collaborator

Success: Queued 7231393efd7d34ee85e8f21b5f969232c3df0c8e with parent f47ec2a, comparison URL.

@rust-timer
Copy link
Collaborator

Finished benchmarking try commit 7231393efd7d34ee85e8f21b5f969232c3df0c8e

@alex
Copy link
Member Author

alex commented Feb 18, 2019

So the regressions here are frankly much larger than I'd expected. Since the only benchmark this really tests is the compiler, before I dig too far into looking at the code generated, are there any reasons the changes I made here would be particularly slow, even if they'd left the generated code the same?

@AaronKutch
Copy link
Contributor

AaronKutch commented Feb 18, 2019

If most of the overflow checks on simple usize increments done in hot loops are not optimized away, then I would expect that to be the biggest perf killer. Are there some small functions with tight loops we could look at the before and after assembly for?

@alex
Copy link
Member Author

alex commented Feb 19, 2019

Ok, one thing to note, comparing instructions (which is the default) is probably a mistake, we're for sure adding some instructions. The question is whether those instructions map to time. So for now I'm going to be looking at cycles, which seems like probably the best thing to consider.

It's not a lot better from this perspective, but it is a bit: https://perf.rust-lang.org/compare.html?start=f47ec2ad5b6887b3d400aee49e2294bd27733d18&end=7231393efd7d34ee85e8f21b5f969232c3df0c8e&stat=cycles%3Au

alex added 2 commits February 19, 2019 23:41
This is a prototype for rust-lang/rfcs#2635 to enable us to collect some measurements.
The failures aren't central to what this patch is about (they're related to linking and panic not being included in the final artifact), so they shouldn't disrupt perf measurements.
alex pushed a commit to alex/rust that referenced this pull request Feb 22, 2019
Instead of (&[u8], position) simply store the &[u8] and reslice.

This was originally written for rust-lang#58475, to see if removing the arithmetic helped with avoiding integer overflow checks, however I think the result is slightly more readable in general -- specifically the removal of set_position is a nice win. I think this might be a hair faster even without the changes in rust-lang#58475, but I haven't measured that.
Instead of (&[u8], position) simply store the &[u8] and reslice.

This was originally written for rust-lang#58475, to see if removing the arithmetic helped with avoiding integer overflow checks, however I think the result is slightly more readable in general -- specifically the removal of set_position is a nice win. I think this might be a hair faster even without the changes in rust-lang#58475, but I haven't measured that.
@alex alex force-pushed the usize-overflow-panic branch from 5680ee2 to 864dd3c Compare February 23, 2019 17:54
@alex
Copy link
Member Author

alex commented Feb 23, 2019

@oli-obk Would you mind kicking off another try + perf run?

@Mark-Simulacrum
Copy link
Member

@bors try

@bors
Copy link
Contributor

bors commented Feb 23, 2019

⌛ Trying commit 864dd3c with merge abc6e24...

bors added a commit that referenced this pull request Feb 23, 2019
[WIP] Make usize overflow always have debug-assertions semantics

This is a prototype for rust-lang/rfcs#2635 to enable us to collect some measurements.
@bors
Copy link
Contributor

bors commented Feb 23, 2019

☀️ Test successful - checks-travis
State: approved= try=True

@scottmcm
Copy link
Member

@rust-timer build abc6e24

@rust-timer
Copy link
Collaborator

Success: Queued abc6e24 with parent 02a4e27, comparison URL.

@rust-timer
Copy link
Collaborator

Finished benchmarking try commit abc6e24

@Dylan-DPC-zz
Copy link

ping from triage @alex you have a few conflicts to resolve

@Dylan-DPC-zz Dylan-DPC-zz added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 4, 2019
@alex
Copy link
Member Author

alex commented Mar 4, 2019

I'm currently working through profiling and finding places to optimize (e.g. #58692) to make this practical, so I have not been worried about mergability for the time being.

@Dylan-DPC-zz
Copy link

Ah okay. That's fine :)

@TimNN TimNN added A-allocators Area: Custom and system allocators and removed A-allocators Area: Custom and system allocators labels Mar 19, 2019
@Centril Centril added S-inactive Status: Inactive and waiting on the author. This is often applied to closed PRs. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Mar 30, 2019
@Centril
Copy link
Contributor

Centril commented Mar 30, 2019

Closing due to inactivity, feel free to reopen whenever.

@alex
Copy link
Member Author

alex commented May 24, 2020

In case anyone is eagerly following after this, I wanted to report that I had not forgotten about this! The first overflow optimization issue in LLVM is resolved, now I'm on to failing tests because of inability to do loop idiom recognition: https://bugs.llvm.org/show_bug.cgi?id=46057

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-inactive Status: Inactive and waiting on the author. This is often applied to closed PRs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.