Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get rid of ConstValue::ScalarPair #55392

Closed
wants to merge 3 commits into from

Conversation

oli-obk
Copy link
Contributor

@oli-obk oli-obk commented Oct 26, 2018

based on #55293 (only the last commit is part of this PR)

@rust-highfive
Copy link
Collaborator

r? @davidtwco

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 26, 2018
@oli-obk
Copy link
Contributor Author

oli-obk commented Oct 26, 2018

This can potentially have adverse effects on performance

@bors try

@bors
Copy link
Contributor

bors commented Oct 26, 2018

⌛ Trying commit 8ee3e22d801ed63af310d87ba0bbb0b1913c2c1d with merge a08ddc905730cc7c3a758585abcdd8f0608cdfff...

@oli-obk
Copy link
Contributor Author

oli-obk commented Oct 26, 2018

@rust-timer build a08ddc905730cc7c3a758585abcdd8f0608cdfff

@rust-timer
Copy link
Collaborator

Success: Queued a08ddc905730cc7c3a758585abcdd8f0608cdfff with parent 694cf75, comparison URL.

@oli-obk
Copy link
Contributor Author

oli-obk commented Oct 26, 2018

r? @RalfJung

@bors
Copy link
Contributor

bors commented Oct 26, 2018

☀️ Test successful - status-travis
State: approved= try=True

@rust-timer
Copy link
Collaborator

Finished benchmarking try commit a08ddc905730cc7c3a758585abcdd8f0608cdfff

@oli-obk
Copy link
Contributor Author

oli-obk commented Oct 26, 2018

Small gains for tuple-stress and ucd, big losses for coercions.

Kinda expected. This is a very naive impl: even though str and slices are now ByRef, if they are used in another constant an "optimization" kicks in and converts it back to Value::ScalarPair only to then write it back into memory.

@RalfJung
Copy link
Member

RalfJung commented Oct 30, 2018

Seems this got out of sync with its base PR? I wanted to look at oli-obk/rust@self_managing_allocations...oli-obk:scalar_single to see what is actually new, but that looks like it still contains #55293. Could you rebase?

@jkordish
Copy link

jkordish commented Nov 7, 2018

Ping from triage. @oli-obk checking for a status update if you have time. thanks!

@oli-obk oli-obk added the S-blocked Status: Blocked on something else such as an RFC or other implementation work. label Nov 8, 2018
@oli-obk
Copy link
Contributor Author

oli-obk commented Nov 8, 2018

This PR is blocked on #55293 (which in turn is blocked, too)

@Mark-Simulacrum Mark-Simulacrum removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Nov 15, 2018
@oli-obk oli-obk added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-blocked Status: Blocked on something else such as an RFC or other implementation work. labels Nov 25, 2018
@oli-obk
Copy link
Contributor Author

oli-obk commented Nov 25, 2018

@RalfJung do you think it would be reasonable to not read a ScalarPair automatically if the backing memory is ByRef and just read it when needed?

Kindof adding a ByRefReadOnly optimization for LocalValue?

@RalfJung
Copy link
Member

I don't know what you mean -- by either of your two paragraphs.^^

ScalarPair is only for pattern matching, right? Can't that have its own local version of Immediate?

@oli-obk
Copy link
Contributor Author

oli-obk commented Nov 25, 2018

So... scalar pairs are ByRef inside constants. So if you write ["foo", "bar"] you end up with a MIR like

let _1 = "foo"; // read `ScalarPair` from `ByRef` constant
let _2 = "bar"; // read `ScalarPair` from `ByRef` constant
let _3 = [_1, _2]; // write `ScalarPair` to `ByRef` local

I want to skip the ScalarPair intermediate representation and just have _1 and _2 take a form of Cow representation for their value. So we only move to a representation other than the source value when trying to write to the value. This only works for values obtained from interned memory, because otherwise we might end up pointing to outdated data.

@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-5.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:1770b8d8:start=1543415461496576628,finish=1543415464715278338,duration=3218701710
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#Pull-Requests-and-Security-Restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
Setting environment variables from .travis.yml
$ export IMAGE=x86_64-gnu-llvm-5.0
---

[00:03:05] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:03:05] tidy error: /checkout/src/librustc_mir/const_eval.rs:146: line longer than 100 chars
[00:03:07] some tidy checks failed
[00:03:07] 
[00:03:07] 
[00:03:07] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:03:07] 
[00:03:07] 
[00:03:07] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:03:07] Build completed unsuccessfully in 0:00:57
[00:03:07] Build completed unsuccessfully in 0:00:57
[00:03:07] Makefile:79: recipe for target 'tidy' failed
[00:03:07] make: *** [tidy] Error 1
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:0e1dcd00
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Wed Nov 28 14:34:20 UTC 2018
---
travis_time:end:1f8ea9ac:start=1543415661250998543,finish=1543415661255737622,duration=4739079
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:2a16b1e7
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:09781cc7
travis_time:start:09781cc7
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:02d4383c
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-5.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:23bf3efe:start=1543493064279087024,finish=1543493119398660625,duration=55119573601
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#Pull-Requests-and-Security-Restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
Setting environment variables from .travis.yml
$ export IMAGE=x86_64-gnu-llvm-5.0
---
[00:46:49] .................................................................................................... 2300/5063
[00:46:53] .................................................................................................... 2400/5063
[00:46:57] .................................................................................................... 2500/5063
[00:47:00] .................................................................................................... 2600/5063
[00:47:04] .........................................................F.......................................... 2700/5063
[00:47:08] .......................................F............................................................ 2800/5063
[00:47:14] .................................................................................................... 3000/5063
[00:47:17] .....................................................................i.............................. 3100/5063
[00:47:17] .....................................................................i.............................. 3100/5063
[00:47:20] .....................................F.F............................................................ 3200/5063
[00:47:27] .................................................................................................... 3400/5063
[00:47:30] .................................................................................................... 3500/5063
[00:47:33] .............ii..................................................................................... 3600/5063
[00:47:35] ................................i................................................................... 3700/5063
---
[00:48:15] failures:
[00:48:15] 
[00:48:15] ---- [ui] ui/issues/issue-46964.rs stdout ----
[00:48:15] 
[00:48:15] error: test compilation failed although it shouldn't!
[00:48:15] status: exit code: 101
[00:48:15] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/issues/issue-46964.rs" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-C" "prefer-dynamic" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/issues/issue-46964/a" "-Crpath" "-O" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/issues/issue-46964/auxiliary" "-A" "unused"
[00:48:15] ------------------------------------------
[00:48:15] 
[00:48:15] ------------------------------------------
[00:48:15] ------------------------------------------
[00:           0,\n                0\n            ],\n            relocations: Relocations(\n                SortedMap {\n                    data: [\n                        (\n                            Size {\n                                raw: 0\n                            },\n                            (\n                                (),\n                                AllocId(\n                                    0\n                                )\n                            )\n                        )\n                    ]\n                }\n            ),\n            undef_mask: UndefMask {\n                blocks: [\n                    65535\n                ],\n                len: Size {\n                    raw: 16\n                }\n            },\n            align: Align {\n                pow2: 3\n            },\n            mutability: Mutable,\n            extra: ()\n        },\n        Size {\n            raw: 0\n        }\n    )\n}\n\n"}
[00:48:15] note: Run with `RUST_BACKTRACE=1` for a backtrace.
[00:48:15] note: Run with `RUST_BACKTRACE=1` for a backtrace.
[00:48:15] {"message":"aborting due to previous error","code":null,"level":"error","spans":[],"children":[],"rendered":"error: aborting due to previous error\n\n"}
[00:48:15] note: the compiler unexpectedly panicked. this is a bug.
[00:48:15] 
[00:48:15] note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports
[00:48:15] 
[00:48:15] 
[00:48:15] note: rustc 1.32.0-dev running on x86_64-unknown-linux-gnu
[00:48:15] 
[00:48:15] note: compiler flags: -Z ui-testing -Z unstable-options -C prefer-dynamic -C rpath
[00:48:15] 
[00:48:15] ------------------------------------------
[00:48:15] 
[00:48:15] thread '[ui] ui/issues/issue-46964.rs' panicked at 'explicit panic', src/tools/compiletest/src/runtest.rs:3282:9
[00:48:15] thread '[ui] ui/issues/issue-46964.rs' panicked at 'explicit panic', src/tools/compiletest/src/runtest.rs:3282:9
[00:48:15] note: Run with `RUST_BACKTRACE=1` for a backtrace.
[00:48:15] 
[00:48:15] ---- [ui] ui/issues/issue-51655.rs stdout ----
[00:48:15] 
[00:48:15] error: test compilation failed although it shouldn't!
[00:48:15] status: exit code: 101
[00:48:15] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/issues/issue-51655.rs" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-C" "prefer-dynamic" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/issues/issue-51655/a" "-Crpath" "-O" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/issues/issue-51655/auxiliary" "-A" "unused"
[00:48:15] ------------------------------------------
[00:48:15] 
[00:48:15] ------------------------------------------
[00:48:15] stderr:
[00:48:15] stderr:
[00:48:15] ------------------------------------------
[00:48:15] {"message":"src/librustc_mir/hair/pattern/_match.rs:1395: unexpected ConstValue: Const {\n    ty: &[u8],\n    val: ByRef(\n        AllocId(\n            4\n        ),\n        Allocation {\n            bytes: [\n                0,\n                0,\n                0,\n                0,\n                0,\n                0,\n                0,\n                0              0,\n                0,\n                0,\n                0,\n                0\n            ],\n            relocations: Relocations(\n                SortedMap {\n                    data: [\n                        (\n                            Size {\n                                raw: 0\n                            },\n                            (\n                                (),\n                                AllocId(\n                                    2\n                                )\n                            )\n                        )\n                    ]\n                }\n            ),\n            undef_mask: UndefMask {\n                blocks: [\n                    65535\n                ],\n                len: Size {\n                    raw: 16\n                }\n            },\n            align: Align {\n                pow2: 3\n            },\n            mutability: Mutable,\n            extra: ()\n        },\n        Size {\n            raw: 0\n        }\n    )\n}\n\n"}
[00:48:15] note: Run with `RUST_BACKTRACE=1` for a backtrace.
[00:48:15] note: Run with `RUST_BACKTRACE=1` for a backtrace.
[00:48:15] {"message":"aborting due to previous error","code":null,"level":"error","spans":[],"children":[],"rendered":"error: aborting due to previous error\n\n"}
[00:48:15] note: the compiler unexpectedly panicked. this is a bug.
[00:48:15] 
[00:48:15] note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports
[00:48:15] 
[00:48:15] 
[00:48:15] note: rustc 1.32.0-dev running on x86_64-unknowr types with a small number of variants\n(like enums) you should probably cover all cases explicitly. Alternatively, the\nunderscore `_` wildcard pattern can be added after all other patterns to match\n\"anything else\". Example:\n\n```\nenum Terminator {\n    HastaLaVistaBaby,\n    TalkToMyHand,\n}\n\nlet x = Terminator::HastaLaVistaBaby;\n\nmatch x {\n    Terminator::TalkToMyHand => {}\n    Terminator::HastaLaVistaBaby => {}\n}\n\n// or:\n\nmatch x {\n    Terminator::TalkToMyHand => {}\n    _ => {}\n}\n```\n"},"level":"error","spans":[{"file_name":"/checkout/src/test/ui/match/match-byte-array-patterns-2.rs","byte_start":519,"byte_end":522,"line_start":14,"line_end":14,"column_start":11,"column_end":14,"is_primary":true,"text":[{"text":"    match buf { //~ ERROR non-exhaustive","highlight_start":11,"highlight_end":14}],"label":"pattern `&[_, _, _, _]` not covered","suggested_replacement":null,"suggestion_applicability":null,"expansion":null}],"children":[],"rendered":"error[E0004]: non-exhaustive patterns: `&[_, _, _, _]` not covered\n  --> /checkout/src/test/ui/match/match-byte-array-patterns-2.rs:14:11\n   |\nLL |     match buf { //~ ERROR non-exhaustive\n   |           ^^^ pattern `&[_, _, _, _]` not covered\n\n"}
[00:48:15] {"message":"non-exhaustive patterns: `&[]` and `&[_]` not covered","code":{"code":"E0004","explanation":"\nThis error indicates that the compiler cannot guarantee a matching pattern for\none or more possible inputs to a match expression. Guaranteed matches are\nrequired in order to assign values to match expressions, or alternatively,\ndetermine the flow of execution. Erroneous code example:\n\n```corevious errors","code":null,"level":"error","spans":[],"children":[],"rendered":"error: aborting due to 2 previous errors\n\n"}
[00:48:15] {"message":"For more information about this error, try `rustc --explain E0004`.","code":null,"level":"","spans":[],"children":[],"rendered":"For more information about this error, try `rustc --explain E0004`.\n"}
[00:48:15] ------------------------------------------
[00:48:15] 
[00:48:15] thread '[ui] ui/match/match-byte-array-patterns-2.rs' panicked at 'explicit panic', src/tools/compiletest/src/runtest.rs:3282:9
[00:48:15] 
[00:48:15] 
[00:48:15] ---- [ui] ui/match/match-byte-array-patterns.rs stdout ----
[00:48:15] 
[00:48:15] error: Error: expected failure status (Some(1)) but received status Some(101).
[00:48:15] status: exit code: 101
[00:48:15] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/match/match-byte-array-patterns.rs" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-C" "prefer-dynamic" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/match/match-byte-array-patterns/a" "-Crpath" "-O" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/match/match-byte-array-patterns/auxiliary" "-A" "unused"
[00:48:15] ------------------------------------------
[00:48:15] 
[00:48:15] ------------------------------------------
[00:48:15] stderr:
---
57240 ./obj/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/lib
56896 ./src/llvm/test/MC
56108 ./obj/build/x86_64-unknown-linux-gnu/stage0-bootstrap-tools/release/build
55828 ./obj/build/x86_64-unknown-linux-gnu/stage1-std/x86_64-unknown-linux-gnu
55824 ./obj/build/x86_64-u-f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:0c418d20
travis_time:start:0c418d20
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:0e9e57aa
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors
Copy link
Contributor

bors commented Nov 29, 2018

☔ The latest upstream changes (presumably #56340) made this pull request unmergeable. Please resolve the merge conflicts.

@RalfJung
Copy link
Member

RalfJung commented Dec 3, 2018

@oli-obk I still don't get it. Are you suggesting we change something in the miri engine? That seems unnecessary, the issue here is just about how to communicate the results of computation to the rest of the compiler. But if you do, could you spell out which miri engine types/operations you intent to change how?

The way I see it, we have ty::Const which is roughly like MPlaceTy and great for just representing arbitrary blobs of data of any type. But in the miri engine we also have ImmTy because we sometimes need to work on data, like when we do casts or arithmetic, and then it's nice to have your data right there and not behind a memory indirection. So when something wants to work with constants of particular types, it should also turn them into such a representation. Maybe the pattern match desugaring can just create an EvalContext and call read_immediate, or maybe we have a ty::ConstImmediate and some fn read(tcx, cnst: ty::Const) -> ty::ConstImmediate, or so. That's what I would expect, anyway.

@oli-obk
Copy link
Contributor Author

oli-obk commented Dec 3, 2018

So when something wants to work with constants of particular types, it should also turn them into such a representation.

I agree, but what I'm suggesting is to do this the first time we actually try to read an immediate from that local and not do any work before that.

I have (since writing #55392 (comment)) implemented a scheme that skips even that step.

Instead of creating a ConstValue::ScalarPair, which we only need for strings, I created a special ConstValue::Slice, which has an AllocId and &'tcx Allocation, similar to ByRef, but that Allocation is not for the &str fat pointer, but just for the string. The length is taken from the Allocation's size and any pointers to the allocation need to be created via Pointer::from(alloc_id).

This is basically a step all the way back to ScalarPair, except that we only use it for the common cases of &str, &[u8] and &[u8; N]. It simplifies some code around these uses cases imo.

As a next step I am considering doing the same for ConstValue::Scalar and change it to ConstValue::Int, without a possibility to store pointers in there.
We'd not use the layout to decide whether to do the optimization, but do it purely type-based for the few types that can actually benefit from the optimization

@RalfJung
Copy link
Member

RalfJung commented Dec 3, 2018

I agree, but what I'm suggesting is to do this the first time we actually try to read an immediate from that local and not do any work before that.

Yes that's exactly what I was saying? When you need to do work on something, you convert it to an immediate form. The entire rest of the codebase doesn't even have no know that this form exists.

ConstValue::Slice

I am increasingly confused. The goal, I thought, was to remove variants from ConstValue. There should be only one variant, ByRef. There will probably also have to be a type that allow for Unevaluated, let's call it LazyConstValue -- but crucially, the const_eval query returns a single-variant type.

The length is taken from the Allocation's size

But what about subslices...?

You lost me entirely, I'm afraid.

@oli-obk
Copy link
Contributor Author

oli-obk commented Dec 3, 2018

Yes that's exactly what I was saying? When you need to do work on something, you convert it to an immediate form. The entire rest of the codebase doesn't even have no know that this form exists.

No, you want to convert it the moment you read from the global to a local. I want to convert it the first time someone wants to read an immediate from that local. So where we currently move from Allocation in the global to Immediate in the local and then immediately back into an Allocation, I want to place the Allocation there directly and just copy the relevant bytes when copying to another Allocation.

But as I said, I have moved to a different scheme.

I am increasing confused. The goal, I thought, was to remove variants from ConstValue. There should be only one variant, ByRef. There will probably also have to be a type that allow for Unevaluated, let's call it LazyConstValue -- but crucially, the const_eval query returns a single-variant type.

That's not gonna work out of performance reasons. I tried, and each time the perf results were quite clear that we e.g. don't want to have to read array lengths from Allocations or move fat pointers from immediate representations to Allocation-backed representations (https://perf.rust-lang.org/compare.html?start=694cf752988728ba5da85c36c2deaeefeb49c05e&end=a08ddc905730cc7c3a758585abcdd8f0608cdfff).

But what about subslices...?

Currently a non-issue, so I left it out of the implementation.

@RalfJung
Copy link
Member

RalfJung commented Dec 3, 2018

No, you want to convert it the moment you read from the global to a local.

Wait, why are we talking about locals? Inside miri, everything already works. ConstValue is for outside miri only. There are no locals or globals, just contants. What is even going on?

But as I said, I have moved to a different scheme.

Okay. So now we are talking about another scheme I don't understand either. Hooray :D

That's not gonna work out of performance reasons. I tried, and each time the perf results were quite clear that we e.g. don't want to have to read array lengths from Allocations

There is no reason that ty::Array should contain the same type that is returned by const_eval. In fact I think there are many reasons it should not. The length in ty::Array should be "either Unevaluated or u64". That's an invariant we already have, we just fail to encode it in the type. Whatever code does the normalization can call const_eval, and then immediately read the u64. In fact it has to because in this proposal we now have a proper type in ty::Array that doesn't let us do anything else. :) (It also is currently a Const and not a ConstValue and hence redundantly stores the fact that the type of the array length is usize, or am I missing something?)

Instead of guessing in const_eval what the caller wants, we should have the caller tell us.

Another invariant we fail to encode in the type is that const_eval never returns Unevaluated. We should fix that, too.

or move fat pointers from immediate representations to Allocation-backed representations

What does this mean?

@oli-obk
Copy link
Contributor Author

oli-obk commented Dec 4, 2018

The length in ty::Array should be "either Unevaluated or u64".

that... makes so much more sense than any of the crazyness I have come up with and failed to make nice.

What does this mean?

Encoding fat pointers in constants by using ByRef instead of ScalarValue causes any use of said constant in another to read it back to Immediate, only to be written back to a ByRef later.

@oli-obk oli-obk closed this Dec 4, 2018
@RalfJung
Copy link
Member

RalfJung commented Dec 4, 2018

Encoding fat pointers in constants by using ByRef instead of ScalarValue causes any use of said constant in another to read it back to Immediate, only to be written back to a ByRef later.

Ah I see. Where does that happen though? And can we do something similar to what I proposed for array lengths there?

@oli-obk
Copy link
Contributor Author

oli-obk commented Dec 4, 2018

Ah I see. Where does that happen though? And can we do something similar to what I proposed for array lengths there?

I don't think so, every constant in MIR can possibly be a str and thus a fat pointer. And these are the very constants that we would like to not have a Scalar and ScalarPair variant.

@eddyb suggested once to do some constant folding during MIR building instead of punting it to const eval later. But that would mean [5, 6] gets turned into a single constant, while [4 + 1, 6] does not. Of course, for strings that's kind of fine. We'd just allow ["foo", "bar"] and not [&"fooo"[0..3], "bar"] to get the optimization.

@RalfJung
Copy link
Member

RalfJung commented Dec 4, 2018

Those constants are only ever accessed for codegen, right? The only difference between what we do now and what we would do then is we would access the Allocation once per monomorphization, instead of once per MIR. Given the cost of monomorphization, how can that even be noticeable?

EDIT: Okay, there's also const_prop. But that should also just do one access per constant.

@RalfJung
Copy link
Member

RalfJung commented Dec 4, 2018

But until we have figured that out, here's how I imagine things could look:

enum LazyConstValue<'tcx, T> {
    Unevaluated(DefId, &'tcx Substs<'tcx>),
    Evaluated(T),
}
enum ConstValue<'tcx> {
    Scalar(Scalar),
    ScalarPair(Scalar, Scalar),
    Indirect(AllocId, &'tcx Allocation, Size),
}

const_eval returns ConstValue. ty::Array uses LazyConstValue<u64>. mir::Constant/ty::Const use LazyConstValue<ConstValue>.

Does that make sense?

@oli-obk
Copy link
Contributor Author

oli-obk commented Dec 4, 2018

Does that make sense?

Yes, this is a 1:1 copy of what I prototyped on the train this morning :D

@eddyb
Copy link
Member

eddyb commented Dec 4, 2018

@RalfJung While that's nicer to work with, it's a bit sad, because it means ty::Array is different from struct Foo<T, const N: usize>, and there have been some efforts to "normalize" the typesystem into being a "type constructor" combined with some applied generics (in the form of Substs).
Someone from @rust-lang/wg-traits might be able to explain better.

@oli-obk
Copy link
Contributor Author

oli-obk commented Dec 4, 2018

We have two ways that I can come up with off the top of my head to have our cake and eat it, too:

  1. add queries (or some faster cache) for converting &'tcx Const<'tcx> into u64 for array lengths
  2. add a "normalized" field to &'tcx Const<'tcx> which contains the Scalar and ScalarPair values of the constant, and change the val field to be the id, offset, allocation tuple of the constant.

@oli-obk oli-obk deleted the scalar_single branch June 15, 2020 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants