Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Turn const eval queries into canonical queries #67717

Conversation

BenLewis-Seequent
Copy link

This is ground work for lazy normalization of consts. This enables invoking const eval queries even if the current param env has inference variable within it, which can occur during trait selection.

r? @nikomatsakis

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Dec 30, 2019
@rust-highfive

This comment has been minimized.

src/librustc/infer/canonical/mod.rs Show resolved Hide resolved
src/librustc/mir/interpret/mod.rs Show resolved Hide resolved
src/librustc_mir/const_eval/eval_queries.rs Outdated Show resolved Hide resolved
src/librustc_mir/interpret/eval_context.rs Outdated Show resolved Hide resolved
src/librustc_traits/resolve_vtable.rs Outdated Show resolved Hide resolved
@Centril
Copy link
Contributor

Centril commented Dec 30, 2019

cc @eddyb @oli-obk

@Centril Centril added the A-lazy-normalization Area: Lazy normalization (tracking issue: #60471) label Dec 30, 2019
@bors

This comment has been minimized.

@bors
Copy link
Contributor

bors commented Jan 4, 2020

☔ The latest upstream changes (presumably #67853) made this pull request unmergeable. Please resolve the merge conflicts.

@nikomatsakis
Copy link
Contributor

I spent time on Friday reviewing this, will finish up on Monday

@varkor
Copy link
Member

varkor commented Jan 6, 2020

Presumably this comment can also be removed now:

rust/src/librustc/ty/sty.rs

Lines 2399 to 2400 in 8f94d9b

// FIXME(eddyb) make `const_eval` a canonical query instead,
// that would properly handle inference variables in `substs`.

@BenLewis-Seequent
Copy link
Author

This doesn't fully resolve that comment, as eval needs to be passed an InferCtxt to take advantage of the canonical query. This can be done in a follow up case. I will adjust that comment to reflect this.

| ty::Predicate::ClosureKind(..)
| ty::Predicate::ConstEvaluatable(..) => true,
ty::Predicate::TypeOutlives(..) | ty::Predicate::RegionOutlives(..) => false,
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we filtering these bounds?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was an initial hack to remove inference variables from the param_env before calling subst_and_normalize_erasing_regions. I since replaced that in a later commit with infcx.at(..).normalize(..), which dies handle inference variables.

/// Stores the `Machine` instance.
pub machine: M,

/// The results of the type checker, from rustc.
pub tcx: TyCtxtAt<'tcx>,

pub(super) infcx: &'infcx InferCtxt<'infcx, 'tcx>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we use this field for anything?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we do, for normalizing, Instance::resolve_xxx calls, and recursive const_eval queries. We need to use this whenever we pass param_env into a query as it can contain inference variables.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we use inference variables? Couldn't miri be executed with the canonical form?
Canonical variables in scope would behave like type parameters, wouldn't they?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I never thought about that, I don't think we actually need inference variables as we aren't actually resolving them. Currently it seems like the canconical form is only ever used in preparing/start of queries, so I'm not sure what problems would there be like what happens to Bound variables if they are part of a canonical query down the line.

It would certainly make InferCtxt less invasive, which seems like a good thing. I would like to hear @nikomatsakis thought on this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really see a problem with it.

@nikomatsakis
Copy link
Contributor

This all looks "basically right" to me. Obviously, it needs to be rebased. I'm wondering also whether we should test performance -- seems like maybe yes?

@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-7 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2020-01-07T08:04:00.0616495Z ##[command]git remote add origin https://github.com/rust-lang/rust
2020-01-07T08:04:00.0627687Z ##[command]git config gc.auto 0
2020-01-07T08:04:00.0634777Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2020-01-07T08:04:00.0636762Z ##[command]git config --get-all http.proxy
2020-01-07T08:04:00.0639979Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/67717/merge:refs/remotes/pull/67717/merge
---
2020-01-07T08:54:21.1792026Z .................................................................................................... 1500/9477
2020-01-07T08:54:26.2360443Z .................................................................................................... 1600/9477
2020-01-07T08:54:30.5025919Z .................................................................................................... 1700/9477
2020-01-07T08:54:38.6253774Z .................................................................................................... 1800/9477
2020-01-07T08:54:45.6515295Z .i.................................................................................................. 1900/9477
2020-01-07T08:54:51.3555388Z .........................................................................................iiiii...... 2000/9477
2020-01-07T08:55:10.3821766Z .................................................................................................... 2200/9477
2020-01-07T08:55:12.2852296Z .................................................................................................... 2300/9477
2020-01-07T08:55:14.2611598Z .................................................................................................... 2400/9477
2020-01-07T08:55:19.3615455Z .................................................................................................... 2500/9477
---
2020-01-07T08:57:54.4490522Z .....................i...............i.............................................................. 4900/9477
2020-01-07T08:58:02.8914729Z .................................................................................................... 5000/9477
2020-01-07T08:58:08.2462595Z ..................................................................i................................. 5100/9477
2020-01-07T08:58:15.1103538Z .................................................................................................... 5200/9477
2020-01-07T08:58:21.7911158Z .................................ii.ii...........i.................................................. 5300/9477
2020-01-07T08:58:29.7790040Z .................................................................................................... 5500/9477
2020-01-07T08:58:38.1657444Z .................................................................................................... 5600/9477
2020-01-07T08:58:44.4443182Z .................i.................................................................................. 5700/9477
2020-01-07T08:58:49.7842155Z .................................................................................................... 5800/9477
2020-01-07T08:58:49.7842155Z .................................................................................................... 5800/9477
2020-01-07T08:58:59.3726530Z .................................................................................................... 5900/9477
2020-01-07T08:59:09.5109026Z .......ii...i..ii...........i....................................................................... 6000/9477
2020-01-07T08:59:24.4343714Z .................................................................................................... 6200/9477
2020-01-07T08:59:30.8781271Z .................................................................................................... 6300/9477
2020-01-07T08:59:30.8781271Z .................................................................................................... 6300/9477
2020-01-07T08:59:44.8105813Z ..................................i..ii............................................................. 6400/9477
2020-01-07T09:00:02.5232807Z .................................................................................................... 6600/9477
2020-01-07T09:00:04.2384480Z .........i.......................................................................................... 6700/9477
2020-01-07T09:00:06.1148732Z .................................................................................................... 6800/9477
2020-01-07T09:00:08.3185103Z .........i.......................................................................................... 6900/9477
---
2020-01-07T09:01:35.2477105Z .................................................................................................... 7500/9477
2020-01-07T09:01:38.8201833Z .................................................................................................... 7600/9477
2020-01-07T09:01:43.4881163Z .................................................................................................... 7700/9477
2020-01-07T09:01:53.5332695Z .................................................................................................... 7800/9477
2020-01-07T09:02:01.4036749Z .............................................iiii................................................... 7900/9477
2020-01-07T09:02:15.3733524Z .................................................................................................... 8100/9477
2020-01-07T09:02:21.6180880Z .................................................................................................... 8200/9477
2020-01-07T09:02:36.6774370Z .................................................................................................... 8300/9477
2020-01-07T09:02:44.0470552Z .................................................................................................... 8400/9477
---
2020-01-07T09:04:37.8508158Z ---- [ui] ui/const-generics/array-size-in-generic-struct-param.rs stdout ----
2020-01-07T09:04:37.8508440Z 
2020-01-07T09:04:37.8508866Z error: test compilation failed although it shouldn't!
2020-01-07T09:04:37.8509097Z status: exit code: 1
2020-01-07T09:04:37.8510232Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/const-generics/array-size-in-generic-struct-param.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-C" "prefer-dynamic" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/const-generics/array-size-in-generic-struct-param/a" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/const-generics/array-size-in-generic-struct-param/auxiliary"
2020-01-07T09:04:37.8511340Z ------------------------------------------
2020-01-07T09:04:37.8511557Z 
2020-01-07T09:04:37.8511975Z ------------------------------------------
2020-01-07T09:04:37.8512203Z stderr:
---
2020-01-07T09:04:37.8517912Z    |            ^^^^^^^^^^^^^^
2020-01-07T09:04:37.8518098Z    |
2020-01-07T09:04:37.8518274Z    = note: `#[warn(incomplete_features)]` on by default
2020-01-07T09:04:37.8518432Z 
2020-01-07T09:04:37.8518643Z error[E0284]: type annotations needed: cannot resolve `the constant `ArithArrayLen::<N>::0::{{constant}}#0` can be evaluated`
2020-01-07T09:04:37.8519480Z    |
2020-01-07T09:04:37.8519480Z    |
2020-01-07T09:04:37.8520070Z LL | struct ArithArrayLen<const N: usize>([u32; 0 + N]); // ok
2020-01-07T09:04:37.8523824Z    |                                      ^^^^^^^^^^^^ cannot resolve `the constant `ArithArrayLen::<N>::0::{{constant}}#0` can be evaluated`
2020-01-07T09:04:37.8524385Z error: aborting due to previous error
2020-01-07T09:04:37.8524604Z 
2020-01-07T09:04:37.8525191Z For more information about this error, try `rustc --explain E0284`.
2020-01-07T09:04:37.8525478Z 
---
2020-01-07T09:04:37.8530781Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:385:22
2020-01-07T09:04:37.8531044Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
2020-01-07T09:04:37.8531227Z 
2020-01-07T09:04:37.8531424Z 
2020-01-07T09:04:37.8533850Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "ui" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-7/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "7.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2020-01-07T09:04:37.8538750Z 
2020-01-07T09:04:37.8539074Z 
2020-01-07T09:04:37.8540252Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2020-01-07T09:04:37.8540331Z Build completed unsuccessfully in 0:54:59
2020-01-07T09:04:37.8540331Z Build completed unsuccessfully in 0:54:59
2020-01-07T09:04:37.8593078Z == clock drift check ==
2020-01-07T09:04:37.8611115Z   local time: Tue Jan  7 09:04:37 UTC 2020
2020-01-07T09:04:38.4041436Z   network time: Tue, 07 Jan 2020 09:04:38 GMT
2020-01-07T09:04:38.4045192Z == end clock drift check ==
2020-01-07T09:04:38.8025697Z 
2020-01-07T09:04:38.8142783Z ##[error]Bash exited with code '1'.
2020-01-07T09:04:38.8193470Z ##[section]Starting: Checkout
2020-01-07T09:04:38.8195214Z ==============================================================================
2020-01-07T09:04:38.8195269Z Task         : Get sources
2020-01-07T09:04:38.8195336Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@varkor
Copy link
Member

varkor commented Jan 7, 2020

@bors try @rust-timer queue

@rust-timer
Copy link
Collaborator

Awaiting bors try build completion

@bors
Copy link
Contributor

bors commented Jan 7, 2020

⌛ Trying commit 92b63b8 with merge 952f90b...

bors added a commit that referenced this pull request Jan 7, 2020
Turn const eval queries into canonical queries

This is ground work for lazy normalization of consts. This enables invoking const eval queries even if the current param env has inference variable within it, which can occur during trait selection.

r? @nikomatsakis
@bors
Copy link
Contributor

bors commented Jan 7, 2020

☀️ Try build successful - checks-azure
Build commit: 952f90b (952f90b6c788103776dcd34188e50d2ae459585a)

@rust-timer
Copy link
Collaborator

Queued 952f90b with parent 4f074de, future comparison URL.

@rust-timer
Copy link
Collaborator

Finished benchmarking try commit 952f90b, comparison URL.

/// Stores the `Machine` instance.
pub machine: M,

/// The results of the type checker, from rustc.
pub tcx: TyCtxtAt<'tcx>,

/// The inference context for inference variables that are within this `InterpCx`. Inference
/// variables may be within either the `param_env` or within `substs` of frames. Many operations
/// outside of mir which takes values that may contain variable require this context.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// outside of mir which takes values that may contain variable require this context.
/// outside of mir which take values that may contain variable require this context.

Also, what is "outside of mir"?

And -- does this basically mean we have to use this (one way or another) whenever we use param_env or a frame subst?

@bjorn3
Copy link
Member

bjorn3 commented Jan 7, 2020

This results in many more queries being invokes in clean incremental builds.

bors added a commit that referenced this pull request Jan 11, 2020
perf: Eagerly convert literals to consts

Previousely even literal constants were being converted to an `Unevaluted` constant for evaluation later. This seems unecessary as no more information is needed to be able to convert the literal to a mir constant.

Hopefully this will also minimise the performance impact of #67717, as far less constant evaluations are needed.
Dylan-DPC-zz pushed a commit to Dylan-DPC-zz/rust that referenced this pull request Jan 14, 2020
perf: Eagerly convert literals to consts

Previousely even literal constants were being converted to an `Unevaluted` constant for evaluation later. This seems unecessary as no more information is needed to be able to convert the literal to a mir constant.

Hopefully this will also minimise the performance impact of rust-lang#67717, as far less constant evaluations are needed.
bors added a commit that referenced this pull request Jan 15, 2020
perf: Eagerly convert literals to consts

Previousely even literal constants were being converted to an `Unevaluted` constant for evaluation later. This seems unecessary as no more information is needed to be able to convert the literal to a mir constant.

Hopefully this will also minimise the performance impact of #67717, as far less constant evaluations are needed.
@mark-i-m
Copy link
Member

Would it make sense to rebase and do another perf run now that #68118 is merged?

@nikomatsakis
Copy link
Contributor

Sounds good to me!

@BenLewis-Seequent
Copy link
Author

Waiting for discussion on whether we can just use the canonical form instead of introducing InferCtxt into miri. #67717 (comment)

@Aaron1011
Copy link
Member

Waiting for discussion on whether we can just use the canonical form instead of introducing InferCtxt into miri. #67717 (comment)

@Skinny121: It looks like that should work, if I understand the linked discussion.

bors added a commit that referenced this pull request Feb 28, 2020
…komatsakis

Canonicalize inputs to const eval where needed

Canonicalize inputs to const eval, so that they can contain inference variables. Which enables invoking const eval queries even if the current param env has inference variable within it, which can occur during trait selection.

This is a reattempt of #67717, in a far less invasive way.

Fixes #68477

r? @nikomatsakis
cc @eddyb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-lazy-normalization Area: Lazy normalization (tracking issue: #60471) S-waiting-on-review Status: Awaiting review from the assignee but also interested parties.
Projects
None yet
Development

Successfully merging this pull request may close these issues.