Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CTFE eval_fn_call: use FnAbi to determine argument skipping and compatibility #91342

Merged
merged 7 commits into from
Dec 24, 2021

Conversation

RalfJung
Copy link
Member

@RalfJung RalfJung commented Nov 29, 2021

This makes use of the FnAbi type in CTFE/Miri, which @eddyb has been saying for years is what we should do.^^ FnAbi is used to

  • determine which arguments to skip (rather than the previous heuristic of skipping ZST arguments with the Rust ABI)
  • impose further restrictions on whether caller and callee are consistent in how a given argument is passed

I was hoping it would also simplify the code, but that is not the case -- the previous type compatibility checks are still required (AFAIK), only the ZST skipping is gone and that took barely any code. We also need some hacks because FnAbi assumes a certain way of implementing caller_location (by passing extra arguments), but Miri can just read the caller location from the call stack so it doesn't need those arguments. (The fact that every backend has to separately implement support for these arguments seems suboptimal -- looks like this might have been better implemented on the MIR level.) To avoid having to implement those unnecessary arguments in Miri, we just compute whether the argument is present on the caller/callee side, but don't actually pass that argument around.

I have no idea if this looks the way @eddyb thinks it should look... but it makes Miri's test suite pass. ;)
One of rustc's tests fails unfortunately (ui/const-generics/issues/issue-67739.rs), some const generic code that is evaluated too early -- I think that should raise TooGeneric but instead it ICEs. My assumption is this is some FnAbi code that has not been properly tested on polymorphic code, but it might also be me calling that FnAbi code the wrong way.

r? @oli-obk @eddyb
Fixes #56166
Miri PR at rust-lang/miri#1928

@rust-highfive
Copy link
Collaborator

Some changes occured to the CTFE / Miri engine

cc @rust-lang/miri

Some changes occured to the CTFE / Miri engine

cc @rust-lang/miri

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Nov 29, 2021
@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@eddyb
Copy link
Member

eddyb commented Nov 29, 2021

(The fact that every backend has to separately implement support for these arguments seems suboptimal -- looks like this might have been better implemented on the MIR level.)

We can't, because whether to pass the argument or not can vary across a trait's impls, so from the caller's perspective it's undecided until monomorphization.

@rust-log-analyzer

This comment has been minimized.

// For comparing the PassMode, we allow the attributes to differ
// (e.g., it is okay for NonNull to differ between caller and callee).
// FIXME: Are there attributes (`call::ArgAttributes`) that do need to be checked?
let mode_compat = || {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there should be a strict mode option that is closer to the old behavior to catch cases where someone depends on details of the rust abi.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new behavior is strictly more strict than the old one, so... I am confused by this comment.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old behavior was to say that both are incompatible when the type doesn't match and either the rust abi isn't used or the rust abi used but it isn't an argument with the direct or pair pass mode. This is much stricter than considering the argument to be compatible when the pass modes match (minus arg attributes).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example the old code would consider struct Foo(u8, u64, u8); and struct Bar(u64, u8, u8); to be incompatible, but the new code will consider them compatible for the rust abi as both get a cast pass mode that passes a 64bit argument (for the u64) and a 16bit argument (packing both u8 fields together).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still also compare the types -- I first thought comparing the modes would be enough but they do not contain remotely enough information (e.g., two Indirect arguments might have completely different types).

The idea is that the old type-based restrictions apply plus the mode has to be the same. Possibly the type-based restrictions could be relaxed (just requiring size and alignment to be the same, or even just the size) -- but so far I am not proposing to do that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After this PR the amount of cases is drastically increased.

For the Rust ABI, it is not. I am confused why you say that. The old conditions must be true and the pass mode must match.

Given that FnAbi does not contain the Abi, I also don't think we should make anything depend on the Abi.

Copy link
Member Author

@RalfJung RalfJung Dec 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also leads to more code being accepted for rust_abi where the indirect and cast pass modes were previously always considered incompatible.

layout_compat implements exactly the old layout checks (except it ignores the Abi), and the requirement for compatibility is layout_compat() && pad_compat() && mode_compat(). So strictly fewer things are accepted. You keep repeating that more things are accepted but that's not what the code does... or I am terribly misunderstanding you.

So, no, what you say is just not true I think. Previously we did not even look at modes so indirect and cast could easily have been considered compatible!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the rust abi it is unstable if an argument will be passed as indirect or cast. I can't think of any valid reasons to allow mismatched types for them even for libstd internal usage.

We accept transmutes between repr(Rust) types whose fields happen to be laid out in the right way. I think we likewise should accept function calls whose FnAbi happens to match. I think anything else would be (a) inconsistent, and (b) not even well-defined since we don't have a clear spec for what is guaranteed to hold true about the Rust ABI into the future.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wish I could replace layout_compat by something more principled that reflects whether under current rustc, the data would actually be passed properly, but I don't know nearly enough about all this FnAbi madness to implement such a check. Unfortunately, ignoring the layout entirely is wrong (PassMode does not capture enough information), and comparing it for full equality is also wrong. It doesn't look like we actually have an abstraction that properly represents if arguments are compatible. Based on what @eddyb had said in the past I thought FnAbi provides that but now I don't think this is the case.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specifically, I am very confused about two points:

  • Your summary even says that after this PR, for the Rust ABI, we accept strictly less than before. Before it was 'types equal or compatible scalar or scalar pair', now it is 'layout, padding, and pass mode compatible'. Here, 'layout compatible' is defined as 'types equal or compatible scalar or scalar pair', i.e., exactly the old definition. Do we agree so far? (I am assuming that equal types implies equal pass mode and padding; if that is not the case your summary is incorrect.)
    So I don't see why it would make any sense to introduce an even stricter mode with this PR. (I also don't think we should even have that mode, or at least I don't want to be the one implementing it -- but even leaving that aside, it is orthogonal to this PR.)
  • It is true that this PR accepts some calls that were previously rejected, namely calls for non-Rust ABIs where the types are different but the layout is compatible and the padding and pass mode match. This might be worth debating (it is the only change here that I see as potentially contentious), but I don't see a good reason to single out the Rust ABI now that we have pass mode information available. From what I understand, however, your 'strict mode' is not about this case.

@apiraino apiraino added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Dec 1, 2021
_ret,
_unwind,
)?
.map(|(body, _instance)| (body, instance)));
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not happy about this hack but it is needed to keep the const-time backtraces the way they were...

@rust-log-analyzer

This comment has been minimized.

@RalfJung RalfJung force-pushed the fn-abi branch 2 times, most recently from 094f199 to 70066b2 Compare December 3, 2021 02:37
@RalfJung
Copy link
Member Author

RalfJung commented Dec 3, 2021

Okay I think this should be mostly set now -- except for @bjorn3's concern about a 'strict mode' which I have not understood yet.

@oli-obk
Copy link
Contributor

oli-obk commented Dec 23, 2021

Did a review pass, too. Considering that and the comments above:

@bors r=eddyb,oli-obk

@bors
Copy link
Contributor

bors commented Dec 23, 2021

📌 Commit 56b7d5f has been approved by eddyb,oli-obk

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 23, 2021
@bors
Copy link
Contributor

bors commented Dec 23, 2021

⌛ Testing commit 56b7d5f with merge 17d141cdaad042d4541355ae007ce2b2279a0a6e...

@rust-log-analyzer
Copy link
Collaborator

The job x86_64-msvc-2 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
9 note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
10 


The actual run.stderr differed from the expected run.stderr.
Actual run.stderr saved to D:\a\rust\rust\build\x86_64-pc-windows-msvc\test\ui\panics\panic-short-backtrace-windows-x86_64\panic-short-backtrace-windows-x86_64.run.stderr
error: 1 errors occurred comparing run output.
status: exit code: 101
status: exit code: 101
command: PATH="D:\a\rust\rust\build\x86_64-pc-windows-msvc\stage2\lib\rustlib\x86_64-pc-windows-msvc\lib;C:\Program Files (x86)\Windows Kits\10\bin\x64;C:\Program Files (x86)\Windows Kits\10\bin\10.0.22000.0\x64;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64;C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX64\x64;D:\a\rust\rust\build\x86_64-pc-windows-msvc\stage0-bootstrap-tools\x86_64-pc-windows-msvc\release\deps;D:\a\rust\rust\build\x86_64-pc-windows-msvc\stage0\bin;D:\a\rust\rust\ninja;D:\a\rust\rust\msys2\mingw64\bin;C:\hostedtoolcache\windows\Python\3.10.1\x64\Scripts;C:\hostedtoolcache\windows\Python\3.10.1\x64;C:\msys64\usr\bin;D:\a\rust\rust\sccache;C:\Program Files\MongoDB\Server\5.0\bin;C:\aliyun-cli;C:\vcpkg;C:\cf-cli;C:\Program Files (x86)\NSIS;C:\tools\zstd;C:\Program Files\Mercurial;C:\hostedtoolcache\windows\stack\2.7.3\x64;C:\cabal\bin;C:\ghcup\bin;C:\tools\ghc-9.2.1\bin;C:\Program Files\dotnet;C:\mysql\bin;C:\Program Files\R\R-4.1.2\bin\x64;C:\SeleniumWebDrivers\GeckoDriver;C:\Program Files (x86)\sbt\bin;C:\Program Files (x86)\GitHub CLI;C:\Program Files\Git\bin;C:\Program Files (x86)\pipx_bin;C:\hostedtoolcache\windows\go\1.15.15\x64\bin;C:\hostedtoolcache\windows\Python\3.7.9\x64\Scripts;C:\hostedtoolcache\windows\Python\3.7.9\x64;C:\hostedtoolcache\windows\Ruby\2.5.9\x64\bin;C:\tools\kotlinc\bin;C:\hostedtoolcache\windows\Java_Temurin-Hotspot_jdk\8.0.312-7\x64\bin;C:\npm\prefix;C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\wbin;C:\ProgramData\kind;C:\Program Files\Eclipse Foundation\jdk-8.0.302.8-hotspot\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0;C:\Windows\System32\OpenSSH;C:\ProgramData\Chocolatey\bin;C:\Program Files\Docker;C:\Program Files\PowerShell\7;C:\Program Files\Microsoft\Web Platform Installer;C:\Program Files\dotnet;C:\Program Files\Microsoft SQL Server\130\Tools\Binn;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\170\Tools\Binn;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit;C:\Program Files (x86)\Microsoft SQL Server\110\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\140\DTS\Binn;C:\Program Files (x86)\Microsoft SQL Server\150\DTS\Binn;C:\Program Files\nodejs;C:\Program Files\OpenSSL\bin;C:\Strawberry\c\bin;C:\Strawberry\perl\site\bin;C:\Strawberry\perl\bin;C:\ProgramData\chocolatey\lib\pulumi\tools\Pulumi\bin;C:\Program Files\TortoiseSVN\bin;C:\Program Files\CMake\bin;C:\ProgramData\chocolatey\lib\maven\apache-maven-3.8.4\bin;C:\Program Files\Microsoft Service Fabric\bin\Fabric\Fabric.Code;C:\Program Files\Microsoft SDKs\Service Fabric\Tools\ServiceFabricLocalClusterManager;C:\Program Files\Git\cmd;C:\Program Files\Git\mingw64\bin;C:\Program Files\Git\usr\bin;C:\tools\php;C:\Program Files (x86)\sbt\bin;C:\SeleniumWebDrivers\ChromeDriver;C:\SeleniumWebDrivers\EdgeDriver;C:\Program Files\Amazon\AWSCLIV2;C:\Program Files\Amazon\SessionManagerPlugin\bin;C:\Program Files\Amazon\AWSSAMCLI\bin;C:\Program Files (x86)\Google\Cloud SDK\google-cloud-sdk\bin;C:\Program Files (x86)\Microsoft BizTalk Server;C:\Program Files\LLVM\bin;C:\Users\runneradmin\.dotnet\tools;C:\Users\runneradmin\.cargo\bin;C:\Users\runneradmin\AppData\Local\Microsoft\WindowsApps" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\test\\ui\\panics\\panic-short-backtrace-windows-x86_64\\a.exe"
------------------------------------------

------------------------------------------
stderr:
stderr:
------------------------------------------
thread 'main' panicked at 'd was called', D:\a\rust\rust\src/test\ui\panics\panic-short-backtrace-windows-x86_64.rs:48:5
   0: std::panicking::begin_panic
   1: d
   2: c
   3: b
---

Some tests failed in compiletest suite=ui mode=ui host=x86_64-pc-windows-msvc target=x86_64-pc-windows-msvc


command did not execute successfully: "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage0-tools-bin\\compiletest.exe" "--compile-lib-path" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage2\\bin" "--run-lib-path" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage2\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "--rustc-path" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\stage2\\bin\\rustc.exe" "--src-base" "D:\\a\\rust\\rust\\src/test\\ui" "--build-base" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\test\\ui" "--stage-id" "stage2-x86_64-pc-windows-msvc" "--suite" "ui" "--mode" "ui" "--target" "x86_64-pc-windows-msvc" "--host" "x86_64-pc-windows-msvc" "--llvm-filecheck" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\llvm\\build\\bin\\FileCheck.exe" "--nodejs" "C:\\Program Files\\nodejs\\node" "--npm" "C:\\Program Files\\nodejs\\npm" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0  -Lnative=D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\native\\rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0  -Lnative=D:\\a\\rust\\rust\\build\\x86_64-pc-windows-msvc\\native\\rust-test-helpers" "--docck-python" "C:\\hostedtoolcache\\windows\\Python\\3.10.1\\x64\\python3.exe" "--lldb-python" "C:\\hostedtoolcache\\windows\\Python\\3.10.1\\x64\\python3.exe" "--gdb" "C:\\msys64\\usr\\bin\\gdb" "--llvm-version" "13.0.0-rust-1.59.0-nightly" "--llvm-components" "aarch64 aarch64asmparser aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils aggressiveinstcombine all all-targets analysis arm armasmparser armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter avr avrasmparser avrcodegen avrdesc avrdisassembler avrinfo binaryformat bitreader bitstreamreader bitwriter bpf bpfasmparser bpfcodegen bpfdesc bpfdisassembler bpfinfo cfguard codegen core coroutines coverage debuginfocodeview debuginfodwarf debuginfogsym debuginfomsf debuginfopdb demangle dlltooldriver dwarflinker dwp engine executionengine extensions filecheck frontendopenacc frontendopenmp fuzzmutate globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo instcombine instrumentation interfacestub interpreter ipo irreader jitlink libdriver lineeditor linker lto m68k m68kasmparser m68kcodegen m68kdesc m68kdisassembler m68kinfo mc mca mcdisassembler mcjit mcparser mips mipsasmparser mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmparser msp430codegen msp430desc msp430disassembler msp430info native nativecodegen nvptx nvptxcodegen nvptxdesc nvptxinfo objcarcopts object objectyaml option orcjit orcshared orctargetprocess passes powerpc powerpcasmparser powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata remarks riscv riscvasmparser riscvcodegen riscvdesc riscvdisassembler riscvinfo runtimedyld scalaropts selectiondag sparc sparcasmparser sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target textapi transformutils vectorize webassembly webassemblyasmparser webassemblycodegen webassemblydesc webassemblydisassembler webassemblyinfo webassemblyutils windowsmanifest x86 x86asmparser x86codegen x86desc x86disassembler x86info xray" "--cc" "" "--cxx" "" "--cflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--channel" "nightly" "--color" "always"


Build completed unsuccessfully in 0:40:36
Build completed unsuccessfully in 0:40:36
make: *** [Makefile:74: ci-subset-2] Error 1

@bors
Copy link
Contributor

bors commented Dec 23, 2021

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Dec 23, 2021
@RalfJung
Copy link
Member Author

There's no way this can affect ui\panics\panic-short-backtrace-windows-x86_64.rs... right?

@bors retry

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 23, 2021
@bors
Copy link
Contributor

bors commented Dec 24, 2021

⌛ Testing commit 56b7d5f with merge 59337cd...

@bors
Copy link
Contributor

bors commented Dec 24, 2021

☀️ Test successful - checks-actions
Approved by: eddyb,oli-obk
Pushing 59337cd to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Dec 24, 2021
@bors bors merged commit 59337cd into rust-lang:master Dec 24, 2021
@rustbot rustbot added this to the 1.59.0 milestone Dec 24, 2021
@rust-highfive
Copy link
Collaborator

📣 Toolstate changed by #91342!

Tested on commit 59337cd.
Direct link to PR: #91342

💔 miri on windows: test-pass → build-fail (cc @RalfJung @eddyb @oli-obk).
💔 miri on linux: test-pass → build-fail (cc @RalfJung @eddyb @oli-obk).

rust-highfive added a commit to rust-lang-nursery/rust-toolstate that referenced this pull request Dec 24, 2021
Tested on commit rust-lang/rust@59337cd.
Direct link to PR: <rust-lang/rust#91342>

💔 miri on windows: test-pass → build-fail (cc @RalfJung @eddyb @oli-obk).
💔 miri on linux: test-pass → build-fail (cc @RalfJung @eddyb @oli-obk).
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (59337cd): comparison url.

Summary: This benchmark run did not return any relevant changes.

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

@rustbot label: -perf-regression

bors added a commit to rust-lang/miri that referenced this pull request Dec 24, 2021
adjust for FnAbi changes

This is the Miri side of rust-lang/rust#91342.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

miri function argument passing should use FnAbi
10 participants