Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[do not merge] Proof of concept implementation of forward compatible v0 symbols (MCP 737) #125487

Closed
wants to merge 2 commits into from
Closed

[do not merge] Proof of concept implementation of forward compatible v0 symbols (MCP 737) #125487

wants to merge 2 commits into from
+132 −39

Conversation

michaelwoerister
Copy link
Member

This PR changes v0 symbol names to enable graceful degradation for demanglers that don't support newer language features yet (as described in MCP 737).

The corresponding changes in rustc-demangle are at https://github.com/michaelwoerister/rustc-demangle/tree/skip_unknown.

This proof-of-concept implementation is meant to help evaluating if MCP 737 is viable.

r? @ghost

@rustbot rustbot added PG-exploit-mitigations Project group: Exploit mitigations S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels May 24, 2024
@michaelwoerister michaelwoerister removed the PG-exploit-mitigations Project group: Exploit mitigations label May 24, 2024
@michaelwoerister
Copy link
Member Author

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

Sign in to view
@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label May 24, 2024
@michaelwoerister michaelwoerister mentioned this pull request May 24, 2024
Closed
bors added a commit to rust-lang-ci/rust that referenced this pull request May 24, 2024
@bors
Auto merge of rust-lang#125487 - michaelwoerister:skippable_v0, r=<try>
df5a899 
[do not merge] Proof of concept implementation of forward compatible v0 symbols (MCP 737)

This PR changes v0 symbol names to enable graceful degradation for demanglers that don't support newer language features yet (as described in [MCP 737](rust-lang/compiler-team#737)).

The corresponding changes in rustc-demangle are at https://github.com/michaelwoerister/rustc-demangle/tree/skip_unknown.

This proof-of-concept implementation is meant to help evaluating if MCP 737 is viable.

r? `@ghost`
@bors
Copy link
Contributor

bors commented May 24, 2024

⌛ Trying commit 17c7fc4 with merge df5a899...

@rust-log-analyzer
Copy link
Collaborator

The job mingw-check-tidy failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot) 
Getting action download info
Download action repository 'msys2/setup-msys2@v2.22.0' (SHA:cc11e9188b693c2b100158c3322424c4cc1dadea)
Download action repository 'actions/checkout@v4' (SHA:a5ac7e51b41094c92402da3b24376905380afc29)
Download action repository 'actions/upload-artifact@v4' (SHA:65462800fd760344b1a7b4382951275a0abb4808)
Complete job name: PR - mingw-check-tidy
git config --global core.autocrlf false
shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
---
COPY scripts/sccache.sh /scripts/
RUN sh /scripts/sccache.sh

COPY host-x86_64/mingw-check/reuse-requirements.txt /tmp/
RUN pip3 install --no-deps --no-cache-dir --require-hashes -r /tmp/reuse-requirements.txt \
    && pip3 install virtualenv
COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
COPY host-x86_64/mingw-check/validate-error-codes.sh /scripts/

# NOTE: intentionally uses python2 for x.py so we can test it still works.
# NOTE: intentionally uses python2 for x.py so we can test it still works.
# validate-toolstate only runs in our CI, so it's ok for it to only support python3.
ENV SCRIPT TIDY_PRINT_DIFF=1 python2.7 ../x.py test \
           --stage 0 src/tools/tidy tidyselftest --extra-checks=py:lint
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile --allow-unsafe --generate-hashes reuse-requirements.in
---

#12 [5/8] COPY host-x86_64/mingw-check/reuse-requirements.txt /tmp/
#12 DONE 0.0s

#13 [6/8] RUN pip3 install --no-deps --no-cache-dir --require-hashes -r /tmp/reuse-requirements.txt     && pip3 install virtualenv
#13 0.422   Downloading binaryornot-0.4.4-py2.py3-none-any.whl (9.0 kB)
#13 0.437 Collecting boolean-py==4.0
#13 0.441   Downloading boolean.py-4.0-py3-none-any.whl (25 kB)
#13 0.454 Collecting chardet==5.1.0
---
#13 3.533 Building wheels for collected packages: reuse
#13 3.534   Building wheel for reuse (pyproject.toml): started
#13 3.855   Building wheel for reuse (pyproject.toml): finished with status 'done'
#13 3.856   Created wheel for reuse: filename=reuse-1.1.0-cp310-cp310-manylinux_2_35_x86_64.whl size=181117 sha256=f5f58750481f69515c2c0d1d503daf565e2565c370d07fc6aeb95fe3498b4269
#13 3.856   Stored in directory: /tmp/pip-ephem-wheel-cache-afyt1o5y/wheels/c2/3c/b9/1120c2ab4bd82694f7e6f0537dc5b9a085c13e2c69a8d0c76d
#13 3.859 Installing collected packages: boolean-py, binaryornot, setuptools, reuse, python-debian, markupsafe, license-expression, jinja2, chardet
#13 3.880   Attempting uninstall: setuptools
#13 3.881     Found existing installation: setuptools 59.6.0
#13 3.882     Not uninstalling setuptools at /usr/lib/python3/dist-packages, outside environment /usr
#13 3.882     Not uninstalling setuptools at /usr/lib/python3/dist-packages, outside environment /usr
#13 3.883     Can't uninstall 'setuptools'. No files were found to uninstall.
#13 4.544 Successfully installed binaryornot-0.4.4 boolean-py-4.0 chardet-5.1.0 jinja2-3.1.2 license-expression-30.0.0 markupsafe-2.1.1 python-debian-0.1.49 reuse-1.1.0 setuptools-66.0.0
#13 4.544 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#13 5.055 Collecting virtualenv
#13 5.095   Downloading virtualenv-20.26.2-py3-none-any.whl (3.9 MB)
#13 5.154      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.9/3.9 MB 69.0 MB/s eta 0:00:00
#13 5.189 Collecting distlib<1,>=0.3.7
#13 5.195   Downloading distlib-0.3.8-py2.py3-none-any.whl (468 kB)
#13 5.201      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 468.9/468.9 KB 121.6 MB/s eta 0:00:00
#13 5.229 Collecting platformdirs<5,>=3.9.1
#13 5.233   Downloading platformdirs-4.2.2-py3-none-any.whl (18 kB)
#13 5.268 Collecting filelock<4,>=3.12.2
#13 5.271   Downloading filelock-3.14.0-py3-none-any.whl (12 kB)
#13 5.356 Installing collected packages: distlib, platformdirs, filelock, virtualenv
#13 5.517 Successfully installed distlib-0.3.8 filelock-3.14.0 platformdirs-4.2.2 virtualenv-20.26.2
#13 DONE 5.6s

#14 [7/8] COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
#14 DONE 0.0s
---
DirectMap4k:      200640 kB
DirectMap2M:     7139328 kB
DirectMap1G:    11534336 kB
##[endgroup]
Executing TIDY_PRINT_DIFF=1 python2.7 ../x.py test            --stage 0 src/tools/tidy tidyselftest --extra-checks=py:lint
+ TIDY_PRINT_DIFF=1 python2.7 ../x.py test --stage 0 src/tools/tidy tidyselftest --extra-checks=py:lint
    Finished `dev` profile [unoptimized] target(s) in 0.03s
##[endgroup]
downloading https://ci-artifacts.rust-lang.org/rustc-builds-alt/464987730ab36e3456ba57c53965372989382deb/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz
extracting /checkout/obj/build/cache/llvm-464987730ab36e3456ba57c53965372989382deb-true/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz to /checkout/obj/build/x86_64-unknown-linux-gnu/ci-llvm
extracting /checkout/obj/build/cache/llvm-464987730ab36e3456ba57c53965372989382deb-true/rust-dev-nightly-x86_64-unknown-linux-gnu.tar.xz to /checkout/obj/build/x86_64-unknown-linux-gnu/ci-llvm
##[group]Building stage0 tool tidy (x86_64-unknown-linux-gnu)
warning: /checkout/library/core/Cargo.toml: unused manifest key: lints.rust.unexpected_cfgs.check-cfg
warning: /checkout/library/std/Cargo.toml: unused manifest key: lints.rust.unexpected_cfgs.check-cfg
warning: /checkout/library/alloc/Cargo.toml: unused manifest key: lints.rust.unexpected_cfgs.check-cfg
    Updating crates.io index
    Updating git repository `https://github.com/michaelwoerister/rustc-demangle.git`
---
    Finished `release` profile [optimized] target(s) in 25.40s
##[endgroup]
fmt check
tidy check
tidy error: invalid source: "git+https://github.com/michaelwoerister/rustc-demangle.git?branch=skip_unknown#7e674e483a7c61803edc8d59f2641a4ad52a1a30"
removing old virtual environment
removing old virtual environment
creating virtual environment at '/checkout/obj/build/venv' using 'python3.10'
Requirement already satisfied: pip in ./build/venv/lib/python3.10/site-packages (24.0)
Collecting black==23.3.0 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 7))
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 52.5 MB/s eta 0:00:00
Collecting click==8.1.3 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 34))
  Downloading click-8.1.3-py3-none-any.whl (96 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 kB 28.5 MB/s eta 0:00:00
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 kB 28.5 MB/s eta 0:00:00
Collecting importlib-metadata==6.7.0 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 38))
  Downloading importlib_metadata-6.7.0-py3-none-any.whl (22 kB)
Collecting mypy-extensions==1.0.0 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 42))
  Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)
Collecting packaging==23.1 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 46))
  Downloading packaging-23.1-py3-none-any.whl (48 kB)
Collecting pathspec==0.11.1 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 50))
  Downloading pathspec-0.11.1-py3-none-any.whl (29 kB)
Collecting platformdirs==3.6.0 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 54))
  Downloading platformdirs-3.6.0-py3-none-any.whl (16 kB)
  Downloading platformdirs-3.6.0-py3-none-any.whl (16 kB)
Collecting ruff==0.0.272 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 58))
  Downloading ruff-0.0.272-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.9 MB)
Collecting tomli==2.0.1 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 77))
  Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
Collecting typed-ast==1.5.4 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 81))
  Downloading typed_ast-1.5.4-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (877 kB)
  Downloading typed_ast-1.5.4-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (877 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 877.7/877.7 kB 129.6 MB/s eta 0:00:00
Collecting typing-extensions==4.6.3 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 107))
  Downloading typing_extensions-4.6.3-py3-none-any.whl (31 kB)
Collecting zipp==3.15.0 (from -r /checkout/src/tools/tidy/config/requirements.txt (line 114))
  Downloading zipp-3.15.0-py3-none-any.whl (6.8 kB)
Installing collected packages: zipp, typing-extensions, typed-ast, tomli, ruff, platformdirs, pathspec, packaging, mypy-extensions, click, importlib-metadata, black
Successfully installed black-23.3.0 click-8.1.3 importlib-metadata-6.7.0 mypy-extensions-1.0.0 packaging-23.1 pathspec-0.11.1 platformdirs-3.6.0 ruff-0.0.272 tomli-2.0.1 typed-ast-1.5.4 typing-extensions-4.6.3 zipp-3.15.0
some tidy checks failed
Build completed unsuccessfully in 0:00:58
  local time: Fri May 24 12:26:21 UTC 2024
  network time: Fri, 24 May 2024 12:26:21 GMT

@bors
Copy link
Contributor

bors commented May 24, 2024

☀️ Try build successful - checks-actions
Build commit: df5a899 (df5a8997fafb6125bad45edd15f03ebfce52e359)

@rust-timer

This comment has been minimized.

Sign in to view
@rust-timer

This comment was marked as outdated.

Sign in to view
@rustbot rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels May 24, 2024
@michaelwoerister
Copy link
Member Author

The above perf run includes the changes due to making v0 the default.

The performance comparison that is actually relevant uses v0=default as baseline:
https://perf.rust-lang.org/compare.html?start=6d746cd09206996abb0d30d184212d972b0328a8&end=df5a8997fafb6125bad45edd15f03ebfce52e359&stat=instructions%3Au

@michaelwoerister
Copy link
Member Author

michaelwoerister commented May 24, 2024 •
edited
Loading

I'm somewhat skeptical about the results though, since check builds show up as being affected the most, but as far as I know check builds don't generate symbol names much, if at all.

I'll investigate with cachegrind diff when I get a chance.

@michaelwoerister
Copy link
Member Author

We discussed this in the compiler team design meeting on Jun 14, and decided to not pursue this any further for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
perf-regression Performance regression. S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@michaelwoerister @rust-timer @bors @rust-log-analyzer @rustbot