[EXPERIMENT] Ownership analysis [DO NOT MERGE] #124756
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`assigning_clones`: add empty line to doc changelog: none This PR adds, for consistency reasons, an empty line to the example in the doc of the `assigning_clones` lint.
…1995 Fix `FormatArgs` storage when `-Zthreads` > 1 Fixes rust-lang#11886 The initial way I thought of was a little gross so I never opened a PR for it, I thought of a nicer way today that no longer involves any `thread_local`s or `static`s `rustc_data_strucutres::sync::{Lrc, OnceLock}` implement `DynSend` + `DynSync` so we can pass them to the lint passes that need the storage changelog: none r? `@flip1995`
…e-init, r=dswij Don't lint assigning_clones on nested late init locals Fixes rust-lang#12741 changelog: none
More testing or something Duck Checking more bbs Checking more bbs Some progress IIRC Handle more teminators Diving into RValue evaluation I have no idea what I'm doing It doesn't crash I guess? Finding the first pattern Refactoring: Extract pattern from automata to simplify states Remove old value usage based pattern detection Refactoring: Better Automata structure More automator nodes :D More states and small progress Detecting conditional assignments Event posting refactoring Adding test output and testing temp borrows Working on field borrows and cleanup :D Different errors yay Use `mir::Place` for events More patterns and more states, just need some pats :3 Getting the first fricking region Collect dependent types for return tys Somehow handle deref calls 😭 Cleanup AnonRef state thing by using an info struct wrapper General Cleanup Working the return state machine Testing borrows in HIR format... Working on something I guess Having a direction to run against walls Control flow information :D CFG with ranges.. CFG: resonable downgrade CFG: Start Caching return Relations I love refactoring in Rust :D Detecting first returns Handling more returns Tracking assignments better Collecting local information Refactor metadata collection for MIR The first return patterns Cleanup console output with envs Fix `LocalInfo` for arguments Some refactorings again Check unused for owned values Complaining about MIR again Why is there a no-op Drop???? Determine a nice traversal order Probably progress?? Detecting manual drops and moves into functions Fix ICE for item names Remove old automata based implementation Dogfood my code (52 Errors...) Detecting temp borrows Abort on feature use Handling assignments better Unify arguments and variables Remove RC from ownership patterns Better anon ref tracking Stash that thing Better function call relation calculation Track relations between mut inputs as well :D Correct `has_projection()` usage Detecting some patterns in function params Detecting two phase borrows Owned `StateInfo` refactoring Update todos Generalize visitors for analyzers Tracking all assignments Dataflow analysis Dataflow analysis with `&mut` loans Dataflow refactoring, this only considers locals Owned: Detect `ModMutShadowUnmut` Reorganize UI tests Include more type information in `VarInfo` Better track return types Refactorings for borrow tracking Allow nested two phase borrows Identify named borrows for owned values better Correct owned value stat tracking Fill body pats based on stats Pause `Return` pattern for owned values Modeling the state as a list Detecting the first patterns with state merging Detecting conditional moves and drops Mark more tests as passing Better detect overwrites for owned values Improve Drop impl info Remove `Unused` detection due to inconsistency Update docs and move test to pass Refactored `state.borrows` Handle double refs correctly Dealing with ref-refs and renaming temp to arg borows Add `Moved` as a base pattern Handle part moves Better separation between borrow and part borrows Track possibly missed relations Closures Add simple closure ty Snapshot Ctor things and stuff Remove lint level gate Avoiding `rand` crashes 🎉 Aggregate stats for bodies Jsonfy analysis output Update todos Improving traversal construction and running regex :D Type cleanupg Feeding the dog
rustbot
added
the
S-waiting-on-review
This comment has been minimized.
This comment has been minimized.
|
It's green :D Now let's see bors destroy all my hopes and dreams. /s JK love you bors @bors try |
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
May 5, 2024
[EXPERIMENT] Ownership analysis [DO NOT MERGE] TL;DR: I want to collect data about ownership in Rust's ecosystem. I modified Clippy to extract things from MIR. This PR is intended for a crater run, but should definitely not be merged. --- *Roses are red,* *Violets are blue,* *Can I borrow,* *A pattern from you?* So, what is this project all about? Well, last year I was looking for a topic for my master's thesis. I had the incredible luck that the programming languages group at my university is not only a collection of incredibly cool people but also has some interest in Rust. It just so happens that `@amandasystems` is part of the group. We then spent some time looking for a thesis project in the domain of Rust's memory ownership model, and finally settled on the idea to analyze how the memory ownership model is used in practice. For Polonius, it could be interesting to know how many bodies contain borrows and if the changes from the NLL RFC are actually used, etc. ### Implementation Disclaimer, this PR contains some of the most convoluted and spaghettified code I've ever written. Do not use this as an example for anything! I'm pretty sure that the insufficiencies of this PR single-handedly increases the compile time by 100%. It's honestly a miracle that the whole thing doesn't ICE on every function. So at the beginning, I decided to use MIR as that is the IR that the borrow check uses, and it contains all the borrows and moves which might not be explicit in the HIR. While all nice and good it turns out that MIR also gets writ of information which I would have wanted for my analysis. To be clear, MIR is an excellent tool for what it was designed to do, but its loss of information has been a struggle for this project specifically. I would like to collect way more metadata, more complex patterns, and data how named references are used, but the deadline for my thesis is coming up. We decided to limit the scope to named owned variables and see how the analysis goes. If I find some interesting things there is potential to extend this work. ### Run I decided to implement the analysis on Clippy for the shorter compile times and familiar codebase. This branch disables all other lints by default to safe on some runtime. The crater run with Clippy from master might also expose some ICEs which could be interesting for us. The results of this will be aggregated and published as part of my thesis. The time imposed limitations of this implementation probably mean that the results have to be taken with a metric ton of salt, but they should be sufficient to pass my thesis and know if more work in this area would be interesting. I don't know if there is much more too to say. Let's hope: * The CI passes * This run goes smoothly * Results in some interesting data * Teaches me how to implement the whole thing better the next time. --- r? `@ghost`
|
☀️ Try build successful - checks-actions |
|
@craterbot run mode=clippy |
|
👌 Experiment ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more |
craterbot
added
S-waiting-on-crater
|
@craterbot abort Need to add clippy to try CI. |
|
🗑️ Experiment ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more |
craterbot
added
S-waiting-on-review
|
@bors try |
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
May 5, 2024
[EXPERIMENT] Ownership analysis [DO NOT MERGE] TL;DR: I want to collect data about ownership in Rust's ecosystem. I modified Clippy to extract things from MIR. This PR is intended for a crater run, but should definitely not be merged. --- *Roses are red,* *Violets are blue,* *Can I borrow,* *A pattern from you?* So, what is this project all about? Well, last year I was looking for a topic for my master's thesis. I had the incredible luck that the programming languages group at my university is not only a collection of incredibly cool people but also has some interest in Rust. It just so happens that `@amandasystems` is part of the group. We then spent some time looking for a thesis project in the domain of Rust's memory ownership model, and finally settled on the idea to analyze how the memory ownership model is used in practice. For Polonius, it could be interesting to know how many bodies contain borrows and if the changes from the NLL RFC are actually used, etc. ### Implementation Disclaimer, this PR contains some of the most convoluted and spaghettified code I've ever written. Do not use this as an example for anything! I'm pretty sure that the insufficiencies of this PR single-handedly increases the compile time by 100%. It's honestly a miracle that the whole thing doesn't ICE on every function. So at the beginning, I decided to use MIR as that is the IR that the borrow check uses, and it contains all the borrows and moves which might not be explicit in the HIR. While all nice and good it turns out that MIR also gets writ of information which I would have wanted for my analysis. To be clear, MIR is an excellent tool for what it was designed to do, but its loss of information has been a struggle for this project specifically. I would like to collect way more metadata, more complex patterns, and data how named references are used, but the deadline for my thesis is coming up. We decided to limit the scope to named owned variables and see how the analysis goes. If I find some interesting things there is potential to extend this work. ### Run I decided to implement the analysis on Clippy for the shorter compile times and familiar codebase. This branch disables all other lints by default to safe on some runtime. The crater run with Clippy from master might also expose some ICEs which could be interesting for us. The results of this will be aggregated and published as part of my thesis. The time imposed limitations of this implementation probably mean that the results have to be taken with a metric ton of salt, but they should be sufficient to pass my thesis and know if more work in this area would be interesting. I don't know if there is much more too to say. Let's hope: * The CI passes * This run goes smoothly * Results in some interesting data * Teaches me how to implement the whole thing better the next time. --- r? `@ghost`
|
☀️ Try build successful - checks-actions |
|
@craterbot run mode=clippy |
|
👌 Experiment ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more |
craterbot
added
S-waiting-on-crater
|
@craterbot p=1 |
|
📝 Configuration of the ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more |
|
Thank you @Mark-Simulacrum, now it will run before the |
|
🚧 Experiment ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more |
|
🎉 Experiment
|
craterbot
added
S-waiting-on-review
|
I'm closing this since the experiment is done running. Thank you for kicking off crater! Now I'll need to take some time to look at the data. :) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
TL;DR: I want to collect data about ownership in Rust's ecosystem. I modified Clippy to extract things from MIR. This PR is intended for a crater run, but should definitely not be merged.
Roses are red,
Violets are blue,
Can I borrow,
A pattern from you?
So, what is this project all about? Well, last year I was looking for a topic for my master's thesis. I had the incredible luck that the programming languages group at my university is not only a collection of incredibly cool people but also has some interest in Rust. It just so happens that @amandasystems is part of the group.
We then spent some time looking for a thesis project in the domain of Rust's memory ownership model, and finally settled on the idea to analyze how the memory ownership model is used in practice.
For Polonius, it could be interesting to know how many bodies contain borrows and if the changes from the NLL RFC are actually used, etc.
Implementation
Disclaimer, this PR contains some of the most convoluted and spaghettified code I've ever written. Do not use this as an example for anything! I'm pretty sure that the insufficiencies of this PR single-handedly increases the compile time by 100%. It's honestly a miracle that the whole thing doesn't ICE on every function.
So at the beginning, I decided to use MIR as that is the IR that the borrow check uses, and it contains all the borrows and moves which might not be explicit in the HIR. While all nice and good it turns out that MIR also gets writ of information which I would have wanted for my analysis.
To be clear, MIR is an excellent tool for what it was designed to do, but its loss of information has been a struggle for this project specifically.
I would like to collect way more metadata, more complex patterns, and data how named references are used, but the deadline for my thesis is coming up. We decided to limit the scope to named owned variables and see how the analysis goes. If I find some interesting things there is potential to extend this work.
Run
I decided to implement the analysis on Clippy for the shorter compile times and familiar codebase. This branch disables all other lints by default to safe on some runtime. The crater run with Clippy from master might also expose some ICEs which could be interesting for us.
The results of this will be aggregated and published as part of my thesis. The time imposed limitations of this implementation probably mean that the results have to be taken with a metric ton of salt, but they should be sufficient to pass my thesis and know if more work in this area would be interesting.
I don't know if there is much more too to say. Let's hope:
r? @ghost