Skip to content

Commit

Permalink
create turbo-static for compile time graph analysis (#8037)
Browse files Browse the repository at this point in the history
### Description

<!--
  ✍️ Write a short summary of your work.
  If necessary, include relevant screenshots.
-->

### Testing Instructions

<!--
  Give a quick description of steps to test your changes.
-->


Closes TURBO-2877
  • Loading branch information
arlyon authored Jun 4, 2024
1 parent 400be27 commit 0bf9e11
Show file tree
Hide file tree
Showing 10 changed files with 1,141 additions and 20 deletions.
101 changes: 81 additions & 20 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ resolver = "2"
members = [
"crates/node-file-trace",
"crates/tower-uds",
"crates/turbo-static",
"crates/turbo-tasks*",
"crates/turbo-prehash",
"crates/turbopack*",
Expand Down
2 changes: 2 additions & 0 deletions crates/turbo-static/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
call_resolver.bincode
graph.cypherl
25 changes: 25 additions & 0 deletions crates/turbo-static/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
[package]
name = "turbo-static"
version = "0.1.0"
edition = "2021"
license = "MPL-2.0"

[dependencies]
bincode = "1.3.3"
clap = { workspace = true, features = ["derive"] }
ctrlc = "3.4.4"
ignore = "0.4.22"
itertools.workspace = true
lsp-server = "0.7.6"
lsp-types = "0.95.1"
proc-macro2 = { workspace = true, features = ["span-locations"] }
serde = { workspace = true, features = ["derive"] }
serde_json.workspace = true
serde_path_to_error = "0.1.16"
syn = { version = "2", features = ["parsing", "full", "visit", "extra-traits"] }
tracing-subscriber = { version = "0.3.18", features = ["env-filter"] }
tracing.workspace = true
walkdir = "2.5.0"

[lints]
workspace = true
33 changes: 33 additions & 0 deletions crates/turbo-static/readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Turbo Static

Leverages rust-analyzer to build a complete view into the static dependency
graph for your turbo tasks project.

## How it works

- find all occurences of #[turbo_tasks::function] across all the packages you
want to query
- for each of the tasks we find, query rust analyzer to see which tasks call
them
- apply some very basis control flow analysis to determine whether the call is
made 1 time, 0/1 times, or 0+ times, corresponding to direct calls,
conditionals, or for loops
- produce a cypher file that can be loaded into a graph database to query the
static dependency graph

## Usage

This uses an in memory persisted database to cache rust-analyzer queries.
To reset the cache, pass the `--reindex` flag. Running will produce a
`graph.cypherl` file which can be loaded into any cypher-compatible database.

```bash
# pass in the root folders you want to analyze. the system will recursively
# parse all rust code looking for turbo tasks functions
cargo run --release -- ../../../turbo ../../../next.js
# now you can load graph.cypherl into your database of choice, such as neo4j
docker run \
--publish=7474:7474 --publish=7687:7687 \
--volume=$HOME/neo4j/data:/data \
neo4j
```
Loading

0 comments on commit 0bf9e11

Please sign in to comment.