Replace Bash ~and Python~ scripts with Rust

[emilk]

We have a bunch of Python scripts in our repository for CI stuff, linting, etc.

We would like to replace them all with Rust, perhaps using something like xtask.

Some things we need to make sure of:

• The helper scripts should still be single-file (not need a Cargo.toml per script).See e.g. eRFC: single-file packages ("cargo script") integration rust-lang/rfcs#3424
• Each file should have very light dependencies and compile quickly (avoid e.g. clap)
• Work hard on keeping the code simple and terse

Related issues:

• Make the repository less unix-centric #2200
• Replace setup_web.sh with something more robust #2174
• Join our Rust "scripts" crates into one #5889

---

If we replace lint.py with Rust, we should consider using ast_grep_core of https://github.com/ast-grep/ast-grep

[abey79]

I feel Bash and Python scripts don't deserve the same treatment. Bash scripts are a plague on Windows, but the same is not true for Python, in the context of a project that otherwise heavily relies on a working Python install.

I, for one, feel like I'm leagues away from being as quick to improve on, e.g., upload_image.py if it was in Rust instead.

[emilk]

True, but python still requires a separate pip install -r … step at the moment which cargo run doesn't, but that may be something pixi will solve for us

[abey79]

I'd argue our ubiquitous use of requirements.txt is a bit the issue there. We should put everything in pyproject.toml for a centralised, single source of truth, using this pattern:

[project.optional-dependencies]
tests = ["pytest==7.1.2"] # we somehow have this here already, not sure we actually use it
dev = [
    # .... put everything we need
]
docs = [
    # ... if we feel necessary, we can sub-categorise
]

Then, pip install -e ".[dev]" should do the trick.

The disadvantage is that these "extra" are published/visible to pypi. Poetry includes another mechanism (optional groups) that would be a better fit.

[emilk]

The point I was trying to make is that we are primarily a Rust shop. To work on e.g. the Rerun viewer, you should ideally need only to install Rust, and that's it. Anything beyond that are extra steps that we should try to avoid, including installing Python, pip, and any pip requirements.

Ideally you should only need to install Python if you are working on the Python SDK.
Just like you should only need to install a CMake if you are working on the C++ SDK.

[abey79]

I'm not totally on-board with that.

Rust users should cargo add rerun and not have to think of Python—I agree with that. Contributors, on the other hand, are by nature exposed to the project's build system (in the broad meaning of the term).

The build system needs to strike a balance between up-front setup for contributor vs. constraints on its day-to-day maintenance/development. My position is that banning Python from the build system would shift that balance to a level that would overall be detrimental to the dev experience. This is obviously a somewhat subjective topic. Maybe less so is the question of being a contributor without the ability to run the Python test suite.

[emilk]

We (well, @jprochazk) have started porting more and more .sh and .py scripts to Rust, and so far I think it has been a win. However, so far each such "Rust script" has become its own separate crate, which I think is the wrong direction:

• It requires many files (crate folder, Cargo.toml, src folder, main.rs, maybe a README.md)
• It makes it harder to add a new quick command (a lot of boilerplate setting up the above files)
• It adds more binaries (Reduce the number of crate binaries in the cargo workspace #4623)

So, I think we should consider an xtask-esque approach, where a single Rust binary multiplexes out into different Rust "scripts", so that each script is just a file added to the same crate.

I've opened an issue for this:

• Join our Rust "scripts" crates into one #5889

[Wumpf]

all subtasks done now!