Client-Server redesign

[sake92]

Disclaimer:
I am not the most knowledgeable person in this area, just want to spark a discussion so we can have a better dev experience with Mill.
This is kind of continuation of the "Fine-grained parallelism" discussion #3714

Current status

Execution mode

In the current world (correct me if I'm wrong), we have 2 modes of calling Mill:

• client-server mode (long running server(s), and usually short-lived clients)
• interactive mode (one "batch" process, runs and exits)

The interactive mode is mostly needed for interactive apps that use terminal IO.
This is not the best experience, because users need to know in advance which mode they need to use.

If you accidentally use the client-server mode for running an interactive app, you will get surprising results.
I made a comparison with other JVM build tools here: https://github.com/sake92/java-build-tool-daemon-interactive
Only sbt does it fine (tho you can run only one app instance, probably because of their weird JSON protocol, not 100% sure).

Multiple clients

We already have a situation where we need many mill clients:

• compile backend code from terminal in watch mode
• compile frontend code from terminal in watch mode
• compile code from IDE via BSP

If the tasks run with no lock, they will step on each other toes.
Hence the global lock added in #3519

Proposal and a POC

Small WIP POC: https://github.com/sake92/mill-client-server-poc

My proposal is to:

• remove interactive mode
• remove in-process running (runLocal)
• introduce a lightweight client-server protocol
• have a singleton server per-workspace (restart itself only if its JVM params change), with a single file lock

The protocol does introduce more complexity, but the benefits are good:

• multiple simultaneous clients supported
• always get the benefits of compile caching etc
• always use locking to prevent corrupted state
• interactive processes work with no issues
• single execution mode (no need for -i)
• only one code path, simplify the Mill codebase

The server would act as a "kernel", it would do the heavyweight background stuff like compiling, caching, locking etc.
The clients would get messages from server and react to them.

Interactivity

Interactive process execution would happen in the client itself.
This would take the burden off server itself:

• no need to give more memory to server in order to execute your app
• watch mode can be implemented quite easily: interrupt current subprocess, wait for server's message.. and run it again (this is often an issue in sbt interactive mode where process threads still stay running for some reason..)

Tasks Locking

I am not sure what is the best approach here, don't have much experience with writing concurrent code on this scale/complexity.
But some kind of in-memory locking could be the simplest to start with.

---

The POC also has a small benchmark for scala-native client vs JVM.
TLDR: it is much snappier (15ms vs 300ms)

---

Of course, this is just a rough sketch, you can play with the POC to experiment more..
What do you think overall?

[lihaoyi]

always use locking to prevent corrupted state

This is really the crux of the issue. The core challenge here is how to effectively implement fine grained locking for parallel topological evaluation of the task graph. Too coarse grained and we lose all benefits, too fine grained and we start getting race conditions

A minor challenge is making the client smart enough to make -i unnecessary. This means the client would need to do all the fancy terminal stuff that the server currently does (indirectly through the Scala/Ammonite REPL). Annoying but not difficult.

Everything you else you described is accurate, and relatively straightforward.

[lefou]

I also think we should move in that direction.

The core challenge here is how to effectively implement fine grained locking for parallel topological evaluation of the task graph.

I guess the nice thing is, that we always know the full task graph ahead of evaluation. So, in the case when there are two concurrent requests, we can easily check if there is any locking necessary at all.

The naive approach could be for the second request to just wait as long as we have some overlaps. This shouldn't be any worse than a global lock. Also, while we wait, part of the work we planned to do is already in progress and already finished once the second request is unblocked.