Improve developer experience - provide aliases and overview

[lefou]

Often, when switching projects, I find it hard to remember which targets needs to be run to start, test or deploy an application. Often, also some parameters need to be applied.

Co-colleagues used to maintain a small Makefile, just to collect the essential build commands. As using GNU Make around Mill doesn't feel right to me, I think we should try to fill the gaps.

• We should provide some nice way to use/remember aliases for often used Mill invocations (see also Add ability to alias Modules (specially external modules) #299)
• We should provide some easy way to display those aliases and possibly additional selected tasks as some help text to a user

It would be great, if we could provide some API to improve the DX out of the box.

Lets visit the example given by @carlosedp in this comment: #299 (comment)

I have an implementation I add to all my build.sc files allowing me to define an alias list that even print a help... I defined the alias runner as the run command but could be set to anything else by changing the def method.

// In your build.sc
...

// -----------------------------------------------------------------------------
// Command Aliases
// -----------------------------------------------------------------------------
// Alias commands are run like `./mill run [alias]`
// Define the alias as a map element containing the alias name and a Seq with the tasks to be executed
val aliases: Map[String, Seq[String]] = Map(
  "lint"     -> Seq("mill.scalalib.scalafmt.ScalafmtModule/reformatAll __.sources", "__.fix"),
  "fmt"      -> Seq("mill.scalalib.scalafmt.ScalafmtModule/reformatAll __.sources"),
  "checkfmt" -> Seq("mill.scalalib.scalafmt.ScalafmtModule/checkFormatAll __.sources"),
  "deps"     -> Seq("mill.scalalib.Dependency/showUpdates"),
  "testall"  -> Seq("__.test"),
  "pub"      -> Seq("io.kipp.mill.ci.release.ReleaseModule/publishAll"),
)

// Alias Runner
def run(ev: eval.Evaluator, alias: String = "") = T.command {
  aliases.get(alias) match {
    case Some(t) =>
      mill.main.MainModule.evaluateTasks(
        ev,
        t.flatMap(x => Seq(x, "+")).flatMap(_.split("\\s+")).init,
        mill.define.SelectMode.Single,
      )(identity)
    case None =>
      Console.err.println("Use './mill run [alias]'."); Console.out.println("Available aliases:")
      aliases.foreach(x => Console.out.println(s"${x._1.padTo(15, ' ')} - Commands: (${x._2.mkString(", ")})"));
      sys.exit(1)
  }
}

For example:

❯ ./mill run
[1/1] run
Use './mill run [alias]'.
Available aliases:
testall         - Commands: (__.test)
pub             - Commands: (io.kipp.mill.ci.release.ReleaseModule/publishAll)
deps            - Commands: (mill.scalalib.Dependency/showUpdates)
fmt             - Commands: (mill.scalalib.scalafmt.ScalafmtModule/reformatAll __.sources)
lint            - Commands: (mill.scalalib.scalafmt.ScalafmtModule/reformatAll __.sources, __.fix)
checkfmt        - Commands: (mill.scalalib.scalafmt.ScalafmtModule/checkFormatAll __.sources)
❯ ./mill run fmt
[1/1] run > [7/7] mill.scalalib.scalafmt.ScalafmtModule.reformatAll
Formatting 2 Scala sources
parsed config (v3.7.3): /Users/cdepaula/repos/scala/mill-docker-nativeimage/.scalafmt.conf

I could submit a PR adding something like this to the core Mill if desired.

Let's find some nice Mill-idiomatic API to replace such workarounds.

And, as it is often easier to start thinking how some API might feel like and also to spot issues, here is a first sketch how it could look like:

/** Run all tests. */
def testall = T.alias("__.test")
/** Publish all modules. */
def pub = T.alias("io.kipp.mill.ci.release.ReleaseModule/publishAll")
def run = T.alias(foo.bar.run)

def help = T.help(testAll, pub, run, foo.run)

$ mill help
testall    Run all tests.
           -> mill __.test
pub        Publish all modules.
           -> mill io.kipp.mill.ci.release.ReleaseModule/publishAll
run        -> mill foo.bar.run
foo.run    -> mill foo.run

Please comment, what you think?

[carlosedp]

I like the idea and the simplicity to define the aliases... I think it would be great if T.alias could receive String* so multiple tasks could be chained in one alias.

Another improvement is if the help command could be defined dynamically from all T.aliases and maybe add some kind of optional annotation to the alias that would be used by the help command.

Also will the alias override an existing task? For example in a module extending RootModule, the run task runs the module... would a run alias override it?

Thanks for using my code as an idea :)

[lefou]

Yeah, I had varargs in mind for T.alias, accepting either string or Task. I think overriding existing tasks would not fit the concept, as aliases should just act as a placeholder when resolving. But RootModule doesn't define a run task, so there shouldn't be many conflicts.

[carlosedp]

I'd love to take a shot on this one... would you point me where to start looking (maybe follow the implementation of other T. ... I'm not versed into Scala macros so this could be a limitation for me...

[lefou]

I'd hoped to get some feedback on the idea from @lihaoyi or @lolgab , so that we agree on whether we want something like that in Mill in general and what it should feel like API wise. E.g. should we refer to the segments of a target or the real tasks or should we allow both.

But I guess, it wouldn't harm to start with a POC, to see the impact on code and usability and to get familiar with the Mill code base. But you should be prepared that your work will not make it into main, if there is no agreement.

As aliases don't provide their own tasks, there are probably no macros needed. I think aliases should be handled by the resolver (mill.resolve.Resolve and mill.resolve.ResolveCore). The actual aliases might be stored in the context of the modules.

[lihaoyi]

I don't like the discussed APIs so far. They work, but they don't feel like the rest of Mill. Specifically, Mill has distinct Scala APIs and CLIs, and mixing query strings in the build.sc Scala code feels out of place (even if it can already be done, and is done in some places). It's concise and familiar, but without IDE support or compile checking. Overall it just feels out of place, even if the same aliases implemented in Bash would fit right in.

Something like this would be better IMO, at least keeping to Scala-like APIs in the build.sc:

/** Run all tests. */
def testall = T.alias(allModules[TestModule](_.test()))

/** Publish all modules. */
def pub = T.alias(io.kipp.mill.ci.release.ReleaseModule.publishAll(???))
def run = T.alias(foo.bar.run)

def help = T.help(testAll, pub, run, foo.run)

I wonder if we can drop the wrappers, and let people just alias tasks directly? This cuts boilerplate further, but we'd need some story for how CLI arguments are handled (does each alias need to duplicate them? Or can we automate that somehow?)

/** Run all tests. */
def testall = allModules[TestModule](_.test())

/** Publish all modules. */
def pub = io.kipp.mill.ci.release.ReleaseModule.publishAll(???)
def run = foo.bar.run()

Some of the need for aliases could be satisfied in other ways, albeit less conveniently:

1. A better package organization e.g. moving MainModule to mill.runner would allow a def idea = T.command{...} to shorten the tedious mill.scalalib.GenIdea/idea, though this won't work for external modules

2. "Bulk" aliases can already be done a bit clunkily via def testall = T.command{ T.traverse(allModules[TestModule])(_.test())() }. Maybe we can simplify it to allow removal of the T.command wrapper?

3. Showing alias metadata like run -> mill foo.bar.run could conceivably be done via the inspect metadata on existing tasks.

4. Showing common commands currently already exists in various readme files specific to each project, albeit in a totally unstructured format

5. Showing the "important" tasks could (should?) also be done via a more general prioritization mechanism to allow some control of the order or grouping of things shown in mill resolve. e.g. even within a single ScalaModule foo.bar.baz, presumably foo.bar.baz.run and foo.bar.baz.publishLocal would be more relevant to users than foo.bar.baz.resolvedAmmoniteReplIvyDeps or foo.bar.baz.artifactNameParts, even if the latter are certainly necessary internally and even sometimes necessary for users to override

Overall I can see the need, but I'm not convinced it's a good ROI in terms of complexity v.s. benefit, at least what has been discussed so far. I feel like we should try to streamline the existing clunky techniques above, rather than introducing a whole new set of APIs just to bypass them and ending up with two ways to do the same thing. There's that saying "every problem can be solved by an additional layer of indirection, except the problem of too many layers of indirection", and one of Mill's primary design goals has been to minimize the number of "layers" that a user needs to think about. If we could better integrate these user workflows by streamlining the core Mill APIs, rather than layering them on top, I think that would put us in a better place

One compromise is we could prototype this as a plugin, either an external plugin we link to on our docsite or as a in-tree plugin in the contrib/ folder. This may still require changes to Mill's core code, depending on how it's done, but probably fewer changes than implementing the whole thing in Mill to begin with. Such an approach would let it evolve more quickly with real broad usage, and we can defer to a later date the question of integrating it into the Mill core wholesale or piecemeal.

[lefou]

How about supporting a lightweight alias mechanism like git has? Just a small textual replacement directly handled by the Mill CLI. It could even allow CLI options.

//> using mill.alias myalias: -j0 -d __.someTask + someOtherCommand
//> using mill.alias testAll: --no-server -j1 app.smoketests
//> using mill.alias find-updates-1: --meta-level 1 -j0 mill.scalalib.Dependency/showUpdates --format PerDependency

[lihaoyi]

I don't think we should push this kind of logic out of the Scala code. The Scala code is messy, but adding more adhoc places to look for stuff only makes it messier.

IMO adding some kind of T.alias shorthand would be preferable, and stuff like magicimports/metabuild/directives should only be used in scenarios where they are truly necessary e.g. if you need to perform logic before the Scala code compiles