Reify tasks in DML

[EvanKirshenbaum]

Currently, DML has syntax for performing actions sequentially:

{
   do_this;
   then_this;
}

and in parallel:

[[
   do_this;
   while_doing_this;
]]

The parallel block ends when all of its statements have finished unless one of them propagates an EvaluationError, in which case that's immediately propagated. (Other unfinished statements continue to execute.)

With the addition of future values (#283) and better support for paths (#282), you can now say something like

[[
   future drop d2 : walk_the_rest_of_d2s_path;
   initial_path : split north as d2 : continue_walking;
]]

This creates d2 but waits for it to get a value before moving. The problem is that because the future drop initialization includes the notion of waiting within it, I suspect that it will seem reasonable to do this inside of a sequential block, but the initialization statement for d2 doesn't end until the injection is done.

Without injections in the initialization, it would have to be written as

future drop d2
[[
   initial_path : split north as d2 : continue_walking;
   d2 : walk_the_rest_of_d2s_path;
]]

which more clearly needs to be done in parallel, but I sort of like the ability to not have to pre-declare.

What I'm thinking of here is the ability to have a basic T task notion to say Do this in the background and give me the ability to wait for it. This would allow background tasks to be spawned in sequential blocks, and would make it easier to decide that some don't need to be waited for. The syntax I'm thinking of is something like

{
    task t1 is future drop d2 : walk_the_rest_of_d2s_path;
    task t2 is initial_path : split north as d2 : continue_walking;
    join with t1, t2;
}

(Normally, I would use wait for, but I already use this in the sense of pause.) Tasks would be first class objects, typed to the value of their expressions, but they wouldn't be assignable. You would be able to ask for the task's value (if it was finished) and whether or not it was finished. I would probably also want a notion of task group for when we don't want to have to individually name tasks or when they might be created in a loop, as

{
   task group tasks;
   repeat with i = 1 to 10 {
      task in tasks is do_phase(i);
   }
   join with tasks;
}

I don't think you can do that with the current syntax.

One other thing we might want to do is have a task be specified to be delayed, meaning it doesn't actually start until you say t : start or group : start. (An alternative would be to have each one explicitly wait on a barrier (#284).)

Migrated from internal repository. Originally created by @EvanKirshenbaum on Jul 06, 2023 at 2:01 PM PDT.