custom mutations

[oliver-sanders]

Follow-on from #339

Something I've had at the back of my mind for a while that should probably be documented.

Api-On-The-Fly (AOTF) allows us to integrate mutations into the UI automatically, adding them to the relevant context menus and creating forms for manual input.

This means that we could very easily support user-defined mutations. For example here is a (slightly contrived) version of a common operation, setting HPC reservations via broadcasts:

mutation set_reservation($workflow: WorkflowID, $cycle: CyclePoint, $reservation: String){
  broadcast(workflows: [$workflow], cyclePoints: [$cycle], namespaces: ["my_model_*", "obs*"], settings: [{"environment":{"RES": $reservation}}])
}

Note: I don't think GraphiQL likes our broadcast settings interface, might need to improvise...

AOTF understands what WorkflowID and CyclePoint mean so would add this mutation to the context menus for cycle points and workflows. Ah the power of automation.

Strengths:

• Allows working practices within groups to be codified, potentially very nice for operators.
• Enables compound mutations (you can wrap multiple actual mutations under one user-defined one).
• These mutations can be developed in GraphiQL then saved for future use, all within the UI.

Weaknesses:

• Documentation.
  • Could add a dictionary of {inputName: helpMarkdown} to go along with the mutation.
  • Trivial to fold that into the mutation stuff in the UI.
• Chaining.
  • Looks like mutations should be executed in order by the server.
  • Would need zmq: implement REQ-REP + PUB-SUB comms cylc-flow#3329 to guarantee execution order (as opposed to queueing order).
• Cylc Flow version changes can break user-defined mutations.
• Configured per-user (though easily shared around teams as plain text).

Here's another operations-like example, it's somewhat a toy example, just intended to demonstrate the point:

mutation stop_in_preperation_of_transfer_to_another_platform($workflow:WorkflowID){
  hold(workflows:[$workflow]) {}
  kill(workflows:[$workflow], tasks:["start_cycle", "my_model*"])
}

mutation restart_after_transfer_to_another_platform($workflow:WorkflowID, $cycle:CyclePoint, $reservation:String){
  # reflow from the start of a defined cycle
  trigger(workflows:[$workflow], tasks:["cycle_start."$cycle], reflow=True)  # does string templating work here?
  # release everything
  release(workflows:[$workflow])
  # ensure the reservation is set
  # if only we had a way to ensure this mutation completed before release...
  broadcast(...)
}

And another toy example:

mutation hold_non_essential_work() {
  # we don't have an exclude workflows option here yet, but easy to add (already implemented for queries)
  hold(exworkflows:["oper*"])
}

Anyway, this is potentially quite simple to implement.

[hjoliver]

Nice idea, but the "weaknesses" you've listed above are significant. Worth thinking about on the back burner though.

[oliver-sanders]

Thinking again about chaining, I guess we might be able to do something at the UIS to run the mutations in order.
Once we have mutation tracking (i.e. the ability to follow the status of a mutation until completion) that could become quite powerful.

[oliver-sanders]

After a quick search it looks like if you define multiple mutations the server should execute them in order, halting if one of them fails.

At the moment this means that the commands are guaranteed to be queued in order.

Once we have mutation tracking that means they will be guaranteed to execute in order.

One weakness removed from the list.

[oliver-sanders]

Good use cases for both workflow-defined macros (e.g. setting reservations) and user/group-defined macros (e.g. managing workflows).