Meta-farm a multi farm concept ?

[nyue]

I am interested in how job description can be use to describe jobs for different farm-setup/cluster from a common submission point. Does OpenJD support or may support some sort of farm session id/key/hash?

Scenario 1:

Master studio offshore/nearshore work to Studio A and Studio B. Master studio has agreement with A and B to be provided with 100 blades each for the work. Master Studio would like to have a farm view of those 100x2 blades however, Master studio and Studio A + B all uses different farm software e.g. Tractor, Qube, Deadline

Sceneario 2:

Master studio has on-prem and datacenter blades number 1000 , 500 respectively. They want to spin up 100 blades on each of AWS, GCP, Azure for 3 separate self-contain shows/projects. Master studio uses Deadline/Qube but would like to use HQueue on the cloud because they provided easier distributed simulation. Are there some meta-farm level conventions/systems ?

Are the two scenario of multi-farm relevant to OpenJD ?

My hope is that if OpenJD includes such per farm meta data, we may look forward to a future where there is a single pane of glass to view all the farms collectively/globally and also allow us to dive into each farm individually to monitor.

Kind regards

[mwiebe]

Hey Nicholas, thanks for the question and ideas!

The goals we have aimed for in developing OpenJD are focused on the input side of your examples. Here's how I picture that:

In Scenario 1, this can apply by creating a job template representing the job, that you can then choose to send to any of the available farms that supports OpenJD.

In Scenario 2, it similarly can apply in the job submission process. It doesn't apply to the other side of the scenario, where you want to view all the farms collectively/globally. This is an interesting question to consider, and to tackle it would need an analysis of the APIs that each farm provides and development of a common interface to query their status. It's not something we've looked at for the scope of OpenJD, but we're always open to contributors interested in questions like this.