Discussing layer architecture and current names #53
Replies: 5 comments 7 replies
-
Yes there are no nested stages if that's what you mean
If I'm not mistaken this class is meant to be further developed to handle the multiple submission modes (HPC/Grid) and also asynchronous execution such as below in 2 |
Beta Was this translation helpful? Give feedback.
-
If you could also pass the parameters, I really like this idea and from what you describe I think it should be a relevant simplification, +1!
Agreed! How about |
Beta Was this translation helpful? Give feedback.
-
Wrapping up what @amjjbonvin @rvhonorato and I discussed today and answering @rvhonorato previous comments: So, if there are no nested I am becoming an FP-dev1l; the need for an HPC/Grid submission routine is not crucial to enforce a I prefer programming in layers where each layer/unit is independent of its "parent" units. For example, if B uses A, A should be independent of B. I like to pass parameters down the line. I have over-used kwargs sometimes for this matter, not sure yet if that is the best approach or if it is best to have parameter namespaces for that. But passing parameters down the line should be our aim, IMHO. @amjjbonvin, which nomenclature do you prefer? 🤖 Which actions do you want to do?
If at the end of the day In today's meeting, we decided that haddock3
Inside A novelty from today's meeting, the different
The order = [
"topology"
"scoring" # this would give the default
"clustering.fcc"
"analysis.whatever"
"scoring.megascore"
] Regarding the defaults, there should not be a module named Pull Request #47 is a dummy PR that we can use to draft and write ideas on the necessary actions (modules) and their categories. Use I will work towards refactoring the config to the execution engines considering the new layering and module structure discussed here. |
Beta Was this translation helpful? Give feedback.
-
Let's say a user wants to go directly to CG. The order in the config file would be:
order = ["coarsegrain", "rigidock", …]
But you still need the AA topo…
And I would still call it: “cgtopo” and “aatopo” - makes things cleared to my opinion
In this case the CG module could handle the AA internally.
Could be indeed, but it should create then a directory with the aa topologies. So could generate 00_cgtopo and 00_aatopo (i.e. both 00)
Probably better to have it automatic. If a user would not specify the allatom in the order, then some module might fail downstream.
|
Beta Was this translation helpful? Give feedback.
-
topoaa and topocg are better organizable than aatopo and cgtopo imho.
agreed!
|
Beta Was this translation helpful? Give feedback.
-
HADDOCK3 hierarchic names and general architecture:
Current configuration
The current configuration at the time of writing is:
workflow
: contains all operations and is somehow managed by theclass WorkflowManager
class Workflow
is assigned as aself.recipe
insideclass WorkflowManager
. So, arecipe
is aworkflow
within theWorkflowManager
context.stage
is an operation done in the workflow. Examples ofstages
are:topology
,scoring
,clustering
.stages
are controlled by theclass Workflow
. Andstages
do nothing, they simply refer to the concept of the operation.stages
have differentsteps
-class Step
.steps
are the actual operations defined by the python modules within eachstage
's folder, likedefault.py
.steps
are also namedmode
inside theclass Step
.stage
is namedmodule
inside theclass Step
.Comments
In the configuration file, and in the code, on top of the haddock module layer there is the
stage
layer. But I don't see more than onestage
item in the[stage]
layer. Is it? Even looking to the original @brianjimenez examples I see no more than onestage
item. There are manymodules
inside thestage
layer, but not another stage. So, why we need the conceptstage
?class WorkflowManager
manages nothing and it can be refactored to a simpler function named, for example,run_haddock()
.My suggestions
So, I suggest refactoring to:
A general
Workflow
that contains a sequence ofmodes
. Nothing more.In this way, we refactor from 5 conceptual layers to only 2. The fact that
modes
are contained withinmodules
does not mean that amodule
per se is a conceptual or organizational layer; it is just a category, a tag, organizing themodes
. Well, it happens to be a folder serving code organization purposes.If we refactor from 5 layers to 2 layers as I am proposing, we will decouple the
stages
and the user can freely compose the workflow considering only themodes
:Yes, we can currently do:
but we have to assign
mode=""
assuming this isdefault
and also using integers for ordering is suboptimal because if we want to reorder we need to change every integer in the list. Furthermore, there is this concept ofstage
in the code and the configuration file for which I don't see a purpose. Note: if there exists a reason for having severalstages
it is not reflected in the current definition of theorder
pameter, not the code itself.In
haddock3/src/haddock/workflow.py
Line 92 in ab01731
self.module
to refer to conceptualhaddock3
modules andself.mode
to refer the actual operations. I suggest we find another name for "haddock modules" because modules refer to*.py
files in a project, and creates confusion.If we remove the
stage
andmodule
layers, we can also renamemode
because mode refers to a mode of a module and we would not separate them anymore in the configuration file. Maybe we can haveaction
?@amjjbonvin @rvhonorato, what do you think? If you agree, I can work on this refactor.
Beta Was this translation helpful? Give feedback.
All reactions