Graph: all nodes overlap on experiments with only independant jobs

[kinow]

In GitLab by @pabretonniere on Sep 14, 2022, 19:13

Hello @cgutierr

My experiment a55y is just a series of independent jobs. But the view is putting all the jobs at the same place in the graph.

https://earth.bsc.es/autosubmitapp/experiment/a55y/graph

Everything is there, as I can move everything manually but this is obviously not optimal.

(after moving the boxes by hand)

Could you please have a look?

Thanks

FYI @portega and @emoreno who have the same issue in a562 and a563

[kinow]

In GitLab by @cgutierr on Sep 15, 2022, 08:23

Hi @pabretonniere I will have a look and come up with an answer ASAP.

[kinow]

In GitLab by @cgutierr on Sep 19, 2022, 08:02

We were not supporting experiments without any dependency at all, this has been fixed, now the nodes will be displayed side by side and will not overlap.

This is how it looks:

This change will be included in the next deployment.

@pabretonniere @portega @emoreno

[kinow]

In GitLab by @pabretonniere on Sep 19, 2022, 12:25

Great, thanks! Do you know when the next deployment will be?

[kinow]

In GitLab by @mcastril on Sep 21, 2022, 13:07

We are still having some issues with the testing VM and CAS, but once that it's sorted out and we can test the new features in the https://earth.bsc.es/gitlab/es/autosubmitreact/-/milestones/2#tab-issues we can do an update

[kinow]

In GitLab by @jberlin on Nov 10, 2022, 16:12

I could not validate this because the issue with the init

[kinow]

In GitLab by @cgutierr on Nov 14, 2022, 08:21

This problem is due to the BasicConfig class. It is unable to read the VM's path correctly and it seems that the esarchive path is hardcoded:

[2022-11-14 08:06:28 +0000] [11299] [INFO] New Graph Representation failed: [Errno 2] No such file or directory: '/esarchive/autosubmit/as_metadata/graph/graph_data_t06v.db'

I've made a temporal fix and right now it's working fine.
I'll try to extend the BasicConfig class to be more dynamic so that this does not happen ever again and I'll create a new branch and merge the corresponding changes.

@jberlin

[kinow]

In GitLab by @jberlin on Nov 14, 2022, 09:11

Thanks @cgutierr, yes there are a lot of hardcoding stuff that in the long run should be removed.

[kinow]

In GitLab by @jberlin on Nov 14, 2022, 15:44

I re deployed all the components after the merges we did and I am still getting the same error for this particular case, maybe we are missing something ? I get:

graph tab:
init() takes exactly 2 arguments (3 given)

treeview
'NoneType' object has no attribute 'strip'

log of the API:

[2022-11-14 15:42:59 +0000] [18067] [INFO] LOGRUN|RTIME|a55y|0.0183820724487
[2022-11-14 15:43:00 +0000] [18067] [INFO] GRAPH|RECEIVED|a55y~none~standard
[2022-11-14 15:43:00 +0000] [18067] [INFO] Could not generate graph with faster method
[2022-11-14 15:43:00 +0000] [18067] [INFO] Could not generate Graph and recieved the following exception: __init__() takes exactly 2 arguments (3 given)
[2022-11-14 15:43:00 +0000] [18067] [INFO] Process: 18067 graph workers: {18067: ['jberlin', 'graph', u'a55y', True]}
[2022-11-14 15:43:00 +0000] [18067] [INFO] GRAPH|RTIME|a55y|0.0745420455933
[2022-11-14 15:43:01 +0000] [18067] [INFO] LOGRUN|RECEIVED|a55y
[2022-11-14 15:43:01 +0000] [18069] [INFO] GRAPH|RECEIVED|a55y~none~standard
[2022-11-14 15:43:01 +0000] [18067] [INFO] LOGRUN|RTIME|a55y|0.0171399116516
[2022-11-14 15:43:01 +0000] [18069] [INFO] Could not generate graph with faster method
[2022-11-14 15:43:01 +0000] [18069] [INFO] Could not generate Graph and recieved the following exception: __init__() takes exactly 2 arguments (3 given)
[2022-11-14 15:43:01 +0000] [18069] [INFO] Process: 18069 graph workers: {}
[2022-11-14 15:43:01 +0000] [18069] [INFO] GRAPH|RTIME|a55y|0.0505800247192
[2022-11-14 15:43:02 +0000] [18068] [INFO] LOGRUN|RECEIVED|a55y
[2022-11-14 15:43:02 +0000] [18067] [INFO] GRAPH|RECEIVED|a55y~date-member~standard
[2022-11-14 15:43:02 +0000] [18068] [INFO] LOGRUN|RTIME|a55y|0.0197179317474
[2022-11-14 15:43:02 +0000] [18067] [INFO] Could not generate graph with faster method
[2022-11-14 15:43:02 +0000] [18067] [INFO] Could not generate Graph and recieved the following exception: __init__() takes exactly 2 arguments (3 given)
[2022-11-14 15:43:02 +0000] [18067] [INFO] Process: 18067 graph workers: {}
[2022-11-14 15:43:02 +0000] [18067] [INFO] GRAPH|RTIME|a55y|0.0447189807892
[2022-11-14 15:43:03 +0000] [18067] [INFO] LOGRUN|RECEIVED|a55y
[2022-11-14 15:43:03 +0000] [18070] [INFO] GRAPH|RECEIVED|a55y~status~standard
[2022-11-14 15:43:03 +0000] [18067] [INFO] LOGRUN|RTIME|a55y|0.0196571350098
[2022-11-14 15:43:03 +0000] [18070] [INFO] Could not generate graph with faster method
[2022-11-14 15:43:03 +0000] [18070] [INFO] Could not generate Graph and recieved the following exception: __init__() takes exactly 2 arguments (3 given)
[2022-11-14 15:43:03 +0000] [18070] [INFO] Process: 18070 graph workers: {}
[2022-11-14 15:43:03 +0000] [18070] [INFO] GRAPH|RTIME|a55y|0.0546960830688

[kinow]

In GitLab by @cgutierr on Nov 16, 2022, 08:54

Thanks @jberlin for pointing those out.

I've pushed the fixes in a Validation_2 branch. With 1 commit for each fix.

• The Graph error was the same we had couple days ago but now the error came from the old graph generation.

  Commit 1/2

• The Tree strip error was due to the preparation I did for checking the Experiment version in the DB... For the Autosubmit 4 YAML validation. I've commented those out because it's not prioritary for this release and we'll have time to think about AS4 validation.

  Commit 2/2

The changes are already present at the rocky API, I hope I don't miss anything.

Pd: I think I've fixed the whitespacing problem so I hope now the code review is way better 😄

[kinow]

In GitLab by @jberlin on Nov 16, 2022, 09:11

Hi @cgutierr , great !! I will have a look, and also thanks for the whitespacing thing. so nows reviews will be more clear

[kinow]

In GitLab by @mcastril on Nov 16, 2022, 12:43

@cgutierr thanks for the details. I think that you can write the commit id's in the message if you had not tagged the issue in the commit and it doesn't automatically appear here.

[kinow]

In GitLab by @jberlin on Nov 18, 2022, 12:12

Validated in the development VM