-
Notifications
You must be signed in to change notification settings - Fork 93
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
UI Server graphql protobuf feed #3122
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Few typos, unused imports, but nothing broken (except one name
variable created in another loop
that looks suspicious.
Finished reading the code diff in GitHub UI, and running a linter in my IDE. Reported issues found here, plus initial questions.
@dwsutherland amazing work! Will take a better look once the PR has been updated. This time will try to run the code.
It would be good, in my opinion, to have unit test for the new code. There are many methods that could be exercised without the need of a feature test, and some conditions in the if/else
blocks are easier to test with custom test data and/or mocking.
👏 👏 🎉
Conflicts after the |
fc6a9d7
to
e66437a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
1st pass through 👀...
dd9fc1f
to
08a1c7a
Compare
Closes #3023 |
08a1c7a
to
6bde666
Compare
6bde666
to
56be0a1
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
1st pass...
56be0a1
to
94b0ba0
Compare
@oliver-sanders : something others have raised, but I guess the hint is the i.e. sort by Hmm... strike that (I think) |
3d8956e
to
013b7ec
Compare
013b7ec
to
b810432
Compare
@dwsutherland we had a quick demo this morning (UK time) for the end to end system. I synced the repositories, and found one issue in 2019-06-27T09:11:10Z INFO - DONE
Traceback (most recent call last):
File "/home/kinow/Development/python/workspace/cylc-uiserver/venv/bin/cylc-run", line 25, in <module>
main(is_restart=False)
File "/home/kinow/Development/python/workspace/cylc-uiserver/venv/lib/python3.7/site-packages/cylc/flow/terminal.py", line 65, in wrapper
function(*args, **kwargs)
File "/home/kinow/Development/python/workspace/cylc-uiserver/venv/lib/python3.7/site-packages/cylc/flow/scheduler_cli.py", line 132, in main
scheduler.start()
File "/home/kinow/Development/python/workspace/cylc-uiserver/venv/lib/python3.7/site-packages/cylc/flow/scheduler.py", line 295, in start
raise exc
File "/home/kinow/Development/python/workspace/cylc-uiserver/venv/lib/python3.7/site-packages/cylc/flow/scheduler.py", line 266, in start
self.run()
File "/home/kinow/Development/python/workspace/cylc-uiserver/venv/lib/python3.7/site-packages/cylc/flow/scheduler.py", line 1545, in run
has_updated = self.update_data_structure()
File "/home/kinow/Development/python/workspace/cylc-uiserver/venv/lib/python3.7/site-packages/cylc/flow/scheduler.py", line 1592, in update_data_structure
self.ws_data_mgr.initiate_data_model()
File "/home/kinow/Development/python/workspace/cylc-uiserver/venv/lib/python3.7/site-packages/cylc/flow/ws_data_mgr.py", line 70, in initiate_data_model
self.update_task_proxies()
File "/home/kinow/Development/python/workspace/cylc-uiserver/venv/lib/python3.7/site-packages/cylc/flow/ws_data_mgr.py", line 364, in update_task_proxies
prereq_obj = prereq.api_dump(self.workflow_id)
File "/home/kinow/Development/python/workspace/cylc-uiserver/venv/lib/python3.7/site-packages/cylc/flow/prerequisite.py", line 258, in api_dump
expression=temp,
UnboundLocalError: local variable 'temp' referenced before assignment I fixed it by adding |
e1bd90a
to
3d0d529
Compare
3d0d529
to
bdc51d9
Compare
bdc51d9
to
ced2124
Compare
ced2124
to
2efac18
Compare
@kinow Sorry - there was a bug in it for about 30min (and it wouldn't have worked with the UI Server at that point anyway, as I had turned the protobuf message to JSON to test it)... All fixed now.. working on the UI Server |
MERGING! 💥 thanks @dwsutherland 💐 We'll put up a bunch of Issues for urgent follow-up... |
The sibling pull-request to the UI Server graphql integration:
cylc/cylc-uiserver#34
And the Cylc 8 component implementation of the GraphQL PoC.
This change is to provide data in structure and format suitable for resolving GraphQL queries/mutations off of, for the consumption/control the web UI.
There will be many more improvements/additions to be incorporated both in this and subsequent pull requests, however, for the sake of UI development and review this needs to be exposed.
The extra dependency is
protobuf
which is just installed via pip, and if the definition filews_messages.proto
is changed with development then the binary for generating the python module can be found here (not needed at runtime) and the following command used:At present graphql (at the UI Server) resolves directly off the decoded protobuf messages.. and the protobuf message for the entire data structure can be requested using:
Which seems just as fast as
$ cylc client --no-input baz api
(the overhead probably setting up the runtime client):At the moment the full data structure is recalculated in the mainloop on changes (just like
state_summary_mgr
updates), and an additional protobuf message is populated with this for encoding and transport.This Pull Request
Additional (to come or future PR?)