-
Notifications
You must be signed in to change notification settings - Fork 93
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Task messaging ssh #4119
Task messaging ssh #4119
Conversation
There is currently an issue with GH actions - looks like the ssh keys are having problems authenticating. Getting a |
8e5944a
to
7a27941
Compare
To update, the ssh comms is currently untestable on our CI build with the current GH actions setup (we are unable to ssh from the remote to the scheduler for security reasons). Hence the codecov/patch diff is low. Tests should run locally, with the addition of cylc-flow/etc/conf/global.cylc Line 10 in ad126f2
global-tests.cylc file.
ssh tests can then be run with: |
Incidentally, @oliver-sanders, I got to the bottom of the |
To add to Mel's comments on why we can't use the Docker images to test SSH comms at the moment. This issue proposes [optionally] running the tests themselves in docker (the test host not just the remote hosts) which should enable Scheduler -> Remote -> Scheduled SSH comms (as well as clearing up much of the configuration junk). (also adds Python and Bash versions to the test matrix) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, tested as working 👍 🎉
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, working on getting the tests passing on local platforms...
Two commands have not been converted to use the new get_client
interface:
cylc scan
.cylc tui
.
Neither are important (as they are unlikely to be used in remote jobs), however, it would be good to convert them to match the other commands. cylc scan
cannot be converted at the moment as is requies the async_request
interface which is not yet implemented. No biggie, have punted to another issue #4129, should be simple, can tackle later.
Could do with addressing the first comment bellow.
Ran tests against local platforms, all good, added these two to my regular test matrix:
|
Two conflicts with the |
Add env variable TASK_COMMS_METHOD. Update contact file with additional fields for ssh task comm.
Add functional test for ssh comms
Make error message more informative in ssh_client.py
… unable to test ssh in CI build)
Update cylc/flow/network/client_factory.py Co-authored-by: Hilary James Oliver <hilary.j.oliver@gmail.com> Update cylc/flow/network/ssh_client.py Co-authored-by: Oliver Sanders <oliver.sanders@metoffice.gov.uk>
7b02036
to
2025469
Compare
Now resolved. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rerun tests, all passing 👍!
Task communication ssh added.
New contact file fields:
SCHEDULER_SSH_COMMAND
,SCHEDULER_CYLC_PATH
,SCHEDULER_USE_LOGIN_SHELL
Removed unused contact file field:
COMMS_PROTOCOL_2
These changes close #3327
In light of discussions about
comms_timeout
at latest Cylc Video Conference, this relate to #4112 (comms_timeout
added to ssh comms in this PR)Requirements check-list
CONTRIBUTING.md
and added my name as a Code Contributor.