Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix SSH task collection folder removal upon failure #1649

Merged
merged 9 commits into from
Jul 17, 2024

Conversation

tcompa
Copy link
Collaborator

@tcompa tcompa commented Jul 15, 2024

Close #1633

Checklist before merging

  • I added an appropriate entry to CHANGELOG.md
  • I merged main into the current branch.

Copy link

github-actions bot commented Jul 15, 2024

Benchmark comparison

GET /api/alive/

User Time current (ms) Time main (ms) Ratio current/main Size current (Kb) Size main (Kb)
vanilla 0.8 0.9 0.89 0.0 0.0
power 0.8 0.8 1.00 0.0 0.0
dataset 0.8 0.8 1.00 0.0 0.0
project 0.8 0.8 1.00 0.0 0.0
job 0.8 0.8 1.00 0.0 0.0

GET /api/v2/dataset/

User Time current (ms) Time main (ms) Ratio current/main Size current (Kb) Size main (Kb)
vanilla 8.6 8.5 1.01 0.7 0.7
power 404.0 415.2 0.97 440.2 440.2
dataset 160.4 163.2 0.98 171.2 171.2
project 48.9 49.0 1.00 58.9 58.9
job 28.7 28.2 1.02 41.7 41.7

GET /api/v2/job/

User Time current (ms) Time main (ms) Ratio current/main Size current (Kb) Size main (Kb)
vanilla 5.6 5.5 1.02 2.2 2.2
power 139.0 140.8 0.99 652.0 652.0
dataset 146.5 143.4 1.02 904.8 904.8
project 44.4 45.8 0.97 283.7 283.7
job 36.2 37.3 0.97 224.8 224.8

GET /api/v2/project/

User Time current (ms) Time main (ms) Ratio current/main Size current (Kb) Size main (Kb)
vanilla 6.0 6.3 0.95 0.1 0.1
power 5.9 5.9 1.00 0.1 0.1
dataset 5.9 5.8 1.02 0.1 0.1
project 8.5 7.8 1.09 2.3 2.3
job 6.0 5.9 1.02 0.1 0.1

GET /api/v2/workflow/

User Time current (ms) Time main (ms) Ratio current/main Size current (Kb) Size main (Kb)
vanilla 12.4 12.8 0.97 0.8 0.8
power 23.6 24.5 0.96 29.8 29.8
dataset 18.5 18.5 1.00 17.1 17.1
project 21.2 21.5 0.99 21.5 21.5
job 10.1 10.1 1.00 0.9 0.9

POST /api/v2/project/3/dataset/404/images/query/

User Time current (ms) Time main (ms) Ratio current/main Size current (Kb) Size main (Kb)
dataset 146.3 151.6 0.97 275.1 275.1

@tcompa tcompa marked this pull request as ready for review July 15, 2024 11:04
Copy link

github-actions bot commented Jul 15, 2024

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  fractal_server/ssh
  _fabric.py
  fractal_server/tasks/v2
  background_operations_ssh.py
Project Total  

This report was generated by python-coverage-comment-action

@tcompa
Copy link
Collaborator Author

tcompa commented Jul 16, 2024

Note that renaming a venv folder breaks its bin/activate script, even though the python interpreter remains valid.
Let's find a better way to proceed, and avoid this renaming.

@tcompa
Copy link
Collaborator Author

tcompa commented Jul 16, 2024

For the record

$ python -m venv MYVENV

$ grep -n MYVENV MYVENV/bin/*
MYVENV/bin/activate:41:VIRTUAL_ENV="/tmp/MYVENV"
MYVENV/bin/activate:58:    PS1="(MYVENV) ${PS1:-}"
MYVENV/bin/activate:60:    VIRTUAL_ENV_PROMPT="(MYVENV) "
MYVENV/bin/activate.csh:11:setenv VIRTUAL_ENV "/tmp/MYVENV"
MYVENV/bin/activate.csh:20:    set prompt = "(MYVENV) $prompt"
MYVENV/bin/activate.csh:21:    setenv VIRTUAL_ENV_PROMPT "(MYVENV) "
MYVENV/bin/activate.fish:36:set -gx VIRTUAL_ENV "/tmp/MYVENV"
MYVENV/bin/activate.fish:59:        printf "%s%s%s" (set_color 4B8BBE) "(MYVENV) " (set_color normal)
MYVENV/bin/activate.fish:68:    set -gx VIRTUAL_ENV_PROMPT "(MYVENV) "
MYVENV/bin/pip:1:#!/tmp/MYVENV/bin/python3
MYVENV/bin/pip3:1:#!/tmp/MYVENV/bin/python3
MYVENV/bin/pip3.10:1:#!/tmp/MYVENV/bin/python3

@tcompa
Copy link
Collaborator Author

tcompa commented Jul 17, 2024

Note that renaming a venv folder breaks its bin/activate script, even though the python interpreter remains valid. Let's find a better way to proceed, and avoid this renaming.

Fixed with 2acff13, by moving to a more explicit strategy

@tcompa tcompa merged commit 6d20e73 into main Jul 17, 2024
19 checks passed
@tcompa tcompa deleted the 1633-fix-removing-remote-venv-folder-over-ssh branch July 17, 2024 07:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove remote folder upon failed SSH task collection
1 participant