-
-
Notifications
You must be signed in to change notification settings - Fork 372
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added Files using tenacity for retrying renaming a directory on a opened filepath. #1826
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -18,6 +18,7 @@ venv*/ | |
.eggs/ | ||
.tox/ | ||
/local | ||
__pycache__/ | ||
|
||
/pip-wheel-metadata | ||
# IntelliJ Idea family of suites | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
Added Files integration using tenacity for path dir rename retry |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
from __future__ import annotations | ||
|
||
from pathlib import Path | ||
|
||
from tenacity import retry, stop_after_attempt, wait_fixed | ||
|
||
from briefcase.integrations.base import Tool, ToolCache | ||
|
||
|
||
class Files(Tool): | ||
name = "files" | ||
full_name = "Files" | ||
|
||
@classmethod | ||
def verify_install(cls, tools: ToolCache, **kwargs) -> Files: | ||
"""Make files available in tool cache.""" | ||
# short circuit since already verified and available | ||
if hasattr(tools, "files"): | ||
return tools.files | ||
|
||
tools.files = Files(tools=tools) | ||
return tools.files | ||
|
||
@retry(wait=wait_fixed(0.2), stop=stop_after_attempt(25)) | ||
def path_rename(self, old_path: Path, new_path: object): | ||
"""Using tenacity for a retry policy on pathlib rename. | ||
|
||
Windows does not like renaming a dir in a path with an opened file. | ||
""" | ||
old_path.rename(new_path) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If this is a "retry on any error" handler - what happens for predictable failure - e.g., attempting to rename a file that doesn't exist? There are some failures that should be surfaced immediately. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
import os | ||
import sys | ||
import threading | ||
import time | ||
|
||
import pytest | ||
import tenacity | ||
|
||
|
||
def test_rename_path(mock_tools, tmp_path): | ||
def openclose(filepath): | ||
handler = filepath.open(encoding="UTF-8") | ||
time.sleep(1) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sleep needs to be shorter - 1 second is a long time in a test |
||
handler.close() | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The close should be in a try:finally to ensure that if other parts of the process fail, the file will still be closed. |
||
|
||
(tmp_path / "orig-dir-1").mkdir() | ||
tl = tmp_path / "orig-dir-1/orig-file" | ||
tl.touch() | ||
file_access_thread = threading.Thread(target=openclose, args=(tl,)) | ||
rename_thread = threading.Thread( | ||
target=mock_tools.files.path_rename, | ||
args=(tmp_path / "orig-dir-1", tmp_path / "new-dir-1"), | ||
) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is the rename thread needed? It should be possible to do this inline. |
||
|
||
file_access_thread.start() | ||
rename_thread.start() | ||
|
||
rename_thread.join() | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should join on the file_access_thread to ensure that it cleans up. |
||
|
||
assert "new-dir-1" in os.listdir(tmp_path) | ||
|
||
|
||
@pytest.mark.xfail( | ||
sys.platform == "win32", | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm a little concerned about this being a win32 only error. macOS and Linux won't fail because of a file rename, but they could fail for other reasons - we need to verify the "will eventually fail" path on all platforms. |
||
raises=tenacity.RetryError, | ||
reason="Windows can't rename folder in filepath when the file is open", | ||
) | ||
def test_rename_path_fail(mock_tools, tmp_path, monkeypatch): | ||
(tmp_path / "orig-dir-2").mkdir() | ||
(tmp_path / "orig-dir-2/orig-file").touch() | ||
|
||
with (tmp_path / "orig-dir-2/orig-file").open(encoding="UTF-8"): | ||
|
||
monkeypatch.setattr(tenacity.nap.time, "sleep", lambda x: True) | ||
mock_tools.files.path_rename(tmp_path / "orig-dir-2", tmp_path / "new-dir-2") |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
from briefcase.integrations.files import Files | ||
|
||
|
||
def test_short_circuit(mock_tools): | ||
"""Tool is not created if already cached.""" | ||
mock_tools.files = "tool" | ||
|
||
tool = Files.verify(mock_tools) | ||
|
||
assert tool == "tool" | ||
assert tool == mock_tools.files |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Comma on the end is intentional.