-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: resolve relative datasample spec path #392
Changes from 4 commits
6092a2a
9cadf7d
753bee0
dcdbb7a
07f012c
9f0e61d
ca7c149
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -147,14 +147,31 @@ def is_many(self): | |
return self.paths and len(self.paths) > 0 | ||
|
||
@pydantic.model_validator(mode="before") | ||
def exclusive_paths(cls, values): # noqa: N805 | ||
def exclusive_paths(cls, values: typing.Any) -> typing.Any: # noqa: N805 | ||
"""Check that one and only one path(s) field is defined.""" | ||
if "paths" in values and "path" in values: | ||
raise ValueError("'path' and 'paths' fields are exclusive.") | ||
if "paths" not in values and "path" not in values: | ||
raise ValueError("'path' or 'paths' field must be set.") | ||
Comment on lines
153
to
156
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could they define both keys but with value There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't get what you're refering too. Could you provide an example plz ? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It looks like we allow There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok I see. I added some tests, and it appears that this case is dealts with in the resolve path, as There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't know if There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yep True... I rm the tests, and +1 to deal with this in a dedicated PR |
||
return values | ||
|
||
@pydantic.model_validator(mode="before") | ||
def resolve_paths(cls, values: typing.Any) -> typing.Any: # noqa: N805 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. can you remove the |
||
"""Resolve given path is relative.""" | ||
if "paths" in values: | ||
paths = [] | ||
for path in values["paths"]: | ||
path = pathlib.Path(path) | ||
paths.append(path.resolve()) | ||
|
||
values["paths"] = paths | ||
|
||
elif "path" in values: | ||
path = pathlib.Path(values["path"]) | ||
values["path"] = path.resolve() | ||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. else: values["path"]=path |
||
return values | ||
|
||
@contextlib.contextmanager | ||
def build_request_kwargs(self, local): | ||
# redefine kwargs builder to handle the local paths | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
import pathlib | ||
import uuid | ||
|
||
import pytest | ||
|
||
from substra.sdk.schemas import DataSampleSpec | ||
|
||
|
||
@pytest.mark.parametrize("path", [pathlib.Path() / "data", "./data", pathlib.Path().cwd() / "data"]) | ||
def test_datasample_spec_resolve_path(path): | ||
datasample_spec = DataSampleSpec(path=path, data_manager_keys=[str(uuid.uuid4())]) | ||
|
||
assert datasample_spec.path == pathlib.Path().cwd() / "data" | ||
|
||
|
||
def test_datasample_spec_resolve_paths(): | ||
paths = [pathlib.Path() / "data", "./data", pathlib.Path().cwd() / "data"] | ||
datasample_spec = DataSampleSpec(paths=paths, data_manager_keys=[str(uuid.uuid4())]) | ||
|
||
assert all([path == pathlib.Path().cwd() / "data" for path in datasample_spec.paths]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you remove the
# noqa: N805
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nop, its because it does not like that the first arg is not "self" :/
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this method receive a class as first argument, pydantic recommends decorate with
@classmethod
for a proper type checking (model_validator)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice thx :)