Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #1240 #1241

Merged
merged 2 commits into from
Nov 26, 2024
Merged

Fixes #1240 #1241

merged 2 commits into from
Nov 26, 2024

Conversation

skrawcz
Copy link
Collaborator

@skrawcz skrawcz commented Nov 24, 2024

The cache store assumed that every persister took a path argument. That is not the case because the savers / loaders wrap external APIs and we decided to not try to create our own abstraction layer around them, and instead mirror them.

E.g. polars takes file, but pandas takes path.

Changes

  • cache/store.py

How I tested this

  • locally

Notes

Checklist

  • PR has an informative and human-readable title (this will be pulled into the release notes)
  • Changes are limited to a single goal (no scope creep)
  • Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future TODOs are captured in comments
  • Project documentation has been updated if adding/changing functionality.

The cache store assumed that every persister took a `path` argument. That is
not the case because the savers / loaders wrap external APIs and we decided
to not try to create our own abstraction layer around them, and instead mirror them.

E.g. polars takes `file`, but pandas takes `path`.
@skrawcz skrawcz linked an issue Nov 24, 2024 that may be closed by this pull request
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Reviewed everything up to 2e46872 in 31 seconds

More details
  • Looked at 36 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. hamilton/caching/stores/file.py:80
  • Draft comment:
    Ensure that saver_cls.name() and loader_cls.name() are valid method calls. If these classes do not have a name() method, consider using saver_cls.__name__ and loader_cls.__name__ to get the class name.
  • Reason this comment was not posted:
    Comment did not seem useful.

Workflow ID: wflow_9zJcc6qIk9G7I0QK


Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

hamilton/caching/stores/file.py Show resolved Hide resolved
To catch case with `file` and without `path` or `file`.
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Incremental review on e12b943 in 40 seconds

More details
  • Looked at 127 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 2 drafted comments based on config settings.
1. tests/caching/test_result_store.py:196
  • Draft comment:
    Using eval to load data is a security risk. Consider using a safer alternative like json.loads or ast.literal_eval if the data format allows.
  • Reason this comment was not posted:
    Marked as duplicate.
2. tests/caching/test_result_store.py:145
  • Draft comment:
    Avoid using eval for loading data as it can execute arbitrary code. Consider using a safer alternative like json.loads if the data format allows.
  • Reason this comment was not posted:
    Marked as duplicate.

Workflow ID: wflow_ZHnp8LhCA8wQPFRN


Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.


def load_data(self, type_: Type[Type]) -> Tuple[Type, Dict[str, Any]]:
with open(self.file, "r") as f:
data = eval(f.read())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using eval to load data is a security risk. Consider using a safer alternative like json.loads or ast.literal_eval if the data format allows.

Suggested change
data = eval(f.read())
data = ast.literal_eval(f.read())

tests/caching/test_result_store.py Show resolved Hide resolved
tests/caching/test_result_store.py Show resolved Hide resolved
@skrawcz skrawcz merged commit 911c3ca into main Nov 26, 2024
24 checks passed
@skrawcz skrawcz deleted the fix_1240 branch November 26, 2024 00:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Caching a polars dataframe into parquet fails
1 participant