Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX pipe_output / mutate naming convention and docstrings #1198

Merged
merged 2 commits into from
Oct 22, 2024

Conversation

jernejfrank
Copy link
Contributor

The naming convention for the first node in the pipe_output chain can potentially conflict with user-defined nodes. For example:

@pipe_output(...)
def features():
    ...

would lead to features_raw which is a common user naming convention. This makes it harder to have the conflict by following the same naming convention as the rest of the chained nodes.

Changes

  • first node is now named foo.with_raw
  • corrected docstring in mutate.__init__ to avoid escape key errors in pytest.

How I tested this

  • unit tests
  • sphinx autoload

Notes

The docstring issue is currently only tracked on slack: https://hamilton-opensource.slack.com/archives/C03M34FM058/p1729431573025409

Checklist

  • PR has an informative and human-readable title (this will be pulled into the release notes)
  • Changes are limited to a single goal (no scope creep)
  • Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future TODOs are captured in comments
  • Project documentation has been updated if adding/changing functionality.

Docstring raised Pytest escape error due to /* so we wrap it in quotes.
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Reviewed everything up to 84b7b2a in 21 seconds

More details
  • Looked at 65 lines of code in 2 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. tests/function_modifiers/test_macros.py:838
  • Draft comment:
    Update the test to reflect the new naming convention for the first node in the pipe_output chain. It should be result_from_downstream_function.with_raw instead of result_from_downstream_function_raw.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable:
    The comment is about a change made in the diff, specifically the renaming of a node in the test. The change has been implemented in the code, so the comment is not necessary as it describes an action that has already been completed.
    The comment might be useful for historical context or for other developers to understand why the change was made, but it doesn't suggest a new action to be taken.
    The purpose of the review is to identify actionable comments, and since the change has already been made, the comment does not serve an actionable purpose.
    The comment should be deleted because it describes a change that has already been implemented and does not suggest any further action.

Workflow ID: wflow_vpMJjWLJbICwbI7q


Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@@ -1299,7 +1299,8 @@ def transform_node(
else:
_namespace = self.namespace

original_node = node_.copy_with(name=f"{node_.name}_raw")
# We pick a reserved prefix that ovoids clashes with user defined functions / nodes
original_node = node_.copy_with(name=f"{node_.name}.with_raw")

def __identity(foo: Any) -> Any:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docstring mentions renaming the original function to function_name_raw, but the code now uses function_name.with_raw. Please update the docstring to reflect this change.

Copy link
Collaborator

@elijahbenizzy elijahbenizzy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, I think we should change it to .raw intead of .with_raw?

@@ -1299,7 +1299,8 @@ def transform_node(
else:
_namespace = self.namespace

original_node = node_.copy_with(name=f"{node_.name}_raw")
# We pick a reserved prefix that ovoids clashes with user defined functions / nodes
original_node = node_.copy_with(name=f"{node_.name}.with_raw")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think this should be .raw instead of .with_raw

Current name convention is very prone to name clashes with user naming.

We assign the same naming convention using namespace.raw to indicate the
first node.
Copy link
Collaborator

@elijahbenizzy elijahbenizzy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Thank you.

@elijahbenizzy elijahbenizzy merged commit 4b286be into DAGWorks-Inc:main Oct 22, 2024
24 checks passed
@jernejfrank jernejfrank deleted the bug/pipe branch October 27, 2024 15:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants