feat(cli): Add run-id option to put sub-command #11023
Conversation
WalkthroughThe recent updates enhance metadata ingestion capabilities by introducing optional parameters for system metadata and run IDs. These improvements facilitate better tracking and management of metadata changes, enriching the data handling process. Key functions are now more versatile, allowing users to provide essential context during metadata operations, which bolsters overall utility and traceability. Changes
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configuration File (
|
github-actions
bot
added
the
ingestion
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
Outside diff range, codebase verification and nitpick comments (1)
metadata-ingestion/src/datahub/cli/cli_utils.py (1)
Line range hint
169-191:
Reminder: Add tests.The function
post_entityis missing test coverage.Do you want me to generate the unit testing code or open a GitHub issue to track this task?
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (2)
- metadata-ingestion/src/datahub/cli/cli_utils.py (3 hunks)
- metadata-ingestion/src/datahub/cli/put_cli.py (5 hunks)
Additional comments not posted (3)
metadata-ingestion/src/datahub/cli/put_cli.py (2)
Line range hint
42-60:
LGTM! But verify the function usage in the codebase.The code changes are approved.
However, ensure that all function calls to
aspectmatch the new signature.
Line range hint
91-117:
LGTM! But verify the function usage in the codebase.The code changes are approved.
However, ensure that all function calls to
platformmatch the new signature.metadata-ingestion/src/datahub/cli/cli_utils.py (1)
Line range hint
169-191:
LGTM! But verify the function usage in the codebase.The code changes are approved.
However, ensure that all function calls to
post_entitymatch the new signature.Verification successful
Verification Successful: Function calls to
post_entitymatch the new signature.The function
post_entityis correctly used with the new signature in the following files:
smoke-test/tests/timeline/timeline_test.pymetadata-ingestion/src/datahub/cli/put_cli.pyScripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify all function calls to `post_entity` match the new signature. # Test: Search for the function usage. Expect: Only occurrences of the new signature. rg --type python -A 5 $'post_entity'Length of output: 66
Script:
#!/bin/bash # Description: Verify all function calls to `post_entity` match the new signature. # Test: Search for the function usage. Expect: Only occurrences of the new signature. rg -A 5 'post_entity'Length of output: 2908
| @click.option("--run-id", type=str, help="Run ID into which we should log the aspect.") | ||
| @upgrade.check_upgrade | ||
| @telemetry.with_telemetry() | ||
| def aspect(urn: str, aspect: str, aspect_data: str) -> None: | ||
| def aspect(urn: str, aspect: str, aspect_data: str, run_id: str) -> None: |
There was a problem hiding this comment.
Reminder: Add tests.
The function aspect is missing test coverage.
Do you want me to generate the unit testing code or open a GitHub issue to track this task?
| @click.option( | ||
| "--run-id", type=str, help="Run ID into which we should log the platform." | ||
| ) | ||
| def platform( | ||
| ctx: click.Context, name: str, display_name: Optional[str], logo: str | ||
| ctx: click.Context, name: str, display_name: Optional[str], logo: str, run_id: str |
There was a problem hiding this comment.
Reminder: Add tests.
The function platform is missing test coverage.
Do you want me to generate the unit testing code or open a GitHub issue to track this task?
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- metadata-ingestion/src/datahub/cli/cli_utils.py (3 hunks)
Files skipped from review as they are similar to previous changes (1)
- metadata-ingestion/src/datahub/cli/cli_utils.py
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- metadata-ingestion/src/datahub/cli/cli_utils.py (3 hunks)
Files skipped from review as they are similar to previous changes (1)
- metadata-ingestion/src/datahub/cli/cli_utils.py
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- metadata-ingestion/src/datahub/cli/put_cli.py (6 hunks)
Files skipped from review as they are similar to previous changes (1)
- metadata-ingestion/src/datahub/cli/put_cli.py
pedro93
force-pushed
the
feature/cus-2323
branch
from
11b4eeb
to
f5aecd6
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Files selected for processing (1)
- metadata-ingestion/src/datahub/cli/put_cli.py (6 hunks)
Files skipped from review as they are similar to previous changes (1)
- metadata-ingestion/src/datahub/cli/put_cli.py
Adds an option to assign
run-idto a given put command execution. This is useful when transformers do not exist for a given ingestion payload, we can follow up with custom metadata and assign it to an ingestion pipeline.Checklist
Summary by CodeRabbit
run_idparameter foraspectandplatformfunctions, allowing for better tracking of metadata changes associated with specific runs.run-id, improving user experience during invocation.