Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sdk): supporting bring your own container for arbitrary input and outputs #8066

Merged
merged 25 commits into from
Aug 8, 2022

Conversation

zichuan-scott-xu
Copy link
Contributor

Description of your changes:
The first 16 commits handling functions with no inputs are already reviewed and approved in #7973.

Checklist:

@google-oss-prow
Copy link

Hi @zichuan-scott-xu. Thanks for your PR.

I'm waiting for a kubeflow member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@zichuan-scott-xu
Copy link
Contributor Author

/cc @connor-mccarthy

@google-oss-prow
Copy link

@zichuan-scott-xu: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/ok-to-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Copy link
Member

@connor-mccarthy connor-mccarthy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great! Just a few suggestions before we merge with master.

sdk/python/kfp/compiler/compiler_test.py Show resolved Hide resolved
sdk/python/kfp/components/component_factory.py Outdated Show resolved Hide resolved
sdk/python/kfp/compiler/compiler_test.py Outdated Show resolved Hide resolved
sdk/python/kfp/dsl/__init__.py Show resolved Hide resolved
sdk/python/kfp/dsl/__init__.py Show resolved Hide resolved
@chensun
Copy link
Member

chensun commented Jul 26, 2022

/ok-to-test

@zichuan-scott-xu
Copy link
Contributor Author

/retest

Copy link
Member

@connor-mccarthy connor-mccarthy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the previous comments, Scott. Just a few more things I noticed from another review.

sdk/python/kfp/compiler/compiler_test.py Show resolved Hide resolved
sdk/python/kfp/components/component_factory.py Outdated Show resolved Hide resolved
Copy link
Member

@chensun chensun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Great work, @zichuan-scott-xu !

I left one question, but it doesn't necessarily block this PR from merging.

'--epochs',
num_epochs,
'--model_path',
model.uri,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if user just puts model here? Do we throw an error or return the whole artifact json? Also what if users access some arbitrary valid or invalid property/method here? Do we throw errors?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the reminder! The current design is to throw error in both cases and I will add the error handling logic before merging the PR.

@google-oss-prow google-oss-prow bot removed the lgtm label Aug 4, 2022
Copy link
Member

@connor-mccarthy connor-mccarthy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zichuan-scott-xu, can you add a test to validate that the exception in bb282dc is raised correctly?

@zichuan-scott-xu
Copy link
Contributor Author

/retest

Copy link
Member

@connor-mccarthy connor-mccarthy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, Scott!
/lgtm

cc: @chensun

@connor-mccarthy
Copy link
Member

/approve

@chensun
Copy link
Member

chensun commented Aug 8, 2022

/approve

Great work, @zichuan-scott-xu !

@google-oss-prow
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: chensun, connor-mccarthy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@connor-mccarthy
Copy link
Member

/retest

@google-oss-prow google-oss-prow bot merged commit 04c827f into kubeflow:master Aug 8, 2022
jlyaoyuli pushed a commit to jlyaoyuli/pipelines that referenced this pull request Jan 5, 2023
…d outputs (kubeflow#8066)

* support container_component decorator for function with no inputs

* resolve review comments

* add sample tests for milestone 1

* modify compiler test data

* resolve reviews

* resolve reviews

* WIP

* implementation of function of no inputs

* fixed sample test

* re-fix sample test

* fix rebase merge conflict

* resolve formatting

* resolve isort error for test data

* resolve comments

* fix nit

* resolve nit

* add implementation for placeholders i/o, sample and compiler tests

* resolve comments and merge logic for constructing container component

* resolve comments

* resolve comments

* fix assertion messages

* add error handling for accessing artifact by itself

* add test for raising error for accessing artifact by itself
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants