Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(datasets): Added PyTorchDataset #735

Merged
merged 7 commits into from
Aug 21, 2024

Conversation

bpmeek
Copy link
Contributor

@bpmeek bpmeek commented Jun 17, 2024

Description

Closes 714

Development notes

Added PyTorchDataset to use pytorch's recommended save and load methods. Added tests to kedro-datasets/tests/kedro_datasets_experimental/pytorch/test_pytorch_dataset.py

Checklist

  • Updated the documentation to reflect the code changes
  • Added a description of this change in the relevant RELEASE.md file
  • Added tests to cover my changes

@ankatiyar
Copy link
Contributor

Hey @bpmeek, could you resolve the DCO failures following the instructions here - https://github.com/kedro-org/kedro-plugins/pull/735/checks?check_run_id=26337551853

@bpmeek bpmeek changed the title Added PyTorchDataset feat(datasets): Added PyTorchDataset Jun 18, 2024
@bpmeek bpmeek force-pushed the feature/pytorch-dataset branch 2 times, most recently from bbbd8df to 2b7a162 Compare June 18, 2024 14:48
Signed-off-by: bpmeek <bpmeek.developer@gmail.com>
Signed-off-by: bpmeek <bpmeek.developer@gmail.com>
@bpmeek
Copy link
Contributor Author

bpmeek commented Jun 18, 2024

I'm not sure what to do about the unit-test failures, none of them are related to my changes.

running pytest kedro-datasets/tests/kedro_datasets_experimental/pytorch/test_pytorch_dataset.py passes successfully.

@ankatiyar
Copy link
Contributor

@bpmeek Yeah the failures don't seem to be related, I'll take a look at the CI failures! Thanks for fixing the DCO check!

Signed-off-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Copy link
Contributor Author

@bpmeek bpmeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

merelcht and others added 2 commits August 21, 2024 11:04
Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Copy link
Member

@merelcht merelcht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the experimental contribution @bpmeek ! I've added the dataset to pyproject.toml as well with torch as dependency.

@merelcht merelcht requested a review from DimedS August 21, 2024 13:17
Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Copy link
Contributor

@DimedS DimedS left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks, @bpmeek !

@merelcht merelcht merged commit 36524d6 into kedro-org:main Aug 21, 2024
14 checks passed
merelcht added a commit to galenseilis/kedro-plugins that referenced this pull request Aug 27, 2024
* Added PyTorchDataset

Signed-off-by: bpmeek <bpmeek.developer@gmail.com>

* updated RELEASE.md

Signed-off-by: bpmeek <bpmeek.developer@gmail.com>

* Add dependencies for PyTorchDataset

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>

* Add PyTorchDataset to API docs

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>

* Fix docs build

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>

---------

Signed-off-by: bpmeek <bpmeek.developer@gmail.com>
Signed-off-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Co-authored-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add PyTorch dataset to save models instead of using pickle
4 participants