Skip to content

Commit

Permalink
fix(datasets): Use put() and get() instead of copy in `TensorFl…
Browse files Browse the repository at this point in the history
…owModelDataset`'s `_save` and `_load` methods. (#844)

* fix(datasets): Use `put()` and `get()` instead of `copy` in `TensorFlowModelDataset`'s `_save` and `_load` methods.

Signed-off-by: gitgud5000 <17186026+gitgud5000@users.noreply.github.com>

* chore: Added comments of changes in `RELEASE.md`

Signed-off-by: gitgud5000 <17186026+gitgud5000@users.noreply.github.com>

---------

Signed-off-by: gitgud5000 <17186026+gitgud5000@users.noreply.github.com>
Signed-off-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Co-authored-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com>
  • Loading branch information
3 people committed Sep 26, 2024
1 parent 06cf752 commit 0a3a381
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 2 deletions.
2 changes: 2 additions & 0 deletions kedro-datasets/RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,15 @@

## Bug fixes and other changes
* Refactored all datasets to set `fs_args` defaults in the same way as `load_args` and `save_args` and not have hardcoded values in the save methods.
* Fixed bug related to loading/saving models from/to remote storage using `TensorFlowModelDataset`.
* Fixed deprecated load and save approaches of GBQTableDataset and GBQQueryDataset by invoking save and load directly over `pandas-gbq` lib

## Breaking Changes
## Community contributions
Many thanks to the following Kedroids for contributing PRs to this release:
* [Brandon Meek](https://github.com/bpmeek)
* [yury-fedotov](https://github.com/yury-fedotov)
* [gitgud5000](https://github.com/gitgud5000)
* [janickspirig](https://github.com/janickspirig)


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ def _load(self) -> tf.keras.Model:
# We assume .keras
path = str(PurePath(tempdir) / TEMPORARY_KERAS_FILE) # noqa: PLW2901

self._fs.copy(load_path, path)
self._fs.get(load_path, path)

# Pass the local temporary directory/file path to keras.load_model
device_name = self._load_args.pop("tf_device", None)
Expand All @@ -169,7 +169,7 @@ def _save(self, data: tf.keras.Model) -> None:

# Use fsspec to take from local tempfile directory/file and
# put in ArbitraryFileSystem
self._fs.copy(path, save_path)
self._fs.put(path, save_path)

def _exists(self) -> bool:
try:
Expand Down

0 comments on commit 0a3a381

Please sign in to comment.