fix(datasets): Use put()
and get()
instead of copy
in TensorFlowModelDataset
's _save
and _load
methods.
#844
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR resolves the issue in #839, where saving
TensorFlowModelDataset
to Azure Blob Storage fails due to incorrect use of.copy()
from thefsspec.filesystem
interface.Development notes
The issue occurs in the
_save
and_load
methods on lines 147 and 172, where.copy()
is used. According to the fsspec documentation,.copy()
is intended for remote-to-remote transfers. Since this involves copying from a local filesystem to remote storage,.put()
should be used for saving and.get()
for loading.The code has been updated to use
.put()
and.get()
accordingly, replacing the use of.copy()
.Both methods work for local-to-local and local-to-remote(& vice versa) transfers based on testing.
Checklist
RELEASE.md
file