Skip to content

Commit

Permalink
Use ensure_ndarray in a few more places (#506)
Browse files Browse the repository at this point in the history
* Use `ensure_ndarray` to check `chunk`'s type

Instead of checking to see if `chunk` is an `ndarray`, use
`ensure_ndarray` to get an `ndarray` viewing the underlying data in
`chunk`. This way we know attributes like `dtype` are available and can
be checked easily. Also makes this a bit more friendly with other
array-like types.

* Use `ensure_ndarray` on `out`

Since we are interested in getting an `ndarray` representing the buffer
within `out` for writing into, go ahead and use `ensure_ndarray` to
coerce the underlying buffer into an `ndarray`. This way we can avoid a
needless check and just write into any array-like value for `out` that
is provided.

* Handle `out` when it doesn't supply a buffer

Appears that `out` can also be a Zarr `Array` or any other array-like
that does not expose a buffer, but does allow writing into. In these
cases `ensure_ndarray` will fail as there is not an underlying buffer
that can be used with an `ndarray`. To also handle this case, catch the
`TypeError` that `ensure_ndarray` will raise in this case and use that
to indicate whether `out` is now an `ndarray` or not. This allows us to
continue to write into arbitrary buffers, but also correctly handle
objects that do not expose buffers.

* Adds release notes entry
  • Loading branch information
jakirkham authored Nov 12, 2019
1 parent f5718c5 commit 58b1786
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 2 deletions.
3 changes: 3 additions & 0 deletions docs/release.rst
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,9 @@ Upcoming Release
* Use ``math.ceil`` for scalars.
By :user:`John Kirkham <jakirkham>`; :issue:`500`.

* Use ``ensure_ndarray`` in a few more places.
By :user:`John Kirkham <jakirkham>`; :issue:`506`.

* Refactor out ``_tofile``/``_fromfile`` from ``DirectoryStore``.
By :user:`John Kirkham <jakirkham>`; :issue:`503`.

Expand Down
10 changes: 8 additions & 2 deletions zarr/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -1572,6 +1572,12 @@ def _chunk_getitem(self, chunk_coords, chunk_selection, out, out_selection,

assert len(chunk_coords) == len(self._cdata_shape)

out_is_ndarray = True
try:
out = ensure_ndarray(out)
except TypeError:
out_is_ndarray = False

# obtain key for chunk
ckey = self._chunk_key(chunk_coords)

Expand All @@ -1590,7 +1596,7 @@ def _chunk_getitem(self, chunk_coords, chunk_selection, out, out_selection,

else:

if (isinstance(out, np.ndarray) and
if (out_is_ndarray and
not fields and
is_contiguous_selection(out_selection) and
is_total_slice(chunk_selection, self._chunks) and
Expand Down Expand Up @@ -1769,7 +1775,7 @@ def _encode_chunk(self, chunk):
chunk = f.encode(chunk)

# check object encoding
if isinstance(chunk, np.ndarray) and chunk.dtype == object:
if ensure_ndarray(chunk).dtype == object:
raise RuntimeError('cannot write object array without object codec')

# compress
Expand Down

0 comments on commit 58b1786

Please sign in to comment.