-
Notifications
You must be signed in to change notification settings - Fork 33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix HdfStorage.read() implementation to work with all column types #288
Conversation
Conflicting PR. Removed from build OMERO-python-superbuild-push#707. See the console output for more details.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One minor item I noticed which we might want to just consider later, otherwise 👍
|
||
def _getrows(self, start, stop): | ||
return self.__mea.read(start, stop) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only thing I'd have to check is if there are any situations where the single read is significantly faster, though I assume it's a time v. space issue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I certainly see how the performance can vary depending of the ratio percentage_of_read_columns/percentage_of_read_rows.
I would assume readCoordinates
will suffer from the same tradeoffs as it it also looping over the individual columns to extract records and combine them into a grid.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Objections to capture this as an issue and merging?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
WFM
Reported by @jburel, the tables.read() method internally uses
col.values
(via_getrows
). For any column type with overrides the default methods ofAbstractColumn
likeMaskColumnI
, this will fail.fe8b863 tried to address the problem by using the
getsize()
API which should be independent of the column implementation details. However,testMaskColumn
is still failing as the logic does not populate the.bytes
attributes.6c27eae is a more robust fix to the underlying issue which uses the fact that the column API effectively has a
read
method and effectively delegates to this API likereadCoordinates
.All the other commits expand the unit tests to cover both
readCoordinates
andread
scenarios for the different column types (long, string, mask) as well as a test for various arguments oftable.read
Proposing for a patch release of
omero-py