forked from PyTables/PyTables
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathRELEASE_NOTES.txt
345 lines (279 loc) · 14.2 KB
/
RELEASE_NOTES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
=======================================
Release notes for PyTables 3.1 series
=======================================
:Author: PyTables Developers
:Contact: pytables@googlemail.com
.. py:currentmodule:: tables
Changes from 3.1.1 to 3.1.2
===========================
Improvements
------------
- :class:`link.SoftLink` objects no longer need to be explicitly dereferenced.
Methods and attributes of the linked object are now automatically accessed
when the user acts on a soft-link (see :issue:`399`).
Thanks to Alistair Muldal.
- Now :program:`ptrepack` recognizes hardlinks and replicates them in the
output (*repacked*) file. This saves disk space and makes repacked files
more conformal to the original one. Closes :issue:`380`.
- New :program:`pttree` script for printing HDF5 file contents as a pretty
ASCII tree (closes :issue:`400`). Thanks to Alistair Muldal.
- The internal Blosc_ library has been updated to version 1.5.2.
In order to fix a threading issue in multiprocess scenarios (see
:issue:`411` and :issue:`412`) now the Blosc_ HDF5 filter only uses one
thread. This limitation will be removed as soon as possible.
The new version of Blosc_ also includes a couple of security fixes and
support for architectures requiring strict access alignment.
- The :func:`print_versions` function now also reports the version of
compression libraries used by blosc
- Now the :file:`setup.py` tries to use the '-march=native' C flag by
default. In falls back on '-msse2' if '-march=native' is not supported
by the compiler. Closes :issue:`379`.
- Fixed a spurious unicode comparison warning (closes :issue:`372` and
:issue:`373`).
- Improved handling of empty string attributes. In previous versions of
PyTables empty string were stored as scalar HDF5 attributes having size 1
and value '\0' (an empty null terminated string).
Now empty string are stored as HDF5 attributes having zero size
- Added a new cookbook recipe and a couple of examples for simple threading
with PyTables.
- The redundant :func:`utilsextension.get_indices` function has been
eliminated (replaced by :meth:`slice.indices`). Closes :issue:`195`.
- Allow negative indices in point selection (closes :issue:`360`)
- Index wasn't being used if it claimed there were no results.
Closes :issue:`351` (see also :issue:`353`)
- Atoms and Col types are no longer generated dynamically so now it is easier
for IDEs and static analysis tool to handle them (closes :issue:`345`)
- Small unit tests re-factoring:
* :func:`print_versions` and :func:`tests.common.print_heavy` functions
moved to the :mod:`tests.common` module
* always use :func:`print_versions` when test modules are called as scripts
* use the unittest2_ package in Python 2.6.x
* removed internal machinery used to replicate unittest2_ features
* always use :class:`tests.common.PyTablesTestCase` as base class for all
test cases
* code of the old :func:`tasts.common.cleanup` function has been moved to
:meth:`tests.common.PyTablesTestCase.tearDown` method
* new implementation of :meth:`tests.common.PyTablesTestCase.assertWarns`
compatible with the one provided by the standard :mod:`unittest` module
in Python >= 3.2
* use :meth:`tests.common.PyTablesTestCase.assertWarns` as context manager
when appropriate
* use the :func:`unittest.skipIf` decorator when appropriate
* new :class:tests.comon.TestFileMixin: class
.. _unittest2: https://pypi.python.org/pypi/unittest2
Bugs fixed
----------
- Fixed compatibility problems with numpy 1.9 and 1.10-dev
(closes :issue:`362` and :issue:`366`)
- Fixed compatibility with Cython_ >= 0.20 (closes :issue:`386` and
:issue:`387`)
- Fixed support for unicode node names in LRU cache (only Python 2 was
affected). Closes :issue:`367` and :issue:`369`.
- Fixed support for unicode node titles (only Python 2 was affected).
Closes :issue:`370` and :issue:`374`.
- Fixed a bug that caused the silent truncation of unicode attributes
containing the '\0' character. Closes :issue:`371`.
- Fixed :func:`descr_from_dtype` to work as expected with complex types.
Closes :issue:`381`.
- Fixed the :class:`tests.test_basics.ThreadingTestCase` test case.
Closes :issue:`359`.
- Fix incomplete results when performing the same query twice and exhausting
the second iterator before the first. The first one writes incomplete
results to *seqcache* (:issue:`353`)
- Fix false results potentially going to *seqcache* if
:meth:`tableextension.Row.update` is used during iteration
(see :issue:`353`)
- Fix :meth:`Column.create_csindex` when there's NaNs
- Fixed handling of unicode file names on windows (closes :issue:`389`)
- No longer not modify :data:`sys.argv` at import time (closes :issue:`405`)
- Fix a performance issue on NFS (closes :issue:`402`)
Other changes
-------------
- The minimum Cython_ version is now 0.14.
.. _Cython: http://cython.org
Changes from 3.1.0 to 3.1.1
===========================
Bugs fixed
----------
- Fixed a critical bug that caused an exception at import time.
The error was triggered when a bug in long-double detection is detected
in the HDF5 library (see :issue:`275`) and numpy_ does not expose
`float96` or `float128`. Closes :issue:`344`.
- The internal Blosc_ library has been updated to version 1.3.5.
This fixes a false buffer overrun condition that made c-blosc to fail,
even if the problem was not real.
Improvements
------------
- Do not create a temporary array when the *obj* parameter is not specified
in :meth:`File.create_array` (thanks to Francesc).
Closes :issue:`337` and :issue:`339`).
- Added two new utility functions
(:func:`tables.nodes.filenode.read_from_filenode` and
:func:`tables.nodes.filenode.save_to_filenode`) for the direct copy from
filesystem to filenode and vice versa (closes :issue:`342`).
Thanks to Andreas Hilboll.
- Removed the :file:`examples/nested-iter.py` considered no longer useful.
Closes :issue:`343`.
- Better detection of the `-msse2` compiler flag.
Changes from 3.0 to 3.1.0
=========================
New features
------------
- Now PyTables is able to save/restore the default value of :class:`EnumAtom`
types (closes :issue:`234`).
- Implemented support for the H5FD_SPLIT driver (closes :issue:`288`,
:issue:`289` and :issue:`295`). Many thanks to simleo.
- New quantization filter: the filter truncates floating point data to a
specified precision before writing to disk. This can significantly improve
the performance of compressors (closes :issue:`261`).
Thanks to Andreas Hilboll.
- Added new :meth:`VLArray.get_row_size` method to :class:`VLArray` for
querying the number of atoms of a :class:`VLArray` row.
Closes :issue:`24` and :issue:`315`.
- The internal Blosc_ library has been updated to version 1.3.2.
All new features introduced in the Blosc_ 1.3.x series, and in particular
the ability to leverage different compressors within Blosc_ (see the `Blosc
Release Notes`_), are now available in PyTables via the blosc filter
(closes: :issue:`324`). A big thank you to Francesc.
Improvements
------------
- The node caching mechanism has been completely redesigned to be simpler and
less dependent from specific behaviours of the ``__del__`` method.
Now PyTables is compatible with the forthcoming Python 3.4.
Closes :issue:`306`.
- PyTables no longer uses shared/cached file handlers. This change somewhat
improves support for concurrent reading allowing the user to safely open the
same file in different threads for reading (requires HDF5 >= 1.8.7).
More details about this change can be found in the `Backward incompatible
changes`_ section.
See also :issue:`130`, :issue:`129` :issue:`292` and :issue:`216`.
- PyTables is now able to detect and use external installations of the Blosc_
library (closes :issue:`104`). If Blosc_ is not found in the system, and the
user do not specify a custom installation directory, then it is used an internal
copy of the Blosc_ source code.
- Automatically disable extended float support if a buggy version of HDF5
is detected (see also `Issues with H5T_NATIVE_LDOUBLE`_).
See also :issue:`275`, :issue:`290` and :issue:`300`.
- Documented an unexpected behaviour with string literals in query conditions
on Python 3 (closes :issue:`265`)
- The deprecated :mod:`getopt` module has been dropped in favour of
:mod:`argparse` in all command line utilities (close :issue:`251`)
- Improved the installation section of the :doc:`../usersguide/index`.
* instructions for installing PyTables via pip_ have been added.
* added a reference to the Anaconda_, Canopy_ and `Christoph Gohlke suites`_
(closes :issue:`291`)
- Enabled `Travis-CI`_ builds for Python_ 3.3
- :meth:`Tables.read_coordinates` now also works with boolean indices input.
Closes :issue:`287` and :issue:`298`.
- Improved compatibility with numpy_ >= 1.8 (see :issue:`259`)
- The code of the benchmark programs (bench directory) has been updated.
Closes :issue:`114`.
- Fixed some warning related to non-unicode file names (the Windows bytes API
has been deprecated in Python 3.4)
Bugs fixed
----------
- Fixed detection of platforms supporting Blosc_
- Fixed a crash that occurred when one attempts to write a numpy_ array to
an :class:`Atom` (closes :issue:`209` and :issue:`296`)
- Prevent creation of a table with no columns (closes :issue:`18` and
:issue:`299`)
- Fixed a memory leak that occured when iterating over
:class:`CArray`/:class:`EArray` objects (closes :issue:`308`,
see also :issue:`309`).
Many thanks to Alistair Muldal.
- Make NaN types sort to the end. Closes :issue:`282` and :issue:`313`
- Fixed selection on float columns when NaNs are present (closes :issue:`327`
and :issue:`330`)
- Fix computation of the buffer size for iterations on rows.
The buffers size was overestimated resulting in a :exc:`MemoryError`
in some cases.
Closes :issue:`316`. Thamks to bbudescu.
- Better check of file open mode. Closes :issue:`318`.
- The Blosc filter now works correctly together with fletcher32.
Closes :issue:`21`.
- Close the file handle before trying to delete the corresponding file.
Fixes a test failure on Windows.
- Use integer division for computing indices (fixes some warning on Windows)
Deprecations
------------
Following the plan for the complete transition to the new (PEP8_ compliant)
API, all calls to the old API will raise a :exc:`DeprecationWarning`.
The new API has been introduced in PyTables 3.0 and is backward incompatible.
In order to guarantee a smoother transition the old API is still usable even
if it is now deprecated.
The plan for the complete transition to the new API is outlined in
:issue:`224`.
Backward incompatible changes
-----------------------------
In PyTables <= 3.0 file handles (objects that are returned by the
:func:`open_file` function) were stored in an internal registry and re-used
when possible.
Two subsequent attempts to open the same file (with compatible open mode)
returned the same file handle in PyTables <= 3.0::
In [1]: import tables
In [2]: print(tables.__version__)
3.0.0
In [3]: a = tables.open_file('test.h5', 'a')
In [4]: b = tables.open_file('test.h5', 'a')
In [5]: a is b
Out[5]: True
All this is an implementation detail, it happened under the hood and the user
had no control over the process.
This kind of behaviour was considered a feature since it can speed up opening
of files in case of repeated opens and it also avoids any potential problem
related to multiple opens, a practice that the HDF5 developers recommend to
avoid (see also H5Fopen_ reference page).
The trick, of course, is that files are not opened multiple times at HDF5
level, rather an open file is referenced several times.
The big drawback of this approach is that there are really few chances to use
PyTables safely in a multi thread program. Several bug reports have been
filed regarding this topic.
After long discussions about the possibility to actually achieve concurrent I/O
and about patterns that should be used for the I/O in concurrent programs
PyTables developers decided to remove the *black magic under the hood* and
allow the users to implement the patterns they want.
Starting from PyTables 3.1 file handles are no more re-used (*shared*) and
each call to the :func:`open_file` function returns a new file handle::
In [1]: import tables
In [2]: print tables.__version__
3.1.0
In [3]: a = tables.open_file('test.h5', 'a')
In [4]: b = tables.open_file('test.h5', 'a')
In [5]: a is b
Out[5]: False
It is important to stress that the new implementation still has an internal
registry (implementation detail) and it is still **not thread safe**.
Just now a smart enough developer should be able to use PyTables in a
muti-thread program without too much headaches.
The new implementation behaves differently from the previous one, although the
API has not been changed. Now users should pay more attention when they open a
file multiple times (as recommended in the `HDF5 reference`__ ) and they
should take care of using them in an appropriate way.
__ H5Fopen_
Please note that the :attr:`File.open_count` property was originally intended
to keep track of the number of references to the same file handle.
In PyTables >= 3.1, despite of the name, it maintains the same semantics, just
now its value should never be higher that 1.
.. note::
HDF5 versions lower than 1.8.7 are not fully compatible with PyTables 3.1.
A partial support to HDF5 < 1.8.7 is still provided but in that case
multiple file opens are not allowed at all (even in read-only mode).
.. _pip: http://www.pip-installer.org
.. _Anaconda: https://store.continuum.io/cshop/anaconda
.. _Canopy: https://www.enthought.com/products/canopy
.. _`Christoph Gohlke suites`: http://www.lfd.uci.edu/~gohlke/pythonlibs
.. _`Issues with H5T_NATIVE_LDOUBLE`: http://hdf-forum.184993.n3.nabble.com/Issues-with-H5T-NATIVE-LDOUBLE-tt4026450.html
.. _Python: http://www.python.org
.. _Blosc: http://www.blosc.org
.. _numpy: http://www.numpy.org
.. _`Travis-CI`: https://travis-ci.org
.. _PEP8: http://www.python.org/dev/peps/pep-0008
.. _`Blosc Release Notes`: https://github.com/FrancescAlted/blosc/wiki/Release-notes
.. _H5Fopen: http://www.hdfgroup.org/HDF5/doc/RM/RM_H5F.html#File-Open
**Enjoy data!**
-- The PyTables Developers
.. Local Variables:
.. mode: rst
.. coding: utf-8
.. fill-column: 72
.. End: