What's new in PyTables 0.9
==========================

On this release you will find a series of quite
exciting new features, being the most important the indexing
capabilities, in-kernel selections, support for complex datatypes and
the possibility to modify values in both tables *and* arrays (yeah,
finally :).

New features:
-------------

- Indexing of columns in tables. That allow to make data selections on
  tables up to 500 times faster than standard selections (for
  ex. doing a selection along an indexed column of 100 million of rows
  takes less than 1 second on a modern CPU). Perhaps the most
  interesting thing about the indexing algorithm implemented by
  PyTables is that the time taken to index grows *linearly* with the
  length of the data, so, making the indexation process to be
  *scalable* (quite differently to many relational databases). This
  means that it can index, in a relatively quick way, arbitrarily
  large table columns (for ex. indexing a column of 100 million of rows
  takes just 100 seconds, i.e. at a rate of 1 Mrow/sec). See more
  detailed info about that in http://www.pytables.org/docs/SciPy04.pdf.

- In-kernel selections. This feature allow to make data selections on
  tables up to 5 times faster than standard selections (i.e. pre-0.9
  selections), without a need to create an index. As a hint of how
  fast these selections can be, they are up to 10 times faster than a
  traditional relational database. Again, see
  http://www.pytables.org/docs/SciPy04.pdf for some experiments on that
  matter.

- Support of complex datatypes for all the data objects (i.e. Table,
  Array, EArray and VLArray). With that, the complete set of datatypes
  of Numeric and numarray packages are supported. Thanks to Tom Hedley
  for providing the patches for Array, EArray and VLArray objects, as
  well as updating the User's Manual and adding unit tests for the new
  functionality.

- Modification of values. You can modify Table, Array, EArray and
  VLArray values. See Table.modifyRows, Table.modifyColumns() and the
  newly introduced __setitem__() method for Table, Array, EArray and
  VLArray entities in the Library Reference of User's Manual.

- A new sub-package called "nodes" is there. On it, there will be
  included different modules to make more easy working with different
  entities (like images, files, ...). The first module that has been
  added to this sub-package is "FileNode", whose mission is to enable
  the creation of a database of nodes which can be used like regular
  opened files in Python.  In other words, you can store a set of
  files in a PyTables database, and read and write it as you would do
  with any other file in Python. Thanks to Ivan Vilata i Balaguer for
  contributing this.

Improvements:
-------------

- New __len__(self) methods added in Arrays, Tables and Columns. This,
  in combination with __getitem__(self,key) allows to better emulate
  sequences.

- Better capabilities to import generic HDF5 files. In particular,
  Table objects (in the HDF5_HL naming schema) with "holes" in their
  compound type definition are supported. That allows to read certain
  files produced by NASA (thanks to Stephen Walton for reporting this).

- Much improved test units. More than 2000 different tests has been
  implemented which accounts for more than 13000 loc (this represents
  twice of the PyTables library code itself (!)).

Backward-incompatible API changes:
----------------------------------

- The __call__ special method has been removed from objects File,
  Group, Table, Array, EArray and VLArray. Now, you should use
  walkNodes() in File and Group and iterrows in Table, Array, EArray
  and VLArray to get the same functionality. This would provide better
  compatibility with IPython as well.

'nctoh5', a new importing utility:

- Jeff Whitaker has contributed a script to easily convert NetCDF
  files into HDF5 files using Scientific Python and PyTables. It has
  been included and documented as a new utility.

Bug fixes:
----------

- A call to File.flush() now invoke a call to H5Fflush() so to
  effectively flushing all the file contents to disk. Thanks to Shack
  Toms for reporting this and providing a patch.

- SF #1054683: Security hole in utils.checkNameValidity(). Reported in
  2004-10-26 by ivilata

- SF #1049297: Suggestion: new method File.delAttrNode(). Reported in
  2004-10-18 by ivilata

- SF #1049285: Leak in AttributeSet.__delattr__(). Reported in
  2004-10-18 by ivilata

- SF #1014298: Wrong method call in examples/tutorial1-2.py. Reported
  in 2004-08-23 by ivilata

- SF #1013202: Cryptic error appending to EArray on RO file. Reported
  in 2004-08-21 by ivilata

- SF #991715: Table.read(field="var1", flavor="List") fails. Reported
  in 2004-07-15 by falted

- SF #988547: Wrong file type assumption in File.__new__. Reported in
  2004-07-10 by ivilata


Bon profit!,

-- Francesc Altet
falted@pytables.org

