h5tools Package

h5tools Package

Created on Sun Aug 18 22:31:59 2013

@author: thomas

This file only serves to signal that the content of the folder is a Python package.

h52np Module

Created on Fri Oct 25 16:30:23 2013

@author: Thomas Schatz

class ABXpy.h5tools.h52np.H52NP(h5file)

Bases: object

add_dataset(group, dataset, buf_size=100, minimum_occupied_portion=0.25)

class ABXpy.h5tools.h52np.H52NPbuffer(parent, group, dataset, buf_size, minimum_occupied_portion)

Bases: object

buffer_empty()

current_tail()

dataset_empty()

isempty()

nb_lower_than(x)

read(amount=None)

refill_buffer()

h5_handler Module

Created on Tue Oct 15 09:48:31 2013

@author: Thomas Schatz

class ABXpy.h5tools.h5_handler.H5Handler(h5file, keygroup, keyset, groups=None, datasets=None)

Bases: object

extract_chunk(i_start, i_end, chunk_id)

sort(buffer_size=1000, o_buffer_size=1000)

class ABXpy.h5tools.h5_handler.H5TMP

Bases: object

ABXpy.h5tools.h5_handler.test()

h5io Module

Created on Sun Jan 19 17:06:15 2014

@author: Thomas Schatz

class ABXpy.h5tools.h5io.H5IO(filename, datasets=None, indexes=None, fused=None, group='/')

Bases: object

find()

read()

sort()

write(data, append=True, iterate=False, indexed=False)

ABXpy.h5tools.h5io.get_dtype(data)

ABXpy.h5tools.h5io.test_h5io()

np2h5 Module

Created on Thu Sep 19 13:46:18 2013

@author: Thomas Schatz

Class for efficiently writing to disk (in a specified dataset of a HDF5 file) simple two-dimensional numpy arrays that are incrementally generated along the first dimension. It uses buffers to avoid small I/O.

It needs to be used within a ‘with’ statement, so as to handle buffer flushing and opening and closing of the underlying HDF5 file smoothly.

Buffer size should be chosen according to speed/memory trade-off. Due to cache issues there is probably an optimal size.

The size of the dataset to be written must be known in advance, excepted when overwriting an existing dataset. Not writing exactly the expected amount of data causes an Exception to be thrown excepted is the fixed_size option was set to False when adding the dataset.

class ABXpy.h5tools.np2h5.NP2H5(h5file)

Bases: object

add_dataset(group, dataset, n_rows=0, n_columns=None, chunk_size=100, buf_size=100, item_type=<Mock id='140696957626192'>, overwrite=False, fixed_size=True)

class ABXpy.h5tools.np2h5.NP2H5buffer(parent, group, dataset, n_rows, n_columns, chunk_size, buf_size, item_type, overwrite, fixed_size)

Bases: object

delete()

flush()

iscomplete()

write(data)

ABXpy.h5tools.np2h5.nb_lines(item_size, n_columns, size_in_mem)