Disable Fletcher32 check before reading

[StewartHolmesVG]

I have a file which has known errors in the fletcher32 checksum. When I try to read these chunks in h5py, an exception is raised. What if I want to force h5py to read these anyway?

I can see there is a function to do just this, H5Pset_edc_check (https://docs.hdfgroup.org/hdf5/develop/group___d_x_p_l.html#ga0d95dfa506784acc9aed850c99713609)... but I can't work out how to call this via h5py

[takluyver]

It looks like H5Pset_edc_check is not currently exposed via h5py. You could try adding it, or you might be able to use dset.id.read_direct_chunk() to get the chunk including the checksum, and then trim it off.

[StewartHolmesVG]

Thanks for the quick answer! Unfortunately read_direct_chunk() would be difficult as I am using compression, so I would have to manually uncompress the data.

How would I go about exposing H5Pset_edc_check?

[takluyver]

We've actually already got a generated wrapper function in Cython for it:

h5py/h5py/api_functions.txt

Line 369 in bb38017

herr_t H5Pset_edc_check(hid_t plist, H5Z_EDC_t check)

So to expose it to use from Python should just be a question of adding it (and the corresponding get_edc_check) as methods to the class for data transfer property lists:

h5py/h5py/h5p.pyx

Lines 1982 to 2003 in bb38017

	cdef class PropDXID(PropInstanceID):
	""" Data transfer property list """
	IF MPI:
	def set_dxpl_mpio(self, int xfer_mode):
	""" Set the transfer mode for MPI I/O.
	Must be one of:
	- h5fd.MPIO_INDEPENDENT (default)
	- h5fd.MPIO_COLLECTIVE
	"""
	H5Pset_dxpl_mpio(self.id, <H5FD_mpio_xfer_t>xfer_mode)
	def get_dxpl_mpio(self):
	""" Get the current transfer mode for MPI I/O.
	Will be one of:
	- h5fd.MPIO_INDEPENDENT (default)
	- h5fd.MPIO_COLLECTIVE
	"""
	cdef H5FD_mpio_xfer_t mode
	H5Pget_dxpl_mpio(self.id, &mode)
	return <int>mode

Then you'd need to use the low-level dset.id.read() method to allow passing a dxpl (Data Transfer Property List) in.