Shared HDF5 Utilities#

hdf5_util is the cross-language HDF5 layer used by the current PySCF, DIAGPT, and HEFFSO contracts. It contains a general OpenMolcas-derived wrapper and MPPT-specific validated readers.

Components#

Source	Responsibility
`hdf5_util/mh5.F90`	General Fortran API for HDF5 files, datasets, and attributes
`hdf5_util/mh5_capi.c`	C bindings used by the `mh5` Fortran API
`hdf5_util/mppt_hdf5_reader.F90`	Typed MPPT dataset/attribute readers and shape validation
`hdf5_util/mppt_heffso_handoff_schema.F90`	Versioned DIAGPT-to-HEFFSO names and conventions
`hdf5_util/mppt_model_space_hash.F90`	Cross-language FNV-1a source and handoff determinant hashes
`hdf5_util/Definitions.F90`	Shared integer and real kind definitions

The extracted OpenMolcas support headers and local stubs keep the wrapper self-contained. New MPPT product contracts should use explicit dataset names, schema/version attributes, dimensions, and indexing conventions rather than adding implicit prefix files.

The MPPT reader checks scalar attribute rank, integer storage class, bounded fixed- and variable-length string storage, rank-one array extent, and dataset rank before copying data. Model-space and handoff integer datasets must use signed int64 storage and match their expected extent before allocation. DIAGPT and HEFFSO use these checked entry points; callers must still validate domain limits such as orbital and configuration counts before allocating schema-sized arrays. HEFFSO matrix staging likewise requires floating-point storage and checks the triangular packed-matrix extent before allocating or fetching it.

The workflow normalizes the pinned CIPSIXX producer’s one-element integer model-space attributes to scalar rank before these readers consume them. This adapter is limited to extent-one integers; missing, multi-value, and wrong-typed attributes still fail the normal contract checks.

Cross-Language Array Layout#

HDF5 dimensions are C-ordered on disk, while Fortran readers expose their natural reversed dimension order. Every multidimensional contract must state both forms and include an asymmetric-axis test. A shape-only test with equal dimensions cannot detect an accidental transpose.

The DIAGPT-to-HEFFSO handoff uses schema mppt_heffso_handoff_v3. It stores metat with the orbital/model-space attributes, determinant arrays det_ne, det_nd, det_trou, and det_part, and the pair-packed h1_packed and h2_packed matrices. Its model_space_source attribute must be hdf5_quasi_restricted, and model_space_content_hash_fnv1a64 binds the handoff to the shared /mppt/model_space/quasi_restricted source. Every handoff requires determinant_content_hash_fnv1a64 and scalar integer selected_remap. A value of 1 requires a normalized one-based signed-int64 selected_idx dataset, while 0 requires that dataset to be absent. HEFFSO recomputes the source hash and requires the computed, stored, and handoff model-space hashes to agree. It also recomputes the determinant-content hash from signed-int64 little-endian norb, noca, metat, ncf, selected_remap, then the size-prefixed selected_idx, det_ne, det_nd, det_trou, and det_part vectors. This second digest binds the exact post-remap, post-order catalog consumed with the pair-packed matrices. Python preflight applies the same serialization and the native checks for contiguous offsets, zero-rank placeholders, spin-orbital ranges, and the final sentinel before hashing. Scalar root vectors and energies are not handoff datasets. On the selected route, native provenance requires the handoff reference occupation to match the full-space root occupation and requires selected occupation plus frozen-core count to reproduce that value. The selected occupation is also bound to the hashed model-space noca.

The HEFFSO effective-operator artifact is an asymmetric-axis example: each split-real/imaginary operator dataset is (3,n,n) on disk and (n,n,3) in the Fortran writer. Readers validate schema names, dimensions, finiteness, ordering, and the required matrix relations before scientific use.

Build And Test#

hdf5_util and HDF5 are mandatory in the root CMake project:

cmake -S . -B build \
  -DCMAKE_PREFIX_PATH="$CONDA_PREFIX" \
  -DHDF5_ROOT="$CONDA_PREFIX"
cmake --build build --parallel 1

Run the focused HDF5 and handoff tests through the root test graph rather than using a standalone stale test directory:

ctest --test-dir build -L DIAGPT --output-on-failure
ctest --test-dir build -R 'hdf5|handoff' --output-on-failure

For user-visible datasets, update the owning workflow page and output contract in the same change.