Shared HDF5 Utilities#
hdf5_util is the cross-language HDF5 layer used by the current PySCF,
DIAGPT, and HEFFSO contracts. It contains a general OpenMolcas-derived wrapper
and MPPT-specific validated readers.
Components#
Source |
Responsibility |
|---|---|
|
General Fortran API for HDF5 files, datasets, and attributes |
|
C bindings used by the |
|
Typed MPPT dataset/attribute readers and shape validation |
|
Versioned DIAGPT-to-HEFFSO names and conventions |
|
Cross-language FNV-1a source and handoff determinant hashes |
|
Shared integer and real kind definitions |
The extracted OpenMolcas support headers and local stubs keep the wrapper self-contained. New MPPT product contracts should use explicit dataset names, schema/version attributes, dimensions, and indexing conventions rather than adding implicit prefix files.
The MPPT reader checks scalar attribute rank, integer storage class, bounded fixed- and variable-length string storage, rank-one array extent, and dataset rank before copying data. Model-space and handoff integer datasets must use signed int64 storage and match their expected extent before allocation. DIAGPT and HEFFSO use these checked entry points; callers must still validate domain limits such as orbital and configuration counts before allocating schema-sized arrays. HEFFSO matrix staging likewise requires floating-point storage and checks the triangular packed-matrix extent before allocating or fetching it.
Array-size queries, empty-array checks and rank-four flattened views in the
MPPT reader use int64 extents. The existing _I8_ build of hdf5_util keeps
dataset dimensions on the 64-bit path; array element types and external ABI
kinds are unchanged. Consumers remain responsible for memory availability.
The workflow normalizes the pinned CIPSIXX producer’s one-element integer model-space attributes to scalar rank before these readers consume them. This adapter is limited to extent-one integers; missing, multi-value, and wrong-typed attributes still fail the normal contract checks.
Prefix-owned HEFFSO outputs are checked against the shared HDF5 input by device and inode before stale files are removed. This rejects direct paths, symbolic links, and hard links to the input. The Fortran-to-C filename bridge keeps external filename lengths and inquiry flags at their default ABI kinds even when MPPT arrays use 64-bit integers; repeated alias checks are covered by the HEFFSO report cleanup test.
Cross-Language Array Layout#
HDF5 dimensions are C-ordered on disk, while Fortran readers expose their natural reversed dimension order. Every multidimensional contract must state both forms and include an asymmetric-axis test. A shape-only test with equal dimensions cannot detect an accidental transpose.
The DIAGPT-to-HEFFSO handoff uses schema mppt_heffso_handoff_v3. It stores
metat with the orbital/model-space attributes, determinant arrays det_ne,
det_nd, det_trou, and det_part, and the pair-packed h1_packed and
h2_packed matrices. Its model_space_source attribute must be
hdf5_quasi_restricted, and model_space_content_hash_fnv1a64 binds the
handoff to the shared /mppt/model_space/quasi_restricted source. Every handoff requires
determinant_content_hash_fnv1a64 and scalar integer selected_remap. A value
of 1 requires a normalized one-based signed-int64 selected_idx dataset,
while 0 requires that dataset to be absent. HEFFSO recomputes the source hash
and requires the computed, stored, and handoff model-space hashes to agree. It
also recomputes the determinant-content hash from signed-int64 little-endian
norb, noca, metat, ncf, selected_remap, then the size-prefixed
selected_idx, det_ne, det_nd, det_trou, and det_part vectors. This
second digest binds the exact post-remap, post-order catalog consumed with the
pair-packed matrices. Python preflight applies the same serialization and the
native checks for contiguous offsets, zero-rank placeholders, spin-orbital
ranges, and the final sentinel before hashing. On the selected route, native
provenance requires the handoff reference occupation to match the full-space
root occupation and requires selected occupation plus frozen-core count to
reproduce that value.
The selected occupation is also bound to the hashed model-space noca.
The strict mppt_heffso_properties_v4 artifact is an asymmetric-axis example:
each split-real/imaginary operator dataset is (3,n,n) on disk and (n,n,3)
in the Fortran writer. Its root stores state-indexed line strengths,
L2/S2/J2/Jz, effective L/S/J, Hermiticity residuals,
angular-commutator residuals, and scalar J = L + S and oscillator-strength-formula
residuals. The required /solver group stores durable PRIMME evidence, while
/determinant_identity stores final eigenvector-row occupations in the minimal
vacuum-relative orbital window. Python and Caspy readers require all three
surfaces to be complete, validate dimensions and ordering, recompute derived
matrix values and the determinant hash.
Build And Test#
Complete the environment-manager setup and root
MPPT installation first. The commands below assume that
the version-pinned mppt environment is active and its GNU compiler exports
are still set.
hdf5_util and HDF5 are mandatory in the root CMake project:
The command below reconfigures the existing canonical build/ tree. For a new
tree, use the complete configuration in the installation guide.
cmake -S . -B build \
-DCMAKE_PREFIX_PATH="$CONDA_PREFIX" \
-DHDF5_ROOT="$CONDA_PREFIX"
cmake --build build --parallel 1
Run the focused HDF5 and handoff tests through the root test graph:
ctest --test-dir build -L DIAGPT --output-on-failure
ctest --test-dir build -R 'hdf5|handoff' --output-on-failure
For user-visible datasets, update the owning workflow page and output contract in the same change.