Unified cipsixx Workflow#

The unified workflow runs determinant selection, scalar DIAGPT, and HEFFSO from one explicit PySCF/MPPT HDF5 dump:

PySCF -> HDF5 -> cipsixx_select -> DIAGPT -> HEFFSO

The HDF5 file is the source of truth for integrals, physical selected determinants, the downstream model-space adapter, and stage checkpoints. Generated Fortran input files, logs, the DIAGPT HEFFSO handoff, and the final HEFFSO eigenvectors remain ordinary files in the workflow directory.

Downstream stages are enabled individually, and this HDF5 pipeline is the only supported route from PySCF into DIAGPT and HEFFSO.

Build#

Clone a release snapshot, which includes the pinned cipsixx source:

git clone https://gitlab.com/seregmikhail/mppt-public.git mppt
cd mppt

Configure against the HDF5 and math libraries in the active mppt conda environment. Start with the portable BLAS/LAPACK profile; optional MKL and CUDA backends are documented separately.

conda activate mppt
cmake -S . -B build \
  -UBLAS_LIBRARIES \
  -ULAPACK_LIBRARIES \
  -DCMAKE_BUILD_TYPE=RelWithDebInfo \
  -DCMAKE_PREFIX_PATH="$CONDA_PREFIX" \
  -DHDF5_ROOT="$CONDA_PREFIX" \
  -DOpenMP_gomp_LIBRARY="$CONDA_PREFIX/lib/libgomp.so.1" \
  -DMPPT_ENABLE_CIPSIXX=ON \
  -DMPPT_COPY_TO_HOME_BIN=OFF \
  -DINSTALL_PYSCF2MPPT=OFF
cmake --build build --parallel 1
python -m pip install -e "./pyscf2mppt[test]"

The build places cipsixx_select, diagpt, and heffso under build/bin. The workflow also accepts explicit executable paths on the command line and the CIPSIXX_SELECT_EXECUTABLE, DIAGPT_EXECUTABLE, and HEFFSO_EXECUTABLE environment variables. See the installation guide for test profiles and all CMake options.

Workflow Configuration#

The following is a complete downstream configuration. Orbital lists in YAML are 1-based. Set only the stages that should run to enabled: true.

mppt:
  prefix: Pr
  output_hdf5: calculation.h5

  cipsixx:
    enabled: true
    initial_space: cas              # hf | cas | determinants
    selection_method: mp2_coefficient
    cas_orbitals: [5, 6, 7, 8, 9, 10, 11, 12]
    core_orbitals: [1, 2, 3, 4]
    frozen_orbitals: []
    cas_alpha_electrons: 1
    cas_beta_electrons: 1
    max_selected: 4000
    batch_size: 250
    max_iterations: 20
    max_candidates: 200000
    min_abs_coefficient: 1.0e-7
    min_abs_energy: 0.0
    denominator_tolerance: 1.0e-12
    n_roots: 144
    selected_root: 0
    multi_state: true
    selection_growth: topk         # topk | threshold
    selection_metric: coefficient  # coefficient | energy
    state_weighting: legacy-taum
    root_weights: null              # exactly n_roots values for custom
    spin_adaptation_closure: spin-projection
    final_diagonalization: always
    adaptive_threshold: true
    threshold_multiplier: 0.75
    threshold_floor: 1.0e-7
    min_new_determinants: 100
    omp_num_threads: 8
    iteration_report: cipsixx_iterations.csv
    root_energy_report: cipsixx_root_energies.csv

  diagpt:
    enabled: true
    title: Pr scalar states
    thrpri: 0.25
    wmupa: 1.0
    gap: 0.05
    zshift: false
    write_h0: true
    state_weights: null             # exactly one value per model-space state
    integral_space: auto           # auto | full | selected
    omp_num_threads: 8

  heffso:
    enabled: true
    effective_operators: true
    title: Pr spin-orbit states
    nvectw: 8
    guess_space: null               # null selects min(n, max(512, 2*nev))
    max_basis: null                 # null selects min(n, max(nev+64, 4*nev))
    max_block: null                 # null uses up to 8 OpenMP threads
    max_matvecs: null               # null selects max(10000, 400*nev)
    residual_tolerance: 1.0e-6
    initial_vector_tolerance: 1.0e-7
    iorder: 2
    print_level: 1                 # 0 summary, 1 compact, 2 per-root, 3+ debug
    progress_interval_seconds: 60
    allow_constrained_basis: false
    preconditioner: true
    direct_sparse: true
    sparse_backend: auto           # auto | fortran | cuda
    omp_num_threads: 8

The workflow supports one C1 scalar handoff. The historical .1 filename suffix is fixed for compatibility and is not a configurable pass number. HEFFSO rejects more than 1000 requested states and never silently reduces nvectw. Explicit bases below 2*nvectw require allow_constrained_basis: true; values below the recommended 4*nvectw remain visible as performance warnings. When effective_operators: true, the PySCF dump also writes the full-space dipole and orbital-angular-momentum tensors consumed by HEFFSO, and a valid <prefix>heffso_effective_operators.h5 becomes part of the restart contract. It contains bare catalog-projected L, S, J = L + S, and dipole matrices in the HEFFSO eigenstate basis, not dressed operators or certified spectroscopy. The default final_diagonalization: always makes final variational energies, coefficients, and weights authoritative. Use never only for a deliberate handoff/performance experiment. Custom cipsixx state weighting requires multi_state: true and exactly one nonnegative root_weights value per root, with at least one positive value. An explicit DIAGPT state_weights list follows the same nonnegative/one-positive rule and must contain one value per model-space state at execution. DIAGPT always uses the mandatory repository-pinned PRIMME eigensolver. The retired mppt.diagpt.solver field is rejected by the strict stage schema.

For manual iterative selection from an existing determinant list, set:

mppt:
  cipsixx:
    enabled: true
    initial_space: determinants
    determinants_group: /cipsixx/selected_determinants

Changing the effective selection configuration changes its fingerprint and runs selection again from the configured group.

Run A New Calculation#

A normal YAML containing molecule, basis, SCF/CASSCF, and the mppt section runs all enabled stages after the integral dump:

export PATH="$PWD/build/bin:$PATH"
pyscf2mppt input.yaml

For an isolated directory with copied file resources, orchestration manifests, locking, and managed resume, use:

pyscf2mppt run input.yaml --output-dir runs/pr

Cartesian parameter scans use pyscf2mppt sweep; see managed runs and sweeps for its separate sweep schema and current sequential-execution limitations.

For direct runs, mppt.output_hdf5 may be any path. No stage requires a file named pyscf_dump.h5, and no symlink is created. Managed runs instead require a portable relative path confined to the point directory and reject .., reserved names, and collisions with their current configured/reserved artifact set. This set always reserves <prefix>heffso_effective_operators.h5, so later enabling effective operators cannot collide with another configured output.

PySCF-stage checkpoints remain available independently:

pyscf2mppt input.yaml --checkpoint-dir ckpt --stop-after scf
pyscf2mppt input.yaml --resume-from ckpt --checkpoint-dir ckpt

Restart From An Existing Dump#

The workflow-only mode does not rerun PySCF or integral transformation. Its YAML may contain only the mppt section:

pyscf2mppt workflow.yaml \
  --from-hdf5 checkpoints/calculation.h5 \
  --selection-executable build/bin/cipsixx_select \
  --diagpt-executable build/bin/diagpt \
  --heffso-executable build/bin/heffso

The explicit --from-hdf5 path selects the shared file; mppt.output_hdf5 does not override it. Unlike the strict full-calculation parser, workflow-only parsing currently ignores top-level keys other than mppt and unknown keys directly under mppt. Nested cipsixx, diagpt, and heffso keys remain strict. Check spelling because an ignored typo can silently select a default. If all three stages remain disabled, the command is a successful no-op and may not open or validate the HDF5 file.

This also preserves the arbitrary-PySCF-object route:

from pyscf2mppt import dump_casscf, dump_scf

dump_scf(
    mf,
    output_path="checkpoints/scf.h5",
    cipsixx_config={"initial_space": "hf"},
)
dump_casscf(
    mc,
    output_path="checkpoints/casscf.h5",
    cipsixx_config={
        "initial_space": "cas",
        "cas_orbitals": [5, 6, 7, 8],
        "cas_electrons": 2,
    },
)

Run either dump later with --from-hdf5. Omitting output_path retains the historical pyscf_dump.h5 destination.

HDF5 Contracts#

The important groups have distinct responsibilities:

Path

Index convention

Responsibility

/cipsixx/integrals

0-based selected or full layout

Native one- and two-electron integrals used by cipsixx; /cipsixx@orbital_space identifies the route

/cipsixx/config

0-based after dump normalization

Reproducible selection metadata

/cipsixx/selected_determinants

0-based physical bitsets

Restartable determinant list, coefficients, weights, and variational results

/mppt/model_space/quasi_restricted

1-based spin orbitals

Validated DIAGPT/HEFFSO compatibility view

/mppt/workflow/stages/<stage>

n/a

Status, fingerprint, command, log path, and failure detail

The physical cipsixx space never contains a nonexistent orbital. For odd-electron downstream contracts, only /mppt/model_space/quasi_restricted adds the historical beta-like electron-at-infinity marker required by the DIAGPT and HEFFSO compatibility contracts. That adapter is round-trip validated against the physical bitsets. Records are ordered by excitation rank, net spin-free spatial occupation, spin-orbital holes, and spin-orbital particles. Opposite occupation changes on one spatial orbital cancel, and the infinity marker uses spatial index n_orb+1. This keeps every spin variant of one spatial configuration contiguous for HEFFSO spin-ladder construction. Workflow validation rejects duplicate, old, or externally reordered compatibility views; rerun selection to regenerate them from the unchanged physical determinant space.

With N_select > 0, native cipsixx integrals use the selected orbital space. With N_select <= 0, they use the complete post-trim space. HEFFSO preflight validates an external raw selected_idx as consistently zero-based or one-based when length, range, and uniqueness are valid. The stored attribute is not rewritten, but workflow identities hash its normalized canonical one-based values and DIAGPT uses the same normalization for determinant remapping. An explicit /cipsixx@orbital_space marker is authoritative, including a selected map whose cardinality equals the full orbital count. Older dumps without that marker are inferred from orbital counts: a full-count match takes precedence, otherwise a selected-count match chooses selected mode, and inconsistent counts are rejected.

DIAGPT receives the shared file through MPPT_HDF5_FILE, reads the mandatory /mppt/model_space/quasi_restricted group, and resolves integral_space: auto from the dump contract. All scalar and two-electron integrals come from that HDF5 file; no prefix-derived formatted integral source is probed. Selected mode requires the compact J/K data and four pair-packed ERI families listed in the selected-route contract. It does not consume or produce int2e_full; the full table remains part of only the non-selected integral route, where DIAGPT loads it into packed bijkl and accesses it through ai().

The pinned CIPSIXX revision emits integer model-space scalars as one-element HDF5 arrays. The workflow canonicalizes those attributes to rank-zero scalars immediately after selection and before checkpoint validation or DIAGPT launch; other non-scalar shapes remain invalid.

DIAGPT writes the fixed <prefix>_heffso.1.h5 handoff with schema mppt_heffso_handoff_v3. It contains the ordered determinant catalog, metat, and pair-packed h1/h2 matrices, but no scalar root vector or energy datasets. Every handoff has scalar integer selected_remap; selected-route handoffs set it to 1 and contain the normalized one-based signed-int64 selected_idx map, while unremapped handoffs set it to 0 and omit that dataset. HEFFSO recomputes the model-space FNV-1a hash from the shared source and requires it to match both the source attribute and handoff hash. It also recomputes determinant_content_hash_fnv1a64 from the exact ordered/remapped handoff catalog, validates the selected-orbital mapping when present, and checks the intso dataset before it starts. Python preflight applies the same handoff catalog digest before checkpoint reuse or HEFFSO launch.

Paths And Runtime Environment#

Fresh direct YAML runs use the process current directory as their default runtime workdir, not the YAML file’s directory. Relative basis/ECP/GRPP files, cipsixx reports, workflow HDF5 paths, and prefix-derived files therefore start there. Resume restores the checkpoint workdir for resources and workflow sidecars, but resolves the current YAML’s relative HDF5 path from the current process directory. The direct HDF5 writer does not expand ~, while downstream HDF5 resolution does, so avoid ~ in mppt.output_hdf5. Managed runs instead resolve file resources relative to the source YAML, copy and hash them, and rewrite the effective input.

For direct SCF/CASSCF stages, omp_num_threads: -1 selects detected cores and 0 becomes one thread. Downstream stages use -1 to inherit the environment, reject 0, and apply positive values to OpenMP, MKL, and OpenBLAS. See the input reference for the exact distinctions.

Checkpoints And Invalidation#

Each stage can record running, complete, or failed under /mppt/workflow/stages. Contract or executable-resolution failure before launch can leave no stage record. A complete stage is reused only when:

  • its normalized configuration and executable identity match;

  • required upstream HDF5 identities match;

  • its durable outputs still pass structural validation.

Executable identity uses resolved path, byte size, and nanosecond modification time; it is not a binary-content hash. Selection identities cover normalized configuration, native cipsixx h1, Fock diagonal and pair ERIs, selected config attributes, and explicit initial determinant data when used. Its durable reuse output is the selected determinant/model-space content, not iteration or root-energy CSVs.

DIAGPT identities cover normalized configuration, exact generated stdin, executable identity, complete model-space identity, effective route, route-specific attributes and datasets, and selected mapping when applicable. The validated <prefix>_heffso.1.h5 handoff is its durable reuse output. The optional <prefix>_h0.1 restart file is not a reuse gate; H1/H2 matrices exist only in the handoff.

HEFFSO additionally fingerprints exact stdin, the complete DIAGPT handoff, intso, selected mapping, enabled effective-operator property tensors, and solver-relevant inherited OpenMP/MKL/OpenBLAS/CUDA environment values. Reuse requires valid eigenvectors and the bounded state/occupation/configuration report with exactly nvectw states. When effective operators are enabled it also requires a complete mppt_heffso_effective_operators_v1 artifact; when they are disabled, a stale effective-operator artifact must be absent.

Rerunning selection invalidates DIAGPT and HEFFSO. Rerunning DIAGPT invalidates HEFFSO. Failed and interrupted stages are never treated as complete.

Force all enabled stages to rerun with:

pyscf2mppt workflow.yaml --from-hdf5 calculation.h5 --force-workflow

Inputs and logs are written under .mppt-workflow/. HEFFSO completion requires a nonempty <prefix>_sovec, the fin heffso marker, and the complete validated report/output contract; process exit status alone is insufficient.

PySCF-Only Dumps#

Leaving all three enabled flags at their default false writes the standard PySCF HDF5 dump without launching downstream stages. Enable the required stages in the same YAML, or run them later with --from-hdf5.