CIPSIXX Selection Algorithm#

CIPSIXX builds a determinant space for the scalar MPPT calculation. In the supported workflow it reads native integrals from the shared HDF5 file, grows a physical variational determinant space, and exports a separate quasi-restricted model-space adapter for DIAGPT and HEFFSO.

This page explains the current algorithm and how the mppt.cipsixx settings change it. Use the YAML input reference for exact types, defaults, and validation rules. Use the unified workflow guide for execution, restart, and checkpoint behavior.

Pinned selector release#

MPPT pins CIPSIXX v0.2.0, commit 2c91153fb61f3c0a14e577ff5da634fdc33d7df7. Selector tags use immutable vMAJOR.MINOR.PATCH names and match the binary version:

cipsixx_select --version
# cipsixx_select 0.2.0

This release adds exact bounded candidate storage, packed generation, fixed-input replay, unchanged-model connection/CI reuse, incremental closure counting and chunked atomic HDF5 export. Signed parent sums and eligible-root threshold filtering retain their scientific meaning; the downstream HDF5 schema is unchanged.

The managed YAML workflow continues to use the default hash storage backend. Bounded resource settings are native CLI options, not new mppt.cipsixx YAML fields. max_candidates limits scored output rather than generation memory.

Selection Flow#

Selection changes the model space and its completeness in S and, when applicable, L/J. Larger selections can increase or decrease MPPT multiplet splittings until convergence; determinant count alone is not a symmetry certificate. See MPPT symmetry and convergence before interpreting differences between selected calculations or rotated inputs.

CIPSIXX selection flow

Each ordinary iteration diagonalizes the space that existed at the start of that iteration. Candidate determinants are appended afterward. A final diagonalization is therefore required when the final coefficients and weights must describe the fully grown space.

Initial Determinant Space#

initial_space chooses the starting variational basis.

Value	Initial basis	Required information
`hf`	One determinant occupying the lowest alpha and beta orbitals	Electron counts embedded by the PySCF dump
`cas`	Every alpha/beta occupation combination in `cas_orbitals`	Active orbitals and either `cas_electrons` or both spin-resolved counts
`determinants`	A normalized determinant list read from HDF5	An existing `determinants_group`

For CAS initialization, core_orbitals are doubly occupied in every reference determinant and are not included in the active electron count. A total cas_electrons value is split as ceil(N/2) alpha and floor(N/2) beta unless the spin-resolved counts are supplied explicitly.

The determinant-list route reads physical alpha/beta bitsets. It uses stored coefficients when present, otherwise weights, otherwise equal values, and normalizes the resulting reference vector. Pointing determinants_group at /cipsixx/selected_determinants restarts growth from an earlier physical selection.

YAML orbital labels are one-based. The dump and selector use zero-based selected-space indices internally.

Variational Diagonalization#

At the start of each iteration CIPSIXX builds the dense Hamiltonian in the current determinant basis and requests the lowest n_roots eigenpairs. The diagonalizer cannot return more roots than the current determinant-space dimension.

selected_root is zero-based. In single-state mode it supplies the reference coefficients used for candidate generation. In multi-state mode every returned root can score candidates, but selected_root still supplies the reported variational energy and the final stored CI coefficients.

The initial physical space must contain at least selected_root + 1 determinants. Custom multistate weighting currently also requires the initial space to provide all n_roots roots because root_weights has one value per configured root. The HF initial space contains one determinant, so excited-root or custom multistate selection must start from a sufficiently large CAS or HDF5 determinant list.

Orbital Populations After Selection#

Use the final physical determinants and coefficients in /cipsixx/selected_determinants to form the density of selected_root. Require final diagonalization first. The determinant selected_root column records which root selected a candidate; it does not supply separate final eigenvectors. Selection weights are not normalized populations.

Map the selected-space indices back to the full orbital IDs and restore the frozen doubly occupied orbitals when checking the total electron count. Keep the physical selected-CI density separate from HEFFSO’s SOC-state occupations. HEFFSO’s retained-state bare occupations do not include an effective density-operator correction from the external perturbative space.

When changing the orbital selection window, compare total CI energies using the native selected-space energy plus E_frozen and PotNEcor. The frozen-core constant changes when an occupied shell enters selection. The last ordinary iteration CSV row precedes its growth batch and need not contain the final variational energy.

Orbital populations depend on the chosen orbitals. Sum complete identified shell groups, retain the character weights, and use the full one-particle density in a common orbital basis for density differences across rotations.

Candidate Generation#

For each current determinant with a nonzero reference coefficient, CIPSIXX generates spin-conserving external:

alpha and beta single excitations;
alpha-alpha and beta-beta double excitations;
alpha-beta double excitations.

Determinants already present in the variational space are excluded. When several parent determinants generate the same external determinant, their matrix-element contributions are accumulated before scoring.

frozen_orbitals cannot be used as excitation holes or as excitation particles. Consequently, freezing an unoccupied orbital also prevents an electron from entering it. When frozen_orbitals is omitted, core_orbitals is used as the frozen list. In the managed workflow, frozen_orbitals: [] emits --frozen-orbitals none and suppresses that fallback. When invoking cipsixx_select directly on a PySCF dump, pass the same CLI option because the current HDF5 contract does not distinguish an omitted list from an explicitly empty list.

Large Active Spaces#

Candidate generation is a current scaling limit, especially with many unfrozen occupied orbitals and many possible electron moves.

Model growth rebuilds contributions and diagonalizes the changed model. Empty adaptive-threshold retries reuse CI for unchanged P; bounded storage also reuses value-validated connections for rescoring.
The default hash backend materializes candidate tables. Coefficient/energy filtering happens after complete signed accumulation; those filters do not bound the tables’ peak memory.
max_selected, batch_size and max_candidates do not cap that intermediate memory. A small final target can still require a large candidate set.
One or a few larger growth steps may be practical when a long iterative sequence is too expensive. This changes the selection trajectory; it is not a convergence guarantee or a fixed program limit on iteration count.

For scale, a 141-orbital, 110-alpha/109-beta space with no frozen orbitals has 17,608,109 possible single/double excitations per parent before integral gates. Twelve parents imply roughly 211 million visits before merging duplicates. Those combinatorial counts explain why a large, mobile electron space can be costly even when the requested selected space is small.

Choose frozen orbitals and active spaces from the scientific model, not merely to reduce runtime. The export memory setting does not impose a native CIPSIXX RSS limit. Use the iteration CSVs to inspect generation time and actual growth.

Native bounded storage#

For standalone selection, add these options to the native command after creating an existing scratch directory:

--candidate-storage bounded
--candidate-buffer-mib 512
--candidate-workspace-mib 16
--candidate-scratch /path/to/existing/scratch

Record/workspace budgets are shared across workers. Partition-owned reduction and spill/merge preserve complete signed parent sums before scoring. The codec supports 1–192 selected spatial orbitals and uint32 parent indices.

These are payload budgets, not a process RSS cap: integrals, final models, dense CI and runtime overhead remain resident. --candidate-output-mib bounds materialized candidate payload when nonzero; 0 retains unlimited vector output. Native threshold streaming avoids a second accepted-candidate vector but still needs final model storage and scratch space. Use cipsixx_select --help for the scratch quota, reducer, partition and file-handle controls.

Perturbative Quantities#

For root m, current determinants I, and an external candidate J, the selector computes:

E0_m       = sum_I(c_Im^2 * E_zeroth_I)
S_Jm       = sum_I(H_JI * c_Im)
Delta_Jm   = E0_m - E_zeroth_J
c1_Jm      = S_Jm / Delta_Jm
E2_Jm      = S_Jm * c1_Jm

E_zeroth_I is the diagonal zeroth-order configuration energy. The current executable obtains it from the selected-space Fock diagonal. A root contribution is discarded when abs(Delta_Jm) <= denominator_tolerance.

selection_metric chooses the primary score:

Value	Primary score
`coefficient`	`abs(c1_Jm)`
`energy`	`abs(E2_Jm)`

Both quantities are always computed. min_abs_coefficient and min_abs_energy are conjunctive filters: a candidate must pass both nonzero thresholds even when only one quantity is the ranking metric. Single-state mode compares the raw absolute quantities. Multi-state mode compares the root-weighted absolute quantities, so a nonuniform weighting policy changes admission as well as ranking. Equality passes.

max_candidates, when set, keeps only the highest-scoring retained candidates for the iteration. It is applied after denominator and threshold filtering.

Single-State And Multi-State Scoring#

With multi_state: false, only selected_root generates and scores candidates.

With multi_state: true, every returned root computes its own perturbative quantities. For each candidate, the root with the largest weighted primary score wins. The candidate stores that root’s coupling, denominator, coefficient, second-order energy, and root index. Ties select the lower root index.

state_weighting defines the score multiplier w_m:

Mode	Weight
`uniform`	`w_m = 1`
`legacy-taum`	`w_m = 1 / max_I(abs(c_Im))`
`inverse-participation`	`w_m = 1 / sqrt(sum_I(c_Im^4))`
`custom`	`w_m = root_weights[m]`

The weights are not normalized. A zero custom weight suppresses that root’s candidate score. Custom weights require exactly one value per configured root and at least one positive value.

The winning root is selected by the weighted primary metric before the secondary threshold is checked. Another root is not reconsidered if the winner then fails that secondary threshold.

Targets, Growth, And Stopping#

max_selected is a stopping target, not a hard maximum. The target count is the size of the output-side spin-closure-expanded model space, while candidates are added to the smaller physical variational space. One newly added physical determinant can therefore add several rows to the target count.

Top-K Growth#

With selection_growth: topk and a target, each iteration adds at most:

min(batch_size, remaining target count, retained candidate count)

Spin closure can still make the resulting target count exceed max_selected. When max_selected is null, there is no remaining target budget, and top-k growth adds all retained candidates; batch_size does not cap that no-target case.

Threshold Growth#

With selection_growth: threshold, every retained candidate is added. batch_size is ignored. max_candidates can still cap the retained list, and the target is checked only after the complete threshold batch, so overshoot is expected.

Adaptive Thresholds#

Adaptive thresholds are valid only with threshold growth. When an iteration after the initial iteration adds fewer than min_new_determinants, the threshold for the selected metric is multiplied by threshold_multiplier and clamped to threshold_floor. The multiplier must be strictly between zero and one.

The iteration report records the threshold used in that iteration. Any reduction takes effect in a later iteration.

Stop Conditions#

Selection stops when the first applicable condition occurs:

the closure-expanded target count reaches max_selected;
no candidates remain under fixed-threshold growth;
no new unique determinants can be added;
max_iterations is exhausted.

There is no energy-convergence or perturbative-energy-convergence stopping criterion. The run can finish without reaching a configured target.

In the current empty-candidate adaptive branch, another iteration can be scheduled while either coefficient or energy threshold is above threshold_floor, even when the threshold for the selected metric cannot be reduced. max_iterations remains the hard guard for this edge case.

With max_selected: null, target stopping is disabled. The managed Python workflow still passes its default max_iterations: 12, so a no-target run can perform up to twelve all-candidate growth iterations before another stop condition ends it.

Final Diagonalization#

final_diagonalization: always diagonalizes the grown space after the last iteration that added determinants. It refreshes final selected-root coefficients and weights, making them authoritative for the complete physical space.

final_diagonalization: never skips only that final pass. It does not skip the diagonalization at the start of every ordinary selection iteration. Existing determinants then retain coefficients from the pre-growth space, while newly added determinants retain perturbative admission coefficients. The executable prints a warning, and such output should be treated as a deliberate handoff/performance experiment rather than a final CI result.

If an explicit target is already satisfied by the initial space, always performs one diagonalization and never returns without building the Hamiltonian.

Physical Selection And Model-Space Export#

CIPSIXX writes two determinant representations with different responsibilities.

HDF5 path	Meaning
`/cipsixx/selected_determinants`	Physical initial and MP2-selected determinants used as the variational basis and restart input
`/mppt/model_space/quasi_restricted`	Spin-closed, reordered, quasi-restricted compatibility view consumed by DIAGPT and HEFFSO

spin_adaptation_closure is applied to target counting and export, not to variational diagonalization:

Mode	Export behavior
`none`	Export only the physical determinants
`spin-complement`	Add missing alpha/beta-swapped partners when alpha and beta counts are equal
`spin-projection`	Add all same-spatial-occupation assignments with the same alpha and beta electron counts

The exporter then applies the quasi-restricted conventions required downstream:

closure determinants are deduplicated;
majority-alpha sources are globally spin-flipped into the required majority-beta convention;
determinants are represented as holes and particles relative to a closed-shell vacuum;
odd-electron records receive the compatibility electron-at-infinity marker;
records are reordered so spin variants of one spatial configuration remain contiguous;
metat is clamped to the smaller of n_roots and the exported row count.

Closure rows are not additional variationally diagonalized determinants and do not have selected-CI coefficients. Do not interpret the exported adapter as a CIPSIXX wavefunction.

The current spin-projection closure skips determinants with more than eight singly occupied spatial orbitals. Export also rejects a quasi-restricted excitation rank above eight. These are current compatibility limits, not requests to truncate the physical selection.

Iteration Reports#

iteration_report defaults to cipsixx_iterations.csv. Each ordinary iteration records:

physical determinant counts before and after growth;
filtered/capped candidate count and actual additions;
closure-expanded target count after growth;
coefficient and energy thresholds used;
largest retained coefficient and energy contributions;
selected-root variational energy before growth;
Hamiltonian-build, diagonalization, and selection timings.

root_energy_report defaults to cipsixx_root_energies.csv and records every root returned by each ordinary iteration.

Neither CSV receives a separate row for the final post-growth rediagonalization. Its selected-root energy appears in the executable’s final log summary. Reports are also not durable checkpoint outputs: a reused selection stage does not regenerate them.

Configuration Guide#

Goal	Recommended controls
Ground-state selection	`n_roots: 1`, `selected_root: 0`, `multi_state: false`
One excited-state selection	Start from enough CAS/HDF5 determinants, increase `n_roots`, choose `selected_root`, and keep `multi_state: false`
Balanced selection across roots	`multi_state: true` plus an explicit weighting policy
Bounded per-iteration physical growth	`selection_growth: topk`, set `batch_size` and `max_selected`
Admit every candidate above scientific thresholds	`selection_growth: threshold`, set the coefficient/energy thresholds and optionally `max_candidates`
Relax a sparse threshold automatically	Threshold growth plus `adaptive_threshold: true`
Restart from an earlier physical selection	`initial_space: determinants` and point `determinants_group` at the selected group
Authoritative final coefficients	Keep `final_diagonalization: always`

Do not copy determinant targets or perturbative thresholds between unrelated systems without convergence studies. See benchmarks for measured scaling and the distinction between requested target, physical determinants, and adapted ncf; see outputs for persisted files and HDF5 groups; and see the selected-route contract for the separate meaning of selected integral routing.