Managed Runs and Cartesian Sweeps#
The run and sweep commands add durable artifact directories around the
existing pyscf2mppt calculation pipeline. The legacy command remains valid:
pyscf2mppt input.yaml
Managed single point#
Run one complete configured calculation in an isolated directory:
pyscf2mppt run input.yaml --output-dir runs/h2
When --output-dir is omitted, the default is runs/<input-stem> below the
current directory. A managed point contains:
runs/h2/
├── .pyscf2mppt.lock
├── effective-input.yaml
├── run.json
├── run.log
├── checkpoints/
├── tmp/
├── resources/ # only for file-based basis, ECP, or GRPP inputs
└── <configured calculation and workflow outputs>
effective-input.yaml is the normalized input actually executed. File-based
basis, ECP, and GRPP resources are resolved relative to the source input and
copied into resources/ before execution. run.json records the input hash,
run directory, attempt number, exact child command, timestamps, return code,
and terminal status. The hash includes the contents of file-based basis, ECP,
and GRPP resources, not only their paths. run.log combines the child process
standard output and standard error and separates repeated attempts. A zero child
exit is accepted only when the configured HDF5 output contains the required
MPPT full- or selected-route metadata and integral datasets for runs that enable
integral dumping. Successful managed runs bind that output to the effective
input hash so a dump copied from another point is rebuilt rather than reused.
A new run rejects an existing output directory. Resume a matching run with:
pyscf2mppt run input.yaml --output-dir runs/h2 --resume
Resume requires the same normalized effective input and the original managed
directory location. Each resume creates a new attempt and launches the child
pipeline again. Direct PySCF checkpoint resume is used only when all declared
checkpoint payloads are readable. A no_dump: true run needs only those
payloads. A dumping run additionally requires a structurally valid configured
HDF5 file carrying the matching managed-input hash. That hash is written only
after a successful managed attempt. An interruption before that point, or a
downstream failure after the dump was created, can therefore make the next
attempt rerun SCF/CASSCF instead of resuming those checkpoints. That fallback
opens a new dump in replacement mode, so prior /mppt/workflow stage records in
the HDF5 are lost rather than reused. Even when direct resume is available, a
pre-dump checkpoint can continue into a replacement dump and erase prior
workflow state. Downstream reuse survives only when the resumed direct path does
not rewrite the HDF5, normally for a dump-complete checkpoint. Use
--force-workflow to rerun otherwise reusable downstream stages deliberately.
A copied or moved managed directory cannot be resumed in place because
run.json records its original absolute location. Keep the copy as an archive,
choose a new nonexistent output path, and start a managed run from the source
YAML. A fresh run rejects every already existing path, including an empty
directory.
Managed directories are single-writer resources. A concurrent run or sweep attempt against a directory already in use is rejected.
Enabled downstream executables are checked before the managed directory is
created. They may be selected explicitly with --selection-executable,
--diagpt-executable, and --heffso-executable or through the existing
environment-variable and PATH resolution.
Managed output paths must remain below the point directory. In particular,
mppt.prefix cannot contain /, \, :, or a Windows drive prefix.
mppt.output_hdf5 and configured cipsixx report paths must be portable relative
paths, cannot escape through .., and cannot collide case-insensitively with
each other, managed control names, or the currently reserved prefix-derived
h0.1, heffso.1.h5, sovec, and heffso_effective_operators.h5
artifacts. Retired f44.1, h1.1, and h2.1 names also remain reserved so a
managed rerun can safely remove stale sidecars from older worktrees.
Reserved top-level names include run.json, run.log, checkpoints,
resources, tmp, and .mppt-workflow. Calculation, base-input, and sweep
files must use a .yaml or .yml extension.
Sweep definition#
A sweep file points to one normal calculation input and maps dotted input paths to non-empty value lists:
base_input: h2.yaml
parameters:
molecule.atoms:
- [[H, 0.0, 0.0, 0.0], [H, 0.0, 0.0, 1.2]]
- [[H, 0.0, 0.0, 0.0], [H, 0.0, 0.0, 1.4]]
scf.conv_tol: [1.0e-8, 1.0e-10]
base_input is resolved relative to the sweep file. The example expands to
four points in YAML parameter order. Each value replaces the complete value at
its dotted mapping path, so molecule.atoms replaces the complete geometry.
List-index paths such as molecule.atoms[1] are not supported.
Validate and inspect the expansion without creating artifacts:
pyscf2mppt sweep sweep.yaml --dry-run
Dry-run parses and normalizes every expanded input, resolves file resources,
and prints deterministic point IDs and overrides. It returns before executable
preflight and does not use --output-dir, --resume, --force-workflow, or
executable overrides.
Execute the points sequentially:
pyscf2mppt sweep sweep.yaml --output-dir runs/h2-sweep
When --output-dir is omitted, the default is runs/<sweep-stem> below the
current directory.
The sweep directory contains sweep.json and one managed directory per point:
runs/h2-sweep/
├── .pyscf2mppt.lock
├── sweep.json
└── points/
├── point-000001/
├── point-000002/
├── point-000003/
└── point-000004/
sweep.json records the definition hash, ordered point IDs, overrides, input
hashes, statuses, return codes, errors, and the current count of points whose
status is interrupted. That count is recomputed and can decrease after resume;
it is not cumulative history. Child-process
failures are recorded and later points continue. The command returns 1 when
any point is incomplete and prints a point’s run.log path when that log was
created. Preflight validates every expanded input and every enabled executable
before a new or resumed non-dry-run sweep starts.
Resume a sweep only when its expanded definition is unchanged:
pyscf2mppt sweep sweep.yaml --output-dir runs/h2-sweep --resume
Completed points are not simply skipped. They receive another managed attempt
and are revalidated through the same single-point resume rules. With
no_dump: true, valid direct checkpoint payloads enable resume. A dumping point
also needs the matching managed HDF5 identity. Otherwise it restarts the direct
calculation path and a replacement dump removes old workflow state. Downstream
stages are reusable only when the resumed path preserves that HDF5. Failed or
interrupted points are attempted again through the same path.
Output and exit behavior#
Managed single points stream child output to the terminal and run.log.
Sweeps keep per-point child output in each point’s run.log and print only
point-level progress and summaries. A complete single point returns 0; a
failed point returns the nonzero child code when available and otherwise 1.
An all-complete sweep returns 0, while any incomplete sweep returns 1.
While a managed child is running, SIGINT or SIGTERM records interrupted,
terminates the child process group, escalates after five seconds when necessary,
and returns 130. A signal during expansion, preflight, materialization, or
between sweep points is not guaranteed to update a manifest. Top-level SIGINT
still returns 130; these process-group guarantees do not apply to direct mode.
Current limitations#
Sweep execution is sequential and does not provide a parallel jobs option. Sweeps are independent single points, not a potential-energy-surface model: there is no state tracking, geometry continuation, or cross-point checkpoint reuse. The sweep manifest reports execution status but does not aggregate energies or other scientific results. Read each point’s outputs for analysis. The complete Cartesian product and every normalized point are materialized in memory before execution or dry-run. There is currently no point-count limit, streaming expansion, or large-sweep confirmation prompt.