Dataset Forging¶

Provides the system-agnostic dataset forging pipeline.

sollertia_forgery.forging.dataset_state_path(dataset)¶

Resolves the path to a dataset’s state artifact.

Parameters:: dataset (DatasetData) – The resolved dataset whose state artifact to locate.
Return type:: Path
Returns:: The path to the dataset’s state .feather file.

sollertia_forgery.forging.define_forging_dataset(name, session_names, project_root, *, display_progress=False, force_recreate=False, recreate_animals=())¶

Builds the dataset hierarchy and materializes each tracked animal’s multi-recording configuration.

Notes

Every forging job runs against a hierarchy this call established, so it precedes them rather than joining them on the processing tracker. Rebuilding an animal resets that animal’s tracked jobs, since its new session set makes every stage outstanding again. The tracked jobs of a session no longer held by the animal fall outside the resulting universe and are discarded when the next pipeline run aligns the tracker.

A rebuild also discards the cross-recording output written by this dataset into the animal’s source sessions, both the sessions held by the animal before the rebuild and the ones it holds after it. The cross-recording stages resume from the output they find, and the output of a previous build describes that build’s recording set. Recreating the whole dataset discards that output from every session held by the dataset before or after the recreation, because the recreation rebuilds every animal.

Each animal that this call adds or rebuilds has its multi-recording configuration materialized, as does an animal already in the dataset that lacks one and whose sessions are still on this machine. Whether a configuration is needed at all follows from the dataset’s recorded session type, so a dataset of sessions with no cross-recording tracking reads no source data here.

An animal that already carries its configuration is left alone, and one whose sessions have moved off this machine is passed over. A dataset therefore keeps growing while part of its source data lives elsewhere, and a large project is forged in passes.

Parameters:

name (str) – The unique name of the dataset.
session_names (tuple[str, ...]) – The session names the dataset must contain. A session the dataset does not hold is appended, subject to the resolution policy.
project_root (Path) – The path to the project’s root directory that stores the animal and session data directories. The dataset hierarchy is also created under this directory.
display_progress (bool, default: False) – Determines whether the configuration materialized for each animal requests progress bars.
force_recreate (bool, default: False) – Determines whether to delete the whole existing dataset hierarchy and rebuild it from the provided session list.
recreate_animals (tuple[str, ...], default: ()) – The identifiers of animals already in the dataset to rebuild from the sessions the provided list holds for them.

Return type:

DatasetData

Returns:

The resolved dataset.

Raises:

ValueError – If the arguments contradict each other, or if the dataset’s acquisition system is unknown. The dataset resolution policy raises for a request it cannot satisfy.
FileNotFoundError – If a provided session name resolves to no directory under the project root, or if the acquisition system’s resolver reports a missing input it needs for an animal.
RuntimeError – If a provided session name resolves to more than one directory under the project root.
OSError – If the existing hierarchy, or the cross-recording output of a rebuilt animal, cannot be removed.

sollertia_forgery.forging.discover_forging_jobs(dataset_path)¶

Resolves the forging pipeline’s job universe and possible subset for an already-defined dataset.

Notes

Reads the dataset marker and each animal’s materialized configuration, loading no session marker. Discovery therefore reads nothing outside the dataset hierarchy, so a dataset whose source sessions have moved off this machine still resolves its jobs. Every job that the universe names is possible, because define_forging_dataset admits a session only once it carries the single-day outputs the forging stages consume, so the possible subset equals the universe.

Every stage is specified by the animals and sessions the dataset hierarchy holds, so a batch is prepared against a dataset that define_forging_dataset has already built. Preparing it afterwards names everything the hierarchy now holds, and the recorded job state decides which of those a run dispatches.

Parameters:: dataset_path (Path) – The path to the dataset’s root directory inside the project hierarchy.
Return type:: tuple[DatasetData, list[tuple[str, str]], list[tuple[str, str]]]
Returns:: A tuple of the loaded dataset, the job universe as a list of (job_name, specifier) pairs, and the possible subset.
Raises:: FileNotFoundError – If the dataset’s own marker is not present under the provided path.

sollertia_forgery.forging.discover_project_datasets(project_root)¶

Loads every forged dataset stored under a project’s root directory.

Notes

A forged dataset is a top-level directory under the project root carrying a dataset marker, so the candidate test costs one directory listing and one marker check per candidate.

Parameters:: project_root (Path) – The path to the project’s root directory holding the animal and dataset directories.
Return type:: list[DatasetData]
Returns:: The loaded datasets, ordered by directory name.

sollertia_forgery.forging.forging_cross_recording_paths(dataset)¶

Resolves the cross-recording output one dataset holds inside each of the source sessions it names.

Notes

The cross-recording stages write each session’s aligned fluorescence into that session’s own cindra output rather than into the dataset, so the dataset owns a directory inside every source session it names. That output describes the registration this dataset performed and nothing else, so it goes wherever the dataset goes.

The directory is named for this dataset alone, through the same qualified name the pipeline configures and through cindra’s own layout resolver. A session belonging to several datasets therefore keeps the sibling directory each of the others owns, alongside its single-recording output.

A session whose marker cannot be read is passed over, since a source session that no longer resolves holds no directory this dataset can name.

Parameters:: dataset (DatasetData) – The loaded dataset whose cross-recording output to locate.
Return type:: tuple[Path, ...]
Returns:: The cross-recording output directory this dataset owns in each source session that resolves, in the order the dataset holds its sessions.

sollertia_forgery.forging.forging_job_prerequisites(dataset, universe)¶

Returns the intra-pipeline job ordering for the forging pipeline.

Notes

cindra owns the ordering of the cross-recording stretch, so the prerequisites of every discovery and extraction job come from its resolver rather than from a chain spelled out here. The resolver orders one animal’s recordings at a time, while the universe interleaves every animal. The cross-recording jobs are therefore regrouped under the animal that owns them before the resolver runs over each group.

This library owns the assembly stage, which reads the aligned fluorescence its session’s extraction wrote, so that edge is added on top of the ordering cindra supplies. A dataset needing no multi-day processing carries assembly jobs that depend on nothing.

The dataset supplies the animal to which each session belongs, which the universe pairs do not carry, since an extraction is specified by its session while its discovery is specified by its animal. A job whose animal is no longer in the dataset keeps an empty prerequisite tuple, so every job in the universe carries an entry.

Parameters:

dataset (DatasetData) – The resolved dataset from which the universe was built.
universe (list[tuple[str, str]]) – The job set over which to build ordering, as returned by _build_forging_universe.

Return type:

dict[tuple[str, str], tuple[tuple[str, str], ...]]

Returns:

A mapping of each job to its tuple of prerequisite jobs, following the discovery to extraction to assembly chain.

sollertia_forgery.forging.forging_tracker_path(dataset)¶

Resolves the forging processing tracker path from a loaded dataset.

Parameters:: dataset (DatasetData) – The loaded dataset whose tracker to locate.
Return type:: Path
Returns:: The path to the dataset’s forging tracker, which sits beside the dataset’s own marker.

sollertia_forgery.forging.generate_dataset_state(dataset, *, display_progress=False)¶

Builds and saves the state artifact for one forged dataset.

Reads the dataset’s forging tracker and writes one row per tracked job, carrying that job’s scope, its resolved subject, and everything the tracker records about it. A file lock serializes concurrent writers.

Parameters:

dataset (DatasetData) – The resolved dataset whose forging state to serialize.
display_progress (bool, default: False) – Determines whether to emit a completion message once the artifact is written. Generation reads a single tracker, so no progress bar is displayed.

Return type:

Path

Returns:

The path to which the state artifact was written.

Raises:

Timeout – If the state file’s lock cannot be acquired within the timeout period.
ValueError – If the dataset’s forging tracker records a job name that declares no scope in _DATASET_JOB_SCOPES.

sollertia_forgery.forging.run_forging_pipeline(name, project_root, job_id=None, *, workers=-1, display_progress=False)¶

Runs the outstanding cross-recording and assembly jobs of an already-defined dataset.

Notes

The forging tracker records every stage as a job. There is one cross-recording discovery job per tracked animal, one extraction job per that animal’s session, and one assembly job per session in the dataset. The cross-recording jobs exist only for animals whose acquisition system resolves a multi-recording configuration, so a dataset of sessions without one carries assembly jobs alone.

define_forging_dataset roots the ordering. It resolves the hierarchy from the requested session list and writes each animal’s multi-recording configuration, so no job runs before it completes.

In local mode every stage that the tracker already records as succeeded is skipped, so an invocation runs only the jobs still outstanding. In remote mode the named job runs regardless of the state that the tracker records for it. Rebuilding an animal resets that animal’s jobs first.

In local mode (job_id is None) every outstanding stage runs in sequence: each animal’s discovery and its per-session extractions, then the assembly jobs across a parallel pool. The assembly stage runs sequentially when the resolved worker count or the outstanding session count is one. In remote mode (job_id is provided) only the single job matching the identifier runs, so an external scheduler drives cross-job ordering by dispatching each identifier in prerequisite order.

Parameters:

name (str) – The unique name of the dataset, which define_forging_dataset has already built.
project_root (Path) – The path to the project’s root directory that stores the animal and session data directories. The dataset hierarchy also lives under this directory.
job_id (str | None, default: None) – The hexadecimal identifier of the single job to execute (remote mode). If not provided, every outstanding job runs (local mode).
workers (int, default: -1) – The number of workers to use. A value less than 1 uses all available CPU cores (minus reserved cores), and 1 forces sequential assembly.
display_progress (bool, default: False) – Determines whether to display a progress bar during the assembly stage. The multi-day stages read the flag recorded in each animal’s configuration by define_forging_dataset.

Raises:

ValueError – If the dataset is not defined, if its acquisition system is unknown, if the provided job_id does not match any job, or if a cross-recording job’s multi-recording configuration names fewer than two recording directories or no dataset name. Also raised when an assembled session’s data.feather carries a column the dataset’s description companion file does not describe.
FileNotFoundError – If the dataset carries no data_descriptions.feather companion file, or if a cross-recording job’s multi-recording configuration is missing, is not a .yaml file, is not a valid multi-recording configuration, or names a recording that holds no combined metadata archive. Also raised when a source session is missing a re-exported shared asset it is required to carry.
RuntimeError – If a cross-recording job’s multi-recording configuration names a recording directory holding several combined metadata archives, names recording paths that carry no unique identifying component, or names a recording whose identifying component contains a colon. It is also raised when the host is macOS and carries no loadable OpenMP runtime for the Numba threading layer.

Return type:

None

sollertia_forgery.forging.state.DATASET_STATE_FILENAME: str = 'dataset_state.feather'¶: The filename of the dataset state artifact, written at the dataset’s root beside its forging tracker.

sollertia_forgery.forging.pipeline.FORGING_JOB_NAME: str = 'session_data_assembly'¶: The job name identifying per-session assembly jobs in the forging tracker.

sollertia_forgery.forging.pipeline.FORGING_JOB_CONCURRENCY_LIMITS: dict[str, int] = {'session_data_assembly': 4}¶

The jobs of each forging type that may run at once regardless of the cores a budget could still supply, keyed by the tracker job name.

Notes

Only assembly declares a ceiling. It holds one core per job, so the ceiling sets the width of the pool it opens. The cross-recording jobs exist for two-photon sessions alone, and cindra sizes each of them itself and declares no ceiling for either of their resource classes, so the core budget alone bounds how many run at once.

sollertia_forgery.forging.pipeline.MULTIDAY_DISCOVERY_JOB_NAME: str = 'multiday_discovery'¶: The job name identifying a per-animal cross-recording cell-discovery job in the forging tracker. The job’s specifier is the animal identifier, and the job exists only for animals whose acquisition system resolves a multi-recording configuration.

sollertia_forgery.forging.pipeline.MULTIDAY_EXTRACTION_JOB_NAME: str = 'multiday_extraction'¶: The job name identifying a per-session aligned-fluorescence extraction job in the forging tracker. The job’s specifier is the session name, and the job runs after its animal’s discovery job.

Project Management¶

Provides system-agnostic management pipelines: raw-data checksum verification and project manifest handling.

class sollertia_forgery.managing.ProjectManifest(manifest_file)¶

Bases: object

Provides methods for visualizing and working with the data stored inside the managed project manifest .feather file.

Parameters:: manifest_file (Path) – The path to the .feather manifest file that stores the snapshot of the target project’s state.

_data¶: The Polars DataFrame that stores the snapshot of the project’s state.

property animals: tuple[str, ...]¶: Returns the unique identifiers for each animal participating in the project, in natural order.

property data: DataFrame¶: Returns the manifest’s session rows, one per discovered session, with every column the schema declares.

get_animal_for_session(session)¶

Returns the unique identifier of the animal that participated in the specified session.

Parameters:: session (str) – The unique identifier of the session for which to retrieve the participating animal’s identifier.
Return type:: str
Returns:: The animal identifier recorded on the session’s manifest row.
Raises:: ValueError – If the specified session is not found in the manifest file.

get_session_data(session)¶

Returns the manifest row recording the current acquisition and processing state of the specified session.

Parameters:: session (str) – The unique identifier of the session for which to retrieve the data.
Return type:: DataFrame
Returns:: Every manifest column held for the specified session, which is animal, date, session, session_path, type, system, notes, complete, and the per-pipeline done columns integrity, two_photon, runtime, microcontroller, and video.

get_sessions(animal=None, *, exclude_incomplete=True)¶

Returns the unique identifiers for all sessions that match the input filtering criteria.

Parameters:

animal (str | None, default: None) – An optional animal identifier for which to retrieve the session identifiers. If set to None, the method returns the session IDs for all animals participating in the project.
exclude_incomplete (bool, default: True) – Determines whether to exclude incomplete sessions from the output tuple.

Return type:

tuple[str, ...]

Returns:

The tuple of session identifiers matching the filtering criteria.

Raises:

ValueError – If the specified animal is not found in the manifest file.

get_system_for_session(session)¶

Returns the data acquisition system used to acquire the specified session’s data.

Parameters:: session (str) – The unique identifier of the session for which to retrieve the data acquisition system.
Return type:: str
Returns:: The acquisition system recorded on the session’s manifest row.
Raises:: ValueError – If the specified session is not found in the manifest file.

print_data()¶

Prints the entire contents of the manifest file to the terminal.

Return type:: None

print_notes(animal=None)¶

Prints the animal ID, per-animal session index, session date, session type, acquisition system, and experimenter notes data for each project’s session to the terminal.

This data view is optimized for determining what data acquisition sessions have been carried out and checking the outcomes of each session recorded in the experimenter notes. The session column shows the per-animal 1-based session index, and the date column shows the UTC acquisition time truncated to the second.

Parameters:: animal (str | None, default: None) – The unique identifier of the animal for which to display the data. If provided, this method only displays the data for that animal. Otherwise, it displays the data for all animals.
Return type:: None

print_summary(animal=None)¶

Prints a summary view of the manifest file to the terminal, excluding the session path and the experimenter notes data for each session.

This data view is optimized for tracking which processing steps have been applied to each of the project’s data acquisition sessions. The session column shows the per-animal 1-based session index, and the date column shows the UTC acquisition time truncated to the second. Every pipeline column reports 1 when that pipeline finished and 0 otherwise, matching the numeric complete column, so the operator reads one done or not-done convention throughout.

Parameters:: animal (str | None, default: None) – The unique identifier of the animal for which to display the data. If provided, this method only displays the data for that animal. Otherwise, it displays the data for all animals.
Return type:: None

summarize()¶

Returns a structured summary of the project manifest for programmatic consumption.

Counts the sessions each pipeline finished, alongside the session type and acquisition system distributions. Reports on sessions only, since a dataset’s forging state is recorded on that dataset’s own artifacts.

Return type:: dict[str, int | list[str] | dict[str, int] | dict[str, dict[str, int]]]
Returns:: A dictionary containing total_sessions, total_animals, animals, session_types, acquisition_systems, complete_count, pipeline_status_counts (the count of sessions that finished and of sessions that did not, keyed by pipeline identifier), columns, and total_rows.

sollertia_forgery.managing.checksum_job_prerequisites(session, universe)¶

Returns the intra-pipeline job ordering for the checksum pipeline.

Notes

The checksum pipeline produces a single job with no upstream dependency, so every job maps to an empty prerequisite tuple.

Parameters:

session (SessionData) – The loaded session, accepted for the shared dispatch contract and not read by this ordering.
universe (list[tuple[str, str]]) – The job universe as returned by discover_checksum_jobs.

Return type:

dict[tuple[str, str], tuple[tuple[str, str], ...]]

Returns:

A mapping of each job to its tuple of prerequisite jobs, which is always empty for the checksum pipeline.

sollertia_forgery.managing.discover_checksum_jobs(session_path)¶

Resolves the checksum pipeline’s job universe and possible subset for the target session.

Notes

The checksum pipeline produces exactly one job, so the universe is always the single (CHECKSUM_JOB_NAME, session_name) pair. Both pipeline modes share that job, because a session carries one integrity state whether the run establishes it or confirms it. That job is possible once the session holds raw data the checksum covers, which excludes the checksum file, the tracker, and the tracker lock. Discovery loads the session marker and walks the raw data directory, leaving the session as it found it.

Parameters:: session_path (Path) – The path to the root session directory containing the session data hierarchy.
Return type:: tuple[SessionData, list[tuple[str, str]], list[tuple[str, str]]]
Returns:: A tuple of the loaded session, the job universe as a list of (job_name, specifier) pairs, and the possible subset of that universe.
Raises:: FileNotFoundError – If the source path does not contain a valid session data hierarchy.

sollertia_forgery.managing.generate_project_manifest(project_directory, *, display_progress=False)¶

Builds and saves the project manifest .feather file under the target project’s root directory.

The manifest captures one row per session with its acquisition metadata and per-pipeline processing status. It also writes the project job artifact, <project>_jobs.feather, into the same root, publishing it before the manifest itself. A file lock serializes concurrent writers, and the outcome is recorded on a manifest processing tracker in the project root.

Parameters:

project_directory (Path) – The path to the processed project’s root directory.
display_progress (bool, default: False) – Determines whether to emit a preamble message when generation starts and a completion message when the manifest is written. Generation is fast enough that no progress bar is displayed.

Raises:

FileNotFoundError – If the project directory does not exist, contains no session data, or contains a session without its descriptor file.
ValueError – If an unsupported session type is encountered.
OSError – If any directory under the project directory cannot be read while its sessions are discovered.
Timeout – If the manifest .feather file lock cannot be acquired within 20 seconds.

Return type:

None

sollertia_forgery.managing.project_jobs_path(project_directory)¶

Resolves the path to the project job artifact under the target project’s root directory.

Parameters:: project_directory (Path) – The path to the project’s root directory.
Return type:: Path
Returns:: The path to the project’s job .feather file.

sollertia_forgery.managing.project_manifest_path(project_directory)¶

Resolves the path to the project manifest .feather file under the target project’s root directory.

Parameters:: project_directory (Path) – The path to the project’s root directory.
Return type:: Path
Returns:: The path to the project manifest .feather file.

sollertia_forgery.managing.run_checksum_processing_pipeline(session_path, *, regenerate_checksum=False, workers=-1, display_progress=False)¶

Verifies the integrity of the session’s raw_data directory by comparing its computed checksum against the value stored in ax_checksum.txt, or regenerates that stored value.

Notes

This is a single-stage pipeline that produces exactly one job, which resolves the whole raw_data directory. The job runs in one of two modes. Verification compares the recomputed checksum against the stored value and records a failure on any mismatch. That failure marks the data as corrupted. Regeneration overwrites the stored value with the freshly computed one, re-baselining a session whose raw data changed by intent.

Every path this pipeline reads and writes lives under raw_data, while every other session pipeline writes only under processed_data. A checksum job therefore runs safely alongside any other pipeline’s job for the same session.

The digest covers the whole raw_data tree or the job fails. A subdirectory the host cannot read therefore records a job failure carrying the underlying error, rather than a digest computed over the readable subset.

Parameters:

session_path (Path) – The path to the root data directory of the session to be processed.
regenerate_checksum (bool, default: False) – Determines whether to overwrite the stored ax_checksum.txt value with the freshly computed checksum instead of verifying against it.
workers (int, default: -1) – The number of parallel worker processes. Values below 1 request all available cores minus reserved cores. A value of 1 disables parallelism.
display_progress (bool, default: False) – Determines whether to emit console messages and a progress bar during calculation.

Raises:

FileNotFoundError – If the source path does not contain a valid session data hierarchy, or if verification is requested for a session that stores no checksum value.
ValueError – If the session’s raw_data directory holds no file the checksum covers, which is refused before the tracker is touched so the session’s last recorded verdict survives.
OSError – If any directory or file under the session’s raw_data directory cannot be read, which the pipeline records as a job failure before re-raising.
RuntimeError – If the host is macOS and carries no loadable OpenMP runtime for the Numba threading layer.

Return type:

None

sollertia_forgery.managing.checksum.CHECKSUM_JOB_NAME: str = 'checksum_resolution'¶: The job name identifying the checksum resolution job in the checksum processing tracker (ProcessingTrackers.CHECKSUM), where this pipeline records the job’s state.

sollertia_forgery.managing.manifest.MANIFEST_JOB_NAME: str = 'manifest_generation'¶: The job name used to identify manifest generation jobs in processing trackers.

sollertia_forgery.managing.manifest.MANIFEST_AXES: tuple[str, ...] = ('animal', 'type', 'system', 'complete', 'integrity', 'runtime', 'microcontroller', 'video', 'two_photon')¶: The manifest columns by which a caller may filter sessions, and the axes its breakdown counts. The five pipeline columns each hold a 0 or a 1, so their breakdown reports how many sessions have finished that pipeline.

sollertia_forgery.managing.manifest.MANIFEST_SEMI_FIELDS: tuple[str, ...] = ('animal', 'session', 'session_path', 'date', 'type', 'system', 'complete', 'integrity', 'runtime', 'microcontroller', 'video', 'two_photon')¶: The session fields a semi-detail listing carries, which is the session’s identity and whether each of its pipelines finished.

Runtime Log Processing¶

Provides the data acquisition system runtime log processing pipeline.

sollertia_forgery.runtime.discover_runtime_jobs(session_path)¶

Resolves the runtime pipeline’s job universe and possible subset for the target session.

Notes

The runtime pipeline produces exactly one job, so the universe is always the single (RUNTIME_JOB_NAME, source_id) pair, where the source id is resolved from the session’s acquisition system. That job is possible only when its DataLogger archive is present on disk. Discovery indexes the archives the logger wrote and reads no message, leaving every output file untouched. An absent archive yields an empty possible subset, so a batch layer can align the tracker slot against the universe and skip the job.

The archives are indexed through the data-structures library, which owns the name under which each source writes its archive, so a session that recorded no runtime archive is reported without this pipeline restating that naming rule. The index covers the logger’s own output directory, which is where a session’s archives are assembled side by side.

Parameters:: session_path (Path) – The path to the root session directory containing the session data hierarchy.
Return type:: tuple[SessionData, list[tuple[str, str]], list[tuple[str, str]]]
Returns:: The loaded session, the job universe as (job_name, specifier) pairs, and the possible subset of that universe.
Raises:: ValueError – If the session’s acquisition system is unknown (not a valid AcquisitionSystems member).

sollertia_forgery.runtime.run_runtime_processing_pipeline(session_path, *, workers=-1, display_progress=False)¶

Decodes and parses the acquisition runtime’s log archive for the target session.

Notes

Runs as a single stage. Locates the acquisition system’s runtime DataLogger archive under the session’s raw behavior-data directory and decodes it into a raw (time_us, payload) message table. The decoded table then reaches the registered runtime parser, which writes the system’s behavior feathers into the session’s processed runtime-data directory (processed_data.runtime_data_path). The runtime source id and parser come from the session’s acquisition system.

The runtime job is the only job this pipeline produces, so it always runs. The registered parser may additionally raise system-specific errors (for example ValueError or RuntimeError) that propagate unchanged.

Parameters:

session_path (Path) – The path to the root session directory containing the session data hierarchy.
workers (int, default: -1) – The number of worker processes the decode stage may use. A value less than 1 uses all available CPU cores (minus reserved cores), and 1 forces a single in-process decode.
display_progress (bool, default: False) – Determines whether to display a progress bar while decoding a multi-batch archive.

Raises:

FileNotFoundError – If the session’s runtime log archive is not present in its raw behavior data directory.
RuntimeError – If the host is macOS and carries no loadable OpenMP runtime for the Numba threading layer.
ValueError – If the session’s acquisition system is unknown (not a valid AcquisitionSystems member), or if the runtime log archive carries no valid onset timestamp message.

Return type:

None

sollertia_forgery.runtime.runtime_job_prerequisites(session, universe)¶

Returns the intra-pipeline job ordering for the runtime pipeline.

Parameters:

session (SessionData) – The loaded session, accepted for the shared dispatch contract and not read by this ordering.
universe (list[tuple[str, str]]) – The job universe as returned by discover_runtime_jobs.

Return type:

dict[tuple[str, str], tuple[tuple[str, str], ...]]

Returns:

A mapping of each job to its tuple of prerequisite jobs, which is always empty for the runtime pipeline.

sollertia_forgery.runtime.pipeline.RUNTIME_JOB_NAME: str = 'runtime_processing'¶: The job name identifying the runtime processing job in the runtime processing tracker (ProcessingTrackers.RUNTIME), where this pipeline records the runtime job’s state.

Microcontroller Log Processing¶

Provides the microcontroller log processing pipeline.

sollertia_forgery.microcontrollers.discover_microcontroller_jobs(session_path)¶

Resolves the microcontroller pipeline’s job universe and possible subset for the target session.

Notes

The universe enumerates every job the session’s microcontroller manifest could produce. It holds one extraction job per controller that declares at least one module that the acquisition system parses and that the session configured for use, plus one parse job per such module. The possible subset narrows the universe to controllers whose log archive resolves to exactly one file under the raw behavior data tree. A controller with no archive, or one whose name matches several archives, can be neither extracted nor parsed. Locating those archives is delegated to the acquisition library’s own resolver, so discovery reads the manifest and indexes the archive names, leaving the archives’ contents and every output file untouched.

Parameters:

session_path (Path) – The path to the root session directory containing the session data hierarchy.

Return type:

tuple[SessionData, list[tuple[str, str]], list[tuple[str, str]]]

Returns:

The loaded session, the job universe as (job_name, specifier) pairs, and the possible subset of that universe. Extraction specifiers are controller IDs and parse specifiers are "{controller_id}-{module_type}-{module_id}".

Raises:

FileNotFoundError – If the session’s raw behavior data directory does not exist, if the session’s microcontroller manifest is not present, or if the registered eligibility accessor cannot load its own inputs, for example an absent hardware state file.
OSError – If any directory beneath the session’s raw behavior data directory cannot be read while the microcontroller manifest and the log archives are located.
ValueError – If the session’s acquisition system is unknown, if the microcontroller manifest is malformed, if the raw behavior data tree holds more than one microcontroller manifest, or if no manifest controller declares a module the acquisition system extracts.

sollertia_forgery.microcontrollers.microcontroller_job_prerequisites(session, universe)¶

Returns the intra-pipeline job ordering for the microcontroller pipeline.

Notes

Each parse job reads the raw per-module feather that its controller’s extraction job writes, so every parse job requires that extraction job to have succeeded. Extraction jobs read the raw archive directly and have no upstream dependency. Extraction jobs use the acquisition library’s CONTROLLER_EXTRACTION_JOB_NAME and each parse specifier encodes its controller as the leading "{controller_id}-..." segment.

Parameters:

session (SessionData) – The loaded session, accepted for the shared dispatch contract and not read by this ordering.
universe (list[tuple[str, str]]) – The job universe as returned by discover_microcontroller_jobs.

Return type:

dict[tuple[str, str], tuple[tuple[str, str], ...]]

Returns:

A mapping of each job to its tuple of prerequisite jobs. Parse jobs map to their controller’s extraction job, and extraction jobs map to an empty tuple.

sollertia_forgery.microcontrollers.run_microcontroller_processing_pipeline(session_path, job_id=None, *, workers=-1, display_progress=False)¶

Discovers, validates, and executes microcontroller log processing jobs for the target session.

Notes

Stage 1 (extraction) reads each controller’s log archive via the ataraxis-communication-interface binding and writes raw per-module feathers into the session’s microcontroller_data directory. Stage 2 (parsing) partitions each raw feather by event code and runs the parser registered for the session’s acquisition system (resolved via resolve_microcontroller_parsers), writing the domain-specific feather into microcontroller_data. The pipeline is system-agnostic.

In local mode (job_id is None) every present controller is extracted, then every eligible module is parsed (across a worker pool when more than one worker is available and more than one module is runnable). In remote mode (job_id is provided) only the single matching job runs. That job still honors the worker budget, so a remote extraction fans intra-archive decoding across the pool while a remote parse runs single-core. The processing tracker is co-located with the extracted and parsed output in microcontroller_data.

The extraction configuration is materialized into microcontroller_data on every invocation, before any job is dispatched. The acquisition binding reads each controller’s extraction targets from that file rather than from an in-memory object. A scheduler may dispatch a single extraction job into a fresh process, so writing the file unconditionally lets a remote job read the configuration the local run used.

Parameters:

session_path (Path) – The path to the root session directory containing the session data hierarchy.
job_id (str | None, default: None) – The hexadecimal identifier of the single job to execute (remote mode). If not provided, the whole pipeline runs (local mode).
workers (int, default: -1) – The number of worker processes to use. A value less than 1 uses all available CPU cores (minus reserved cores), and 1 forces sequential processing.
display_progress (bool, default: False) – Determines whether to display progress bars during processing.

Raises:

FileNotFoundError – If the session’s raw behavior data directory does not exist, if the session’s microcontroller manifest is missing, or if the registered eligibility accessor cannot load its own inputs, for example an absent hardware state file. In remote mode, also raised when a requested extraction job’s log archive is not present.
OSError – If any directory beneath the session’s raw behavior data directory cannot be read while the microcontroller manifest and the log archives are located.
RuntimeError – If the host is macOS and carries no loadable OpenMP runtime for the Numba threading layer.
ValueError – If the session’s acquisition system is unknown, if the microcontroller manifest is malformed, or if the raw behavior data tree holds more than one microcontroller manifest. Also raised when no manifest controller declares a module the session’s acquisition system extracts, when no processable controllers are discovered, or when the provided job_id matches no available job.

Return type:

None

sollertia_forgery.microcontrollers.pipeline.PARSE_JOB_NAME: str = 'module_parsing'¶: The job name identifying per-module parsing (Stage 2) jobs in the microcontroller processing tracker. Stage 1 extraction jobs use the acquisition library’s own CONTROLLER_EXTRACTION_JOB_NAME instead.

ataraxis_communication_interface.orchestration.jobs.CONTROLLER_EXTRACTION_JOB_NAME: str = 'microcontroller_data_extraction'¶

The job name under which microcontroller data extraction is registered in a ProcessingTracker.

Notes

The value is hashed into every persisted job identifier, so changing the string invalidates every identifier a tracker already holds and every identifier a scheduler derived independently.

Video Processing¶

Provides the camera video-processing pipeline.

sollertia_forgery.video.discover_video_jobs(session_path)¶

Resolves the video pipeline’s job universe and possible subset for the target session.

Notes

The universe is the full acquisition-time job set the camera manifest defines. It holds one timestamp job and one motion-energy job per registered camera, plus the single rename and tracking jobs. The possible subset is the job set the session’s own data supports. A timestamp job is possible only when its camera’s {source_id}_log.npz archive resolves to exactly one file, and the rename job joins them when at least one is possible. The tracking job is possible when the system’s donated locator resolves a pose-prediction file, so a session carrying none keeps that job in the universe alone. The energy jobs are always possible because they read only their own recordings. A camera with a recording but no log archive therefore keeps its energy job possible while its timestamp job stays in the universe alone. This is discovery only, reading the manifest, indexing archive names, and locating the session’s pose prediction, while decoding no data and mutating nothing.

Parameters:

session_path (Path) – The path to the root session directory containing the session data hierarchy.

Return type:

tuple[SessionData, list[tuple[str, str]], list[tuple[str, str]]]

Returns:

A tuple of the loaded session, the job universe as a list of (job_name, specifier) pairs, and the possible subset of that universe. Timestamp and energy specifiers are camera source IDs, and the rename and tracking specifiers are empty.

Raises:

FileNotFoundError – If the session’s camera manifest is not present.
OSError – If any directory under the raw behavior data directory cannot be read while the camera manifest and the log archives are located.
ValueError – If the session’s acquisition system is not a supported AcquisitionSystems member, if the camera manifest registers no cameras, or the raw behavior data tree holds more than one camera manifest.

sollertia_forgery.video.resolve_camera_video(camera_data_directory, session_name, camera_name)¶

Resolves the recording that a camera produced within the session’s raw camera-data directory.

Each recording reaches the session’s raw camera-data directory named {session_name}_{camera_name}.mp4, so the recording is resolved by reconstructing that exact name rather than by pattern-matching the camera name against the directory. Matching on a suffix would be ambiguous: a camera named camera would match a left_camera recording, since that name also ends in _camera.

Parameters:

camera_data_directory (Path) – The session’s raw camera-data directory (session.raw_data.camera_data_path), which holds the recordings themselves.
session_name (str) – The name of the session that owns the recording, which prefixes every recording filename.
camera_name (str) – The colloquial camera name recorded in the acquisition-time camera manifest.

Return type:

Path | None

Returns:

The path to the camera’s recording, or None if the directory or the recording is absent, which means there is nothing for this camera to analyze.

sollertia_forgery.video.run_video_processing_pipeline(session_path, job_id=None, *, timestamp=False, track=False, energy=False, target_camera=-1, workers=-1, display_progress=False)¶

Discovers, validates, and executes the camera video-processing pipeline for the target session.

Notes

The pipeline runs four kinds of job sharing one processing tracker, grouped under three flags. The timestamp flag runs one parse job per camera whose {source_id}_log.npz archive is on disk, each extracting that camera’s frame timestamps, then the single rename job that republishes every parsed feather under its canonical manifest name. The track flag runs the single job that applies the acquisition system’s donated tracking function to the session’s pose predictions, which no-ops when none are present. The energy flag runs one job per camera, measuring its recording into a motion-energy feather, and no-ops when the recording is absent.

The camera manifest defines the full job universe, one parse and one energy job per registered camera plus the single rename and tracking jobs, so tracker alignment does not depend on which archives are on disk.

Two runtimes select which jobs execute. In local mode (job_id is None) the pipeline runs each requested stage sequentially over one shared worker pool, and runs every stage when no flag is set or all three are set, mirroring the cindra pipeline’s resolution. The parse and energy jobs honor target_camera to narrow that pass to a single camera. In remote mode (a job_id is provided) only the single job matching that identifier runs, chosen entirely by the identifier, so the flags and target_camera are ignored. Either way a parse job fans its archive decoding across the worker pool once the archive is large enough, an energy job fans its recording decoding the same way, and the rename and tracking jobs run single-core.

Parameters:

session_path (Path) – The path to the root session directory containing the session data hierarchy.
job_id (str | None, default: None) – The unique hexadecimal identifier for the job to execute. If provided, only the matching job is executed (remote mode). If not provided, all requested jobs are executed (local mode).
timestamp (bool, default: False) – Determines whether to run the timestamp stage, which parses each camera’s log archive into a frame-timestamp feather and republishes it under its canonical manifest name.
track (bool, default: False) – Determines whether to run the video-tracking job (the system’s donated tracking function).
energy (bool, default: False) – Determines whether to run the per-camera motion-energy job.
target_camera (int, default: -1) – The numeric source ID of the single camera to process when running the timestamp or motion-energy stages. Set to -1 to process every camera with a discovered log archive for the timestamp stage, and every camera registered in the manifest for the motion-energy stage. Ignored by the tracking job, and in remote mode (when job_id is provided), where the job to run is selected entirely by job_id.
workers (int, default: -1) – The number of worker processes that the extraction binding may use per archive, and that the motion-energy job uses to decode each recording. Set to -1 to use all available CPU cores (minus reserved cores).
display_progress (bool, default: False) – Determines whether to display progress during each archive’s parsing and each recording’s motion-energy measurement.

Raises:

ValueError – If the session’s acquisition system is not a supported AcquisitionSystems member, if the camera manifest registers no cameras, or if the raw behavior data tree holds more than one camera manifest. Also raised when no camera log archives are discovered for the timestamp stage, or when job_id does not match an available job. The same error covers a target_camera with no discovered log archive while the timestamp stage runs, or one that is not registered in the camera manifest while the motion-energy stage runs. It also covers a camera recording that cannot be opened, reports no frames, cannot decode the frame preceding a decode chunk, or ends early at a chunk other than the last. Two cameras resolving the same canonical timestamp filename raise it as well, as does one camera’s canonical filename colliding with another camera’s parsed feather.
OSError – If any directory under the raw behavior data directory cannot be read while the camera manifest and the log archives are located.
FileNotFoundError – If the camera manifest is missing, or if the job_id selects a timestamp-parsing job whose camera has no log archive.
RuntimeError – If the host is macOS and carries no loadable OpenMP runtime for the Numba threading layer.

Return type:

None

sollertia_forgery.video.video_job_prerequisites(session, universe)¶

Returns the intra-pipeline job ordering for the video pipeline.

Notes

The rename job hardlinks the parsed timestamp feathers under their canonical names, so it must run after the timestamp parse jobs that write them and depends on every timestamp job present in the given set. The tracking and energy jobs read only their own inputs and have no upstream dependency. Passing the possible subset scopes the rename job to the timestamp jobs that can actually produce feathers, while passing the full universe scopes it to all registered cameras.

Parameters:

session (SessionData) – The loaded session, accepted for the shared dispatch contract and not read by this ordering.
universe (list[tuple[str, str]]) – The job set that the ordering covers, as returned by discover_video_jobs (either the universe or its possible subset).

Return type:

dict[tuple[str, str], tuple[tuple[str, str], ...]]

Returns:

A mapping of each job to its tuple of prerequisite jobs. The rename job maps to the timestamp jobs in the set, and every other job maps to an empty tuple.

sollertia_forgery.video.motion_energy.MINIMUM_CHUNK_FRAMES: int = 4000¶: The smallest frame count a parallel decode chunk is allowed to cover. Seeking into a chunk decodes from the preceding keyframe, so each chunk discards up to one group of frames’ worth of decoded frames. The floor sits high enough relative to the keyframe interval to keep those discarded frames a small fraction of the chunk. A recording shorter than this threshold decodes in the job’s own process and opens no pool at all.

sollertia_forgery.video.pipeline.ENERGY_JOB_NAME: str = 'motion_energy'¶: The job name used to identify a single camera’s motion-energy job in the video processing tracker. The job uses the camera’s source ID as its specifier, so each camera’s recording is measured as an independently schedulable job, mirroring the per-camera timestamp parsing job. It reads only that camera’s recording, so it is independent of every other job.

sollertia_forgery.video.pipeline.RENAME_JOB_NAME: str = 'camera_timestamp_rename'¶: The job name used to identify the single timestamp renaming job in the video processing tracker. The job uses an empty specifier because it publishes every camera’s parsed feather in one pass. It is the pipeline’s only job that depends on another: it links the feathers that the per-camera parse jobs write, so it must run after them.

sollertia_forgery.video.pipeline.TRACKING_JOB_NAME: str = 'pose_tracking'¶: The job name used to identify the single video-tracking job in the video processing tracker. The job uses an empty specifier because the acquisition system’s donated function performs all of that session’s tracking in one pass. It reads only the raw pose predictions, so it is independent of the timestamp and motion-energy jobs that share its tracker.

ataraxis_video_system.orchestration.jobs.CAMERA_EXTRACTION_JOB_NAME: str = 'camera_timestamp_extraction'¶

The job name under which camera timestamp extraction is registered in a ProcessingTracker.

Notes

The value is hashed into every persisted job identifier, so changing the string invalidates every identifier a tracker already holds and every identifier a scheduler derived independently.

Two-Photon Processing¶

Provides the single-recording two-photon (calcium-imaging) processing pipeline based on cindra.

class sollertia_forgery.two_photon.SingleRecordingJobNames(*values)¶

Bases: StrEnum

Defines the job names for the single-recording processing pipeline components.

Notes

The members are declared in execution order, and that order is rendered into the error messages that list the valid job names. The authoritative phase order and prerequisite graph live in SINGLE_RECORDING_PHASES.

BINARIZE = 'binarization'¶: The name for the binarization (step 1) processing job.

COMBINE = 'combination'¶: The name for the combination (step 4) processing job.

PROCESS = 'processing'¶: The generic name for the plane-processing (step 3) job, which discovers ROIs and extracts their fluorescence. During runtime, the processed plane is identified by the tracker’s specifier field using the format ‘plane_{plane_index}’.

REGISTER = 'registration'¶: The generic name for the plane-registration (step 2) job, which removes motion and computes the registration-quality principal components. During runtime, the registered plane is identified by the tracker’s specifier field using the format ‘plane_{plane_index}’.

sollertia_forgery.two_photon.discover_two_photon_jobs(session_path)¶

Resolves the two-photon pipeline’s job universe and possible subset for the target session.

Notes

cindra owns this pipeline’s job model, so the universe is the single binarization job, one registration job and one processing job per virtual imaging plane, and the single combination job. The virtual-plane count is a property of the recording’s acquisition parameters (ROI x physical plane for MROI data). It is read from the copy the bootstrap wrote, falling back to the raw parameters file when the session has yet to be primed.

Possibility here states what this session can run rather than what it has already produced, so every stage that the recording declares is possible and the possible subset equals the universe. A freshly acquired session therefore reports its whole four-stage universe, and the tracker and the prerequisite graph decide when each stage’s turn comes rather than the output tree’s current contents.

Every resolver reads the acquisition parameters and stats the output tree, so resolving a session’s jobs costs the same no matter how often it is requested.

Parameters:

session_path (Path) – The path to the root session directory containing the session data hierarchy.

Return type:

tuple[SessionData, list[tuple[str, str]], list[tuple[str, str]]]

Returns:

A tuple of the loaded session, the job universe as a list of (job_name, specifier) pairs, and the possible subset, which equals the universe. Registration and processing specifiers come from cindra’s plane specifiers and the binarization and combination specifiers are empty.

Raises:

FileNotFoundError – If neither the session’s cindra output directory nor its raw two-photon imaging directory carries the acquisition parameters from which the recording’s imaging planes follow.
ValueError – If the session’s acquisition system is not a supported AcquisitionSystems member.

sollertia_forgery.two_photon.prime_two_photon_recording(session_path)¶

Materializes the session’s cindra configuration and per-plane bootstrap so its jobs can be discovered and run.

Notes

cindra requires the shared configuration and every plane’s runtime data to be written by one single-threaded step before any job reads them, because every per-job stage loads that bootstrap rather than writing it. This is that step.

Priming is idempotent, so a session that already carries a complete bootstrap is read rather than rewritten and this call returns without touching what its jobs recorded there. That keeps repeated preparation of the same session free of writes.

Parameters:

session_path (Path) – The path to the root session directory containing the session data hierarchy.

Raises:

FileNotFoundError – If the session’s raw two-photon imaging directory or its cindra acquisition parameters file is not present, or if the acquisition system’s resolver reports missing inputs it needs to resolve the configuration.
ValueError – If the session’s acquisition system is not a supported AcquisitionSystems member, or if the acquisition system’s resolver cannot resolve a configuration for the session.
OSError – If any directory under the session’s raw two-photon imaging directory cannot be read while searching for the acquisition parameters file.
RuntimeError – If the host is macOS and carries no loadable OpenMP runtime for the Numba threading layer.

Return type:

None

sollertia_forgery.two_photon.run_two_photon_processing_pipeline(session_path, job_id=None, *, binarize=False, register=False, process=False, combine=False, target_plane=-1, workers=-1, display_progress=False)¶

Materializes a session-bound cindra configuration and runs the single-recording two-photon processing pipeline.

Resolves the session’s raw imaging directory (cindra input) through the two-photon data registry and its processed-data root (cindra output) from the session hierarchy. Obtains the cindra single-recording configuration from the acquisition system’s donated resolver, overrides its data path, output path, and progress flag, and materializes the result in the session’s cindra directory. Then owns the two-photon processing tracker and dispatches each stage to cindra as a single tracked job.

Notes

The pipeline runs the single binarization job, one registration job and one processing job per virtual imaging plane, and the single combination job, all sharing one tracker. The virtual-plane count is a property of the recording’s acquisition parameters on disk. In local mode this invocation primes the recording, writing the shared bootstrap that every per-job stage reads and reporting the plane count back. In remote mode it only loads the bootstrap primed by the preparation step and fails when that bootstrap is absent. A job that primed alongside its peers would otherwise overwrite each peer plane’s runtime data with its own stale snapshot. The full four-stage universe defines tracker alignment. A partial invocation therefore keeps every sibling job in the shared tracker.

cindra records each dispatched job’s start, completion, and failure directly on this tracker, which lives in the session’s cindra output directory (session.processed_data.two_photon_tracker_path). Two runtimes select which jobs execute. In local mode (job_id is None) the requested stages run in binarization to registration to processing to combination order, and every stage runs when no flag is set. The registration and processing stages honor target_plane to narrow the pass to one plane. In remote mode (a job_id is provided) only the single job matching that identifier runs, so the stage flags and target_plane are ignored.

The worker count reaches cindra as a call argument, so it applies to every stage this invocation dispatches rather than being baked into the configuration.

Parameters:

session_path (Path) – The path to the root session directory containing the session data hierarchy.
job_id (str | None, default: None) – The unique hexadecimal identifier for the cindra job to execute. If provided, only the matching job runs (remote mode). Otherwise, every requested stage runs (local mode).
binarize (bool, default: False) – Determines whether to run the binarization stage. Ignored in remote mode.
register (bool, default: False) – Determines whether to run the per-plane motion-correction stage. Ignored in remote mode.
process (bool, default: False) – Determines whether to run the per-plane ROI-detection and trace-extraction stage. Ignored in remote mode.
combine (bool, default: False) – Determines whether to run the multi-plane combination stage. Ignored in remote mode.
target_plane (int, default: -1) – The imaging plane for which to run the per-plane stages. Set to -1 to cover all planes. Ignored in remote mode, where the job to run is selected entirely by job_id.
workers (int, default: -1) – The number of workers cindra allocates to each dispatched stage. Set to -1 to accept cindra’s measured default for the stage it runs.
display_progress (bool, default: False) – Determines whether to display progress bars during processing.

Raises:

FileNotFoundError – If the session’s raw two-photon imaging directory does not exist, if no cindra acquisition parameters file is available for the recording, or if the acquisition system’s resolver reports missing inputs it needs to resolve the configuration. In remote mode, also raised when the session carries no materialized cindra configuration and no per-plane bootstrap.
ValueError – If the session’s acquisition system is not a supported AcquisitionSystems member, if the acquisition system’s resolver cannot resolve a configuration for the session, or if job_id does not match any available job.
OSError – If any directory under the session’s raw two-photon imaging directory cannot be read while searching for the acquisition parameters file.
RuntimeError – If the host is macOS and carries no loadable OpenMP runtime for the Numba threading layer.

Return type:

None

sollertia_forgery.two_photon.two_photon_job_prerequisites(session, universe)¶

Returns the intra-pipeline job ordering for the two-photon pipeline.

Notes

cindra publishes the ordering as SINGLE_RECORDING_PHASES, and this call reads the chain from that model. Registration and processing are both per-plane, so a processing job waits on the registration job for its own plane alone. One plane therefore reaches detection while another is still being registered.

Parameters:

session (SessionData) – The loaded session, accepted for the shared dispatch contract and not read by this ordering.
universe (list[tuple[str, str]]) – The job universe as returned by discover_two_photon_jobs. cindra draws each prerequisite from the jobs it is handed, so this must be the full universe.

Return type:

dict[tuple[str, str], tuple[tuple[str, str], ...]]

Returns:

A mapping of each job to its tuple of prerequisite jobs, following the binarization to registration to processing to combination chain.

Job Orchestration¶

Provides the orchestration layer that plans, prepares, dispatches, and closes a batch of processing jobs.

class sollertia_forgery.orchestration.AllocationResolution(batch_id, submission, scheduler_state, claim_state, tracker, verdict, remediation)¶

Bases: object

Resolves one recorded allocation to the verdict its scheduler state and its tracker state together carry.

batch_id: str¶: The identifier of the ledger batch that recorded this allocation.

claim_state: str¶: The state in which those same records place the allocation this job’s tracker claims, or empty when it claims none.

remediation: str¶: The remediation that verdict prescribes.

scheduler_state: str¶: The state in which the scheduler’s records place the recorded allocation.

submission: RemoteSubmission¶: The recorded submission, naming both the job and the allocation that carries it.

tracker: TrackerClaim¶: What the job’s own tracker recorded, which is the record that gates rerunning it.

verdict: str¶: The verdict to which the two states resolve.

class sollertia_forgery.orchestration.ExecutionHost(*args, **kwargs)¶

Bases: Protocol

Declares the operations that preparation runs against the host that holds a project’s data.

Notes

Every implementation runs the same underlying functions, so the artifacts that a caller reads describe the same project state either way. A local host calls them in this process and a remote host runs the command line that calls them on the server, so one preparation path serves both.

One operation differs by design. resolve_tracker_paths yields an empty mapping on a remote host, because a remotely dispatched job’s trackers are read and written on the server alone.

clean(pipeline, unit_paths)¶

Removes a pipeline’s output and tracker for the named units, reporting what each removal freed.

Return type:: list[dict[str, Any]]

define_dataset(project_root, dataset_name, session_names, recreate_animals, *, force_recreate)¶

Builds the dataset hierarchy against which a forging batch is resolved.

Return type:: None

fetch(path, destination)¶

Delivers a stored artifact to this machine durably, returning where it landed, or None when it is absent.

Return type:: Path | None

generate_state(project_root, unit_paths, unit_kind)¶

Rewrites the artifacts recording every tracked job’s current status.

Return type:: None

property label: str¶: Returns the name under which this host is reported.

materialize(project_root, unit_paths, unit_kind, *, replan)¶

Rewrites every artifact from which a batch is resolved, for the named units and their project.

Return type:: None

plan(project_root, unit_paths, unit_kind, *, replan)¶

Records what the named units’ jobs will cost and projects every plan cache into the table that ships.

Return type:: list[dict[str, Any]]

read_rows(path)¶

Reads a stored table into plain rows, returning an empty list when the host holds no such table.

Return type:: list[dict[str, Any]]

reset_jobs(pipeline, job_ids_by_unit)¶

Resets each unit’s own tracked jobs to the scheduled state, in one operation.

Return type:: None

resolve_tracker_paths(pipeline, unit_paths)¶

Resolves where each unit’s pipeline tracker sits, for a caller that will open it directly.

Return type:: dict[str, str]

class sollertia_forgery.orchestration.GenericPendingJob(tracker_path, job_id, unit_path=<factory>, job_name='', core_weight=1, memory_mb=0, resident_mb=0, prerequisite_ids=(), pipeline='', name='', specifier='', project_root=None, status='', executor_id='', options=<factory>)¶

Bases: PendingJob

Describes a single batch processing job for the system-agnostic processing tools.

Notes

Extends the shared PendingJob base with the descriptor set every registered pipeline worker needs, so one descriptor serves every pipeline. The shared worker routes on pipeline, every worker reads unit_path, and a multi-job pipeline’s worker reads job_id as well. Fields that a pipeline does not use stay at their defaults, and a descriptor missing a field the engine requires is rejected before dispatch.

executor_id: str¶: The executor the same record named, which is a scheme-tagged identifier such as a scheduler allocation. Carried so reconciliation resolves a running job’s allocation from the batch rather than from a tracker it would have to open, which on a remote host would mean reading one across the transport mid-run.

name: str¶: The human-readable unit name used for logging and status reporting.

options: dict[str, Any]¶: The pipeline-specific parameters the caller chose for this job, such as the mode in which a multi-mode pipeline runs. The mapping is carried through to the pipeline’s own worker, which interprets whichever keys it declares. A pipeline that takes no parameters leaves it empty.

pipeline: str¶: The pipeline that owns this job. The shared worker routes on it, so one pool serves every pipeline.

project_root: Path | None¶: The project root directory, which stays unset while every worker resolves its output location from unit_path. A pipeline whose worker needs the root above that path reads it from here.

specifier: str¶: The specifier that differentiates jobs of the same type within one unit, such as a camera or controller source identifier, a controller-module triple, or a plane index.

status: str¶: The status this job’s tracker recorded when preparation last regenerated the host’s state artifact, named by its ProcessingStatus member. Empty for a descriptor built without one.

class sollertia_forgery.orchestration.JobExecutionState(*, worker, all_jobs=<factory>, pending_jobs=<factory>, active_jobs=<factory>, core_budget=1, memory_budget_mb=1024, concurrency_limits=<factory>, concurrency_reservations=<factory>, dispatch_priorities=<factory>, pool_size=1, thread_ceiling=1, succeeded_job_keys=<factory>, failed_job_keys=<factory>, blocked_jobs=<factory>, lock=<factory>, manager_thread=None, canceled=False)¶

Bases: Generic

Tracks runtime state for one batch execution session budgeted by both cores and memory.

The manager owns one ProcessPoolExecutor and admits each pending job once the running set has room for both its cores and its memory.

Notes

Subclasses of PendingJob carry the fields a worker callable needs at dispatch time, such as the path of the unit the job processes.

active_jobs: list[_ActiveJob]¶: Jobs currently executing on the shared process pool.

all_jobs: dict[tuple[str, str], PendingJobT]¶: All submitted jobs keyed by (unit_path, job_id) dispatch key.

blocked_jobs: list[PendingJobT]¶: Jobs dropped without dispatch because a prerequisite failed or never ran.

canceled: bool¶: Determines whether the execution session has been canceled.

concurrency_limits: dict[str, int]¶: The jobs of each type that may run at once, keyed by tracker job name, for the types that declare a ceiling beyond the two budgets. This ceiling holds however much capacity is idle, since a type recorded here waits on a resource that spare cores and spare memory do not supply. A job type absent from this mapping is bounded by the budgets alone.

concurrency_reservations: dict[str, int]¶: The jobs of each type that run at once while other work can still use the capacity the type gives up, keyed by tracker job name. Admission offers that capacity to every other runnable job first and then releases the reservation over whatever remains, so a reserved type widens rather than idling the host.

core_budget: int¶: The cores the batch may commit across all concurrently running jobs.

dispatch_priorities: dict[tuple[str, str], int]¶: The cores each job’s transitive dependents commit, keyed by dispatch key, which is the weight by which admission orders candidates. Resolved once from the job set when the manager starts, since the batch’s dependency graph does not change while it runs.

failed_job_keys: set[tuple[str, str]]¶: The dispatch keys of the jobs known to have failed, whose dependents can never become runnable.

lock: lock¶: The lock guarding every mutation of the job queues and the recorded outcomes.

manager_thread: Thread | None¶: The background thread running the execution manager, or None before the session starts it.

memory_budget_mb: int¶: The memory the batch may commit across all concurrently running jobs.

pending_jobs: deque¶: Jobs awaiting dispatch, held in the order the next admission pass considers them.

pool_size: int¶: The number of worker processes the pool spawns.

succeeded_job_keys: set[tuple[str, str]]¶: The dispatch keys of the jobs known to have succeeded, seeded from the trackers and extended as jobs finish.

thread_ceiling: int¶: The thread count to which each worker pins its library thread pools.

worker: Callable[[PendingJobT], None]¶: The picklable module-level function invoked by ProcessPoolExecutor.submit for each pending job.

class sollertia_forgery.orchestration.LocalHost¶

Bases: object

Runs a project’s preparation steps in this process, against data this machine holds.

Notes

Calls the same functions the command line exposes, so a locally prepared batch and a remotely prepared one are resolved from artifacts written by identical code.

static clean(pipeline, unit_paths)¶

Removes a pipeline’s output and tracker for the named units.

Parameters:

pipeline (str) – The pipeline whose output to remove.
unit_paths (Sequence[Path]) – The processing units to clean.

Return type:

list[dict[str, Any]]

Returns:

One entry per removed path, carrying the path and the removed_bytes it held.

static define_dataset(project_root, dataset_name, session_names, recreate_animals, *, force_recreate)¶

Builds the dataset hierarchy against which a forging batch is resolved.

Parameters:

project_root (Path) – The path to the project holding the sessions and the dataset.
dataset_name (str) – The name of the dataset to build.
session_names (Sequence[str]) – The sessions to admit into the dataset.
recreate_animals (Sequence[str]) – The animals to rebuild from the named sessions.
force_recreate (bool) – Determines whether to rebuild the whole hierarchy from the named sessions.

Raises:

ValueError – If the arguments contradict each other, or if the dataset’s acquisition system is unknown.
FileNotFoundError – If a named session resolves to no directory under the project root, or if the acquisition system’s resolver reports a missing input it needs for an animal.
RuntimeError – If a named session resolves to more than one directory under the project root.

Return type:

None

static fetch(path, destination)¶

Reports where a stored artifact already sits, since this machine holds it.

Notes

The artifact is already durable on this machine, so nothing is copied and the destination is ignored. A caller recording where a snapshot landed therefore records the artifact’s own location.

Parameters:

path (Path) – The path to the artifact.
destination (Path) – The directory in which a copy would land, unused here.

Return type:

Path | None

Returns:

The artifact’s own path, or None when it is absent.

static generate_state(project_root, unit_paths, unit_kind)¶

Rewrites the artifacts recording every tracked job’s current status.

Notes

A session batch reads the project manifest’s walk, which writes the project job artifact alongside it, while a dataset batch reads each named dataset’s own state artifact.

Parameters:

project_root (Path) – The path to the project’s root directory.
unit_paths (Sequence[Path]) – The unit root directories the batch covers.
unit_kind (str) – Whether the units are sessions or datasets.

Return type:

None

property label: str¶: Returns the name under which this host is reported.

classmethod materialize(project_root, unit_paths, unit_kind, *, replan)¶

Rewrites every artifact from which a batch is resolved, for the named units and their project.

Notes

Three steps run in a fixed order. Planning each unit records what its jobs will cost and registers them on their processing trackers. Projecting the project’s plan caches gathers those figures into the single table that ships. Refreshing the state serializes what the trackers now record.

The order matters, because planning is what registers a unit’s jobs and the state step reads those registries. Planning also re-estimates nothing a unit’s cache already holds unless a caller asks for it, so a submission’s sizing figures never change underneath it.

A unit for which no pipeline resolves a job is reported and skipped, leaving the units that did resolve jobs planned.

Parameters:

project_root (Path) – The path to the project’s root directory.
unit_paths (Sequence[Path]) – The unit root directories the batch covers.
unit_kind (str) – Whether the units are sessions or datasets.
replan (bool) – Determines whether to re-estimate the figures a cache already holds.

Return type:

None

static plan(project_root, unit_paths, unit_kind, *, replan)¶

Records what the named units’ jobs will cost and projects every plan cache into the table that ships.

Notes

Estimation reads each unit’s raw acquisition data, so it costs far more than reading the cache it writes. Only the jobs that a unit’s cache does not already hold are estimated unless a caller asks for the recorded figures to be replaced, so a submission’s sizing figures never move underneath it.

A unit for which no pipeline resolves a job is reported in its own entry rather than aborting the others. A unit is planned without any job that the sizing pass refuses. Each refusal’s reason is reported beside the figures the unit did plan.

Parameters:

project_root (Path) – The path to the project’s root directory.
unit_paths (Sequence[Path]) – The unit root directories to plan.
unit_kind (str) – Whether the units are sessions or datasets.
replan (bool) – Determines whether to re-estimate the figures a cache already holds.

Return type:

list[dict[str, Any]]

Returns:

One entry per named unit, carrying its unit_path, unit_name, job_count, and summed_memory_mb, and summed_resident_mb, or its unit_path, a job_count of zero, and the error that stopped it. An entry whose plan recorded a sizing refusal also carries unsized_jobs, mapping each refusal to the reason it gave. A refusal raised for one job is keyed by its pipeline and job, and a refusal that ended a pipeline’s one-pass sizing is keyed by that pipeline and all jobs.

static read_rows(path)¶

Reads a stored table into plain rows.

Parameters:: path (Path) – The path to the table.
Return type:: list[dict[str, Any]]
Returns:: The table’s rows, or an empty list when the table is absent.

static reset_jobs(pipeline, job_ids_by_unit)¶

Resets each unit’s own tracked jobs to the scheduled state.

Notes

The trackers sit on this machine, so the reset runs in this process rather than over a command line.

Each unit is reset against its own identifiers alone. A job identifier carries no unit, so two units of one project share the identifier of the same stage, and a flat set applied to both would clear a record the caller never named.

Parameters:

pipeline (str) – The pipeline whose jobs to reset.
job_ids_by_unit (Mapping[Path, Sequence[str]]) – The identifiers to reset, keyed by the processing unit that records them. A unit mapped to an empty sequence has every job it tracks reset.

Return type:

None

static resolve_tracker_paths(pipeline, unit_paths)¶

Resolves where each unit’s pipeline tracker sits on this machine.

Notes

The local execution engine opens these files directly. It reads the recorded outcomes to decide which prerequisites are satisfied, and re-reads them to report a running batch’s status. A batch dispatched here therefore carries the locations on its descriptors, and a unit that cannot be loaded contributes no entry.

Parameters:

pipeline (str) – The pipeline whose tracker to locate.
unit_paths (Sequence[Path]) – The unit root directories whose trackers to locate.

Return type:

dict[str, str]

Returns:

The tracker path of each unit, keyed by the unit path as a string.

class sollertia_forgery.orchestration.RemoteHost(server)¶

Bases: object

Runs a project’s preparation steps on the remote compute server and copies its artifacts home.

Notes

Each operation issues the command line that calls the same function a local host calls in-process, so the two hosts write the same artifacts from the same code. Reading a table copies it into a temporary directory and parses it here, because the graph that a batch dispatches is always built on this machine.

Parameters:: server (Server) – The connected compute server holding the project.

_server¶: The connected server through which every operation is issued.

clean(pipeline, unit_paths)¶

Removes a pipeline’s output and tracker for the named units, on the server.

Notes

The command reports each removed path with the bytes it held, so a remote cleanup returns the same figures a local one does.

Parameters:

pipeline (str) – The pipeline whose output to remove.
unit_paths (Sequence[Path]) – The paths, on the server, to the processing units to clean.

Return type:

list[dict[str, Any]]

Returns:

One entry per removed path, carrying the path and the removed_bytes it held.

Raises:

RuntimeError – If the server-side command fails.

define_dataset(project_root, dataset_name, session_names, recreate_animals, *, force_recreate)¶

Builds the dataset hierarchy against which a forging batch is resolved, on the server.

Notes

The invocation calls define_forging_dataset inside the server’s processing environment, because the forging command couples the definition to the tracked forging jobs that follow it. A remote definition therefore performs the same work its local counterpart does, and the resulting jobs are dispatched by a prepared batch.

Parameters:

project_root (Path) – The path, on the server, to the project holding the sessions and the dataset.
dataset_name (str) – The name of the dataset to build.
session_names (Sequence[str]) – The sessions to admit into the dataset.
recreate_animals (Sequence[str]) – The animals to rebuild from the named sessions.
force_recreate (bool) – Determines whether to rebuild the whole hierarchy from the named sessions.

Raises:

RuntimeError – If the server-side command fails.

Return type:

None

fetch(path, destination)¶

Copies a stored artifact off the server so this machine keeps it.

Notes

The copy stays in place, so a snapshot taken at a run’s closure survives the server regenerating its own artifacts afterward.

Parameters:

path (Path) – The path to the artifact on the server.
destination (Path) – The local directory in which the copy lands.

Return type:

Path | None

Returns:

The local path at which the copy landed, or None when the server holds no such artifact.

generate_state(project_root, unit_paths, unit_kind)¶

Rewrites the artifacts recording every tracked job’s current status.

Parameters:

project_root (Path) – The path to the project’s root directory on the server.
unit_paths (Sequence[Path]) – The unit root directories the batch covers.
unit_kind (str) – Whether the units are sessions or datasets.

Raises:

RuntimeError – If the server-side command fails.

Return type:

None

property label: str¶: Returns the name under which this host is reported.

materialize(project_root, unit_paths, unit_kind, *, replan)¶

Rewrites every artifact from which a batch is resolved, for the named units and their project.

Notes

The steps ship as one invocation, chained so each runs only after the one before it succeeded. Naming no unit drops the per-unit planning step and ships the projection and the state refresh alone. Chaining costs a single round trip, and it holds the order the steps require, since planning registers the jobs the state step reads.

Parameters:

project_root (Path) – The path to the project’s root directory on the server.
unit_paths (Sequence[Path]) – The unit root directories the batch covers.
unit_kind (str) – Whether the units are sessions or datasets.
replan (bool) – Determines whether to re-estimate the figures a cache already holds.

Raises:

RuntimeError – If the server-side commands fail.

Return type:

None

plan(project_root, unit_paths, unit_kind, *, replan)¶

Records what the named units’ jobs will cost and projects every plan cache into the table that ships.

Notes

The per-unit figures are read back out of the projection rather than parsed from the command’s output, so a remote plan reports the same numbers a local one returns. A unit for which the projection holds no row planned nothing.

Parameters:

project_root (Path) – The path to the project’s root directory on the server.
unit_paths (Sequence[Path]) – The unit root directories to plan.
unit_kind (str) – Whether the units are sessions or datasets.
replan (bool) – Determines whether to re-estimate the figures a cache already holds.

Return type:

list[dict[str, Any]]

Returns:

Raises:

RuntimeError – If the server-side commands fail.

read_rows(path)¶

Copies a stored table home and reads it into plain rows.

Parameters:: path (Path) – The path to the table on the server.
Return type:: list[dict[str, Any]]
Returns:: The table’s rows, or an empty list when the server holds no such table.

reset_jobs(pipeline, job_ids_by_unit)¶

Resets each unit’s own tracked jobs to the scheduled state on the server.

Notes

One invocation is issued per unit, carrying that unit’s identifiers alone. A job identifier carries no unit, so two units of one project share the identifier of the same stage, and a flat set applied to both would clear a record the caller never named.

The invocations are chained into one round trip, so a batch spanning many units still costs a single connection.

Parameters:

pipeline (str) – The pipeline whose jobs to reset.
job_ids_by_unit (Mapping[Path, Sequence[str]]) – The identifiers to reset, keyed by the path, on the server, to the processing unit that records them. A unit mapped to an empty sequence has every job it tracks reset.

Raises:

RuntimeError – If the server-side command fails.

Return type:

None

static resolve_tracker_paths(pipeline, unit_paths)¶

Returns an empty mapping, because this host’s trackers are read and written on the server alone.

Notes

A remotely dispatched job records its own outcome on the server, its reset runs there over a command line, and its progress is read from the scheduler. Nothing on this machine opens one of these trackers, so recording a path this machine cannot read would only mislead.

Parameters:

pipeline (str) – The pipeline whose tracker would be located.
unit_paths (Sequence[Path]) – The unit root directories that would be located.

Return type:

dict[str, str]

Returns:

An empty mapping.

property server: Server¶: Returns the connected server through which this host operates.

class sollertia_forgery.orchestration.SchedulerReading(statuses=<factory>, queued=frozenset({}), canceled=frozenset({}), unreadable_reason='')¶

Bases: object

Records what the scheduler’s two records reported about one set of allocations.

Notes

Accounting says what the scheduler has committed and the queue says what it holds right now. Neither answers for the other, so an allocation is resolved against both and a source that did not answer withholds nothing but certainty.

canceled: frozenset[str]¶: The allocations this process has since canceled. The scheduler applies a cancellation to the queued and running allocations alike, so a canceled allocation is the scheduler’s no longer whatever the reading that preceded the cancellation said.

canceling(allocations)¶

Returns this reading with the named allocations recorded as canceled.

Notes

A cancellation is issued rather than observed, since the scheduler applies it asynchronously and an accounting query issued straight afterwards may still report the allocation as running. Recording the cancellation here is what lets the verdicts be resolved again from what this process actually did.

Parameters:: allocations (Sequence[str]) – The allocations the cancellation named.
Return type:: SchedulerReading
Returns:: The reading, which resolves every named allocation as settled.

queued: frozenset[str]¶: The allocations the queue currently holds, which is empty for a user whose queue holds none of them.

resolve_state(allocation)¶

Resolves one allocation to the state in which the scheduler’s two records place it.

Notes

The invariant this holds is that an allocation’s state follows from what each record says about it and never from the order in which the two were read. Accounting and the queue are read one after the other rather than as one snapshot, so they answer for different moments and can disagree about the same allocation. A resolution that let the earlier read win would answer differently on two reads of one unchanged batch.

The BLOCKED state is where that disagreement bites, and it is why the accounting map is consulted for it ahead of the queue. Accounting calls such an allocation pending, so the state is derived from the queue’s own reason field. Every allocation carrying it was therefore in the queue when accounting was read, while the later queue snapshot may already have released it. A dependency that can never be satisfied is permanent. The allocation never runs, so it never writes to its job’s tracker and nothing it holds can change again. That is the settled state, and resolving it here answers the same way on whichever side of the queue’s own release of the allocation the two reads fall. Deciding it from the queue instead would hold the allocation on one read and settle it on the next, leaving its batch reported as progressing for as long as the queue kept a job that will never start.

Parameters:: allocation (str) – The scheduler identifier to resolve, or an empty string for a record naming no allocation.
Return type:: str
Returns:: One of held, settled, or gone.

resolve_status(allocation)¶

Returns the state accounting reported for one allocation, which is UNRESOLVED when it reported none.

Parameters:: allocation (str) – The scheduler identifier whose accounting state to report.
Return type:: JobStatus
Returns:: The reported state.

statuses: dict[str, JobStatus]¶: The state accounting reported for each queried allocation, keyed by its scheduler identifier.

unreadable_reason: str¶: What stopped one of the scheduler’s records from being read, or empty when both answered. Every allocation this reading has neither canceled nor found permanently blocked resolves as held while this is set, because a record that did not answer is no evidence that the scheduler has finished with an allocation.

class sollertia_forgery.orchestration.SubmissionBatch(batch_id='', batch_ids=<factory>, batch_directory='', submitted_at=0, walltime_minutes=0, submissions=<factory>)¶

Bases: object

Records one submitted batch and every allocation it holds.

batch_directory: str¶: The path, on the server, to the directory holding this batch’s scripts and logs.

batch_id: str¶: The identifier the preparation issued, which also names the batch’s directory on the server.

batch_ids: list[str]¶: Every prepared batch this submission dispatched, since one submission may span several. An empty list covers the single batch that batch_id names.

property covered_batch_ids: list[str]¶: Returns every prepared batch this submission dispatched.

property pipelines: list[str]¶: Returns the pipelines whose jobs this batch holds.

submissions: list[RemoteSubmission]¶: The allocations this batch holds, in the order they were accepted.

submitted_at: int¶: The UTC timestamp (microsecond-precision epoch) at which the batch was submitted.

walltime_minutes: int¶: The wall-time every allocation of this batch requested.

class sollertia_forgery.orchestration.SubmissionLedger(batches=<factory>)¶

Bases: YamlConfig

Records every batch this host submitted to the remote compute server’s scheduler.

batches: list[SubmissionBatch]¶: The recorded batches, in the order they were last recorded.

resolve_batch(batch_id)¶

Returns the recorded batch with the given identifier, or None when the ledger holds no such batch.

Parameters:: batch_id (str) – The identifier of the batch to resolve.
Return type:: SubmissionBatch | None
Returns:: The recorded batch, or None.

sollertia_forgery.orchestration.batch_directory()¶

Returns the directory holding every prepared batch this host has recorded.

This is the single source of the directory’s location, so every reader and writer derives the same path.

Return type:: Path
Returns:: The path to the prepared-batch directory under the Sollertia platform working directory.

sollertia_forgery.orchestration.build_pending_job(job)¶

Builds one job descriptor into the pending job that both backends dispatch.

Parameters:: job (dict[str, Any]) – A job descriptor carrying job_id, unit_path, cores, memory_mb, and resident_mb, and optionally tracker_path, job_name, unit_name, specifier, pipeline, prerequisite_ids, options, status, and executor_id.
Return type:: GenericPendingJob
Returns:: The pending job.
Raises:: KeyError – If the descriptor omits a field the engine requires.

sollertia_forgery.orchestration.cancel_allocations(server, allocations)¶

Cancels every named allocation in one call, which the scheduler applies to the queued and running ones alone.

Notes

The allocations are named rather than resolved from the ledger, because the allocation running a job is not always the one this host recorded for it. A job whose tracker claims another submitter’s allocation is reached through this call, since the ledger holds no record naming that allocation.

Parameters:

server (Server) – The connected server that runs the allocations.
allocations (Sequence[str]) – The scheduler identifiers to cancel.

Return type:

list[str]

Returns:

The allocation identifiers the cancellation named.

sollertia_forgery.orchestration.cancel_submissions(server, submissions)¶

Cancels every allocation the given submissions hold in one call, which the scheduler applies to the queued and running ones alone.

Parameters:

server (Server) – The connected server that runs the batch.
submissions (Sequence[RemoteSubmission]) – The submissions to cancel.

Return type:

list[str]

Returns:

The allocation identifiers the cancellation named.

sollertia_forgery.orchestration.classify_batch(resolutions)¶

Resolves one batch’s verdict from the verdicts of the allocations it holds.

Notes

The verdict rests on the states the batch’s allocations hold rather than on how long the batch has been outstanding, so nothing has to be tuned and a slow run is never mistaken for a stopped one.

A batch every allocation of which has settled is awaiting closure rather than progressing or stalled, since the read that observes a batch resolving entirely to the plain drop also closes it. Such a batch is outstanding only because it holds an entry that closure may not drop or because that closure failed. A batch holding no allocation at all resolves the same way, since there is nothing left for the scheduler to advance.

Parameters:: resolutions (Sequence[AllocationResolution]) – The resolutions of the allocations the batch holds.
Return type:: str
Returns:: One of progressing, stalled, or awaiting_closure.

sollertia_forgery.orchestration.clean_pipeline_output(pipeline, unit_paths)¶

Removes a pipeline’s output and processing tracker for the named units.

Notes

Removes the directory the pipeline owns outright alongside its tracker, so a later preparation rediscovers every job from the acquired data rather than resuming a partial run. A pipeline that writes into a directory it shares with the acquired data owns none, so cleaning it removes its tracker alone and leaves the inputs in place.

A pipeline whose stages also write outside the unit declares those directories separately, and a cleanup removes them on the same terms. Each of them names the unit being cleaned, so a tree that holds the directories of several units keeps the ones the other units own.

The unit is loaded rather than discovered, since a cleanup needs the unit’s own locations and a unit whose pipeline never ran still has output to remove. A unit that cannot be loaded is reported and skipped, leaving the others cleaned.

Parameters:

pipeline (str) – The pipeline whose output to remove.
unit_paths (Sequence[Path]) – The processing units to clean.

Return type:

list[dict[str, Any]]

Returns:

One entry per removed path, carrying the path and the removed_bytes it held. Naming a pipeline that the dispatch table does not support removes nothing.

sollertia_forgery.orchestration.close_batch(host, batch_id)¶

Snapshots what one finished batch’s jobs recorded and stores the result on the batch itself.

Notes

Rewrites the project’s artifacts on the host, reads each of the batch’s jobs out of them where the host holds them, and delivers a copy of each artifact to this machine. Regenerating first is what makes the snapshot describe the state after the run rather than the state against which the run was prepared.

The outcome is written into the batch’s own outcome file, so a finished batch stays answerable once nothing is running and nothing is queued. The prepared document is retired behind that write, because it describes work for which the outcome now answers.

Parameters:

host (ExecutionHost) – The host that holds the data the batch’s jobs read.
batch_id (str) – The identifier of the batch to close.

Return type:

_BatchOutcome | None

Returns:

The recorded outcome, or None when this host holds no batch under that identifier.

Raises:

RuntimeError – If a step fails on the host.
Timeout – If the outcome file’s lock cannot be acquired within the timeout period.

sollertia_forgery.orchestration.close_covered_batches(host, batch)¶

Snapshots what every prepared batch of one submission recorded, then retires each prepared document.

Notes

One submission may dispatch several prepared batches, and each carries its own document, so each is snapshotted separately rather than folded into the identifier that keys the ledger.

The submission’s own ledger entry is left alone. Retiring that entry is the caller’s decision, whether the caller is the closure of a settled batch or the explicit retirement of a stalled one.

Parameters:

host (ExecutionHost) – The host that holds the data the batch’s jobs read.
batch (SubmissionBatch) – The recorded submission whose prepared batches to snapshot.

Return type:

list[_BatchOutcome]

Returns:

The recorded outcomes, holding one entry per prepared batch this host still held a document for.

Raises:

RuntimeError – If a step fails on the host.
Timeout – If an outcome file’s lock cannot be acquired within the timeout period.

sollertia_forgery.orchestration.close_settled_batches(host, batches, resolutions)¶

Closes every batch whose resolved allocations all prescribe a plain drop, then retires the ones that closed.

Notes

A batch closes here exactly when every entry the resolution holds for it prescribes drop, which is the one remediation that leaves every tracker as it stands and needs nothing canceled. That is a derivation from the caller’s own resolution rather than a second reading of the scheduler. It keeps this closure from diverging from the verdicts a caller is shown. The caller resolves once and hands the same entries to both, and nothing here consults a record of its own. An entry prescribing none holds its batch open because the work may still be live, and one prescribing a reset holds it open because a tracker still claims a run no allocation is carrying. Each of those is released by the explicit remediation rather than automatically.

A batch holding no allocation carries no entry at all, so nothing it holds prescribes anything but a drop and it closes. That is the exit such a record needs, since closure is what drops it.

Retirement is issued per batch identifier, and only for a batch whose own closure completed. A batch that fails to close therefore stays in the submission ledger however its siblings fared. The entry keeps it answerable and lets the next query try again.

Parameters:

host (ExecutionHost) – The host that holds the data the batches’ jobs read.
batches (Sequence[SubmissionBatch]) – The batches the resolution covered.
resolutions (Sequence[AllocationResolution]) – The verdict and remediation resolved for every allocation those batches hold, as resolve_allocations returned it for these same batches.

Return type:

list[_BatchOutcome]

Returns:

The outcomes of the batches that were closed. A closed batch with no prepared record on this host is retired without producing one.

sollertia_forgery.orchestration.connect_to_server()¶

Opens a connection to the configured remote compute server.

Return type:: Server
Returns:: The connected server, which the caller closes or uses as a context manager.

sollertia_forgery.orchestration.current_timestamp()¶

Returns the current UTC timestamp as a microsecond-precision epoch, matching every other timestamp in this stack.

Return type:: int

sollertia_forgery.orchestration.forget_batch_records(batch_ids)¶

Removes every file this host holds for the named batches, which are the prepared document and the recorded outcome.

Parameters:: batch_ids (list[str]) – The identifiers to remove.
Return type:: list[str]
Returns:: The identifiers this host held under either file, in the order the caller named them.

sollertia_forgery.orchestration.forget_batches(batch_ids)¶

Drops the named batches from the submission ledger, whatever state their allocations hold.

Notes

This clears the ledger alone, which is the record of the allocations this host has outstanding on the scheduler. The prepared documents and the recorded outcomes live in the separate batch registry that forget_batch_records clears.

Parameters:: batch_ids (Sequence[str]) – The identifiers of the batches to drop.
Return type:: list[str]
Returns:: The identifiers the ledger actually held and dropped.
Raises:: Timeout – If the ledger’s lock cannot be acquired within the timeout period.

sollertia_forgery.orchestration.generate_project_plan(project_directory, *, display_progress=False)¶

Projects every plan cache under a project into one table and saves it at the project root.

Notes

Reads the caches alone, so this is the cheap half of planning and the half that ships. A unit carrying no cache contributes no rows, so a reader sizing a submission against this table treats an absent job as unplanned rather than as free.

Parameters:

project_directory (Path) – The path to the project whose plan caches to project.
display_progress (bool, default: False) – Determines whether to report what the projection covered once it is written.

Return type:

Path

Returns:

Where the projection was written.

Raises:

FileNotFoundError – If the project directory does not exist, since a projection has nowhere to be written and no unit to read.
Timeout – If the projection file’s lock cannot be acquired within the timeout period.

sollertia_forgery.orchestration.group_jobs_by_tracker(state)¶

Groups all jobs in an execution state by their tracker file path.

Minimizes redundant file reads by batching jobs that share the same tracker, so each tracker YAML file is deserialized only once when iterating over the groups.

Parameters:: state (JobExecutionState[TypeVar(PendingJobT, bound= PendingJob)]) – The active job execution state containing the job registry.
Return type:: dict[Path, list[TypeVar(PendingJobT, bound= PendingJob)]]
Returns:: A dictionary mapping each tracker path to its list of pending job descriptors.

sollertia_forgery.orchestration.job_execution_manager(state)¶

Dispatches queued jobs against a shared ProcessPoolExecutor under the batch’s core and memory budgets.

Notes

Runs as a daemon thread for the lifetime of a single execution session. The loop wakes when a running job finishes, so capacity is refilled the moment it is released and a long-running batch costs nothing while its jobs run. Each pass reaps finished futures, refreshes the recorded outcomes when one finished, and admits whatever the freed capacity and the pipelines’ own orderings allow. Cancellation stops new admissions but lets running futures finish naturally.

A pass that admits nothing while nothing is running means every remaining job waits on a prerequisite that neither succeeded nor is queued. The remainder is recorded as blocked and the session ends rather than waiting forever.

Parameters:: state (JobExecutionState[TypeVar(PendingJobT, bound= PendingJob)]) – The active job execution state containing the pending queue, active jobs, worker callable, and budgets. Mutated under state.lock as jobs move between queues.
Return type:: None

sollertia_forgery.orchestration.prepare_batch(host, pipeline, unit_paths, options=None, *, replan=False)¶

Resolves a pipeline’s dispatchable jobs for the named units out of the project’s own artifacts.

Notes

Materialization runs on the host that holds the data, since estimating a job’s cost and reading a tracker both need the data itself. The resulting artifacts are then read onto this machine and the batch is built here, so the graph that a run dispatches is resolved in one place whichever host prepared it.

Planning re-estimates nothing a unit’s cache already holds unless a caller asks for it, because a submission’s sizing figures must not change underneath it. Recorded status is always refreshed, since that is the part that changes between runs.

A job that the unit cannot run never reaches its processing tracker, so its absence from the state artifact is what rules it out. A job whose upstream stage this run can neither dispatch nor find already succeeded is reported as blocked rather than dispatched.

Parameters:

host (ExecutionHost) – The host holding the units, which materializes the artifacts and delivers them here.
pipeline (str) – The batch pipeline to prepare.
unit_paths (Sequence[str]) – The processing unit directories on that host whose jobs to prepare. Every unit must belong to one project, since the artifacts from which a batch is resolved are written per project.
options (dict[str, Any] | None, default: None) – The pipeline-specific parameters given to the prepared jobs.
replan (bool, default: False) – Determines whether to re-estimate the resource figures the units’ caches already hold.

Return type:

BatchDocument

Returns:

The prepared batch document.

Raises:

ValueError – If the named pipeline is not a supported batch pipeline, if no unit is named, if the named units span more than one project, or if a named unit sits too few directory levels below its project for that project to be resolved from it.
FileNotFoundError – If the host holds no plan table for the units’ project.
RuntimeError – If a step fails on the host.

sollertia_forgery.orchestration.project_plan_path(project_directory)¶

Resolves the path to a project’s job plan projection.

Parameters:: project_directory (Path) – The path to the project’s root directory.
Return type:: Path
Returns:: The path to the project’s plan .feather file.

sollertia_forgery.orchestration.read_batch_outcome(batch_id)¶

Reads what a batch’s jobs finally recorded.

Parameters:: batch_id (str) – The identifier of the batch to read.
Return type:: dict[str, Any] | None
Returns:: The stored outcome, or None when this host holds no outcome under that identifier.

sollertia_forgery.orchestration.read_ledger()¶

Reads the submission ledger, treating an absent ledger as holding no batches.

Return type:: SubmissionLedger
Returns:: The recorded ledger.

sollertia_forgery.orchestration.read_prepared_batch(batch_id)¶

Reads one recorded batch.

Parameters:: batch_id (str) – The identifier of the batch to read.
Return type:: BatchDocument | None
Returns:: The recorded batch document, or None when this host holds no batch under that identifier.

sollertia_forgery.orchestration.read_prepared_batches(batch_ids)¶

Reads several recorded batches, reporting the identifiers this host does not hold.

Parameters:: batch_ids (list[str]) – The identifiers to read.
Return type:: tuple[list[BatchDocument], list[str]]
Returns:: A tuple of the documents that were found and the identifiers that were not, the latter sorted.

sollertia_forgery.orchestration.read_scheduler_records(server, allocations)¶

Reads what accounting and the queue each report about the named allocations.

Notes

Accounting is read first and a failure there is raised, because a failed query writes nothing to standard output and reporting that as ‘no row for anything’ would resolve every allocation as gone at once.

The queue is read second and a failure there is carried rather than raised, because the reading it leaves is still usable. Every allocation the reading has neither canceled nor found permanently blocked resolves as held, which is the reading that disturbs nothing.

Parameters:

server (Server) – The connected server that runs the allocations.
allocations (Sequence[str]) – The scheduler identifiers to read.

Return type:

SchedulerReading

Returns:

The reading, carrying accounting’s answer, the queue’s answer, and what stopped the queue when it could not be read.

Raises:

RuntimeError – If the accounting query fails.

sollertia_forgery.orchestration.reconcile_local_jobs(jobs)¶

Decides what to do with each job of a batch prepared against this machine.

Notes

Nothing else on this machine should be running a job this batch holds, so a record left in the running state describes a pool that died without recording an outcome. Every job is therefore dispatched and its record cleared.

A recorded process identifier is not checked for liveness, because a tracker on shared storage can hold a process identifier belonging to another machine and identifiers are reused.

Parameters:: jobs (Sequence[GenericPendingJob]) – The batch’s jobs.
Return type:: _Reconciliation
Returns:: The reconciliation, which adopts nothing, withholds nothing, and dispatches every job.

sollertia_forgery.orchestration.reconcile_remote_jobs(server, jobs)¶

Decides what to do with each job of a batch prepared against the remote compute server.

Notes

Every job is resolved through the same published resolution a status read and a remediation resolve, and the verdict that resolution carries is what decides the job’s fate. Nothing here reads a scheduler record on its own account, so what this dispatch adopts and what a status read reports as running cannot disagree.

Two records can show that a job already has an allocation, and they cover different windows. The submission ledger names allocations this host submitted, including ones still queued, but it knows nothing about a batch submitted from another machine. The executor identifier that a job’s tracker recorded travels with the data and so covers every submitter, but it only appears once the allocation starts running. Both therefore reach the resolution, the ledger’s as the recorded allocation of each job’s submission and the tracker’s as that job’s claim, which is the pairing the resolution already reads. The tracker’s record is taken off the batch’s own descriptors, which preparation filled from the host’s state artifact, rather than by opening a tracker across the transport while the batch runs.

A job whose verdict is running is left alone. Every allocation the resolution reports that way is one the scheduler still holds, one whose job a still-held allocation claims on its tracker, or one whose tracker claims to be running under an executor outside the scheduler. The first two name an allocation, so the job is adopted onto it and its dependents wait on that allocation rather than on a second one this run would submit. The third names none, so there is nothing to adopt and nothing on which a dependent could wait. That job is withheld with its dependents, because dispatching it would run a second copy over a tracker to which its own executor may still be writing. That is the one verdict whose remediation is to leave everything as it stands.

Every other verdict releases the job. finished, failed, and abandoned mean nothing carries it any longer, and stranded means its tracker claims a run no allocation is carrying, which is exactly the claim the dispatch’s own reset clears.

The scheduler is read only when a job carries an allocation to resolve. A batch whose jobs carry none resolves entirely from what their trackers recorded, so a routine dispatch costs no scheduler round trip at all. A queue that cannot be read is carried rather than raised, which leaves every claimed job adopted and disturbs nothing.

Parameters:

server (Server) – The connected server that runs the batch.
jobs (Sequence[GenericPendingJob]) – The batch’s jobs.

Return type:

_Reconciliation

Returns:

The reconciliation, naming which jobs to submit, which allocations to adopt, which jobs to withhold, and which records to clear.

Raises:

RuntimeError – If the accounting query fails.

sollertia_forgery.orchestration.record_prepared_batch(document)¶

Records one prepared batch and returns the identifier that resolves it.

Parameters:: document (BatchDocument) – The prepared batch to record.
Return type:: str
Returns:: The identifier under which the batch was recorded.
Raises:: Timeout – If the batch file’s lock cannot be acquired within the timeout period.

sollertia_forgery.orchestration.remote_batch_directory(server, batch_id)¶

Resolves the server-side directory holding one batch’s job scripts and logs.

Parameters:

server (Server) – The server that runs the batch.
batch_id (str) – The identifier of the batch.

Return type:

Path

Returns:

The path to the batch’s directory on the server.

sollertia_forgery.orchestration.render_allocation(resolution, reading)¶

Renders one resolved allocation as a response payload.

Parameters:

resolution (AllocationResolution) – The resolution to render.
reading (SchedulerReading) – The reading against which the resolution was resolved, which carries the state accounting reported.

Return type:

dict[str, Any]

Returns:

The submission’s own fields alongside the batch that recorded it, the state in which each scheduler record placed it, what its job’s tracker holds, the verdict, and the remediation that verdict prescribes.

sollertia_forgery.orchestration.reset_stranded_jobs(host, resolutions)¶

Returns every stranded job of the given resolutions to the scheduled state on its own tracker.

Notes

Only a stranded job is reset. A job that recorded success would lose that result, a job that recorded failure would lose the verdict its operator has yet to see, and a job that never left the scheduled state is already runnable. A write to any of those trackers would destroy or invent a record rather than release one.

The identifiers are grouped by pipeline and by unit, because a job identifier names a stage rather than a unit and two units of one project share the identifier of the same stage. A unit named with no identifier at all has every job it tracks reset, so a unit contributing no stranded job is never named.

Parameters:

host (ExecutionHost) – The host holding the units whose trackers to write.
resolutions (Sequence[AllocationResolution]) – The resolutions whose stranded jobs to reset.

Return type:

set[tuple[str, str]]

Returns:

The unit path and job identifier of each job that was reset.

Raises:

RuntimeError – If a step fails on the host.

sollertia_forgery.orchestration.reset_tracked_jobs(pipeline, unit_paths, job_ids=())¶

Returns the named tracked jobs of several units to the scheduled state.

Notes

Resolves each tracker from its unit rather than taking a path, so a caller names what it wants to be reset without knowing where the record sits. Identifiers that a unit does not track are dropped, because a tracker rejects a request naming a job it does not hold and would then reset nothing at all.

Every named identifier is applied to every named unit. A job identifier is derived from the job name and the specifier alone, so two units of one project share the identifier of the same stage. A caller holding per-unit identifiers passes one unit at a time rather than a flat set.

Naming no identifier resets every job the unit tracks, which is how a caller returns a unit to a clean slate.

Parameters:

pipeline (str) – The pipeline whose jobs to reset.
unit_paths (Sequence[Path]) – The processing units whose records to clear.
job_ids (Sequence[str], default: ()) – The identifiers to reset, or empty to reset every job each unit tracks.

Return type:

list[str]

Returns:

The identifiers that were reset, which repeats an identifier held by more than one unit. Naming a pipeline that the dispatch table does not support returns nothing.

sollertia_forgery.orchestration.resolve_allocations(batches, reading, claims)¶

Resolves every allocation of the given batches to one verdict and the remediation it prescribes.

Notes

Every allocation is resolved from the batch’s own submissions rather than from the answer a query returned, so an allocation for which neither record answered is still resolved rather than passed over.

An allocation resolves as running whenever the scheduler holds it, holds the allocation its job’s tracker claims, or that tracker claims to be running under an executor for which neither of the scheduler’s records answers. Everything else is resolved by what that tracker recorded, because the tracker is what gates rerunning the job.

Parameters:

batches (Sequence[SubmissionBatch]) – The recorded batches whose allocations to resolve.
reading (SchedulerReading) – What the scheduler’s records reported.
claims (Mapping[tuple[str, str], TrackerClaim]) – What each job’s tracker recorded, keyed by unit path and job identifier.

Return type:

list[AllocationResolution]

Returns:

One resolution per recorded allocation, in the order the batches hold them.

sollertia_forgery.orchestration.resolve_batch_host(documents)¶

Resolves the single host on which a set of prepared batches runs.

Notes

A batch runs where it was prepared, because its jobs read data that host holds. Executing batches prepared against different hosts together is therefore rejected rather than resolved to one of them.

Parameters:: documents (list[BatchDocument]) – The prepared batches whose host is resolved.
Return type:: str
Returns:: The name of the host against which every named batch was prepared.
Raises:: ValueError – If no batch is named, or if the named batches were prepared against different hosts.

sollertia_forgery.orchestration.resolve_batches(ledger, batch_ids=None)¶

Resolves the batches to which a status or cancellation applies.

Parameters:

ledger (SubmissionLedger) – The recorded ledger.
batch_ids (Sequence[str] | None, default: None) – The identifiers to resolve, or None to resolve every outstanding batch.

Return type:

list[SubmissionBatch]

Returns:

The resolved batches.

sollertia_forgery.orchestration.resolve_concurrency_limits(job_names)¶

Resolves the concurrent-job ceiling each queued job type carries beyond the batch’s two budgets.

Notes

Only the job types that declare a limit appear in the result, so a caller reads an absent name as bounded by the core and memory budgets alone. Declared limits are raised to one, since a ceiling of zero would leave a queued job with no admission path at all.

Parameters:: job_names (set[str]) – The job type names present in the batch.
Return type:: dict[str, int]
Returns:: The jobs of each limited type that may run at once, keyed by job name.

sollertia_forgery.orchestration.resolve_concurrency_reservations(job_names)¶

Resolves the concurrency reserved for each queued job type while other work can use the capacity it gives up.

Notes

Only the job types that declare a reservation appear in the result, so a caller reads an absent name as competing at its full core-derived width. Declared reservations are raised to one, since a reservation of zero would keep a type out of the first admission pass entirely.

Parameters:: job_names (set[str]) – The job type names present in the batch.
Return type:: dict[str, int]
Returns:: The jobs of each reserved type admitted before the reservation lifts, keyed by job name.

sollertia_forgery.orchestration.resolve_core_allocations(job_cores, job_names, core_budget, job_limits=None, job_reservations=None)¶

Resolves the cores the host can supply for one job of each queued job type and how many of its jobs run at once.

Notes

A type’s core count is the allocation its representative job declares, narrowed to the budget so a small host never promises a job more cores than it has. The caller supplies the widest job of each type as that representative, because sizing is per job and one type can hold jobs of several widths. Dispatch then caps each job at the count resolved here rather than dispatching every job of the type at it, so a narrower job keeps the width its own sizing pass chose. The concurrency that follows is the budget divided by that count, narrowed again by any ceiling the type declares for itself. The engine treats the core term as a guide, since admission weighs every running job against the same budget at that job’s own width, while it enforces the declared ceiling exactly.

A job type with no registered allocation stops the batch, since dispatching it would run it at a width nobody chose.

Parameters:

job_cores (dict[str, int]) – The cores the widest job of each type occupies, keyed by tracker job name, which is the representative width from which the type’s allocation is resolved.
job_names (set[str]) – The job type names present in the batch.
core_budget (int) – The cores the batch may commit across all concurrently running jobs.
job_limits (dict[str, int] | None, default: None) – The concurrent-job ceilings the job types declare beyond the budgets, keyed by tracker job name. Only the types that declare one appear, and passing nothing bounds every type by the budgets alone.
job_reservations (dict[str, int] | None, default: None) – The concurrency reserved for the job types while other work can use the capacity they give up, keyed by tracker job name. Only the types that declare one appear, and passing nothing lets every type compete at its full width.

Return type:

dict[str, _JobAllocation]

Returns:

A dictionary mapping each job name to its resolved allocation.

Raises:

ValueError – If any queued job type has no registered core allocation.

sollertia_forgery.orchestration.resolve_dataset_plan(dataset_path, *, regenerate_plan=False, display_progress=False)¶

Plans every dataset-scoped job of one dataset, estimating only the jobs the cache does not already hold.

Notes

A dataset’s figures follow from the single-day outputs its jobs consume, and admission requires a session to carry those outputs already, so a dataset is plannable as soon as its hierarchy is defined.

Every pipeline whose dispatch entry declares the dataset unit is planned, so a pipeline added over datasets reaches this pass by declaring that kind rather than by being named here.

Parameters:

dataset_path (Path) – The path to the dataset’s root directory to plan.
regenerate_plan (bool, default: False) – Determines whether to re-estimate the jobs the cache already holds. A cache stamped with another model version is re-estimated whatever this asks.
display_progress (bool, default: False) – Determines whether to report the pipelines that resolved no jobs for this dataset and why.

Return type:

_JobPlan

Returns:

The dataset’s plan, holding an entry for every job its pipelines resolve.

Raises:

ValueError – If no pipeline resolves any job for this dataset, since a dataset with no plannable job names no path to a plan file.
TimeoutError – If a pipeline’s processing tracker lock or the plan file’s own lock cannot be acquired within the timeout period.

sollertia_forgery.orchestration.resolve_host_memory_mb()¶

Reads the host’s total physical memory.

Return type:: int
Returns:: The host’s total physical memory in megabytes.

sollertia_forgery.orchestration.resolve_job_cores(job_name)¶

Resolves the cores the named job type declares for one of its jobs.

Notes

Reports the declared allocation itself. Every stage this package sizes for itself runs at that width. A stage that a dependency owns is sized whole by that dependency, so for those the declared allocation restates the dependency’s own figure and the sizing pass is what a plan records. Narrowing an allocation to what a host can supply belongs to the execution layer, because the host that plans a unit and the host that runs its jobs need not be the same one.

Parameters:: job_name (str) – The tracker job name whose allocation to resolve.
Return type:: int
Returns:: The cores one job of that type occupies.
Raises:: ValueError – If the job type declares no core allocation.

sollertia_forgery.orchestration.resolve_live_allocations(resolutions)¶

Resolves every allocation of the given resolutions that the scheduler still holds.

Notes

A resolution names two allocations, and either of them being held is what makes its verdict running. The recorded one is this host’s own submission, while the one the job’s tracker claims may be another submitter’s, so both are named here. A cancellation covering the recorded allocation alone would leave the allocation actually running the job free to write into the tracker that same remediation then resets.

Parameters:: resolutions (Sequence[AllocationResolution]) – The resolutions whose held allocations to name.
Return type:: list[str]
Returns:: The allocation identifiers, in natural order and holding no duplicates.

sollertia_forgery.orchestration.resolve_project_root(unit_paths, unit_kind)¶

Resolves the project that owns the named units.

Parameters:

unit_paths (Sequence[Path]) – The processing unit directories whose project is resolved.
unit_kind (str) – Whether the units are sessions or datasets, which sets how far above a unit its project sits.

Return type:

Path

Returns:

The path to the project root.

Raises:

ValueError – If no unit is named, if a named unit’s path holds fewer parent directories than its kind sits below its project, or if the named units span more than one project.

sollertia_forgery.orchestration.resolve_queried_allocations(submissions, claims)¶

Resolves every allocation a verdict reads, which is the one each submission recorded and the one each job’s tracker claims.

Notes

A tracker claim is read alongside the ledger’s own record because the two cover different submitters. A job another machine submitted is claimed on the tracker alone, and resetting that job while its allocation still runs is exactly the destruction this resolution exists to refuse.

Parameters:

submissions (Sequence[RemoteSubmission]) – The submissions whose recorded allocations to read.
claims (Mapping[tuple[str, str], TrackerClaim]) – What each job’s tracker recorded, keyed by unit path and job identifier.

Return type:

list[str]

Returns:

The allocation identifiers to query, in natural order and holding no duplicates.

sollertia_forgery.orchestration.resolve_session_plan(session_path, *, regenerate_plan=False, display_progress=False)¶

Plans every processing job of one session, estimating only the jobs the cache does not already hold.

Notes

Estimation reads the session’s raw acquisition data, so it costs far more than reading the cache it writes. A pipeline that resolves no jobs for this session is skipped, which is the ordinary case for a session carrying no imaging or no camera data.

Parameters:

session_path (Path) – The path to the session root directory to plan.
regenerate_plan (bool, default: False) – Determines whether to re-estimate the jobs the cache already holds. Leave False to keep every recorded figure, since a submission may already have been sized against it. A cache stamped with another model version is re-estimated whatever this asks.
display_progress (bool, default: False) – Determines whether to report the pipelines that resolved no jobs for this session and why.

Return type:

_JobPlan

Returns:

The session’s plan, holding an entry for every job its pipelines resolve.

Raises:

ValueError – If no pipeline resolves any job for this session, since a session with no plannable job names no path to a plan file.
TimeoutError – If a pipeline’s processing tracker lock or the plan file’s own lock cannot be acquired within the timeout period.

sollertia_forgery.orchestration.resolve_tracker_claims(host, submissions)¶

Reads what each submitted job’s own processing tracker currently records.

Notes

The trackers are rewritten into the host’s state artifacts before they are read, because a job records its outcome on its tracker and nothing regenerates those artifacts while a batch runs. Reading them as they stand would report the state against which the batch was prepared, which is what would let a finished job read as one still running and be reset.

The regeneration is issued once per project and unit kind, since a session artifact covers a whole project while a dataset artifact covers one dataset. The rows it produces are then narrowed to each pipeline the submissions name, the way a batch is resolved out of the same artifacts.

A submission naming no unit is skipped rather than resolved, since a record that does not say which unit it ran against names no tracker to read.

Parameters:

host (ExecutionHost) – The host holding the units whose trackers to read.
submissions (Sequence[RemoteSubmission]) – The submissions whose jobs to read.

Return type:

dict[tuple[str, str], TrackerClaim]

Returns:

What each job’s tracker records, keyed by the unit path and job identifier that name the job. A job the artifacts hold no row for carries an empty claim.

Raises:

RuntimeError – If a step fails on the host.

sollertia_forgery.orchestration.run_batch_job(job)¶

Runs one prepared job of any pipeline, routing on the pipeline the job carries.

Notes

This is the single picklable entry point the shared pool dispatches, so one pool holds jobs from every pipeline at once. Each pipeline’s own worker is looked up rather than bound into the job, so the descriptor stays a plain data record that pickles cheaply.

The image-decode ceiling is written before the pipeline’s own worker runs, bounding any read that names no decode width of its own. A reader that names one, as cindra does, sizes its threads from the cores the batch handed the job.

The native BLAS and OpenMP pools are held at the job’s own width for as long as it runs. Those pools are built when the worker imports numpy, scipy, and scikit-learn, and they size themselves from the machine’s core count because that import precedes anything the worker does for itself. The variables naming their width are read only at that import, so a pool already built ignores them, and resizing the pools through their own runtime interfaces is what actually holds a job to its admitted cores.

Parameters:: job (GenericPendingJob) – The pending job carrying its pipeline, its target job identifier, and its planned cores.
Raises:: ValueError – If the job names a pipeline the dispatch table does not support.
Return type:: None

sollertia_forgery.orchestration.submit_batch(server, jobs, batch_id, adopted=None, covered_batch_ids=(), *, walltime_minutes=480, verbose=False)¶

Submits a prepared batch to the scheduler as a dependency graph.

Notes

The scheduler sequences the graph itself, so this process may exit as soon as the last job is queued.

The batch is recorded before the first allocation is queued, so a submission that does not return still leaves a batch the remote tools name and resolve. A submission the scheduler refuses partway through records the allocations it had accepted, because the record is written on the way out of the submission either way. A submission the host kills outright records none of them, and the batch it leaves resolves through the jobs’ own trackers and the scheduler queue rather than from allocations the ledger names.

Every accepted allocation is recorded in the submission ledger, including when the scheduler rejects a later job of the same batch, since the allocations it already accepted stay queued.

The record is merged into whatever the ledger already holds for this batch, so re-running a batch the scheduler only partly accepted keeps the allocations the first attempt queued. An entry this call re-submitted is replaced rather than duplicated. The merge is left to the recording call, which reads the entries it carries forward under the same lock that writes them. An entry a concurrent writer committed against the same batch is therefore never dropped by a list this call read before that lock was taken.

An adopted job’s allocation seeds the dependency map before anything is submitted, so a dependent of a job that is already running waits on the allocation running it rather than on a second one.

The concurrency ceilings that the local engine applies do not reach the scheduler. Expressing one natively needs a job array, whose tasks share a single memory request, so the scheduler is left to sequence the whole batch.

Parameters:

server (Server) – The connected server that receives the submission.
jobs (Sequence[dict[str, Any]]) – The job descriptors to submit.
batch_id (str) – The identifier of the batch, which names the directory that holds the scripts and logs.
adopted (dict[tuple[str, str], str] | None, default: None) – The allocation already running each adopted job, keyed by dispatch key. These jobs are not submitted, and their dependents wait on those allocations.
covered_batch_ids (Sequence[str], default: ()) – Every prepared batch this submission dispatches. Closure snapshots an outcome for each. Leave empty for a submission covering the batch batch_id names alone.
walltime_minutes (int, default: 480) – The wall-time every allocation requests.
verbose (bool, default: False) – Determines whether to report each submission as it is accepted.

Return type:

list[RemoteSubmission]

Returns:

The submissions, in the order they were accepted.

Raises:

RuntimeError – If the scheduler rejects a submission.
Timeout – If the submission ledger’s lock cannot be acquired within the timeout period.

sollertia_forgery.orchestration.sync_project_state(server, project, local_directory, *, regenerate=True)¶

Regenerates a remote project’s state artifacts and mirrors them onto this host.

Notes

Regeneration precedes the pull, so the mirrored state tables describe the state after the runs rather than before them. The plan projection is mirrored as the server last wrote it, since replanning is a preparation step rather than a mirroring one.

Parameters:

server (Server) – The connected server holding the project.
project (str) – The name of the project whose state to mirror.
local_directory (Path) – The local directory that receives the mirrored artifacts.
regenerate (bool, default: True) – Determines whether to regenerate the artifacts on the server before pulling them.

Return type:

list[Path]

Returns:

Where the artifacts were written, holding one entry per artifact the server carried.

Raises:

FileNotFoundError – If the server holds no directory for the named project.
RuntimeError – If the server-side search for the project’s datasets reached only part of its tree.

sollertia_forgery.orchestration.batches.OUTCOME_FILE_SUFFIX: str = '.outcome.yaml'¶: The suffix that separates a batch’s outcome file from the prepared document the outcome describes.

sollertia_forgery.orchestration.dispatch.BATCH_PIPELINES: frozenset[ProcessingPipelines] = frozenset({ProcessingPipelines.CHECKSUM, ProcessingPipelines.FORGING, ProcessingPipelines.MICROCONTROLLER, ProcessingPipelines.RUNTIME, ProcessingPipelines.TWO_PHOTON, ProcessingPipelines.VIDEO})¶: The pipelines the generic batch tools support. An import-time check holds this to the dispatch table.

sollertia_forgery.orchestration.dispatch.SESSION_UNIT: str = 'session'¶: The unit label of a pipeline whose jobs process one acquisition session.

sollertia_forgery.orchestration.dispatch.DATASET_UNIT: str = 'dataset'¶: The unit label of a pipeline whose jobs process one forged dataset.

sollertia_forgery.orchestration.local.RESERVED_CORES: int = 2¶: The number of CPU cores held back for host-system operations when the core budget auto-resolves. The batch tools forward this to resolve_worker_count, which applies it only to a non-positive budget and honors an explicit budget up to the logical core count.

sollertia_forgery.orchestration.planning.PROJECT_PLAN_SCHEMA: dict[str, DataTypeClass | DataType] = {'animal': String, 'cores': UInt16, 'dataset': String, 'job_id': String, 'job_name': String, 'memory_mb': UInt32, 'memory_modeled': Boolean, 'pipeline': String, 'prerequisite_ids': List(String), 'resident_mb': UInt32, 'session': String, 'specifier': String, 'unit_kind': String}¶

The column layout of the project plan projection, one row per planned job.

Notes

The subject columns mirror the dataset state artifact, so a reader joins a job’s figures against its recorded state on the same keys. A session row carries no dataset and a dataset row carries neither animal nor session.

The project job artifact keys its own rows by job_id, so the two tables join on it directly. Carrying the ordering here as well lets a scheduler build a job’s dependency graph from this table alone, without resolving the unit that owns the jobs.

sollertia_forgery.orchestration.reconcile.LOCAL_HOST_LABEL: str = 'local'¶: The label a batch carries when it was prepared against this machine.

sollertia_forgery.orchestration.reconcile.REMOTE_HOST_LABEL: str = 'remote'¶: The label a batch carries when it was prepared against the remote compute server.

sollertia_forgery.orchestration.remote.REMOTE_JOB_WALLTIME_MINUTES: int = 480¶

The wall-time a remote allocation requests when a caller names none, in minutes.

Notes

One figure covers every job type, because this bound exists to stop a run that has stopped making progress rather than to describe how long a stage takes.

sollertia_forgery.orchestration.remote.HELD_ALLOCATION: str = 'held'¶

The scheduler state of an allocation the scheduler still holds, which is one its queue carries or one accounting reports in a state it has yet to leave.

Notes

A queue row and a non-terminal accounting row each prove the allocation is still the scheduler’s, and neither is contradicted by the other record saying nothing about it. A record that could not be read at all resolves here too, because a source that did not answer is no evidence of absence. A queue row is not evidence of this state for an allocation the queue itself reports as permanently blocked, since such an allocation never runs however long the queue carries it.

sollertia_forgery.orchestration.remote.SETTLED_ALLOCATION: str = 'settled'¶: The scheduler state of an allocation that accounting reports in a state it never leaves and that the queue no longer carries, of one the queue reports as permanently blocked, and of one this reading has since canceled. The scheduler has finished with it, or will never start it, so nothing it holds can change again.

sollertia_forgery.orchestration.remote.GONE_ALLOCATION: str = 'gone'¶: The scheduler state of an allocation accounting returns no row for and the queue does not carry. Both records disclaim it, and only both of them together are evidence that the scheduler no longer holds it.

sollertia_forgery.orchestration.remote.RUNNING_ALLOCATION: str = 'running'¶: The verdict on an allocation the scheduler still holds, on one whose job a still-held allocation claims on its tracker, and on one whose job’s tracker claims to be running under an executor outside the scheduler. Nothing is remediated for it, because every remediation this module applies would disturb work that may still be live.

sollertia_forgery.orchestration.remote.FINISHED_ALLOCATION: str = 'finished'¶: The verdict on an allocation the scheduler no longer holds whose job recorded success. Its tracker is left exactly as it stands, since resetting it would discard a result the run actually produced.

sollertia_forgery.orchestration.remote.FAILED_ALLOCATION: str = 'failed'¶: The verdict on an allocation the scheduler no longer holds whose job recorded failure. Its tracker is left exactly as it stands, because a failure is a real verdict the operator must see and clear deliberately.

sollertia_forgery.orchestration.remote.ABANDONED_ALLOCATION: str = 'abandoned'¶: The verdict on an allocation the scheduler no longer holds, whose job either never left the scheduled state or carries no record on its tracker at all. Nothing claims that job, so it is already runnable once the ledger entry is gone.

sollertia_forgery.orchestration.remote.STRANDED_ALLOCATION: str = 'stranded'¶: The verdict on an allocation the scheduler no longer holds whose job’s tracker still claims to be running it under an executor for which the scheduler can answer. This is the one verdict whose remediation writes to a tracker, because that claim is what no rerun can otherwise clear.

sollertia_forgery.orchestration.remote.NO_REMEDIATION: str = 'none'¶: The remediation prescribed for a running allocation, which is to leave it alone.

sollertia_forgery.orchestration.remote.DROP_REMEDIATION: str = 'drop'¶: The remediation prescribed for a finished, failed, or abandoned allocation, which is to snapshot what its batch’s jobs recorded and drop the ledger entry, leaving every tracker as it stands.

sollertia_forgery.orchestration.remote.RESET_REMEDIATION: str = 'reset_and_drop'¶: The remediation prescribed for a stranded allocation, which returns its job to the scheduled state on its own tracker before the snapshot and the drop.

sollertia_forgery.orchestration.remote.CANCEL_REMEDIATION: str = 'cancel_reset_and_drop'¶: The remediation applied to an allocation a caller overriding the refusal canceled and whose job the cancellation left stranded on its own tracker. The cancellation runs before any tracker is written, so no tracker is reset underneath an allocation the scheduler was never told to stop.

sollertia_forgery.orchestration.remote.PROGRESSING_BATCH: str = 'progressing'¶: The verdict a batch carries while at least one of its allocations resolves as running. Holding an allocation is the only evidence that the work can still advance, and it is evidence no elapsed period can contradict, so a batch that carries this verdict is left alone however long it has been queued.

sollertia_forgery.orchestration.remote.STALLED_BATCH: str = 'stalled'¶: The verdict a batch carries when none of its allocations resolves as running and at least one of them is gone from both of the scheduler’s records. Neither record answers for such an allocation again, so nothing the scheduler does advances the batch. A stalled batch stays outstanding past a read only when it holds an entry the automatic closure may not drop, which is a stranded job, or when that closure failed, and an explicit remediation is what releases it.

sollertia_forgery.orchestration.remote.AWAITING_CLOSURE_BATCH: str = 'awaiting_closure'¶: The verdict a batch carries when every allocation it holds has settled. The read that observes a batch resolving entirely to the plain drop also closes it. A batch outstanding under this verdict is therefore one holding an entry that closure may not drop or one whose closure failed, and the next read retries it.

Compute Server Transport¶

Provides the remote compute-server transport: SSH/SLURM execution, non-interactive job assembly, project session discovery, and server configuration.

class sollertia_forgery.server.Job(job_name, output_log, error_log, working_directory, conda_environment, cpu_threads=10, ram=10, time=60, dependencies=())¶

Bases: object

Defines a non-interactive SLURM-managed job to be executed on the remote compute server.

Notes

A job that names dependencies runs only after every named allocation completes successfully, and is canceled outright once any of them can no longer do so. Sequencing a pipeline this way lets the submitting process exit as soon as the whole graph is queued.

Parameters:

job_name (str) – The descriptive name of the SLURM job to be created.
output_log (Path) – The absolute path to the file on the compute server to use for storing the messages sent by the job to the ‘stdout’ pipe.
error_log (Path) – The absolute path to the file on the compute server to use for storing the messages sent by the job to the ‘stderr’ pipe.
working_directory (Path) – The absolute path to the compute server’s directory in which the temporary job files are stored.
conda_environment (str) – The name of the mamba / conda environment to activate on the server before running the job.
cpu_threads (int, default: 10) – The number of CPU threads to use for the job.
ram (int, default: 10) – The amount of RAM to allocate for the job, in gigabytes.
time (int, default: 60) – The maximum period of time to run the job, in minutes.
dependencies (Sequence[str], default: ()) – The SLURM-assigned identifiers of the allocations that must complete successfully before this job runs. Leave empty for a job that waits on nothing.

remote_script_path¶: The path to the job’s script file on the remote compute server.

job_id¶: The unique job identifier assigned by the SLURM manager to this job when it is accepted for execution.

job_name¶: The descriptive name of the SLURM job.

_command¶: The job’s batch script under assembly, rendered into shell text on demand.

add_command(command)¶

Adds the input command string to the end of the job’s command sequence.

Notes

The instance generates the job’s SLURM directive header and a Conda activation preamble during class initialization. Do not submit additional SLURM or Conda commands via this method, as this may produce unexpected behavior.

Commands added through this method run under shell error checking, so the job exits with the status of the first command that fails. The scheduler reads that status and sequences a dependent job behind this one.

Parameters:: command (str) – The command string to append to the job’s command sequence, for example ‘python main.py –input 1’.
Return type:: None

property command_script: str¶: Returns the managed job rendered as a shell-script-writable string.

class sollertia_forgery.server.JobStatus(*values)¶

Bases: StrEnum

Defines the set of status codes this library resolves for managed jobs.

These are the states SLURM reports, plus the BLOCKED, UNKNOWN, and UNRESOLVED states resolved locally.

BLOCKED = 'BLOCKED'¶: The job is queued behind a dependency that can no longer be satisfied, so it will never run. Resolved from the queue’s reason field rather than from accounting, which still reports such a job as pending.

BOOT_FAIL = 'BOOT_FAIL'¶: The job terminated because a node allocated to it failed to boot.

CANCELLED = 'CANCELLED'¶: The job was canceled by the user or administrator.

COMPLETED = 'COMPLETED'¶: The job finished successfully.

DEADLINE = 'DEADLINE'¶: The job was terminated on reaching the deadline its partition enforces.

FAILED = 'FAILED'¶: The job terminated with a non-zero exit code.

NODE_FAIL = 'NODE_FAIL'¶: The job terminated due to node failure.

OUT_OF_MEMORY = 'OUT_OF_MEMORY'¶: The job was terminated for exceeding memory limits.

PENDING = 'PENDING'¶: The job is queued and waiting for resources.

PREEMPTED = 'PREEMPTED'¶: The job was terminated to release its resources to a higher-priority allocation.

REVOKED = 'REVOKED'¶: The job’s allocation was revoked, which a federated scheduler does once a sibling cluster starts the job.

RUNNING = 'RUNNING'¶: The job is currently executing.

TIMEOUT = 'TIMEOUT'¶: The job exceeded its time limit.

UNKNOWN = 'UNKNOWN'¶: Accounting returned a row for the allocation whose state string this enumeration does not model, which covers every live state beyond PENDING and RUNNING, such as SUSPENDED, CONFIGURING, or COMPLETING. A row exists, so the scheduler still holds the allocation and this state is never terminal.

UNRESOLVED = 'UNRESOLVED'¶: Accounting was queried successfully and returned no row for the allocation, which covers a submission it has not registered yet as well as one it has purged. Nothing observable separates those two, so this state is never terminal, and the batch on which an allocation holds this state is retired by an explicit caller rather than on a timer.

class sollertia_forgery.server.Server(configuration)¶

Bases: object

Establishes and maintains a bidirectional interface that allows working with a remote compute server.

Submits SLURM-managed jobs to the server, monitors their execution status, and manages the data stored on the server over the SFTP protocol.

Notes

This class assumes that the target server has the SLURM job manager installed and accessible to the user whose credentials are used to connect to the server as part of class initialization.

Parameters:: configuration (ServerConfiguration) – The ServerConfiguration instance that contains the server hostname and access credentials.

_open¶: Tracks whether the connection to the server is open.

_client¶: The authenticated SSH session over which the remote commands run.

_sftp¶: The file-transfer channel opened over that session.

_configuration¶: The hostname, access credentials, data root, and environment name of the server.

abort_job(slurm_job_id)¶

Aborts the job with the specified SLURM-assigned ID if it is currently running or pending on the server.

Parameters:: slurm_job_id (str) – The SLURM-assigned job ID to abort.
Raises:: RuntimeError – If the accounting query that reads the allocation’s state fails.
Return type:: None

abort_jobs(slurm_job_ids)¶

Aborts every named allocation that is still queued or running on the server.

Notes

The scheduler answers a cancellation naming an allocation with which it has already finished as a success, so naming a settled allocation is not a failure here. A non-zero exit therefore reports that the cancellation did not reach the scheduler at all. That failure is raised the way every other command in this class raises, because a caller that resets a job’s tracker behind this call depends on the cancellation having been issued.

Parameters:: slurm_job_ids (Sequence[str]) – The SLURM-assigned job IDs to abort.
Raises:: RuntimeError – If the cancellation fails.
Return type:: None

close()¶

Closes the SFTP and SSH connections to the server.

Return type:: None

create(remote_path, *, is_dir=True, parents=True)¶

Creates a file or directory on the remote server.

Notes

An existing target is left as it is, so a repeated call succeeds.

Parameters:

remote_path (Path) – The absolute path to the file or directory to create on the remote server.
is_dir (bool, default: True) – Determines whether the created target is a directory rather than an empty file.
parents (bool, default: True) – Determines whether missing parent directories are created alongside a requested directory. Parents are always created for a file.

Raises:

RuntimeError – If the remote directory creation command fails.
FileNotFoundError – If parents is False and the target’s parent directory does not exist.

Return type:

None

property environment: str¶: Returns the name of the shared conda environment, on the server, that every remote compute job activates before invoking the slf CLI.

execute_command(command)¶

Executes the specified command on the remote server and returns the result.

Notes

Both streams are drained concurrently. They share the channel’s flow-control window, so draining either to its end before the other lets a command fill that window with the unread stream and block forever. This call then waits on output the command cannot finish writing.

Parameters:: command (str) – The shell command to execute on the remote server.
Return type:: _CommandResult
Returns:: The command’s standard output, its standard error output, and the exit code it reported.

exists(remote_path)¶

Returns True if the target file or directory exists on the remote server.

Parameters:: remote_path (Path) – The path to check on the remote server.
Return type:: bool
Returns:: True if the path exists, False otherwise.

find_paths(remote_path, names, *, minimum_depth, maximum_depth)¶

Returns every path under the target directory whose final component matches one of the given names.

Notes

The search runs as one shell command rather than as a walk over the file-transfer protocol, which trades one query per directory and per candidate for a single round trip. Symbolic links are followed, and a link whose target does not resolve is reported as absent, so the answer matches the one that exists() gives for the same path.

Parameters:

remote_path (Path) – The absolute path to the directory to search on the remote server.
names (Sequence[str]) – The final path components to match. Each is matched as a shell name pattern, so a name carrying a glob metacharacter matches as a pattern rather than as a literal.
minimum_depth (int) – The lowest depth, counted in path components below the searched directory, at which a match is reported.
maximum_depth (int) – The highest depth, counted in path components below the searched directory, at which a match is reported. The search never descends past this depth.

Return type:

list[Path]

Returns:

The absolute paths of every match, in natural sort order.

Raises:

FileNotFoundError – If the searched path is not a directory on the remote server.
RuntimeError – If the search command failed, which leaves it having covered only part of the tree, or if the search reported an entry that does not sit under the searched directory.

get_blocked_job_ids()¶

Returns the identifiers of this user’s queued allocations whose dependencies can no longer be satisfied.

Notes

Queries the user’s whole queue, since naming an allocation that the queue no longer holds makes the command report an error for it.

Return type:: set[str]
Returns:: The SLURM-assigned job IDs the queue reports as permanently blocked.

get_job_status(slurm_job_id)¶

Queries the managed server’s SLURM manager for the runtime status of the job with the specified SLURM-assigned ID.

Notes

Uses the ‘sacct’ command to determine the current state of the job, returning the actual status (e.g., PENDING, RUNNING, COMPLETED, FAILED) assigned by the SLURM manager. A pending allocation is additionally checked against the queue’s reason field, so a job whose dependency can no longer be satisfied is reported as BLOCKED.

Parameters:: slurm_job_id (str) – The SLURM-assigned job ID for which to query the runtime status.
Return type:: JobStatus
Returns:: The allocation’s current status, reported as blocked when its dependency can no longer be satisfied.
Raises:: RuntimeError – If the accounting query fails.

get_job_statuses(slurm_job_ids)¶

Queries the runtime status of every named allocation in one accounting call.

Notes

Accounting reports each allocation alongside its steps, and only the allocation rows are read.

A pending allocation whose dependency can no longer be satisfied is reported as blocked. Accounting still calls that job pending, so the queue’s reason field is the only source of that distinction.

A query that fails reports no state at all rather than a state per identifier. Accounting that cannot answer writes nothing to standard output, which is indistinguishable from an answer that holds no row for any of the requested allocations, so reporting that answer would write off every live allocation at once.

Parameters:: slurm_job_ids (Sequence[str]) – The SLURM-assigned job IDs to query.
Return type:: dict[str, JobStatus]
Returns:: A dictionary mapping every requested job ID to its status. An allocation for which the successful query returned no row reports as UNRESOLVED.
Raises:: RuntimeError – If the accounting query fails.

get_queued_job_ids()¶

Returns the identifiers of every allocation of this user that the scheduler’s queue currently holds.

Notes

The queue records what the scheduler holds right now, while accounting records what it has committed. The controller queues an allocation before slurmdbd commits a row for it, so accounting alone cannot tell a freshly queued allocation from a purged one. Reading the queue is what separates those two.

Queries the user’s whole queue, since naming an allocation that the queue no longer holds makes the command report an error for it.

A user holding nothing answers with an empty set, which is the truthful reading of an empty queue. A command that fails raises instead, because a failure writes nothing to standard output and answering that as an empty queue would report every outstanding allocation as one the scheduler no longer holds.

Return type:: set[str]
Returns:: The SLURM-assigned job IDs the queue holds.
Raises:: RuntimeError – If the queue query fails.

property host: str¶: Returns the hostname or IP address of the server accessible through this class.

is_directory(remote_path)¶

Returns True if the target path is a directory on the remote server.

Parameters:: remote_path (Path) – The path to check on the remote server.
Return type:: bool
Returns:: True if the path exists and is a directory, False otherwise.

list_directory(remote_path)¶

Lists the contents of a directory on the remote server.

Parameters:: remote_path (Path) – The path to the directory on the remote server.
Return type:: list[str]
Returns:: A list of filenames (not full paths) in the directory.
Raises:: FileNotFoundError – If the directory does not exist.

pull(local_path, remote_path)¶

Downloads a file or directory from the remote server to the local machine.

Detects whether the remote path points to a file or directory and handles the transfer accordingly. For directories, all contents are recursively downloaded.

Parameters:

local_path (Path) – The path on the local machine where the file or directory will be saved.
remote_path (Path) – The path to the file or directory on the remote server to download.

Raises:

FileNotFoundError – If the remote path does not exist on the server.

Return type:

None

push(local_path, remote_path)¶

Uploads a file or directory from the local machine to the remote server.

Detects whether the local path points to a file or directory and handles the transfer accordingly. For directories, all contents are recursively uploaded.

Parameters:

local_path (Path) – The path to the file or directory on the local machine to upload.
remote_path (Path) – The path on the remote server where the file or directory will be saved.

Raises:

FileNotFoundError – If the local path does not exist.

Return type:

None

remove(remote_path, *, is_dir, recursive=False)¶

Removes a file or directory from the remote server.

Parameters:

remote_path (Path) – The path to the file or directory on the remote server to be removed.
is_dir (bool) – Determines whether the input path represents a directory or a file.
recursive (bool, default: False) – Determines whether the directory’s contents are deleted before the directory itself. A non-recursive removal succeeds only for an empty directory.

Return type:

None

property root: Path¶: Returns the absolute path to the single root directory of the remote compute server that stores all Sollertia data (raw and processed) accessible through this instance.

submit_job(job, *, verbose=True)¶

Submits the input job to the managed remote compute server via the SLURM job manager.

Parameters:

job (Job) – The Job instance that defines the job to be executed.
verbose (bool, default: True) – Determines whether to emit the submission start and success messages. The warning issued for a job that was already submitted is emitted regardless.

Return type:

Job

Returns:

The submitted job, carrying the identifier the scheduler assigned to its allocation.

Raises:

RuntimeError – If the job cannot be submitted to the server for any reason.

property user: str¶: Returns the username used to authenticate with the server.

class sollertia_forgery.server.ServerConfiguration(username='', password='', host='', root='', environment='')¶

Bases: YamlConfig

Defines the access credentials and data root for the Sollertia platform remote compute server.

environment: str = ''¶: The name of the shared conda environment, on the remote compute server, in which sollertia-forgery and all of its processing dependencies are installed.

host: str = ''¶: The hostname or IP address used to reach the server.

password: str = ''¶: The password to use for server authentication.

root: str = ''¶: The absolute path, on the remote compute server, to the single root directory that stores all Sollertia data (raw and processed). All server-side data operations resolve their paths relative to this root.

username: str = ''¶: The username to use for server authentication.

sollertia_forgery.server.create_server_configuration_file(username, password, host, root, environment)¶

Creates the server_configuration.yaml file for the Sollertia platform compute server at the canonical path every server-related call resolves.

Parameters:

username (str) – The username to use for server authentication.
password (str) – The password to use for server authentication.
host (str) – The hostname or IP address used to reach the server.
root (str) – The absolute path, on the remote compute server, to the root directory that stores all Sollertia data.
environment (str) – The name of the shared conda environment, on the remote compute server, in which sollertia-forgery and all of its processing dependencies are installed.

Return type:

None

sollertia_forgery.server.discover_project_data(project)¶

Discovers and reports all sessions stored under the project’s directory on the remote compute server.

Parameters:: project (str) – The name of the project whose data to discover.
Return type:: tuple[DatasetSession, ...]
Returns:: The project’s acquired sessions, in natural sort order.

sollertia_forgery.server.discover_project_markers(project_path, server, *, include_sessions=True)¶

Discovers the dataset and session marker files a remote project holds.

Notes

Both marker kinds are read in one server-side search, since both answer the same question about the same tree. A directory carrying a dataset marker is a forged dataset rather than an animal, so the sessions it holds are never reported as an animal’s sessions.

The search reads every directory above the depth it covers, so a caller that needs the datasets alone narrows it to the depth at which their markers sit. That keeps the answer from depending on the session and output directories every animal holds, which a project shared between accounts need not leave readable.

Parameters:

project_path (Path) – The absolute path to the project’s root directory on the remote compute server.
server (Server) – The Server instance used to communicate with the remote compute server.
include_sessions (bool, default: True) – Determines whether the search covers the project’s acquired sessions alongside its datasets.

Return type:

_ProjectMarkers

Returns:

The project’s forged dataset directories and its acquired sessions. The sessions are empty when the search did not cover them.

Raises:

FileNotFoundError – If the server holds no directory at the project path.
RuntimeError – If the server-side search reached only part of the project’s tree.

sollertia_forgery.server.get_server_configuration()¶

Resolves and validates the Sollertia platform compute server’s configuration data.

Return type:

ServerConfiguration

Returns:

The access credentials, data root, and environment name recorded for the remote compute server.

Raises:

FileNotFoundError – If the ‘server_configuration.yaml’ file does not exist in the local Sollertia platform working directory.
ValueError – If the loaded server configuration is unconfigured or contains placeholder access credentials.

sollertia_forgery.server.get_server_configuration_path()¶

Returns the path to the server_configuration.yaml file under the Sollertia platform working directory.

Return type:: Path

Notes

The path is composed rather than read, so it resolves for a working directory that holds no such file yet.

sollertia_forgery.server.remote_state_directory(project)¶

Returns the local directory mirroring one remote project’s state artifacts.

Notes

The mirror reproduces the project directory by name, so an artifact pulled into it keeps the filename its writer derived from the project.

Parameters:: project (str) – The name of the project whose remote state is mirrored.
Return type:: Path

sollertia_forgery.server.remote_state_path()¶

Returns the local directory holding everything this host records about remote runs.

Return type:: Path

Notes

One directory holds both halves of what a remote run leaves behind, namely the state artifacts pulled from the server and this host’s own record of what it submitted.

sollertia_forgery.server.server.TERMINAL_JOB_STATUSES: frozenset[JobStatus] = frozenset({JobStatus.BLOCKED, JobStatus.BOOT_FAIL, JobStatus.CANCELLED, JobStatus.COMPLETED, JobStatus.DEADLINE, JobStatus.FAILED, JobStatus.NODE_FAIL, JobStatus.OUT_OF_MEMORY, JobStatus.PREEMPTED, JobStatus.REVOKED, JobStatus.TIMEOUT})¶

The statuses a job never leaves. Reaching one tells a caller that a polled submission has settled.

Notes

UNKNOWN and UNRESOLVED are both absent. A row this enumeration cannot parse still proves that the scheduler holds the allocation, and a successful query that returned no row for it covers a submission accounting has yet to register as well as one it has purged. Nothing observable separates those two, so an allocation that stays unresolvable never settles its batch on its own. The remote status read classifies such a batch as stalled and names the allocations, and the caller retires it explicitly.

Agnostic Substrate¶

Provides the system-agnostic substrate shared across the acquisition systems.

class sollertia_forgery.shared_assets.AssemblyGeometry(reference_samples, source_samples)¶

Bases: object

Describes the shape one session’s assembly job takes, as the system that assembles the session reports it.

Notes

An assembly holds two families of arrays at once, and they are counted on different clocks. The frame it builds carries one column per assembled value at the height of the reference clock, while the sources it reads arrive on the clock on which each of them was sampled. A fast camera writes far more samples than the reference clock on which the assembly settles, so the two families diverge by the ratio between those rates and neither one bounds the other.

This record states sample counts alone. What a sample of the assembled frame costs, and what a sample of a source costs, are properties of the assembly stage rather than of the data. The system that donates this record answers only for its own data.

reference_samples: int¶: The samples the reference clock holds, which is the height of the frame the assembly job builds.

source_samples: tuple[int, ...]¶: The samples held by each source the assembly reads, counted on that source’s own clock, one entry per source. A source the assembly reads through several files of equal height contributes one entry, since those files share a clock.

class sollertia_forgery.shared_assets.OpenMPStatus(*values)¶

Bases: StrEnum

Defines the outcome of a request to make the OpenMP runtime loadable.

AVAILABLE = 'available'¶: The runtime already loads, so nothing was changed.

LINKED = 'linked'¶: The discovered runtime was linked into a directory the dynamic loader searches.

PREVIEWED = 'previewed'¶: The link was resolved and reported as a dry run, so nothing was changed.

UNRESOLVED = 'unresolved'¶: No OpenMP runtime was found to link, so nothing was changed.

class sollertia_forgery.shared_assets.ProcessingPipelines(*values)¶

Bases: StrEnum

Enumerates the data processing and management pipelines an agent orchestrates over a project, its sessions, and its datasets.

Notes

The member names mirror the corresponding members of sollertia-shared-assets’ ProcessingTrackers enum. The values here are short pipeline identifiers, so a caller resolves a tracker filename through that enum. Cindra’s multi-recording stage has no member in either enum, since its tracker is written once per dataset inside a dataset-named directory that no fixed per-session path addresses. This library resolves that stage’s paths through cindra’s own resolve_dataset_path instead.

CHECKSUM = 'checksum'¶: The raw data integrity (checksum) verification pipeline.

FORGING = 'forging'¶: The dataset assembly (forging) pipeline.

MANIFEST = 'manifest'¶: The project manifest generation pipeline.

MICROCONTROLLER = 'microcontroller'¶: The microcontroller log processing pipeline.

RUNTIME = 'runtime'¶: The acquisition-runtime log processing pipeline.

TWO_PHOTON = 'two_photon'¶: The single-recording two-photon (calcium-imaging) processing pipeline.

VIDEO = 'video'¶: The camera video-processing pipeline.

sollertia_forgery.shared_assets.count_feather_rows(feather_path)¶

Counts the rows one feather holds, reading the file’s own metadata rather than its values.

Notes

A feather’s IPC footer states the rows held by every record batch the file carries, so a length query projects no column at all and answers from that footer. The read therefore costs the same on a feather of any height.

Parameters:: feather_path (Path) – The path to the feather whose height is read.
Return type:: int
Returns:: The rows the feather holds.

sollertia_forgery.shared_assets.delay_terminal()¶

Delays the runtime execution for 100 milliseconds, so consecutive terminal printouts stay visually separated.

Return type:: None

sollertia_forgery.shared_assets.merge_event_streams(timestamps_a, values_a, timestamps_b, values_b)¶

Merges two chronologically-sorted event streams into a single timestamp-sorted stream.

Notes

Uses NumPy’s stable sort (kind="stable"), so events sharing a timestamp keep the first stream ahead of the second.

Parameters:

timestamps_a (NDArray[uint64]) – The timestamps of the first event stream.
values_a (NDArray[TypeVar(ScalarT, bound= generic)]) – The values of the first event stream.
timestamps_b (NDArray[uint64]) – The timestamps of the second event stream.
values_b (NDArray[TypeVar(ScalarT, bound= generic)]) – The values of the second event stream.

Return type:

tuple[NDArray[uint64], NDArray[TypeVar(ScalarT, bound= generic)]]

Returns:

The chronologically sorted timestamps and the values reordered to match them.

sollertia_forgery.shared_assets.multi_recording_dataset_name(animal_id, dataset_name)¶

Returns the cindra multi-recording dataset name under which one animal’s recordings are tracked within a forged dataset.

Notes

Qualifying the dataset name with the animal identifier keeps an animal’s multi-recording outputs separate when a dataset spans several animals.

Parameters:

animal_id (str) – The identifier of the animal whose recordings are tracked together.
dataset_name (str) – The unqualified forged dataset name.

Return type:

str

Returns:

The {animal_id}_{dataset_name} name under which cindra records the animal’s multi-recording output.

sollertia_forgery.shared_assets.natural_sort(frame, by, *, nulls_last=False)¶

Orders a frame on the named string columns the way a reader reads them, so 2 precedes 10.

Notes

Each column is ranked over its distinct values alone and the frame is then ordered on the resulting integers. A column holds far fewer distinct identifiers than rows, so the text comparison runs over the small set and the row ordering stays a native integer sort.

Parameters:

frame (DataFrame) – The frame to order.
by (Sequence[str]) – The names of the columns on which to order, in precedence order.
nulls_last (bool, default: False) – Determines whether rows holding no value for a column sort after the rows that do.

Return type:

DataFrame

Returns:

The ordered frame, carrying the same columns it was given.

sollertia_forgery.shared_assets.posix_text(path)¶

Renders one path as the forward-slash text that every host and every stored record reads.

Notes

A path composed on Windows separates its components with a backslash, which a POSIX compute server reads as a literal character inside a name rather than as a separator. Rendering here holds every path this library sends to the server, and every path it records as part of a job’s identity, to the one form both platforms parse.

Parameters:: path (PurePath) – The path to render.
Return type:: str
Returns:: The path as forward-slash separated text.

sollertia_forgery.shared_assets.resolve_openmp_runtime(*, runtime_path=None, link_path=None, execute=False, force=False)¶

Links a discovered OpenMP runtime into a directory the dynamic loader searches by default.

The link is what makes Numba’s OpenMP threading layer resolve on macOS, because the omppool extension shipped in the Numba wheel names its dependency through an rpath for which it carries no entries. A host whose runtime already loads is left alone unless the link is forced. Only macOS reaches the linking path, because every other platform runs the TBB threading layer and gains nothing from an OpenMP runtime.

Parameters:

runtime_path (Path | None, default: None) – The absolute path to the OpenMP runtime to link, or None to search the macOS package manager directories, the active conda environment, and the installed Python distributions for one.
link_path (Path | None, default: None) – The absolute path at which to write the link, or None to derive it from the directory the loader searches by default.
execute (bool, default: False) – Determines whether to create the resolved link, where a dry run reports it and changes nothing.
force (bool, default: False) – Determines whether to link a runtime on a host whose runtime already loads.

Return type:

_OpenMPSummary

Returns:

The summary describing the resolved runtime, the resolved link, and what the call changed.

Raises:

RuntimeError – If the host runs a platform other than macOS, if the resolved link path already holds the runtime itself, if the link directory cannot be created, or if the link cannot be written.

sollertia_forgery.shared_assets.resolve_session_tracker_path(session, pipeline)¶

Resolves the processing tracker on which one per-session pipeline records its jobs.

Parameters:

session (SessionData) – The loaded session whose tracker location to resolve.
pipeline (ProcessingPipelines) – The pipeline whose tracker to resolve. Must be one of SESSION_PIPELINES.

Return type:

Path

Returns:

The path to that pipeline’s processing tracker for this session.

Raises:

ValueError – If the pipeline operates on a unit other than a single session.

sollertia_forgery.shared_assets.verify_openmp_runtime()¶

Aborts the runtime when Numba’s OpenMP threading layer has no runtime to load on this macOS host.

Return type:: None

Notes

A host that cannot run a parallelized kernel fails while it has done no work rather than partway through a session. The error replaces the threading-layer error Numba raises at the first parallelized call, which names no remedy. The platforms whose threading layer needs no separately installed runtime return without checking anything.

This library selects the threading layer for the whole process, so the guarantee covers every kernel the process may compile, including the parallelized stages its dependencies own.

Raises:: RuntimeError – If the host is macOS and no OpenMP runtime is loadable.

sollertia_forgery.shared_assets.pipelines.SESSION_PIPELINES: tuple[ProcessingPipelines, ...] = (ProcessingPipelines.CHECKSUM, ProcessingPipelines.RUNTIME, ProcessingPipelines.MICROCONTROLLER, ProcessingPipelines.VIDEO, ProcessingPipelines.TWO_PHOTON)¶

The pipelines that process a single session, in the order a per-pipeline report presents them.

Notes

Derived from the tracker mapping, so the two can never disagree about which pipelines a session carries. The manifest and forging pipelines are absent, since one operates on a project and the other on a dataset.

Command-Line Interface¶

slf¶

Processes and manages data acquired with the Sollertia data acquisition platform.

Exposes system-agnostic management commands (‘manifest’, ‘checksum’, ‘dataset-state’, ‘server’, ‘reset’, ‘clean’, ‘omp’), the agentic MCP server (‘mcp’), and the generic processing, forging, and planning commands (‘process’, ‘forge’, ‘plan’). The acquisition system is inferred from the data, so no command takes a system selector.

Usage

slf [OPTIONS] COMMAND [ARGS]...

checksum¶

Resolves the data integrity checksum for the target session’s ‘raw_data’ directory.

Verifies the stored checksum by default, and recalculates it to absorb expected changes when –regenerate-checksum is given.

Usage

slf checksum [OPTIONS]

Options

-sp, --session-path <session_path>¶: Required The absolute path to the processed session’s root data directory.

-rc, --regenerate-checksum¶

Determines whether to recalculate and overwrite the cached session’s checksum value. When the command is called with this flag, it re-checksums the data instead of verifying its integrity.

Default:: False

-w, --workers <workers>¶

The number of parallel worker processes to use for hashing the session’s files. Values below 1 request all available cores minus the reserved system cores, and a value of 1 disables parallelism.

Default:: -1

-np, --no-progress¶

Determines whether to suppress the preamble message and progress bar during checksum resolution. These are displayed by default.

Default:: False

clean¶

Removes a pipeline’s output and processing tracker for the named units.

Returns each unit to an unprocessed state, so a later preparation rediscovers every job from the acquired data rather than resuming a partial run. Each removed path is reported with the bytes it held, one per line.

Usage

slf clean [OPTIONS]

Options

-p, --pipeline <pipeline>¶

Required The pipeline whose output to remove.

Options:: checksum | forging | microcontroller | runtime | two_photon | video

-up, --unit-path <unit_path>¶: Required The absolute path to a processing unit to clean. Can be specified multiple times.

dataset-state¶

Snapshots each named dataset’s forging job state into a shippable table at the dataset root.

The snapshot is what carries a dataset’s forging progress to another host, since the project manifest reports one row per session while a dataset’s jobs sit at differing scopes.

Usage

slf dataset-state [OPTIONS]

Options

-dp, --dataset-path <dataset_path>¶: Required The absolute path to a dataset root directory to snapshot. Can be specified multiple times.

forge¶

Forges a dataset by assembling per-session data from a project’s processed sessions.

Provided sessions that the dataset does not hold are appended to it, so a dataset grows by naming the sessions to add. An animal already in the dataset is frozen, because widening its session set invalidates the outputs already forged for it and requires rebuilding the animal as a whole. Name that animal with –recreate-animal to rebuild it from the provided sessions while every other animal keeps its data. A rebuild discards the cross-recording output written by this dataset into the animal’s source sessions, so its recordings are tracked afresh.

The forging pipeline is system-agnostic: it resolves the dataset’s system-specific assembly worker internally from the central registry and infers the acquisition system from the resolved dataset, so the command carries no system selector.

Usage

slf forge [OPTIONS]

Options

-dn, --dataset-name <dataset_name>¶: Required The unique name for the dataset to create or use.

-pp, --project-path <project_path>¶: Required The path to the project’s root data directory storing the animal and session data directories.

-s, --session <session>¶: The session name the dataset must contain. A session the dataset does not hold is appended to it. Can be specified multiple times.

-id, --job-id <job_id>¶: The unique hexadecimal identifier for this job. If provided, runs only the matching job (remote mode).

-w, --workers <workers>¶

The number of worker processes to use for the multi-day and assembly stages. Set to -1 for automatic resolution.

Default:: -1

-f, --force-recreate¶

Determines whether to delete the whole existing dataset hierarchy and rebuild it from the provided sessions.

Default:: False

-ra, --recreate-animal <recreate_animal>¶: The identifier of an animal already in the dataset to rebuild from the sessions provided for it, leaving every other animal untouched. Can be specified multiple times.

-np, --no-progress¶

Determines whether to suppress the progress bar during the assembly stage. It also sets the flag recorded in each animal’s multi-recording configuration when this call defines or rebuilds the dataset. These are displayed by default.

Default:: False

manifest¶

Generates and inspects the project manifest .feather file that snapshots a project’s state.

The project path is parsed on this group and shared by every subcommand, so it must be given before the subcommand name.

Usage

slf manifest [OPTIONS] COMMAND [ARGS]...

Options

-pp, --project-path <project_path>¶: The absolute path to the project’s root data directory.

create¶

Creates the manifest .feather file that captures the snapshot of the target project’s state.

An existing manifest for the project is recreated (overwritten) with a fresh snapshot. The project job artifact, ‘<project>_jobs.feather’, is published into the same root ahead of the manifest itself.

Usage

slf manifest create [OPTIONS]

Options

-np, --no-progress¶

Determines whether to suppress the preamble and completion messages during manifest generation. These messages are displayed by default.

Default:: False

print¶

Prints the requested data from the target project’s manifest file to the terminal as a formatted table.

Usage

slf manifest print [OPTIONS]

Options

-a, --animal <animal>¶: The identifier of the animal for which to print the manifest data. If not provided, this command prints the data for all animals participating in the target project.

-n, --notes¶

Determines whether to print the ‘experimenter notes’ view of the available manifest data. This data view is optimized for checking the outcome of each data acquisition session conducted for the target project.

Default:: False

-s, --summary¶

Determines whether to print the ‘data processing’ view of the available manifest data. This view is optimized for tracking the data processing state of each data acquisition session conducted for the target project.

Default:: False

mcp¶

Starts the agentic Model Context Protocol server using the requested transport.

The ‘stdio’ transport exchanges messages over the standard input and output streams of this process, which is the transport that local agent clients are expected to use. The ‘sse’ and ‘streamable-http’ transports instead serve the same tools over the network, which is how remote agent clients reach a server running on the processing host.

Usage

slf mcp [OPTIONS]

Options

-t, --transport <transport>¶

The transport protocol the MCP server uses to communicate with the connected client.

Default:: 'stdio'
Options:: stdio | sse | streamable-http

omp¶

Links the OpenMP runtime that the Numba threading layer loads on macOS into a directory the loader searches.

The Numba macOS wheel names its OpenMP dependency through an rpath that carries no entries. The loader therefore expands that name against the entries the running interpreter carries, and the only directory it reaches is that interpreter’s own library directory. This command finds an installed runtime and links it into that directory. A conda environment grants that write without sudo, while a system-wide interpreter needs it. Running the command on any other platform errors, because those platforms run the TBB threading layer instead.

Usage

slf omp [OPTIONS]

Options

-s, --source <source>¶: The path to the OpenMP runtime to link. Omit to search the macOS package manager directories, the active conda environment, and the installed Python distributions for one.

-t, --target <target>¶: The path receiving the link. Omit to derive it from the directory the dynamic loader searches by default.

-f, --force¶: Determines whether to link a runtime on a host whose OpenMP runtime already loads.

-y, --yes¶: Determines whether to create the resolved link. Without this flag the command reports what it would do and changes nothing.

plan¶

Estimates and records the cores and memory every processing or forging job will occupy.

Each unit’s figures are cached beside its outputs and are kept across later runs, so a submission is never sized against figures that changed after it was planned. Use ‘project’ to project every cache under a project into the single table that ships.

Usage

slf plan [OPTIONS] COMMAND [ARGS]...

dataset¶

Records what every forging job of each named dataset will cost.

Usage

slf plan dataset [OPTIONS]

Options

-dp, --dataset-path <dataset_path>¶: Required The absolute path to a dataset root directory to plan. Can be specified multiple times.

-rp, --regenerate-plan¶

Determines whether to recalculate and overwrite the figures the dataset’s plan already holds. When the command is called with this flag, it re-estimates every job instead of only the ones the plan lacks.

Default:: False

project¶

Projects every plan cache under the project into one table at the project root.

Reads the plan caches already on disk, so the table covers exactly the units that have been planned.

Usage

slf plan project [OPTIONS]

Options

-pp, --project-path <project_path>¶: Required The absolute path to the project root directory whose plan caches are collected.

session¶

Records what every processing job of each named session will cost.

Usage

slf plan session [OPTIONS]

Options

-sp, --session-path <session_path>¶: Required The absolute path to a session root directory to plan. Can be specified multiple times.

-rp, --regenerate-plan¶

Determines whether to recalculate and overwrite the figures the session’s plan already holds. When the command is called with this flag, it re-estimates every job instead of only the ones the plan lacks.

Default:: False

process¶

Runs the requested data extraction pipelines on a single session.

The session path, job id, worker budget, and no-progress flag are parsed on this group and shared by every subcommand, so they must be given before the subcommand name.

Usage

slf process [OPTIONS] COMMAND [ARGS]...

Options

-sp, --session-path <session_path>¶: The absolute path to the session root directory to process.

-id, --job-id <job_id>¶: The unique hexadecimal identifier for this processing job. If provided, runs only the matching job (remote mode). If not provided, discovers and runs every available job for the session (local mode). The ‘runtime’ subcommand ignores it, since its single-job pipeline has no remote-dispatch job.

-w, --workers <workers>¶

The number of parallel workers to use for processing. Set to -1 for automatic resolution. Set to 1 for sequential execution. Jobs that parallelize internally consume this budget as their per-job core allotment, and single-core jobs treat it as a no-op.

Default:: -1

-np, --no-progress¶

Determines whether to suppress the progress bar during processing. The progress bar is displayed by default.

Default:: False

microcontroller¶

Extracts the microcontroller module log archives and parses them into domain-specific behavior feathers.

Usage

slf process microcontroller [OPTIONS]

runtime¶

Decodes the acquisition runtime log archive and parses it into the session’s runtime behavior feathers.

Usage

slf process runtime [OPTIONS]

two-photon¶

Runs the single-recording two-photon (calcium-imaging) processing pipeline for a session.

The acquisition system resolves the cindra processing configuration. When none of ‘–binarize’, ‘–register’, ‘–process’, or ‘–combine’ is requested, all four stages run in sequence (local mode). Supplying ‘–job-id’ instead runs only the matching job.

Usage

slf process two-photon [OPTIONS]

Options

-b, --binarize¶: Determines whether to run the binarization stage.

-r, --register¶: Determines whether to run the per-plane registration stage.

-p, --process¶: Determines whether to run the per-plane processing stage.

-cb, --combine¶: Determines whether to run the multi-plane combination stage.

-tp, --target-plane <target_plane>¶

The imaging plane that the per-plane stages process. Set to -1 to cover all planes.

Default:: -1

video¶

Extracts camera frame timestamps, processes pose predictions, and measures per-camera motion energy.

When none of ‘–timestamp’, ‘–track’, or ‘–energy’ is requested, all three stages run (local mode). Supplying ‘–job-id’ instead runs only the matching job.

Usage

slf process video [OPTIONS]

Options

-ts, --timestamp¶: Determines whether to run the timestamp stage, which parses each camera’s log archive into a frame-timestamp feather and republishes it under its canonical manifest name. Ignored when ‘–job-id’ is provided (remote mode selects the job by ID).

-tr, --track¶: Determines whether to run the video-tracking stage, which post-processes externally-produced pose predictions (e.g. DeepLabCut ‘.h5’ files) into tracking feathers. It is a no-op when no predictions are present. Ignored when ‘–job-id’ is provided (remote mode selects the job by ID).

-en, --energy¶: Determines whether to run the motion-energy stage, which measures each camera’s recording into a per-frame movement signal. It is a no-op for a camera whose recording is absent. Ignored when ‘–job-id’ is provided (remote mode selects the job by ID).

-tc, --target-camera <target_camera>¶

The numeric source ID of the single camera to process for the timestamp and motion-energy stages. Set to -1 to process every camera. Ignored when ‘–job-id’ is provided (remote mode selects the job by ID).

Default:: -1

reset¶

Returns tracked jobs of the named units to the scheduled state, leaving every untargeted job’s record intact.

One invocation covers every named unit, and each unit resets only the identifiers it actually tracks, so a batch spanning many units costs a single call.

Usage

slf reset [OPTIONS]

Options

-p, --pipeline <pipeline>¶

Required The pipeline whose tracked jobs to reset.

Options:: checksum | forging | microcontroller | runtime | two_photon | video

-up, --unit-path <unit_path>¶: Required The absolute path to a processing unit whose jobs to reset, which is a session root for a session pipeline and a dataset root for ‘forging’. Can be specified multiple times.

-id, --job-id <job_id>¶: The hexadecimal identifier of a tracked job to reset. Can be specified multiple times. Omit to reset every job each named unit tracks.

server¶

Interacts with the remote Sollertia compute server.

Authors the server access configuration, discovers a project’s sessions, reports SLURM queue and job accounting data, and resolves and remediates the batches this machine has outstanding on the scheduler.

Usage

slf server [OPTIONS] COMMAND [ARGS]...

batches¶

Reports the batches this machine still has outstanding on the remote compute server’s scheduler.

Resolves every allocation the submission ledger holds against three records, which are the scheduler’s accounting, the scheduler’s queue, and the processing tracker of the job the allocation carries. Reports per batch how long it has been outstanding and the state to which those records resolve. A batch is ‘stalled’ when none of its allocations resolves as running and at least one is gone from both scheduler records, which no later query changes, and the reported remedy names the command that remediates it. Closes and retires any batch whose allocations all resolve to a plain drop, exactly as the agentic status read does.

Usage

slf server batches [OPTIONS]

Options

-b, --batch-id <batch_id>¶: The identifier of an outstanding batch to report. Can be specified multiple times. Omit to report every outstanding batch.

-a, --allocations¶

Determines whether to additionally report every allocation on its own row, carrying the state each scheduler record places it in, what its job’s tracker holds, and the verdict they carry together.

Default:: False

configure¶

Creates the remote compute server configuration file in the Sollertia platform working directory.

Usage

slf server configure [OPTIONS]

Options

-u, --username <username>¶: Required The username to use for server authentication.

-p, --password <password>¶: The password to use for server authentication. Prompted interactively (with hidden input) if not provided.

-h, --host <host>¶: Required The host name or IP address of the server.

-r, --root <root>¶: Required The absolute path, on the remote server, to the root directory that stores all Sollertia data.

-e, --environment <environment>¶: Required The name of the shared conda environment, on the remote server, in which sollertia-forgery and all of its processing dependencies are installed.

discover¶

Discovers and prints the sessions stored under the project’s directory on the remote compute server.

Usage

slf server discover [OPTIONS]

Options

-p, --project <project>¶: Required The name of the project whose sessions to discover under the server’s data root.

print¶

Displays the remote server’s SLURM queue status or job data as a formatted table.

Usage

slf server print [OPTIONS]

Options

-j, --job-data¶

Determines whether to display the remote server’s job accounting history (runtime statistics) using SLURM’s ‘sacct’ command.

Default:: False

-q, --queue¶

Determines whether to display the remote server’s job queue status using SLURM’s ‘squeue’ command.

Default:: False

-u, --user <user>¶: Allows filtering the displayed queue and job data to only include the jobs submitted by the specified user. Set to ‘all’ to display data for all users. Defaults to the username used for the server authentication.

-id, --job-id <job_id>¶: Determines the job for which to display the queue and accounting data. Bypasses user and date filtering options.

-st, --start-time <start_time>¶: Allows filtering displayed job data to only include the jobs that started on or after this date (format: YYYY-MM-DD or YYYY-MM-DD HH:MM:SS). Only applies to the job accounting (’–job-data’) data, as the queue reports the currently active jobs and supports no date window.

-et, --end-time <end_time>¶: Allows filtering displayed job data to only include the jobs that ended on or before this date (format: YYYY-MM-DD or YYYY-MM-DD HH:MM:SS). Only applies to the job accounting (’–job-data’) data, as the queue reports the currently active jobs and supports no date window.

pull¶

Copies a file or directory off the remote compute server onto this machine.

A directory is copied whole. The copy lands inside the destination directory under the remote path’s own final component, which is how a session’s processed data, one feather, or a batch’s logs are brought back. The destination is created when it does not exist, and a destination that already holds a file of that name is overwritten.

Usage

slf server pull [OPTIONS]

Options

-r, --remote-path <remote_path>¶: Required The absolute path, on the server, to the file or directory to copy.

-d, --destination <destination>¶: Required The absolute path to the local directory that receives the copy.

retire-batch¶

Remediates the named batches and drops them from this machine’s submission ledger.

This is the remedy for a batch the ‘batches’ report calls stalled, whose ledger entry would otherwise keep it outstanding forever and keep its jobs claimed against a rerun. Every allocation is resolved exactly as that report resolves it, and a job its own tracker still claims to be running while no allocation is, which the report calls stranded, is returned to the scheduled state. A job that recorded success or failure keeps that record, so nothing a run produced is discarded here. What the batch’s jobs recorded is then snapshotted, through the same closure applied to a finished batch, so the run stays answerable once the entry is gone. A batch holding an allocation that resolves as running is refused without ‘–force’, and a snapshot that fails refuses the drop without ‘–drop-without-outcome’. This drops the ledger entries and replaces each covered batch’s prepared document with its recorded outcome, leaving the outcomes themselves in the batch registry.

Usage

slf server retire-batch [OPTIONS]

Options

-b, --batch-id <batch_id>¶: Required The identifier of an outstanding batch to retire. Can be specified multiple times.

-f, --force¶

Determines whether to remediate batches holding an allocation that resolves as running. Cancels each such allocation before any tracker is written, and abandons one whose state could not be read at all.

Default:: False

-do, --drop-without-outcome¶

Determines whether to drop the ledger entries when what their jobs recorded cannot be snapshotted, which an unreachable server is one cause of.

Default:: False

Dispatch Registries¶

Collects every acquisition-system-keyed dispatch registry in one place and runs the import-time checks that guard them.

class sollertia_forgery.registries.ForgingAssembler(*args, **kwargs)¶

Bases: Protocol

Defines the call signature of the per-session forging data-assembly worker an acquisition system donates.

class sollertia_forgery.registries.MicrocontrollerParser(*args, **kwargs)¶

Bases: Protocol

Defines the call signature of the microcontroller module parsers an acquisition system donates.

sollertia_forgery.registries.resolve_assembly_geometry_resolver(system)¶

Resolves the assembly-geometry resolver registered for the target acquisition system.

Parameters:: system (str | AcquisitionSystems) – The acquisition system that recorded the dataset being forged.
Return type:: _AssemblyGeometryResolver
Returns:: The registered resolver. It reports the samples held by the reference clock on which its system’s assembler settles, together with the samples held by each source that assembler reads. It refuses a session for which that assembler would settle on no reference clock at all.
Raises:: ValueError – If the acquisition system is unknown.

sollertia_forgery.registries.resolve_assembly_source_resolver(system)¶

Resolves the assembly-source resolver registered for the target acquisition system.

Parameters:: system (str | AcquisitionSystems) – The acquisition system that recorded the dataset being forged.
Return type:: _AssemblySourceResolver
Returns:: The registered resolver, which reports the samples held by each source the system’s assembler reads for one session, counted on that source’s own clock and routed by the session’s own type.
Raises:: ValueError – If the acquisition system is unknown.

sollertia_forgery.registries.resolve_eligible_microcontroller_modules(system, session)¶

Resolves the microcontroller modules the target session configured for use.

Parameters:

system (str | AcquisitionSystems) – The acquisition system that recorded the session being processed.
session (SessionData) – The loaded session whose hardware state determines module eligibility.

Return type:

set[tuple[int, int]]

Returns:

The (module_type, module_id) pairs the session configured for use.

Raises:

ValueError – If the acquisition system is unknown.

sollertia_forgery.registries.resolve_forging_admission_pipelines(system)¶

Resolves the per-session-type pipeline requirements a session must satisfy to join the system’s datasets.

Parameters:: system (str | AcquisitionSystems) – The acquisition system that recorded the session.
Return type:: dict[SessionTypes, frozenset[ProcessingPipelines]]
Returns:: The mapping from session type to the pipelines that must report every job as succeeded. A session type absent from the mapping joins no dataset for this system.
Raises:: ValueError – If the acquisition system is unknown.

sollertia_forgery.registries.resolve_forging_assembly_worker(system)¶

Resolves the per-session forging data-assembly worker registered for the target acquisition system.

Parameters:: system (str | AcquisitionSystems) – The acquisition system that recorded the dataset being forged.
Return type:: ForgingAssembler
Returns:: The registered assembly worker.
Raises:: ValueError – If the acquisition system is unknown.

sollertia_forgery.registries.resolve_forging_column_descriptions(system)¶

Resolves the dataset column-description mapping registered for the target acquisition system.

Parameters:: system (str | AcquisitionSystems) – The acquisition system that recorded the dataset being forged.
Return type:: dict[str, str]
Returns:: The mapping from each column name that the system’s assembly worker can emit into data.feather to its human-readable description.
Raises:: ValueError – If the acquisition system is unknown.

sollertia_forgery.registries.resolve_microcontroller_event_codes(system)¶

Resolves the microcontroller module event codes registered for the target acquisition system.

Parameters:: system (str | AcquisitionSystems) – The acquisition system that recorded the session being processed.
Return type:: dict[tuple[int, int], tuple[int, ...]]
Returns:: A mapping from each (module_type, module_id) pair that the system parses to the event codes its parser reads.
Raises:: ValueError – If the acquisition system is unknown.

sollertia_forgery.registries.resolve_microcontroller_parsers(system)¶

Resolves the microcontroller module parsers registered for the target acquisition system.

Parameters:: system (str | AcquisitionSystems) – The acquisition system that recorded the session being processed.
Return type:: dict[tuple[int, int], MicrocontrollerParser]
Returns:: A mapping from each (module_type, module_id) pair that the system parses to its parser.
Raises:: ValueError – If the acquisition system is unknown.

sollertia_forgery.registries.resolve_multi_recording_configuration_resolver(system)¶

Resolves the multi-recording cindra configuration resolver registered for the target acquisition system.

Parameters:: system (str | AcquisitionSystems) – The acquisition system that recorded the dataset being forged.
Return type:: Callable[[SessionData], MultiRecordingConfiguration | None]
Returns:: The registered resolver, which returns None when the system performs no cross-recording tracking for the session.
Raises:: ValueError – If the acquisition system is unknown.

sollertia_forgery.registries.resolve_multi_recording_session_types(system)¶

Resolves the session types the target acquisition system tracks across recordings.

Notes

Answers whether a session type carries cross-recording tracking without the loaded session that the multi-recording configuration resolver requires. A caller holding a dataset therefore reads the answer from the dataset’s own recorded session type rather than from its animals’ source data.

Parameters:: system (str | AcquisitionSystems) – The acquisition system that recorded the dataset being forged.
Return type:: frozenset[SessionTypes]
Returns:: The session types whose animals the system registers against each other, which is empty for a system that performs no cross-recording tracking.
Raises:: ValueError – If the acquisition system is unknown.

sollertia_forgery.registries.resolve_pose_prediction_locator(system)¶

Resolves the pose-prediction locator registered for the target acquisition system.

Parameters:: system (str | AcquisitionSystems) – The acquisition system that recorded the session being processed.
Return type:: _PosePredictionLocator
Returns:: The registered locator, which returns the path to the session’s pose-prediction file, or None when the session carries none.
Raises:: ValueError – If the acquisition system is unknown.

sollertia_forgery.registries.resolve_runtime_binding(system)¶

Resolves the runtime source id and parser registered for the target acquisition system.

Parameters:: system (str | AcquisitionSystems) – The acquisition system that recorded the session being processed.
Return type:: tuple[str, _RuntimeParser]
Returns:: A (source_id, parser) tuple. The source id locates the system’s runtime DataLogger archive, and the parser interprets the decoded runtime messages into the system’s behavior feathers.
Raises:: ValueError – If the acquisition system is unknown.

sollertia_forgery.registries.resolve_single_recording_configuration_resolver(system)¶

Resolves the single-recording cindra configuration resolver registered for the target acquisition system.

Parameters:: system (str | AcquisitionSystems) – The acquisition system that recorded the session being processed.
Return type:: Callable[[SessionData], SingleRecordingConfiguration]
Returns:: The registered resolver, which returns the system’s single-recording cindra configuration for one session.
Raises:: ValueError – If the acquisition system is unknown.

sollertia_forgery.registries.resolve_two_photon_data_locator(system)¶

Resolves the raw two-photon imaging directory locator registered for the target acquisition system.

Parameters:: system (str | AcquisitionSystems) – The acquisition system that recorded the session being processed.
Return type:: _TwoPhotonDataLocator
Returns:: The registered locator, which resolves the session’s raw two-photon imaging directory.
Raises:: ValueError – If the acquisition system is unknown.

sollertia_forgery.registries.resolve_video_tracking(system)¶

Resolves the video-tracking function registered for the target acquisition system.

Parameters:: system (str | AcquisitionSystems) – The acquisition system that recorded the session being processed.
Return type:: _VideoTracker
Returns:: The registered tracking function, which performs all of that system’s video tracking and no-ops when no DeepLabCut predictions are present.
Raises:: ValueError – If the acquisition system is unknown.

Mesoscope-VR Assets¶

Provides the Mesoscope-VR system-specific assets donated to the system-agnostic worker packages.

class sollertia_forgery.mesoscope_vr.DatasetColumn(*values)¶

Bases: StrEnum

Defines every column name that can appear in the assembled session data feather produced by the forging pipeline.

Notes

Each member’s value is the column name as it appears in data.feather, so members compare equal to the raw column strings.

Several columns are conditional. The runtime columns (TRIAL, TRIAL_TYPE, CUE, IN_TRIGGER_ZONE, RUNTIME_STATE), the fluorescence columns (FRAME and the SINGLE_DAY_* and MULTI_DAY_* members), BRAKE, and SCREENS are present only for mesoscope experiment sessions. REINFORCING_GUIDED and AVERSIVE_GUIDED are present only when the corresponding guidance events were recorded. TORQUE_N_CM is absent for run training. DISTANCE_CM and SPEED_CM_S are absent for lick training. The per-camera video columns are present only when that camera’s feathers were produced, and the pupil columns only when the face camera’s pose predictions were processed. The remaining behavior columns (TIME_US, ELAPSED_MINUTES, LICK, WATER_UL, REWARD, SYSTEM_STATE) are present in every forged session.

AVERSIVE_GUIDED = 'aversive_guided'¶: Aversive guidance state at each sample. Present only when aversive guidance was recorded.

BLINKING_STATE = 'blinking_state'¶: Blink state at each sample, marking a closed or covered eye. Encoded as 1 during a blink and 0 otherwise.

BODY_CAMERA_FRAME_LUMINANCE = 'body_camera_frame_luminance'¶: Body camera mean frame intensity at each sample in gray levels, tracking scene illumination.

BODY_CAMERA_MOTION_ENERGY = 'body_camera_motion_energy'¶: Body camera motion energy at each sample, the mean absolute inter-frame intensity change in gray levels. A within-session movement magnitude, high during movement and low during stillness.

BRAKE = 'brake'¶: The running wheel brake engagement at each sample.

CUE = 'cue'¶: Active virtual reality cue identifier at each sample. 255 marks samples outside the run state.

DILATION_STATE = 'dilation_state'¶: Dilation-clip state at each sample, marking a pupil dilated past the eye aperture. Encoded as 1 when clipped and 0 otherwise.

DISTANCE_CM = 'distance_cm'¶: Cumulative distance traveled by the animal in centimeters at each sample.

ELAPSED_MINUTES = 'elapsed_minutes'¶: Elapsed time in minutes since the first sample of the session’s reference clock. The clock starts during setup, so the clipped dataset’s first row carries a value above zero.

EYE_CENTER_X_PX = 'eye_center_x_px'¶: Horizontal position of the fitted eye ellipse center in face-camera pixels at each sample.

EYE_CENTER_Y_PX = 'eye_center_y_px'¶: Vertical position of the fitted eye ellipse center in face-camera pixels at each sample.

EYE_HEIGHT_PX = 'eye_height_px'¶: Length of the fitted eye ellipse’s top-bottom chord in face-camera pixels at each sample.

EYE_OPENNESS = 'eye_openness'¶: Ratio of the fitted eye ellipse’s height to its width at each sample, a distance-invariant measure of eye openness.

EYE_WIDTH_PX = 'eye_width_px'¶: Length of the fitted eye ellipse’s left-right chord in face-camera pixels at each sample.

FACE_CAMERA_FRAME_LUMINANCE = 'face_camera_frame_luminance'¶: Face camera mean frame intensity at each sample in gray levels, tracking scene illumination.

FACE_CAMERA_MOTION_ENERGY = 'face_camera_motion_energy'¶: Face camera motion energy at each sample, the mean absolute inter-frame intensity change in gray levels. A within-session movement magnitude, high during movement and low during stillness.

FRAME = 'frame'¶: One-based mesoscope acquisition frame index at each sample.

IN_TRIGGER_ZONE = 'in_trigger_zone'¶: Boolean flag indicating whether the animal is inside a stimulus trigger zone at each sample, as the Virtual Reality runtime registers it. The zone extends one actor lead beyond each declared boundary, because the actor’s collider surface crosses the zone before and after its tracked position does.

LICK = 'lick'¶: Lick sensor engagement state at each sample.

MULTI_DAY_CELL_FLUORESCENCE = 'multi_day_cell_fluorescence'¶: Multi-recording raw cell fluorescence trace per ROI, over the cells tracked across every session of this animal in the dataset.

MULTI_DAY_NEUROPIL_FLUORESCENCE = 'multi_day_neuropil_fluorescence'¶: Multi-recording raw neuropil fluorescence trace per ROI, over the cells tracked across every session of this animal in the dataset.

MULTI_DAY_SPIKES = 'multi_day_spikes'¶: Multi-recording OASIS-deconvolved spike rates per ROI, over the cells tracked across every session of this animal in the dataset.

MULTI_DAY_SUBTRACTED_FLUORESCENCE = 'multi_day_subtracted_fluorescence'¶: Multi-recording neuropil- and baseline-subtracted fluorescence trace per ROI, over the cells tracked across every session of this animal in the dataset.

PUPIL_AREA_PX2 = 'pupil_area_px2'¶: Area enclosed by the fitted pupil ellipse in square face-camera pixels at each sample.

PUPIL_CENTER_X_PX = 'pupil_center_x_px'¶: Horizontal position of the fitted pupil ellipse center in face-camera pixels at each sample.

PUPIL_CENTER_Y_PX = 'pupil_center_y_px'¶: Vertical position of the fitted pupil ellipse center in face-camera pixels at each sample.

PUPIL_DIAMETER_PX = 'pupil_diameter_px'¶: Mean of the fitted pupil ellipse’s two axis diameters in face-camera pixels at each sample, the primary arousal proxy. NaN marks blink and dilation samples.

PUPIL_FIT_CONDITION = 'pupil_fit_condition'¶: Condition number of the fitted pupil ellipse at each sample, where higher values indicate a less reliable fit. NaN marks unmeasured samples.

PUPIL_FIT_RESIDUAL_PX = 'pupil_fit_residual_px'¶: Root-mean-square distance in pixels between the confident pupil-perimeter points and the fitted ellipse at each sample, a fit-quality measure.

PUPIL_IN_EYE_X = 'pupil_in_eye_x'¶: Horizontal offset of the pupil center from the eye center at each sample, normalized to the eye ellipse’s horizontal semi-axis and dimensionless.

PUPIL_IN_EYE_Y = 'pupil_in_eye_y'¶: Vertical offset of the pupil center from the eye center at each sample, normalized to the eye ellipse’s vertical semi-axis and dimensionless.

PUPIL_REFLECTION_OFFSET_X_PX = 'pupil_reflection_offset_x_px'¶: Horizontal offset of the pupil center from the corneal reflection in pixels at each sample, a motion-robust horizontal eye-position signal.

PUPIL_REFLECTION_OFFSET_Y_PX = 'pupil_reflection_offset_y_px'¶: Vertical offset of the pupil center from the corneal reflection in pixels at each sample, a motion-robust vertical eye-position signal.

REFLECTION_X_PX = 'reflection_x_px'¶: Horizontal position of the corneal reflection in face-camera pixels at each sample.

REFLECTION_Y_PX = 'reflection_y_px'¶: Vertical position of the corneal reflection in face-camera pixels at each sample.

REINFORCING_GUIDED = 'reinforcing_guided'¶: Reinforcing guidance state at each sample. Present only when reinforcing guidance was recorded.

REWARD = 'reward'¶: The reward-classification state at each sample, one of ‘no’ (no reward tone playing), ‘tone’ (reward tone playing with no water delivered during the tone), or ‘yes’ (reward tone playing with water delivered during the tone).

RUNTIME_STATE = 'runtime_state'¶: Experiment runtime state label at each sample.

SCREENS = 'screens'¶: The Virtual Reality display state at each sample.

SINGLE_DAY_CELL_FLUORESCENCE = 'single_day_cell_fluorescence'¶: Single-recording raw cell fluorescence trace per ROI, over the cells detected in this session alone.

SINGLE_DAY_NEUROPIL_FLUORESCENCE = 'single_day_neuropil_fluorescence'¶: Single-recording raw neuropil fluorescence trace per ROI, over the cells detected in this session alone.

SINGLE_DAY_SPIKES = 'single_day_spikes'¶: Single-recording OASIS-deconvolved spike rates per ROI, over the cells detected in this session alone.

SINGLE_DAY_SUBTRACTED_FLUORESCENCE = 'single_day_subtracted_fluorescence'¶: Single-recording neuropil- and baseline-subtracted fluorescence trace per ROI, over the cells detected in this session alone.

SPEED_CM_S = 'speed_cm_s'¶: Animal’s running speed in cm/s at each sample.

SYSTEM_STATE = 'system_state'¶: Acquisition system state at each sample (idle, rest, run for experiment sessions, lick training or run training for training sessions).

TIME_US = 'time_us'¶: Microsecond-precision sample timestamps from the acquisition reference clock.

TORQUE_N_CM = 'torque_N_cm'¶: The torque applied by the animal to the running wheel in N·cm at each sample, forced to zero during ‘run’ periods upstream.

TRIAL = 'trial'¶: One-based trial identifier at each sample. 65535 marks samples outside the run state.

TRIAL_TYPE = 'trial_type'¶: Trial type label at each sample (e.g. ‘ABC’, ‘ABCD’). ‘undefined’ marks non-run samples.

WATER_UL = 'water_uL'¶: The cumulative water reward volume delivered to the animal at each sample in microliters.

sollertia_forgery.mesoscope_vr.assemble_mesoscope_session(source_session_path, output_path, dataset_name)¶

Assembles a single Mesoscope-VR session’s unified data feather, routing by session type.

Parameters:

source_session_path (Path) – The path to the source session’s root directory in the project hierarchy.
output_path (Path) – The path to the data.feather file to write inside the forged dataset hierarchy.
dataset_name (str) – The unqualified dataset name, forwarded to the experiment assembler to resolve the cindra multi-recording output directory.

Raises:

FileNotFoundError – If a required processed-data directory or reference clock is missing (propagated from the resolved assembler).
ValueError – If the session type is not a supported forging session type, or if a sub-dataset cannot be assembled (propagated from the resolved assembler).

Return type:

None

sollertia_forgery.mesoscope_vr.get_eligible_modules(session)¶

Returns the Mesoscope-VR hardware modules the target session configured for use.

Parameters:: session (SessionData) – The loaded session whose hardware state determines module eligibility.
Return type:: set[tuple[int, int]]
Returns:: The (module_type, module_id) pairs the session configured for use.

sollertia_forgery.mesoscope_vr.get_module_event_codes()¶

Returns the axci event codes each Mesoscope-VR hardware module’s parser reads.

Notes

The mapping is rebuilt on every call, so callers may mutate the returned dictionary freely.

Return type:: dict[tuple[int, int], tuple[int, ...]]
Returns:: A mapping from each (module_type, module_id) pair this system parses to the tuple of event codes its parser reads.

sollertia_forgery.mesoscope_vr.locate_mesoscope_pose_predictions(session)¶

Locates the externally-produced DeepLabCut prediction file the Mesoscope-VR tracking stage reads.

Notes

The acquisition rig writes the prediction beside the face-camera video during preprocessing, under a name carrying the eye-tracking project. A re-run leaves several files matching, and the natural-sort-first one is the file the tracking stage opens.

Parameters:: session (SessionData) – The loaded session whose raw camera_data directory holds the prediction.
Return type:: Path | None
Returns:: The path to the prediction file, or None when the session carries none.

sollertia_forgery.mesoscope_vr.locate_two_photon_data(session)¶

Resolves the Mesoscope-VR session’s raw two-photon imaging directory.

Parameters:: session (SessionData) – The loaded session whose raw two-photon imaging directory is resolved.
Return type:: Path
Returns:: The path to the session’s mesoscope_data directory under its raw-data root.

sollertia_forgery.mesoscope_vr.parse_brake(event_partition, output_directory, session)¶

Parses the brake module (type 3, id 1) into the session’s brake behavior feather.

Parameters:

event_partition (dict[int, DataFrame]) – The event-code-keyed partition dictionary holding the module’s extracted event data.
output_directory (Path) – The path to the session’s processed microcontroller-data directory where the feather is written.
session (SessionData) – The loaded session whose hardware state determines module eligibility.

Raises:

FileNotFoundError – If the session’s hardware state YAML file is absent.

Return type:

None

sollertia_forgery.mesoscope_vr.parse_encoder(event_partition, output_directory, session)¶

Parses the wheel-encoder module (type 2, id 1) into the session’s encoder behavior feather.

Parameters:

event_partition (dict[int, DataFrame]) – The event-code-keyed partition dictionary holding the module’s extracted event data.
output_directory (Path) – The path to the session’s processed microcontroller-data directory where the feather is written.
session (SessionData) – The loaded session whose hardware state determines module eligibility.

Raises:

FileNotFoundError – If the session’s hardware state YAML file is absent.
ValueError – If either rotation event code carries no data payload, or stores a null payload inside an otherwise decodable stream. Also raised when a code spreads its payloads across more than one dtype, or decodes into a value count that is not a whole multiple of its message count.

Return type:

None

sollertia_forgery.mesoscope_vr.parse_gas_puff(event_partition, output_directory, session)¶

Parses the gas-puff valve module (type 5, id 2) into the session’s gas-puff behavior feather.

Parameters:

event_partition (dict[int, DataFrame]) – The event-code-keyed partition dictionary holding the module’s extracted event data.
output_directory (Path) – The path to the session’s processed microcontroller-data directory where the feather is written.
session (SessionData) – The loaded session whose hardware state determines module eligibility.

Raises:

FileNotFoundError – If the session’s hardware state YAML file is absent.

Return type:

None

sollertia_forgery.mesoscope_vr.parse_lick(event_partition, output_directory, session)¶

Parses the lick-sensor module (type 4, id 1) into the session’s lick behavior feather.

Parameters:

event_partition (dict[int, DataFrame]) – The event-code-keyed partition dictionary holding the module’s extracted event data.
output_directory (Path) – The path to the session’s processed microcontroller-data directory where the feather is written.
session (SessionData) – The loaded session whose hardware state determines module eligibility.

Raises:

FileNotFoundError – If the session’s hardware state YAML file is absent.
ValueError – If the lick event code carries no data payload or stores a null payload inside an otherwise decodable stream. Also raised when the code spreads its payloads across more than one dtype, or decodes into a value count that is not a whole multiple of its message count.

Return type:

None

sollertia_forgery.mesoscope_vr.parse_mesoscope_frame(event_partition, output_directory, session)¶

Parses the mesoscope-frame TTL module (type 1, id 1) into the session’s mesoscope-frame behavior feather.

Parameters:

event_partition (dict[int, DataFrame]) – The event-code-keyed partition dictionary holding the module’s extracted event data.
output_directory (Path) – The path to the session’s processed microcontroller-data directory where the feather is written.
session (SessionData) – The loaded session whose hardware state determines module eligibility.

Raises:

FileNotFoundError – If the session’s hardware state YAML file is absent.
ValueError – If the extracted event data carries no rising or no falling TTL edge timestamps.

Return type:

None

sollertia_forgery.mesoscope_vr.parse_runtime(decoded_messages, output_directory, session)¶

Parses the decoded Mesoscope-VR runtime archive into the session’s runtime behavior feathers.

Routes each decoded payload by its leading code (or by length for VR wall cue sequences) and writes the resulting behavior feathers. The experiment-only feathers are written only for experiment sessions.

Parameters:

decoded_messages (DataFrame) – The decoded runtime messages as a Polars DataFrame with a time_us UInt64 column and a payload Binary column, in archive order.
output_directory (Path) – The path to the session’s processed runtime-data directory where the runtime feathers are written.
session (SessionData) – The loaded session, from which the experiment configuration is resolved.

Raises:

FileNotFoundError – If the session is an experiment session but its experiment configuration or VR task template YAML file is missing.
ValueError – If the recorded VR wall cue sequences are absent, if their distance breakpoints are inconsistent, or if the experiment configuration references a trial name absent from the VR task template.
RuntimeError – If a VR wall cue sequence cannot be fully decomposed into trial motifs.

Return type:

None

sollertia_forgery.mesoscope_vr.parse_screen(event_partition, output_directory, session)¶

Parses the screen module (type 7, id 1) into the session’s screen behavior feather.

Parameters:

event_partition (dict[int, DataFrame]) – The event-code-keyed partition dictionary holding the module’s extracted event data.
output_directory (Path) – The path to the session’s processed microcontroller-data directory where the feather is written.
session (SessionData) – The loaded session whose hardware state determines module eligibility.

Raises:

FileNotFoundError – If the session’s hardware state YAML file is absent.

Return type:

None

sollertia_forgery.mesoscope_vr.parse_torque(event_partition, output_directory, session)¶

Parses the torque-sensor module (type 6, id 1) into the session’s torque behavior feather.

Parameters:

event_partition (dict[int, DataFrame]) – The event-code-keyed partition dictionary holding the module’s extracted event data.
output_directory (Path) – The path to the session’s processed microcontroller-data directory where the feather is written.
session (SessionData) – The loaded session whose hardware state determines module eligibility.

Raises:

FileNotFoundError – If the session’s hardware state YAML file is absent.
ValueError – If either rotation event code carries no data payload, or stores a null payload inside an otherwise decodable stream. Also raised when a code spreads its payloads across more than one dtype, or decodes into a value count that is not a whole multiple of its message count.

Return type:

None

sollertia_forgery.mesoscope_vr.parse_valve(event_partition, output_directory, session)¶

Parses the water-valve module (type 5, id 1) into the session’s valve behavior feather.

Parameters:

event_partition (dict[int, DataFrame]) – The event-code-keyed partition dictionary holding the module’s extracted event data.
output_directory (Path) – The path to the session’s processed microcontroller-data directory where the feather is written.
session (SessionData) – The loaded session whose hardware state determines module eligibility.

Raises:

FileNotFoundError – If the session’s hardware state YAML file is absent.

Return type:

None

sollertia_forgery.mesoscope_vr.process_mesoscope_video_tracking(session, output_directory)¶

Post-processes the Mesoscope-VR face-camera DLC predictions into per-frame pupil and eye metrics.

Locates the pupil project’s externally-produced DLC .h5 beside the face-camera video in the session’s raw camera_data directory, where the acquisition rig writes it during preprocessing. If none is present, returns without doing anything: the stage is optional and gated on detecting the prediction file.

Otherwise, reads the thirteen canonical bodyparts and fits an ellipse to the pupil and to the eye for each frame. Flags occluded frames as blinks and derives motion-robust eye-position signals from the pupil relative to the eye and the corneal reflection. Writes the results into a {camera}_pupil.feather in the processed video-data directory.

Notes

The predictions are produced upstream rather than here because DeepLabCut pins numpy<2 while sollertia-forgery runs numpy>=2 on Python 3.14, so DeepLabCut cannot be imported in-process. This worker only reads DeepLabCut’s .h5 output, keeping the deeplabcut library out of this environment.

Parameters:

session (SessionData) – The loaded session whose pupil DLC predictions are post-processed. Its raw camera_data directory supplies the DLC .h5.
output_directory (Path) – The processed video-data directory (session.processed_data.video_data_path) into which the pupil feather is written.

Raises:

ValueError – If a DLC .h5 is present but is missing a canonical bodypart or has an unrecognized layout.

Return type:

None

sollertia_forgery.mesoscope_vr.resolve_mesoscope_assembly_geometry(session)¶

Reports the shape taken by the assembly of a Mesoscope-VR session that records no imaging.

States the two heights at which assemble_training_dataset works: the reference clock on which its columns are placed, and the clock on which each source it reads was sampled. The two are different heights and neither bounds the other, since the reference clock is the slowest camera’s while the sources include every faster camera the session recorded.

Notes

The reference clock is resolved through the same selection the assembler itself makes, so the height reported here is the height of the frame that assembler builds rather than that of any other clock the session holds.

The sources are reported through the system’s own source resolver rather than enumerated again here, so the two heights this record carries stay drawn from one statement of what the assemblers read. That resolver routes the source set by session type, and this record is built for the session types whose assembly settles its reference clock on a camera.

Reads the IPC metadata of each source feather and, for the camera clocks alone, two timestamps of each. No column is materialized, so measuring a session costs the same whatever its length.

Parameters:

session (SessionData) – The loaded session whose assembly geometry is measured.

Return type:

AssemblyGeometry

Returns:

The samples the reference clock holds, and the samples that each source read by the assembly holds.

Raises:

FileNotFoundError – If no camera timestamp feather with at least two frames spanning a positive duration is present, in which case the assembler settles on no reference clock either and the job cannot run.
ValueError – If the session’s type is not one the Mesoscope-VR assemblers cover.

sollertia_forgery.mesoscope_vr.resolve_mesoscope_assembly_sources(session)¶

Reports the samples that each source read by one Mesoscope-VR session’s assembly holds on that source’s clock.

Notes

A source arrives at the clock on which it was sampled rather than at the clock on which the assembled frame is placed. A camera’s timestamp, motion-energy and pupil feathers are read at that camera’s frame count, and a behavior feather is read at the count of samples its own parser logged, both before anything is interpolated onto the reference clock. A camera faster than the reference one therefore contributes arrays taller than the frame the job builds, and neither family bounds the other.

A source is reported once per clock rather than once per file. The three feathers of one camera carry one row per acquired frame, so that camera’s frame count describes every array it contributes, while each behavior feather carries its own clock and is reported on its own. A feather the session did not produce is left out, because the assembler reads whichever of the optional sources its session type wrote and no others.

Reads the IPC metadata of each source feather alone. No column is materialized, so measuring a session costs the same whatever its length.

Parameters:: session (SessionData) – The loaded session whose assembly sources are measured.
Return type:: tuple[int, ...]
Returns:: The samples that each source read by the assembly holds, one entry per source, cameras first. Empty when the session carries none of the sources its assembler reads.
Raises:: ValueError – If the session’s type is not one the Mesoscope-VR assemblers cover, in which case no assembly of it exists whose sources could be reported.

sollertia_forgery.mesoscope_vr.resolve_multi_recording_configuration(session)¶

Resolves the multi-recording cindra configuration for a Mesoscope-VR session.

Notes

Mesoscope-VR tracks cells across recordings only for experiment sessions, so a session of any other type returns no configuration. When it applies, the configuration is selected from the animal’s genotype, read from the session’s surgery metadata.

Parameters:

session (SessionData) – The loaded session whose multi-recording configuration is resolved.

Return type:

MultiRecordingConfiguration | None

Returns:

The multi-recording configuration for the session’s calcium indicator, or None when the session is not a mesoscope experiment session and therefore carries no cross-recording cell tracking.

Raises:

FileNotFoundError – If the session’s surgery metadata file is missing.
ValueError – If the animal’s genotype does not map to a recognized calcium indicator.

sollertia_forgery.mesoscope_vr.resolve_single_recording_configuration(session)¶

Resolves the single-recording cindra configuration for a Mesoscope-VR session.

Notes

The configuration is selected from the animal’s genotype, read from the session’s surgery metadata, so a GCaMP6f and a jGCaMP8s recording receive their indicator-tuned parameters automatically.

Parameters:

session (SessionData) – The loaded session whose single-recording configuration is resolved.

Return type:

SingleRecordingConfiguration

Returns:

The single-recording configuration for the session’s calcium indicator.

Raises:

FileNotFoundError – If the session’s surgery metadata file is missing.
ValueError – If the animal’s genotype does not map to a recognized calcium indicator.

class sollertia_forgery.mesoscope_vr.metadata.DatasetColumn(*values)¶

Defines every column name that can appear in the assembled session data feather produced by the forging pipeline.

Notes

Each member’s value is the column name as it appears in data.feather, so members compare equal to the raw column strings.

AVERSIVE_GUIDED = 'aversive_guided'¶: Aversive guidance state at each sample. Present only when aversive guidance was recorded.

BLINKING_STATE = 'blinking_state'¶: Blink state at each sample, marking a closed or covered eye. Encoded as 1 during a blink and 0 otherwise.

BODY_CAMERA_FRAME_LUMINANCE = 'body_camera_frame_luminance'¶: Body camera mean frame intensity at each sample in gray levels, tracking scene illumination.

BODY_CAMERA_MOTION_ENERGY = 'body_camera_motion_energy'¶: Body camera motion energy at each sample, the mean absolute inter-frame intensity change in gray levels. A within-session movement magnitude, high during movement and low during stillness.

BRAKE = 'brake'¶: The running wheel brake engagement at each sample.

CUE = 'cue'¶: Active virtual reality cue identifier at each sample. 255 marks samples outside the run state.

DILATION_STATE = 'dilation_state'¶: Dilation-clip state at each sample, marking a pupil dilated past the eye aperture. Encoded as 1 when clipped and 0 otherwise.

DISTANCE_CM = 'distance_cm'¶: Cumulative distance traveled by the animal in centimeters at each sample.

ELAPSED_MINUTES = 'elapsed_minutes'¶: Elapsed time in minutes since the first sample of the session’s reference clock. The clock starts during setup, so the clipped dataset’s first row carries a value above zero.

EYE_CENTER_X_PX = 'eye_center_x_px'¶: Horizontal position of the fitted eye ellipse center in face-camera pixels at each sample.

EYE_CENTER_Y_PX = 'eye_center_y_px'¶: Vertical position of the fitted eye ellipse center in face-camera pixels at each sample.

EYE_HEIGHT_PX = 'eye_height_px'¶: Length of the fitted eye ellipse’s top-bottom chord in face-camera pixels at each sample.

EYE_OPENNESS = 'eye_openness'¶: Ratio of the fitted eye ellipse’s height to its width at each sample, a distance-invariant measure of eye openness.

EYE_WIDTH_PX = 'eye_width_px'¶: Length of the fitted eye ellipse’s left-right chord in face-camera pixels at each sample.

FACE_CAMERA_FRAME_LUMINANCE = 'face_camera_frame_luminance'¶: Face camera mean frame intensity at each sample in gray levels, tracking scene illumination.

FACE_CAMERA_MOTION_ENERGY = 'face_camera_motion_energy'¶: Face camera motion energy at each sample, the mean absolute inter-frame intensity change in gray levels. A within-session movement magnitude, high during movement and low during stillness.

FRAME = 'frame'¶: One-based mesoscope acquisition frame index at each sample.

IN_TRIGGER_ZONE = 'in_trigger_zone'¶: Boolean flag indicating whether the animal is inside a stimulus trigger zone at each sample, as the Virtual Reality runtime registers it. The zone extends one actor lead beyond each declared boundary, because the actor’s collider surface crosses the zone before and after its tracked position does.

LICK = 'lick'¶: Lick sensor engagement state at each sample.

MULTI_DAY_CELL_FLUORESCENCE = 'multi_day_cell_fluorescence'¶: Multi-recording raw cell fluorescence trace per ROI, over the cells tracked across every session of this animal in the dataset.

MULTI_DAY_NEUROPIL_FLUORESCENCE = 'multi_day_neuropil_fluorescence'¶: Multi-recording raw neuropil fluorescence trace per ROI, over the cells tracked across every session of this animal in the dataset.

MULTI_DAY_SPIKES = 'multi_day_spikes'¶: Multi-recording OASIS-deconvolved spike rates per ROI, over the cells tracked across every session of this animal in the dataset.

MULTI_DAY_SUBTRACTED_FLUORESCENCE = 'multi_day_subtracted_fluorescence'¶: Multi-recording neuropil- and baseline-subtracted fluorescence trace per ROI, over the cells tracked across every session of this animal in the dataset.

PUPIL_AREA_PX2 = 'pupil_area_px2'¶: Area enclosed by the fitted pupil ellipse in square face-camera pixels at each sample.

PUPIL_CENTER_X_PX = 'pupil_center_x_px'¶: Horizontal position of the fitted pupil ellipse center in face-camera pixels at each sample.

PUPIL_CENTER_Y_PX = 'pupil_center_y_px'¶: Vertical position of the fitted pupil ellipse center in face-camera pixels at each sample.

PUPIL_DIAMETER_PX = 'pupil_diameter_px'¶: Mean of the fitted pupil ellipse’s two axis diameters in face-camera pixels at each sample, the primary arousal proxy. NaN marks blink and dilation samples.

PUPIL_FIT_CONDITION = 'pupil_fit_condition'¶: Condition number of the fitted pupil ellipse at each sample, where higher values indicate a less reliable fit. NaN marks unmeasured samples.

PUPIL_FIT_RESIDUAL_PX = 'pupil_fit_residual_px'¶: Root-mean-square distance in pixels between the confident pupil-perimeter points and the fitted ellipse at each sample, a fit-quality measure.

PUPIL_IN_EYE_X = 'pupil_in_eye_x'¶: Horizontal offset of the pupil center from the eye center at each sample, normalized to the eye ellipse’s horizontal semi-axis and dimensionless.

PUPIL_IN_EYE_Y = 'pupil_in_eye_y'¶: Vertical offset of the pupil center from the eye center at each sample, normalized to the eye ellipse’s vertical semi-axis and dimensionless.

PUPIL_REFLECTION_OFFSET_X_PX = 'pupil_reflection_offset_x_px'¶: Horizontal offset of the pupil center from the corneal reflection in pixels at each sample, a motion-robust horizontal eye-position signal.

PUPIL_REFLECTION_OFFSET_Y_PX = 'pupil_reflection_offset_y_px'¶: Vertical offset of the pupil center from the corneal reflection in pixels at each sample, a motion-robust vertical eye-position signal.

REFLECTION_X_PX = 'reflection_x_px'¶: Horizontal position of the corneal reflection in face-camera pixels at each sample.

REFLECTION_Y_PX = 'reflection_y_px'¶: Vertical position of the corneal reflection in face-camera pixels at each sample.

REINFORCING_GUIDED = 'reinforcing_guided'¶: Reinforcing guidance state at each sample. Present only when reinforcing guidance was recorded.

REWARD = 'reward'¶: The reward-classification state at each sample, one of ‘no’ (no reward tone playing), ‘tone’ (reward tone playing with no water delivered during the tone), or ‘yes’ (reward tone playing with water delivered during the tone).

RUNTIME_STATE = 'runtime_state'¶: Experiment runtime state label at each sample.

SCREENS = 'screens'¶: The Virtual Reality display state at each sample.

SINGLE_DAY_CELL_FLUORESCENCE = 'single_day_cell_fluorescence'¶: Single-recording raw cell fluorescence trace per ROI, over the cells detected in this session alone.

SINGLE_DAY_NEUROPIL_FLUORESCENCE = 'single_day_neuropil_fluorescence'¶: Single-recording raw neuropil fluorescence trace per ROI, over the cells detected in this session alone.

SINGLE_DAY_SPIKES = 'single_day_spikes'¶: Single-recording OASIS-deconvolved spike rates per ROI, over the cells detected in this session alone.

SINGLE_DAY_SUBTRACTED_FLUORESCENCE = 'single_day_subtracted_fluorescence'¶: Single-recording neuropil- and baseline-subtracted fluorescence trace per ROI, over the cells detected in this session alone.

SPEED_CM_S = 'speed_cm_s'¶: Animal’s running speed in cm/s at each sample.

SYSTEM_STATE = 'system_state'¶: Acquisition system state at each sample (idle, rest, run for experiment sessions, lick training or run training for training sessions).

TIME_US = 'time_us'¶: Microsecond-precision sample timestamps from the acquisition reference clock.

TORQUE_N_CM = 'torque_N_cm'¶: The torque applied by the animal to the running wheel in N·cm at each sample, forced to zero during ‘run’ periods upstream.

TRIAL = 'trial'¶: One-based trial identifier at each sample. 65535 marks samples outside the run state.

TRIAL_TYPE = 'trial_type'¶: Trial type label at each sample (e.g. ‘ABC’, ‘ABCD’). ‘undefined’ marks non-run samples.

WATER_UL = 'water_uL'¶: The cumulative water reward volume delivered to the animal at each sample in microliters.

class sollertia_forgery.mesoscope_vr.video_tracking.PupilColumn(*values)¶

Defines every column written into the Mesoscope-VR pupil-tracking feather by the donated video-tracking worker.

Notes

Positions and lengths are expressed in the face camera’s pixel coordinate frame (_px), matching the frame in which DeepLabCut reports its predictions, and are never converted to physical units: the camera is not calibrated against a physical scale. Dimensionless ratios and flags carry no unit suffix.

Every geometric column is NaN on frames whose feature kept too few confident points to fit, so consumers see NaN rather than a confidently-wrong ellipse. For the pupil columns those frames are exactly the ones that blinking_state and dilation_state flag. The eye columns answer only to their own fit, so they stay populated on a blink flagged by low openness or by a lost corneal reflection.

BLINKING_STATE = 'blinking_state'¶: Boolean flag marking frames where the eye is closed or covered, leaving no measurable pupil. True when a fitted eye opens less than half its session-median amount, and when a lost eye ring or corneal reflection leaves no pupil fit either. A pupil that resolves keeps the frame open, since a covered eye presents no pupil ring to fit.

DILATION_STATE = 'dilation_state'¶: Boolean flag marking frames where the pupil dilated past the eye’s aperture and was clipped too far to fit a diameter. True only on non-blink frames whose pupil is otherwise unmeasurable. A fully visible dilated pupil is measured normally and flagged False.

EYE_CENTER_X_PX = 'eye_center_x_px'¶: Horizontal position of the fitted eye ellipse center in pixels at each frame.

EYE_CENTER_Y_PX = 'eye_center_y_px'¶: Vertical position of the fitted eye ellipse center in pixels at each frame.

EYE_HEIGHT_PX = 'eye_height_px'¶: Length of the fitted eye ellipse’s top-bottom chord in pixels at each frame.

EYE_OPENNESS = 'eye_openness'¶: Ratio of the fitted eye ellipse’s height to its width at each frame, a distance-invariant measure of how open the eye is. NaN wherever the eye ring cannot be fit.

EYE_WIDTH_PX = 'eye_width_px'¶: Length of the fitted eye ellipse’s left-right chord in pixels at each frame.

PUPIL_AREA_PX2 = 'pupil_area_px2'¶: Area enclosed by the fitted pupil ellipse in square pixels at each frame.

PUPIL_CENTER_X_PX = 'pupil_center_x_px'¶: Horizontal position of the fitted pupil ellipse center in pixels at each frame.

PUPIL_CENTER_Y_PX = 'pupil_center_y_px'¶: Vertical position of the fitted pupil ellipse center in pixels at each frame.

PUPIL_DIAMETER_PX = 'pupil_diameter_px'¶: Mean of the fitted pupil ellipse’s two axis diameters in pixels at each frame. The pipeline’s primary arousal proxy.

PUPIL_FIT_CONDITION = 'pupil_fit_condition'¶: Condition number of the fitted pupil ellipse at each frame, a fit-precision proxy on which higher values indicate a less reliable fit. NaN wherever the pupil is unmeasured.

PUPIL_FIT_RESIDUAL_PX = 'pupil_fit_residual_px'¶: Root-mean-square distance in pixels between the confident pupil-perimeter points and their positions on the fitted ellipse at each frame, a fit-quality measure. NaN on unmeasured frames and on exactly-determined three-point fits. Only overdetermined fits report a residual.

PUPIL_IN_EYE_X = 'pupil_in_eye_x'¶: Horizontal offset of the pupil center from the eye center at each frame, normalized to the eye ellipse’s horizontal semi-axis. Dimensionless and therefore comparable across animals.

PUPIL_IN_EYE_Y = 'pupil_in_eye_y'¶: Vertical offset of the pupil center from the eye center at each frame, normalized to the eye ellipse’s vertical semi-axis. Dimensionless and therefore comparable across animals.

PUPIL_REFLECTION_OFFSET_X_PX = 'pupil_reflection_offset_x_px'¶: Horizontal offset in pixels of the pupil center from the corneal reflection at each frame. Referencing the reflection cancels the eye’s common-mode motion relative to the camera, making this a motion-robust horizontal eye-position signal.

PUPIL_REFLECTION_OFFSET_Y_PX = 'pupil_reflection_offset_y_px'¶: Vertical offset in pixels of the pupil center from the corneal reflection at each frame. Referencing the reflection cancels the eye’s common-mode motion relative to the camera, making this a motion-robust vertical eye-position signal.

REFLECTION_X_PX = 'reflection_x_px'¶: Horizontal position of the corneal reflection in pixels at each frame. NaN wherever the reflection falls below the likelihood threshold, which flags the frame as a blink unless the pupil still resolves.

REFLECTION_Y_PX = 'reflection_y_px'¶: Vertical position of the corneal reflection in pixels at each frame. NaN wherever the reflection falls below the likelihood threshold, which flags the frame as a blink unless the pupil still resolves.

sollertia_forgery.mesoscope_vr.forging.MESOSCOPE_ADMISSION_PIPELINES: dict[SessionTypes, frozenset[ProcessingPipelines]] = {SessionTypes.LICK_TRAINING: frozenset({ProcessingPipelines.CHECKSUM, ProcessingPipelines.MICROCONTROLLER, ProcessingPipelines.RUNTIME, ProcessingPipelines.VIDEO}), SessionTypes.MESOSCOPE_EXPERIMENT: frozenset({ProcessingPipelines.CHECKSUM, ProcessingPipelines.MICROCONTROLLER, ProcessingPipelines.RUNTIME, ProcessingPipelines.TWO_PHOTON, ProcessingPipelines.VIDEO}), SessionTypes.RUN_TRAINING: frozenset({ProcessingPipelines.CHECKSUM, ProcessingPipelines.MICROCONTROLLER, ProcessingPipelines.RUNTIME, ProcessingPipelines.VIDEO})}¶

Maps each Mesoscope-VR session type to the pipelines that must have completed before the session may join a forged dataset.

Notes

Every pipeline resolves its own job universe from the acquisition manifests, so a completed tracker already means every source recorded by the session was processed. Admission therefore checks which pipelines completed rather than counting sources.

A training session records no imaging, so the two-photon pipeline is absent from its requirement. A session type absent from this mapping joins no dataset, which is the case for window checking.

sollertia_forgery.mesoscope_vr.metadata.MESOSCOPE_COLUMN_DESCRIPTIONS: dict[str, str] = {'aversive_guided': 'Aversive guidance state at each sample. Present only when aversive guidance was recorded.', 'blinking_state': 'Blink state at each sample, marking a closed or covered eye. Encoded as 1 during a blink and 0 otherwise.', 'body_camera_frame_luminance': 'Body camera mean frame intensity at each sample in gray levels, tracking scene illumination.', 'body_camera_motion_energy': 'Body camera motion energy at each sample, the mean absolute inter-frame intensity change in gray levels. A within-session movement magnitude, high during movement and low during stillness.', 'brake': 'The running wheel brake engagement at each sample.', 'cue': 'Active virtual reality cue identifier at each sample. 255 marks samples outside the run state.', 'dilation_state': 'Dilation-clip state at each sample, marking a pupil dilated past the eye aperture. Encoded as 1 when clipped and 0 otherwise.', 'distance_cm': 'Cumulative distance traveled by the animal in centimeters at each sample.', 'elapsed_minutes': "Elapsed time in minutes since the first sample of the session's reference clock. The clock starts during setup, so the clipped dataset's first row carries a value above zero.", 'eye_center_x_px': 'Horizontal position of the fitted eye ellipse center in face-camera pixels at each sample.', 'eye_center_y_px': 'Vertical position of the fitted eye ellipse center in face-camera pixels at each sample.', 'eye_height_px': "Length of the fitted eye ellipse's top-bottom chord in face-camera pixels at each sample.", 'eye_openness': "Ratio of the fitted eye ellipse's height to its width at each sample, a distance-invariant measure of eye openness.", 'eye_width_px': "Length of the fitted eye ellipse's left-right chord in face-camera pixels at each sample.", 'face_camera_frame_luminance': 'Face camera mean frame intensity at each sample in gray levels, tracking scene illumination.', 'face_camera_motion_energy': 'Face camera motion energy at each sample, the mean absolute inter-frame intensity change in gray levels. A within-session movement magnitude, high during movement and low during stillness.', 'frame': 'One-based mesoscope acquisition frame index at each sample.', 'in_trigger_zone': "Boolean flag indicating whether the animal is inside a stimulus trigger zone at each sample, as the Virtual Reality runtime registers it. The zone extends one actor lead beyond each declared boundary, because the actor's collider surface crosses the zone before and after its tracked position does.", 'lick': 'Lick sensor engagement state at each sample.', 'multi_day_cell_fluorescence': 'Multi-recording raw cell fluorescence trace per ROI, over the cells tracked across every session of this animal in the dataset.', 'multi_day_neuropil_fluorescence': 'Multi-recording raw neuropil fluorescence trace per ROI, over the cells tracked across every session of this animal in the dataset.', 'multi_day_spikes': 'Multi-recording OASIS-deconvolved spike rates per ROI, over the cells tracked across every session of this animal in the dataset.', 'multi_day_subtracted_fluorescence': 'Multi-recording neuropil- and baseline-subtracted fluorescence trace per ROI, over the cells tracked across every session of this animal in the dataset.', 'pupil_area_px2': 'Area enclosed by the fitted pupil ellipse in square face-camera pixels at each sample.', 'pupil_center_x_px': 'Horizontal position of the fitted pupil ellipse center in face-camera pixels at each sample.', 'pupil_center_y_px': 'Vertical position of the fitted pupil ellipse center in face-camera pixels at each sample.', 'pupil_diameter_px': "Mean of the fitted pupil ellipse's two axis diameters in face-camera pixels at each sample, the primary arousal proxy. NaN marks blink and dilation samples.", 'pupil_fit_condition': 'Condition number of the fitted pupil ellipse at each sample, where higher values indicate a less reliable fit. NaN marks unmeasured samples.', 'pupil_fit_residual_px': 'Root-mean-square distance in pixels between the confident pupil-perimeter points and the fitted ellipse at each sample, a fit-quality measure.', 'pupil_in_eye_x': "Horizontal offset of the pupil center from the eye center at each sample, normalized to the eye ellipse's horizontal semi-axis and dimensionless.", 'pupil_in_eye_y': "Vertical offset of the pupil center from the eye center at each sample, normalized to the eye ellipse's vertical semi-axis and dimensionless.", 'pupil_reflection_offset_x_px': 'Horizontal offset of the pupil center from the corneal reflection in pixels at each sample, a motion-robust horizontal eye-position signal.', 'pupil_reflection_offset_y_px': 'Vertical offset of the pupil center from the corneal reflection in pixels at each sample, a motion-robust vertical eye-position signal.', 'reflection_x_px': 'Horizontal position of the corneal reflection in face-camera pixels at each sample.', 'reflection_y_px': 'Vertical position of the corneal reflection in face-camera pixels at each sample.', 'reinforcing_guided': 'Reinforcing guidance state at each sample. Present only when reinforcing guidance was recorded.', 'reward': "The reward-classification state at each sample, one of 'no' (no reward tone playing), 'tone' (reward tone playing with no water delivered during the tone), or 'yes' (reward tone playing with water delivered during the tone).", 'runtime_state': 'Experiment runtime state label at each sample.', 'screens': 'The Virtual Reality display state at each sample.', 'single_day_cell_fluorescence': 'Single-recording raw cell fluorescence trace per ROI, over the cells detected in this session alone.', 'single_day_neuropil_fluorescence': 'Single-recording raw neuropil fluorescence trace per ROI, over the cells detected in this session alone.', 'single_day_spikes': 'Single-recording OASIS-deconvolved spike rates per ROI, over the cells detected in this session alone.', 'single_day_subtracted_fluorescence': 'Single-recording neuropil- and baseline-subtracted fluorescence trace per ROI, over the cells detected in this session alone.', 'speed_cm_s': "Animal's running speed in cm/s at each sample.", 'system_state': 'Acquisition system state at each sample (idle, rest, run for experiment sessions, lick training or run training for training sessions).', 'time_us': 'Microsecond-precision sample timestamps from the acquisition reference clock.', 'torque_N_cm': "The torque applied by the animal to the running wheel in N·cm at each sample, forced to zero during 'run' periods upstream.", 'trial': 'One-based trial identifier at each sample. 65535 marks samples outside the run state.', 'trial_type': "Trial type label at each sample (e.g. 'ABC', 'ABCD'). 'undefined' marks non-run samples.", 'water_uL': 'The cumulative water reward volume delivered to the animal at each sample in microliters.'}¶: The Mesoscope-VR column-description binding donated to the forging pipeline. Maps every column name that the Mesoscope-VR assembly worker can emit into data.feather to its human-readable description. Derived from DatasetColumn and _COLUMN_DESCRIPTIONS.

sollertia_forgery.mesoscope_vr.runtime.RUNTIME_SOURCE_ID: str = '1'¶: The source ID used by the Mesoscope-VR runtime DataLogger for its log archive. Every processable session contains exactly one runtime archive, and it always uses this ID.

sollertia_forgery.mesoscope_vr.two_photon.MESOSCOPE_MULTI_RECORDING_SESSION_TYPES: frozenset[SessionTypes] = frozenset({SessionTypes.MESOSCOPE_EXPERIMENT})¶

The Mesoscope-VR session types whose animals are tracked across recordings.

Notes

Cross-recording cell tracking needs calcium imaging, which only an experiment session records, so every other session type resolves no multi-recording configuration.