Skip to content

salvus.flow.acta.memo.records

salvus.flow.acta.memo.records salvus flow acta memo records

SQLite storage for cache records, provenance, leases, and run events.

Each instance serializes access to its connection with a lock. Databases use WAL mode and a busy timeout. Manifests and file observations are append-only; cache records, labels, leases, and invalidations are mutable. Explicit idle pruning may also remove disposable provenance.

Classes

FileObservation

class FileObservation(builtins.object):
    def __init__(
        self, digest: str, path: str, host: str, observed_at: str
    ) -> None: ...

A recorded file digest, path, and hashing time.

Parameters
  • digest str — The content digest the file hashed to.
  • path str — The absolute path it was read from.
  • host str — Hostname that read the file.
  • observed_at str — ISO-8601 UTC timestamp of the hashing.

Manifest

class Manifest(builtins.object):
    def __init__(
        self,
        manifest_id: int,
        key: str,
        task: str,
        envelope: dict[str, Any],
        arg_values: dict[str, Any],
        output_refs: dict[str, Any],
        timings: dict[str, float],
        hostname: str,
        versions: dict[str, str],
        created_at: str,
        provenance: dict[str, Any],
        fingerprint_parts: dict[str, str],
        run_id: str | None = None,
        input_snapshot: dict[str, Any] | None = None,
    ) -> None: ...

Provenance for a computed or adopted result.

Parameters
  • manifest_id int — Database row ID.
  • key str — Computation’s envelope key.
  • task str — Task name.
  • envelope dict[str, Any] — Complete cache-key envelope.
  • arg_values dict[str, Any] — Normalized argument values with defaults omitted; artifact arguments use digests. Diagnostic only; empty for legacy manifests.
  • output_refs dict[str, Any] — Output names mapped to digests or lists of digests.
  • timings dict[str, float] — Stage names mapped to elapsed seconds.
  • hostname str — Host that committed the result.
  • versions dict[str, str] — Package names mapped to installed versions.
  • created_at str — ISO-8601 UTC insertion time.
  • provenance dict[str, Any] — Execution or adoption metadata, excluded from cache keys.
  • fingerprint_parts dict[str, str] — Qualified names and source digests for diagnostics. Empty for tasks with an explicit impl.
  • run_id str | None = None — Producing run ID, or None without a run context.
  • input_snapshot dict[str, Any] | None = None — Dispatch inputs before normalization and default elision, or None if unavailable.

Record

class Record(builtins.object):
    def __init__(
        self,
        key: str,
        output_refs: dict[str, Any],
        manifest_id: int,
        retention: Retention = Retention.STANDARD,
        last_used_at: str = "",
    ) -> None: ...

A committed key -> outputs record.

Parameters
  • key str — The envelope key.
  • output_refs dict[str, Any] — Mapping from output name to digest, or to a list of digests for multi-output tasks.
  • manifest_id int — The manifest that produced this record.
  • retention Retention = Retention.STANDARD — Retention class recorded at commit.
  • last_used_at str = '' — ISO-8601 UTC time of the last hit, stamped at commit and re-stamped (coalesced) by protocol.lookup. Empty for a row written before the column existed, which means “fall back to the manifest’s created_at”.

RecordStore

class RecordStore(builtins.object):
    def __init__(self, db_path: pathlib.Path | str) -> None: ...

SQLite-backed store for records and append-once manifests.

Call close() when finished. Abandoned stores also close their connections when collected or at interpreter exit.

Parameters
  • db_path pathlib.Path | str — Path to the SQLite database file. Created on first use, along with its parent directory.
Attributes
write_count int

Return the number of transactions committed by this instance.

Methods
in_memory()
def in_memory() -> RecordStore: ...

Return a records store held entirely in memory (no file).

Returns RecordStore
acquire_lock()
def acquire_lock(self, key: str, holder: str, ttl_seconds: float) -> bool: ...

Try to acquire the single-flight lock for key.

Parameters
  • key str — The envelope key to acquire.
  • holder str — This caller’s opaque identity. Must be passed again to release_lock/renew_lock, and only that same holder can release or renew.
  • ttl_seconds float — Lease duration from now; renew before expiry to avoid duplicate computation.
Returns bool — True if this call acquired the lock (fresh or stolen), False if a live holder already has it.
active_ref_digests()
def active_ref_digests(self, now: float | None = None) -> set[str]: ...

Return digests whose execution leases have not expired.

Parameters
  • now float | None = None — Unix timestamp to compare against; current time by default.
Returns set[str]
add_active_refs()
def add_active_refs(
    self, holder: str, digests: Sequence[str], ttl_seconds: float
) -> None: ...

Lease content refs needed by an execution that is still running.

Parameters
  • holder str — Opaque execution identity.
  • digests Sequence[str] — Content digests to keep reachable.
  • ttl_seconds float — Lease duration from now.
Returns None
append_events()
def append_events(self, rows: Sequence[EventRow]) -> list[Event]: ...

Append a batch of events in one transaction, in order.

Parameters
  • rows Sequence[EventRow] — Tuples of (run_id, attempt, task, key, parent_key, kind, at, detail).
Returns list[Event] — The events as stored, each carrying the seq the database assigned it.
apply_prune()
def apply_prune(
    self,
    record_keys: Sequence[str],
    manifest_ids: Sequence[int],
    observation_digests: Sequence[str],
    run_ids: Sequence[str],
) -> None: ...

Apply a collector’s idle-maintenance database plan atomically.

Use through Session.prune, which computes reachability and verifies exclusive idle maintenance before touching either store.

Parameters
  • record_keys Sequence[str] — Cache entries to remove.
  • manifest_ids Sequence[int] — Disposable provenance rows to remove.
  • observation_digests Sequence[str] — File observations no longer needed.
  • run_ids Sequence[str] — Entire disposable runs, including their events.
Returns None
close()
def close(self) -> None: ...

Close the underlying SQLite connection.

Returns None
commit_record()
def commit_record(
    self,
    key: str,
    output_refs: dict[str, Any],
    manifest_id: int,
    retention: Retention = Retention.STANDARD,
    last_used_at: str | None = None,
    from_binding: bool = False,
) -> None: ...

Insert or replace the record for key.

Parameters
  • key str — The envelope key.
  • output_refs dict[str, Any] — Mapping from output name to digest(s).
  • manifest_id int — The manifest that produced this record.
  • retention Retention = Retention.STANDARD — Retention class to record.
  • last_used_at str | None = None — ISO-8601 UTC stamp to start the clock from; defaults to now, since committing is a use.
  • from_binding bool = False — Adopt a published result only if this database has not invalidated it. Only fresh computation clears an invalidation marker.
Returns None
delete_label()
def delete_label(self, name: str) -> bool: ...

Remove the label called name.

Parameters
  • name str — The label name to remove.
Returns bool — True if a label was removed, otherwise False.
delete_record()
def delete_record(self, key: str) -> None: ...

Delete the record for key, if any.

Parameters
  • key str — The envelope key whose record should be removed.
Returns None
end_run()
def end_run(self, run_id: str, ended_at: str, outcome: str) -> None: ...

Close a run with the time and outcome it ended on.

Parameters
  • run_id str — The run to close.
  • ended_at str — ISO-8601 UTC time it ended.
  • outcome str — "succeeded" or "failed".
Returns None
events_for_run()
def events_for_run(self, run_id: str) -> list[Event]: ...

Return one run’s events, in seq order.

Parameters
  • run_id str — Run ID; use get_run to resolve a name.
Returns list[Event]
file_observations_for()
def file_observations_for(self, digest: str) -> list[FileObservation]: ...

Return every recorded sighting of digest, oldest first.

Parameters
  • digest str — The content digest to look up.
Returns list[FileObservation]
get_label()
def get_label(self, name: str) -> Label | None: ...

Return the label called name, or None if there is none.

Parameters
  • name str — The label name to resolve.
Returns Label | None
get_manifest()
def get_manifest(self, manifest_id: int) -> Manifest | None: ...

Return the manifest with manifest_id, or None if absent.

Parameters
  • manifest_id int — Row id of the manifest to fetch.
Returns Manifest | None
get_record()
def get_record(self, key: str) -> Record | None: ...

Return the record for key, or None if absent.

Parameters
  • key str — The envelope key to look up.
Returns Record | None
get_run()
def get_run(self, ref: str) -> Run | None: ...

Return the run with this id or this name, or None.

Parameters
  • ref str — A run id or a run name.
Returns Run | None
has_active_work()
def has_active_work(self) -> bool: ...

Return whether live task locks or object leases forbid maintenance.

This check does not prevent new executions.

Returns bool
invalidate_record()
def invalidate_record(self, key: str) -> None: ...

Remove a record and prevent its adoption from a published binding.

Parameters
  • key str — Computation to invalidate in this database.
Returns None
is_invalidated()
def is_invalidated(self, key: str) -> bool: ...

Return whether fresh computation is required for a key.

Parameters
  • key str — Computation to inspect.
Returns bool
iter_records()
def iter_records(self) -> list[Record]: ...

Return every current record.

Returns list[Record]
latest_manifest_for_key()
def latest_manifest_for_key(self, key: str) -> Manifest | None: ...

Return the most recent manifest committed for key, or None.

Parameters
  • key str — The envelope key to look up.
Returns Manifest | None — The matching manifest with the highest ID, or None.
latest_manifest_for_task()
def latest_manifest_for_task(
    self, task: str, before_manifest_id: int | None = None
) -> Manifest | None: ...

Return the most recent manifest for task, or None.

Parameters
  • task str — Task name to filter on.
  • before_manifest_id int | None = None — If given, only consider manifests with a strictly smaller id (i.e. earlier than this run).
Returns Manifest | None — The manifest with the highest manifest_id matching the filters, or None if no manifest matches.
list_labels()
def list_labels(self, prefix: str = "") -> list[Label]: ...

Return every label whose name starts with prefix, by name.

Parameters
  • prefix str = '' — Literal name prefix; the empty default returns all labels and is not a special case.
Returns list[Label]
list_manifests()
def list_manifests(self, limit: int | None = None) -> list[Manifest]: ...

Return manifests in insertion order, most recent last.

Parameters
  • limit int | None = None — If given, only the most recent limit manifests are returned (still in ascending id order); must be a positive integer.
Returns list[Manifest]
list_runs()
def list_runs(
    self, limit: int | None = None, failed_only: bool = False
) -> list[Run]: ...

Return runs, newest first.

Parameters
  • limit int | None = None — Keep at most this many; None returns all of them.
  • failed_only bool = False — Keep only runs whose outcome was a failure.
Returns list[Run]
manifest_for_output_ref()
def manifest_for_output_ref(self, digest: str) -> Manifest | None: ...

Return the newest manifest that produced digest, if any.

Parameters
  • digest str — The content digest to find a producer for.
Returns Manifest | None
manifests_for_task()
def manifests_for_task(
    self, task: str, limit: int | None = None
) -> list[Manifest]: ...

Return task’s manifests, newest first.

Parameters
  • task str — Task name to select.
  • limit int | None = None — If given, at most this many, still newest first.
Returns list[Manifest]
observation_counts()
def observation_counts(self) -> dict[str, int]: ...

Return file observation counts by digest for maintenance previews.

Returns dict[str, int]
prune_runs()
def prune_runs(self, before: datetime) -> tuple[int, int]: ...

Delete runs started before before, and their events.

Parameters
  • before datetime — Cutoff; a run started at or after it survives.
Returns tuple[int, int] — The number of runs and the number of events deleted.
put_file_observation()
def put_file_observation(
    self, digest: str, path: str, host: str, observed_at: str
) -> int: ...

Record a file digest observation.

Observations are appended without deduplication.

Parameters
  • digest str — The content digest the file hashed to.
  • path str — The absolute path it was read from.
  • host str — The hostname that read it.
  • observed_at str — ISO-8601 UTC timestamp.
Returns int — The new observation’s row id.
put_label()
def put_label(
    self,
    name: str,
    key: str | None = None,
    ref: str | None = None,
    force: bool = False,
    created_at: str | None = None,
) -> Label: ...

Point name at exactly one key or ref.

Parameters
  • name str — The label name; validated before anything is written.
  • key str | None = None — The envelope key to name, or None.
  • ref str | None = None — The content digest to name, or None.
  • force bool = False — Allow replacing an existing label.
  • created_at str | None = None — ISO-8601 UTC timestamp; defaults to now.
Returns Label — The label as stored.
put_manifest()
def put_manifest(
    self,
    key: str,
    task: str,
    envelope: dict[str, Any],
    arg_values: dict[str, Any],
    output_refs: dict[str, Any],
    timings: dict[str, float],
    hostname: str,
    versions: dict[str, str],
    created_at: str,
    provenance: dict[str, Any],
    fingerprint_parts: Mapping[str, str] | None = None,
    run_id: str | None = None,
    input_snapshot: dict[str, Any] | None = None,
) -> int: ...

Append a new manifest row and return its id.

Parameters
  • key str — The envelope key this manifest was produced for.
  • task str — Task name.
  • envelope dict[str, Any] — The full envelope mapping.
  • arg_values dict[str, Any] — The call’s raw argument values, diagnostic-only (see Manifest).
  • output_refs dict[str, Any] — Mapping from output name to digest(s).
  • timings dict[str, float] — Mapping from stage name to elapsed seconds.
  • hostname str — The hostname of the calling (not necessarily computing) process.
  • versions dict[str, str] — Mapping from package name to version string.
  • created_at str — ISO-8601 UTC timestamp.
  • provenance dict[str, Any] — Execution or adoption metadata.
  • fingerprint_parts Mapping[str, str] | None = None — Source digests for diagnostics. None is stored as an empty mapping.
  • run_id str | None = None — The run that produced it, or None for a session with no run context.
  • input_snapshot dict[str, Any] | None = None — Full dispatch inputs, or None if unavailable.
Returns int — The new manifest’s row id.
records_for_task()
def records_for_task(self, task: str) -> list[Record]: ...

Return every current record whose manifest’s task is task.

Parameters
  • task str — Task name to filter on.
Returns list[Record]
release_active_refs()
def release_active_refs(self, holder: str) -> None: ...

Release every active-ref lease owned by holder.

Parameters
  • holder str — Opaque execution identity.
Returns None
release_lock()
def release_lock(self, key: str, holder: str) -> None: ...

Release the lock on key, only if holder still owns it.

Parameters
  • key str — The envelope key whose lock should be released.
  • holder str — This caller’s opaque identity, as passed to acquire_lock.
Returns None
renew_active_refs()
def renew_active_refs(self, holder: str, ttl_seconds: float) -> None: ...

Extend every active-ref lease owned by holder.

Parameters
  • holder str — Opaque execution identity.
  • ttl_seconds float — New lease duration from now.
Returns None
renew_lock()
def renew_lock(self, key: str, holder: str, ttl_seconds: float) -> bool: ...

Extend the lease on key, only if holder still owns it.

Sessions renew automatically; low-level callers must arrange renewal.

Parameters
  • key str — The envelope key whose lease should be extended.
  • holder str — This caller’s opaque identity, as passed to acquire_lock.
  • ttl_seconds float — New lease duration from now.
Returns bool — True if renewed, False if holder no longer owns the lock (e.g. it was already stolen).
schema_version()
def schema_version(self) -> int: ...

Return the schema version this database is at.

Returns int
start_run()
def start_run(self, name: str | None = None) -> Run: ...

Open a run and return it.

Parameters
  • name str | None = None — An explicit name; by default it is derived from the new id (see salvus.flow.acta.observability.names.run_name).
Returns Run
touch_record()
def touch_record(self, key: str, last_used_at: str) -> None: ...

Set a record’s last-used stamp.

Parameters
  • key str — The envelope key to stamp.
  • last_used_at str — ISO-8601 UTC stamp to write.
Returns None