salvus.flow.ng.in_process_job
salvus.flow.ng.in_process_job salvus flow ng in_process_job Job-like wrapper around in-process runs.
Functions
run_in_process()
run_in_process()def run_in_process(
solver: Solver,
input_file: typing.Any,
output_folder: str | pathlib.Path | None = None,
overwrite: bool = False,
job_name: str | None = None,
ranks: int | None = None,
verbosity: int = 1,
telemetry_interval_seconds: float = 2.0,
wait: bool = True,
progress: FrontendOverride = "auto",
) -> NGInProcessJob: ...Run a simulation in-process with the solver.
The backend of salvus.flow.functions.run and run_async for
Solver site names. run() calls this with wait=True (the
default): synchronous, as before T2 — the returned job has
already finished. run_async() calls this with wait=False: the
solve runs on a daemon worker thread and this function returns the
NGInProcessJob immediately, before the run finishes — poll
job.update_status()/job.telemetry(), iterate job. telemetry_stream(), or call job.wait() to block until it does.
Either way, a live progress display (see
salvus.flow.ng.progress_display) starts immediately and runs on its own
daemon
thread — wait=True still shows it, since this function itself
always runs the solve on a worker thread internally and only
blocks (job.wait()) once the display has started; the native
session bindings release the GIL during advance()/run_segment()
so that display thread keeps rendering while a solve chunk runs.
A TelemetryCollector and, whenever output_folder is given, a
telemetry.FileSink AND a telemetry.ZmqSink are always built
(cheap: an in-memory dataclass swap plus, at most, one small JSON
write every telemetry_interval_seconds, plus one daemon thread
idling on a socket poll) — see notes/ng-run-telemetry-design.md
(control-project repo) for the design this implements (T1+T2).
progress.json is written into output_folder continuously while
the run is in progress, not only once at the end, so a process
watching the folder from outside sees live progress; the
ZmqSink’s own telemetry_endpoint.json lets a separate Python
process on the same machine query (snapshot/history/ping) and
cancel this run directly, without polling files. Without an
output_folder telemetry still runs in-memory only (no sinks —
no folder to write progress.json/telemetry_endpoint.json into),
so job.telemetry()/job.cancel() still work for a same-process
caller, just not for a separate process.
solverSolver — The runtime solver configuration.input_filetyping.Any — The simulation object. Currently this must be asimple_config.simulation.Waveforminstance with sources, receivers, and a mesh attached.output_folderstr | pathlib.Path | None =None— If given,receivers.h5,meta.json,progress.json, andtelemetry_endpoint.jsonare written there.overwritebool =False— Allow writing into an already existing output folder. Existing files are not removed first, matching the command-line solverrunbehavior.job_namestr | None =None— An optional custom job name. Also becomesprogress.json’s owntelemetry.run.job_id.ranksint | None =None— Accepted for compatibility; the solver runs single-process, so anything but 1 warns.verbosityint =1— The verbosity level.telemetry_interval_secondsfloat =2.0— Target spacing, in seconds, betweenprogress.jsonrewrites and between adaptive-chunkingprogress_callbackfirings (solve_forward_through_sessions’s ownprogress_interval_seconds) — seetelemetry.FileSink.waitbool =True—True(the default) blocks until the solve finishes and returns the finished job, as before T2 (internally, the solve still runs on a worker thread — this is what lets a live display run at the same time; see above).Falsereturns theNGInProcessJobimmediately;job.wait()blocks until it finishes.progressFrontendOverride ='auto'—"auto"(the default) resolves a live display viaprogress_display.detect_frontend()(anipywidgetspanel in Jupyter, arichTUI panel on an interactive terminal, atqdmbar otherwise);"widget"/"panel"/"tqdm"force that frontend;"plain"shows no live display but leaves the old, verbosity-gated print lines (P6) alone;"none"shows nothing at all, including those print lines, regardless ofverbosity.
wait=True, still running if wait=False.Classes
NGInProcessJob
NGInProcessJobclass NGInProcessJob(builtins.object):
def __init__(
self,
event_data: EventData | None = None,
recorded_data: typing.Any = None,
event_block: EventBlock | None = None,
time_axis: typing.Any = None,
runtime_in_seconds: float | None = None,
job_name: str | None = None,
boundaries: typing.Sequence[dict] | None = None,
timing_information: dict | None = None,
telemetry_snapshot: TelemetrySnapshot | None = None,
collector: TelemetryCollector | None = None,
) -> None: ...The result of an in-process run, mimicking a SalvusJob.
Returned by salvus.flow.functions.run (and run_async) when a
Solver object is passed instead of a site name. run_async
(run_in_process(..., wait=False)) runs the solve on a worker
thread and returns this object immediately, before the run has
finished — wait()/update_status() reflect the run’s real,
live status via the same TelemetryCollector the solve loop
updates; get_as_event()/copy_output() raise until the run
actually finishes. run() (the default, wait=True) keeps the
pre-T2 behavior: the run has already finished by the time this
object exists. Either way this is not backed by the job database.
Wrap an in-process run, finished or still running.
event_dataEventData | None =None— The recorded seismograms.Noneuntil the run finishes (always the case at construction for an async,wait=Falserun).recorded_datatyping.Any =None— The raw recorded receiver values, shape [n_time_steps + 1, n_receivers, n_components].Noneuntil the run finishes.event_blockEventBlock | None =None— The event block defining sources and receivers.Noneuntil the run finishes.time_axistyping.Any =None— Theng_salvustime axis of the run.Noneuntil the run finishes.runtime_in_secondsfloat | None =None— Wall time of the solve.Noneuntil the run finishes.job_namestr | None =None— An optional custom job name.boundariestyping.Sequence[dict] | None =None— The boundary conditions the solver applied (NGSetup.boundaries), recorded inmeta.json.timing_informationdict | None =None— A serializedTimertree, as_serialize_timerproduces.telemetry_snapshotTelemetrySnapshot | None =None— The run’s finalTelemetrySnapshot, for a job constructed without a livecollector(e.g. directly, in a test) —update_status()falls back to mapping this whencollectorisNone.collectorTelemetryCollector | None =None— The run’s liveTelemetryCollector, when one exists (every run throughrun_in_processbuilds one).update_status(),telemetry(),telemetry_stream(), andcancel()all go through this directly rather than thetelemetry_snapshotfallback above, since it is always at least as current.
cancel()
cancel()def cancel(self) -> None: ...Request cancellation of a still-running async (wait=False)
run — honored at the next progress-callback chunk boundary
(see solve_forward_through_sessions’s own cancel_event
parameter), not immediately. A no-op if the run has already
finished, or has no live collector at all (e.g. a synchronous
run, which has already finished by the time this object
exists).
A separate process does the equivalent by sending {"op": "cancel"} to this run’s ZmqSink (see telemetry_endpoint.json) —
both paths set the exact same threading.Event.
copy_output()
copy_output()def copy_output(
self,
destination: str | pathlib.Path,
allow_existing_destination_folder: bool = False,
status: str | None = None,
) -> tuple[dict[str, int], bool]: ...Write the run’s output files to a folder.
Produces the same receivers.h5, meta.json, and job_info.json
files a SalvusCompute run would leave behind; a project only treats a
simulation as computed once job_info.json exists.
destinationstr | pathlib.Path — The output folder.allow_existing_destination_folderbool =False— Allow writing into an existing folder.statusstr | None =None— Written under meta.json’s ownstatuskey.None(the default) uses the freshest knownTelemetrySnapshot.run.status("finished"if none is known at all) —run_in_processpasses this explicitly instead, since it calls this method before the collector’s own final snapshot (which would otherwise still read"running") is built.
delete()
delete()def delete(self) -> None: ...Do nothing; there are no remote files or database entries.
display()
display()def display(self, kind: FrontendOverride = "auto") -> DisplayHandle | None: ...Start a live, environment-adapted progress display for this
job (an ipywidgets panel in Jupyter, a rich TUI panel on an
interactive terminal, a tqdm bar otherwise — see
salvus.flow.ng.progress_display), fed from telemetry_stream() on
its own daemon thread.
Safe to call on a job at any point in its life: already
finished (renders the final snapshot once, synchronously, and
starts no thread), still running (streams live until a
terminal snapshot arrives, then self-closes), or one built
with no live collector at all (same one-shot behavior as the
finished case, via telemetry_stream()’s own fallback).
kindFrontendOverride ='auto'—"auto"(the default) resolves viaprogress_ display.detect_frontend(); otherwise"widget","panel","tqdm","plain", or"none"forces that frontend.
DisplayHandle, or None for kind in ("plain", "none") — both mean “no live display.”get_as_event()
get_as_event()def get_as_event(self) -> EventData: ...Get the simulation results.
EventData object.get_diagnosis_string()
get_diagnosis_string()def get_diagnosis_string(self) -> str: ...Get a diagnosis string for the job.
get_status_from_db()
get_status_from_db()def get_status_from_db(self) -> JobStatus: ...Return the current status — matches command-line solver
SalvusJob. get_status_from_db() in shape (for job-agnostic callers
that read it unconditionally, e.g. SimulationStore._to_info_dict),
but there is no database behind an in-process run: this simply returns
update_status()’s own always-live result.
JobStatus.telemetry()
telemetry()def telemetry(self) -> TelemetrySnapshot | None: ...The freshest known TelemetrySnapshot for this run, or None
if none has been recorded yet (immediately after an async
construction, before the worker thread’s first publish).
This same-process job object reads its own TelemetryCollector
directly — the cheapest and always-current path. A separate
process on the same machine reaches the identical live state
via the ZmqSink/telemetry_endpoint.json handshake this
run’s run_in_process call also sets up (see telemetry.ZmqSink), not
through this method.
telemetry_stream()
telemetry_stream()def telemetry_stream(
self, poll_interval_seconds: float = 0.1
) -> typing.Iterator[TelemetrySnapshot]: ...Yield each new TelemetrySnapshot as it is recorded, until the
run reaches a terminal status (finished/failed/
cancelled), then stop.
Polls the live collector (there is no push notification
in-process) at poll_interval_seconds; for a job with no
collector at all (e.g. constructed directly in a test), any
telemetry_snapshot given at construction is yielded once,
then the generator ends immediately.
poll_interval_secondsfloat =0.1— How often to check for a new snapshot.
update_status()
update_status()def update_status(self) -> JobStatus: ...Return the job status.
Maps the freshest known TelemetrySnapshot.run.status (the
live collector’s own latest, when one is attached — which
every run through run_in_process does — otherwise the
telemetry_snapshot a job was directly constructed with) onto
a JobStatus; falls back to JobStatus.finished when neither
exists (e.g. a job constructed with no telemetry at all, in a
test), matching this method’s behavior before telemetry
existed.
JobStatus.wait()
wait()def wait(self, timeout: float | None = None) -> JobStatus: ...Block until the run finishes (a no-op if it already has), then return the job status.
timeoutfloat | None =None— Maximum seconds to wait for the worker thread, asthreading.Thread.join.None(the default) waits indefinitely. Has no effect on a synchronous (wait=True) run, which has already finished.
JobStatus — see update_status().