Skip to content

feat: osa ingestion status / list — observe a run after starting it #204

Description

@rorybyrne

osa ingestion has only a start subcommand. Once a run is started there is no supported way to find out what it did.

Environment: osa-py 0.8.0.

Current behaviour

$ osa ingestion start --convention global-seismic-events --limit 25
 ✓ Starting ingestion for global-seismic-events  urn:osa:localhost:ing:4df450f2-...  0.1s

$ osa ingestion --help
╭─ Commands ──────────────────────────────────────╮
│ start  Start an ingestion run for a convention. │
╰─────────────────────────────────────────────────╯

It prints a URN and exits. There's no status, no list, and the URN can't be passed to anything.

What checking a run actually took

docker logs my-archive-server-1 2>&1 | grep -E "4df450f2"

and then, because the run's own COMPLETE line reported a count that turned out to be wrong (linked below), querying Postgres directly to establish the truth:

select count(*) from records;
select count(*) from metadata.seismic_event_v2;
select count(*) from features.estimate_energy;

That's three levels below the CLI to answer "did my ingestion work". osa logs server helps, but it means reading interleaved request logs and knowing which line matters.

Suggested fix

  • osa ingestion status <urn> — stage, pulled / passed / published / failed counts, and any batch error.
  • osa ingestion list — recent runs per convention with their outcome.

The data is already there: ingest_runs is a table, the workflow logs per-stage transitions, and process_batch already computes the counts. This is mostly surfacing.

Related: the counts these commands report should mean queryable rows, per the linked reporting bug — otherwise status inherits the same problem the log line has.

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew functionality

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions