feat(spatial): add Ouster point cloud upload and import API by bonsairobo · Pull Request #760 · nominal-io/nominal-client

bonsairobo · 2026-05-07T18:37:30Z

Adds an end-to-end workflow for spatial point-cloud assets: Ouster PCAP → CSV preprocessing → upload → dagger import → scout spatial resource.

Public surface

nominal.thirdparty.ouster.convert_ouster_dataset(...) — PCAP → CSV with optional nav-pose correction. Standalone, usable independently.
nominal.core.upload_point_cloud(client, csv_path, ...) — uploads a point-cloud CSV and creates the corresponding scout spatial resource. Returns the spatial RID.
Adds an ouster optional dependency group (ouster-sdk, numpy, pyyaml).
Adds dagger-client>=30,<31 to core deps.

upload_point_cloud keeps the metadata kwargs the original revision shipped (description, labels, properties, sensor_model, coordinate_system, resolution_mm, scan_pattern).

How `upload_point_cloud` works

Multipart-upload the CSV to scout — bytes go directly to S3 via short-lived presigned PUT URLs; scout never streams the payload.
Ask scout for a 15-minute signed GET URL via POST /upload/v1/sign-download (nominal-io/scout#13918) so dagger can pull the file.
Infer column layout from the first two lines of the CSV (geometry on x/y/z; remaining columns classified int / real / string from the first data row — mirrors scout's pre-removal DaggerSpatialService.ColumnLayout).
Talk to dagger directly through scout's /api/dagger reverse proxy using dagger-client:
- PUT /v1/object-spaces/{workspace.id}?tenant={org_uuid} (idempotent)
- POST /v1/imports/{model_uuid}?tenant={org_uuid}&object-space={workspace.id} (fire-and-forget, 202)
POST /spatial/v1/spatials on scout with dagger_uuid = model_uuid, source_handle = Handle(s3=s3_path), and the metadata kwargs. Returns the spatial RID.

tenant is the org RID's UUID locator. object_space is workspace.id (the workspace locator string — dagger-client's type hint says UUID but the runtime stringifies, matching scout's pre-removal Java behaviour).

Prerequisites — DRAFT until all three land

This PR is not mergeable until:

nominal-io/scout#13407 — scout's spatial CRUD refactor (drops importFile, makes daggerUuid required on CreateSpatialRequest, adds sourceHandle).
nominal-io/scout#13918 — POST /upload/v1/sign-download on UploadService.
A nominal-api release that regenerates against both.

Until then, CreateSpatialRequest(dagger_uuid=..., source_handle=...) will TypeError at runtime (the call carries a # type: ignore[call-arg]), and the presign call uses raw clients.upload._request instead of a typed clients.upload.sign_download(...). Both swap to the typed surface post-regen.

Test plan

New tests/core/test_spatial.py (20 tests): RID locator parsing, geometry / column inference, _classify() heuristics, archetype assembly for the Ouster CSV shape (x, y, z, time, reflectivity, signal, near_infrared), dagger base-URL construction, and the end-to-end mocked flow — asserting CreateSpatialRequest field mapping, presign request shape, dagger client construction, ImportRequest body, and the returned spatial RID.
just verify green on the branch (ruff check, ruff format --check, mypy across all 145 source files, pytest across all 272 tests).
End-to-end smoke against a real scout + dagger stack — pending the prerequisite PRs deploying.

devin-ai-integration

Devin Review found 1 potential issue.

View 5 additional findings in Devin Review.

semgrep-code-nominal-io · 2026-05-13T01:57:01Z

Legal Risk

The following dependencies were released under a license that
is currently prohibited by your organization. Merging is blocked until this is resolved.

Recommendation

Reach out to your security team or Semgrep admin to address this issue. In special cases, exceptions may be made for dependencies with violating licenses, however, the general recommendation is to avoid using a dependency under such a license

LGPL-2.1-or-later

zeroconf 0.148.0

ZPL-2.1

waitress 3.0.0

non-standard

cursor

Cursor Bugbot has reviewed your changes using default mode and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 390fb7b. Configure here.}

bonsairobo · 2026-06-10T17:32:17Z

Rebased this PR onto current main and pushed the updated branch.

Follow-up commits added here:

chore(deps): bump nominal-api to 0.1256.0
fix(spatial): validate scan pattern values
fix(ouster): harden conversion error handling
fix(compute): adapt rolling aggregation API
fix(spatial): tighten generated model typing

Feedback addressed:

Upgraded to the released nominal-api==0.1256.0 package and adjusted compute rolling aggregation serialization for the generated API shape.
Validated scan_pattern before upload side effects and added docstring/test coverage.
Improved Ouster conversion dependency errors, invalid buf3 diagnostics, invalid record logging, and empty-nav trajectory handling.
Added focused tests for the new scan-pattern and Ouster failure modes.

Verification run locally:

uv run ruff format --check
uv run ruff check
uv run mypy
uv run pytest (302 passed)

Add a single-function API for the end-to-end workflow: Ouster PCAP dataset → CSV preprocessing → upload → Dagger spatial import. - NominalClient.upload_point_cloud(): uploads a CSV point cloud and triggers spatial import via Scout's spatial service (import-file endpoint) - nominal.thirdparty.ouster.upload_ouster_point_cloud(): orchestrates Ouster PCAP preprocessing then calls upload_point_cloud() for each sensor - nominal.thirdparty.ouster.convert_ouster_dataset(): standalone preprocessing (PCAP → CSV with nav pose correction), usable independently - Register SpatialService in ClientsBunch - Add 'ouster' optional dependency group (ouster-sdk, numpy, pyyaml) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The dagger reverse proxy is mounted at `/api/dagger` in combined-service. Previously `_dagger_base_url` assumed `_api_base_url` already ended in `/api` and appended a literal `/dagger`. That broke for bare-host base URLs ("https://api.gov.nominal.io") and silently produced `/dagger` instead of `/api/dagger`. Carry the full `/api/dagger` prefix in the constant and strip a trailing `/api` from the base before appending, so both forms work.

The single-row classifier mis-tagged columns whose first row happens to be integer-valued but later rows are float (`stress=1` then `0.998`), which broke downstream dagger ingest when the typed column failed to accept the float values. Symmetrically, a numeric-looking first row followed by a non-numeric row needs to demote to string rather than fail on the server. Read up to 1000 non-empty data rows and pick the most permissive type that covers every sample value: any non-numeric forces string, any mixed int/float promotes to real.

Reductions are the per-partition aggregations the renderer samples at coarse LOD zoom levels. Without them, Int / Real attributes can't drive `ColorSource::Ramp` (Geometry coloring falls back to solid white) and `VolumetricFilter::ValueRange` is a no-op. Attach Min + Max + Mean to every Int / Real attribute on ingest. Min and Max satisfy ValueRange's two-sided filter; any single reduction is enough to drive Ramp coloring. String attributes have no useful scalar aggregation, so their reductions stay empty.

Sample-based inference still has blind spots: an all-integer-valued sample for a column that is *semantically* float, or a numeric-looking string column that the caller knows shouldn't be coerced. Give callers an escape hatch. `column_types={"stress": "real"}` forces the named columns to the given type; unknown column names raise ValueError, invalid type values raise ValueError, and geometry columns (x/y/z) are silently ignored if listed.

`SamplerType.MEAN` is not a valid pairing with an Int-typed attribute, so Int attributes get Min + Max only. Real attributes still get Min + Max + Mean. Min/Max alone are sufficient to drive `VolumetricFilter::ValueRange` and `ColorSource::Ramp` for Int columns.