[travon-speller]

Description

Adds guided Clean data and Transform data authoring to Automation, with typed Purge filters, operation-specific configuration, upstream table inspection, and improved active-run result selection.

Required backend: SEMOSS/Semoss#3163. Deploy the paired branches and refresh the backend node catalog after restart.

Changes Made

• Expose clean.process in the Clean category and data.transform in Data, using backend catalog definitions.
• Add operation-specific forms and persisted-graph adapters for six Clean operations and seventeen Transform operations, including joins, Pivot/unpivot, case changes, split, schema changes, and percent/timestamp operations.
• Add typed Purge filter Builder/JSON modes. Preserve scalar value types, validate malformed rules, respect read-only state, and prevent stale rules from silently replacing invalid edits.
• Restore Inspect table as an explicit, bounded read of previous upstream outputs through existing authorized SDK/Pixel APIs—not a Python preview or an execution of draft nodes.
• Update active-run automatic selection to follow running work and settle on the final successful or failed node, while preserving explicit user selection. Include pure selection-resolver tests.
• Add API, catalog, adapter/round-trip, validation, Purge, and selection tests, plus Clean/Transform capability documentation.

The current merge-base diff is limited to packages/automation (26 files). It does not add a new service, permission system, frame runtime, or Python-node execution mode.

Authoring and Inspection Flow

flowchart TD
    A[Backend node catalog] --> B[Add Clean or Transform node]
    B --> C[Choose upstream table and operation]
    C --> D[Guided fields or typed Purge Builder / JSON]
    D --> E[Validate and serialize node configuration]
    E --> F[Existing authorized save and Java execution]
    C --> G[User clicks Inspect table]
    G --> H[Read up to 20 recent runs through existing APIs]
    H --> I{Matching successful root-node table}
    I -->|Retained result| J[Read schema and bounded result page]
    I -->|Available live frame| K[Use existing frame pager]
    I -->|No usable output| L[Show actionable empty state]
    J --> M[Table viewer; never execute draft nodes]
    K --> M
    F --> N[Active run: automatic latest/final selection or explicit user selection]

Loading

Behavior and Limits

• Inspect table displays previous run data, not a newly computed preview of an unsaved draft. It prefers retained table results, with existing live-frame paging when available.
• The authoring lookup currently handles root-node results; it does not select loop iterations. Do not treat it as verified per-iteration input inspection.
• Backend validation/RBAC remain authoritative. Unsupported frame backends and operation settings are not made executable by a frontend control.
• These changes include new active-run selection code; historical run viewing is a separate path. Pure resolver tests do not replace testing stream completion and run switching in the UI.
• Use the operation support matrix in packages/automation/docs/clean-nodes.md; R/Polars and shared BI importer fixes are outside this PR.

Validation Status — 2026-10-28

Reviewed head: 74332a94313ea2b500bc42409b9f59acb1e8bd2f.

• pnpm --filter @semoss/automation test: 18 files / 109 tests passed.
• Biome on all 25 changed TypeScript/TSX files: passed, no fixes applied.
• Automation production Vite build to an isolated review directory: passed, with the large-chunk warning.
• Current GitHub build and Snyk checks: passed.
• Diff whitespace check: passed.
• Full @semoss/automation type-check: not passing (29 diagnostics in unchanged dev audit-log/run-data-browser/viewer files and a local untracked tabular-file form). These diagnostics are not attributed to this PR's changed forms; unrelated files were not modified.
• Authenticated internal-browser save/run/inspection/accessibility regression: unverified in this review; the connected browser exposed only a blank tab, not the user's signed-in internal browser.

Open Review Findings Before Merge

• Fix Pivot's guided Columns to keep editor: filtering blank values in onChange immediately removes the row added by Add column, so users cannot enter a second maintained column. A focused review-only test using the actual form/list handlers reproduces the failure. JSON entry is only a workaround.
• Add a component regression for adding/editing/deleting maintained columns and switching form/JSON modes.
• Resolve the companion backend's space-delimiter and Join-cleanup findings, then complete paired live regression.
• Verify active completion, explicit selection, failure selection and run switching; table inspection after save/reopen; read-only behavior; keyboard/focus and narrow layouts.

How to Test

1. Build/reload both branches and open an authorized disposable Automation. Add Query, Clean, Transform and a downstream consumer; save and reopen to verify all settings round-trip.
2. Test each operation-specific form and both Purge modes using text, numeric, boolean, variable-bound and invalid values. Confirm invalid edits cannot save stale valid settings.
3. Run representative root and loop paths. Inspect original/transformed data, schema changes, empty results, retained paging and reopened history.
4. Click Inspect table with retained data, no previous output, and an unavailable live frame. Confirm it reads history without launching a run and correctly identifies the source run.
5. Test Pivot with multiple maintained columns and concatenate/split with a space delimiter after the open fixes; retest downstream values.
6. Verify automatic active-run selection, explicit earlier-node selection, failed runs and switching between runs. Check read-only users and inaccessible projects, then keyboard/focus/responsive behavior.

[snyk-io]

✅ Snyk checks have passed. No issues have been found so far.

Status	Scan Engine	Critical	High	Medium	Low	Total (0)
✅	Open Source Security	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.