Repository navigation
feat(agg,cli,plugin): v0.1.12 — R-31–R-37 (xlsx, parquet, provenance, corr --by, category keys, agg budget) - #10
Merged
Merged
Conversation
…e, corr --by, category keys, agg row budget
…quet string fidelity, corr --by caps - tool layer: runAnalyzeCli returns stdout JSON on non-zero exit when it parses with an ok field, so dataset_verify delivers checks/mismatches on mismatch instead of throwing a misclassified invalid-args
…back, mock/Rust parity, README sync to v0.1.12 - cmd_verify falls back to parentDatasetId when parentDatasetIds is absent (matches interface.md, load/list, and the mock); legacy-parent test covers present and missing parent
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
v0.1.12 full requirements batch from the post-v0.1.11 lists (R-31–R-37):
agg/transform_aggalways return up to 20 inline preview rows (min(20, rowCount)), with or without--out. Fixes the silent empty-rowshole whenrowCount > 20and no--out, and returns full rows for small results that also persist via--out.truncatedis now simplyrowCount > 20.ingest --format xlsxvia calamine — first sheet or--sheet <name|index>(exact sheet-name match wins, so all-digit names like2024stay selectable; digits fall back to the 0-based index); multi-sheet →sheet-skippedwarning; corrupt/password → exit 2 actionable;.xlsmstays exit 4.{id}.parquet); legacy{id}.csvremains readable (lazy fallback);rmremoves both. Internal Utf8 cells use an unambiguous encoding — strings stored as quoted JSON, scalars raw — so string values like"007"/"NA"survive save/load unchanged and meta dtypes stay consistent; exported parquet keeps raw strings (ingest round-trip unchanged).sourceChecksum(sha256:…); derived sets recordparentDatasetIds(join/union/diff =[left, right];parentDatasetIdstays first parent). Newverify/dataset_verifycommand + capability token (--check-source), including fallback toparentDatasetIdfor legacy metas.dataset_listsurfaces both fields.corr --by g1[,g2…]— per-group pairwise pairs withbykeys; caps 50 groups / 100 pairs + truncation warnings (both warning codes can co-occur);pairCountreports all candidate pairs (groupCount × column-pairs).1/1.0share one1label;1.5stays exact.Frozen contract preserved: tool names, one-line stdout JSON,
analysis/*codes,.dsh-analyzedata root. Capability list appendsverify.Review rounds folded in: tool layer surfaces
verify's exit-1 JSON (checks/mismatches), mock CLI kept in parity with Rust (capabilities, id validation, real row counts, source status), READMEs synced to v0.1.12.Verification
cargo test -p dsh-analyze— 201/201 (64 unit + 39 agg + 35 periphery + 5 quality + 22 relation + 18 series + 18 transform)tsc -p tsconfig.json— passnode --testpolicy/tools/schema — 98/98verify --check-sourceclean; R-32.parquetwritten; R-31 xlsx round-trip 21 rows;versionincludesverifyTest plan
e2e-v016-real-cli.mjscases after CLI sync (follow-up, external repo)