Skip to content

feat: in-place row edits for Arrow tables and GeoJSON layers - #3688

Open
igorDykhta wants to merge 2 commits into
masterfrom
igr/row-edit-for-arrow-containers
Open

feat: in-place row edits for Arrow tables and GeoJSON layers#3688
igorDykhta wants to merge 2 commits into
masterfrom
igr/row-edit-for-arrow-containers

Conversation

@igorDykhta

@igorDykhta igorDykhta commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • addToDataset / removeFromDataset now mutate Arrow tables with primitive columns in place (concat append, keyed upsert, slice remove). DuckDB still warns and no-ops until INSERT exists.
  • GeoJSON layers rebuild dataToFeature when the table revision or row count changes, so appended features show on the map. Filter-only updates still skip a re-parse.
  • Live-data example: Arrow and GeoJSON remount a matching table; GeoJSON fetches buildings-australia.geojson at runtime.

Test plan

  • yarn test vis-state + data-container + DuckDB no-op
  • yarn start:live-dataArrow: Add / Remove last / Insert then Move tracker (row count stays put)
  • GeoJSON: seed buildings appear; Add 1 / Add 5 draw new footprints; Move tracker swaps track-01 without growing the table

Ihor Dykhta added 2 commits September 1, 2026 17:47
Signed-off-by: Ihor Dykhta <ihordykhta@Ihors-MacBook-Pro.local>
Signed-off-by: Ihor Dykhta <ihordykhta@Ihors-MacBook-Pro.local>
Copilot AI lite review requested due to automatic review settings September 1, 2026 17:37
@igorDykhta igorDykhta changed the title Igr/row edit for arrow containers feat: in-place row edits for Arrow tables and GeoJSON layers Sep 1, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends Kepler’s in-place row editing flow (addToDataset / removeFromDataset) to work with Arrow-backed datasets (for supported “primitive” Arrow column types), while keeping DuckDB tables as a warned no-op. It also adjusts GeoJSON layer metadata rebuilding so feature parsing stays correct when datasets change via row edits.

Changes:

  • Add append / replace / remove support to ArrowDataContainer, with explicit rejection for nested/binary/geoarrow columns.
  • Keep DuckDB row edits as a warned no-op at the table layer, and update action/reducer/docs text to reflect Arrow support.
  • Rebuild GeoJSON dataToFeature when row count or table revision changes (not on filter-only updates), plus expand tests and the live-data example to cover Arrow/GeoJSON cases.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
test/node/utils/duckdb-utils-test.js Adds coverage ensuring DuckDB row edits remain a warned no-op.
test/node/utils/data-container-test.js Adds unit tests for ArrowDataContainer append/replace/remove semantics and rejection cases.
test/node/reducers/vis-state-test.js Adds integration coverage for Arrow dataset row edits and GeoJSON feature rebuild behavior.
src/utils/src/data-container-interface.ts Updates DataContainerInterface docs to reflect Arrow support for row edits.
src/utils/src/arrow-data-container.ts Implements Arrow in-place row edits with type gating and schema/metadata preservation.
src/table/src/kepler-table.ts Updates table-level row edit behavior/docs; tightens upsert replace failure handling.
src/reducers/src/vis-state-updaters.ts Updates reducer docs to include Arrow behavior and rejection rules.
src/layers/src/geojson-layer/geojson-layer.ts Rebuilds parsed GeoJSON features when row count or meta revision changes.
src/layers/src/base-layer.ts Plumbs trigger-change metadata into updateLayerMeta calls.
src/duckdb/src/table/duckdb-table.ts Adds explicit DuckDB table overrides for row edits that warn + return false.
src/actions/src/vis-state-actions.ts Updates action docs to reflect Arrow support and DuckDB/no-op constraints.
src/actions/src/actions.ts Updates addDataToMap JSDoc to clarify Arrow vs DuckDB behavior.
examples/live-data/src/app.tsx Extends the live-data example to demo CSV vs Arrow vs GeoJSON remounted sessions.
examples/live-data/README.md Documents the new live-data example modes and limitations.
docs/api-reference/reducers/vis-state.md Updates public reducer docs to reflect Arrow support and rejection rules.
docs/api-reference/actions/actions.md Updates public action docs to reflect Arrow support and DuckDB behavior.
CHANGELOG.md Notes Arrow support for row edits and GeoJSON rebuild fix in Unreleased.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/table/src/kepler-table.ts
Comment thread test/node/utils/duckdb-utils-test.js
@igorDykhta
igorDykhta requested a review from lixun910 September 1, 2026 17:48
@igorDykhta igorDykhta self-assigned this Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants