Skip to content

[GH-3329] Preserve dimensions of empty Points, LineStrings, and Polygons - #3335

Open
jiayuasu wants to merge 4 commits into
apache:masterfrom
jiayuasu:fix/empty-linestring-dimensions
Open

[GH-3329] Preserve dimensions of empty Points, LineStrings, and Polygons#3335
jiayuasu wants to merge 4 commits into
apache:masterfrom
jiayuasu:fix/empty-linestring-dimensions

Conversation

@jiayuasu

@jiayuasu jiayuasu commented Sep 8, 2026

Copy link
Copy Markdown
Member

Did you read the Contributor Guide?

Yes.

Is this PR related to a ticket?

Closes #3329. Alternative to #3332 that fixes geometry serialization shared by the conversion methods.

What changes were proposed in this PR?

An empty 2D LineString can acquire a Z dimension when Sedona deserializes it with older GEOS versions. This changes the WKB type codes of an all-empty GeometryCollection during conversion to GeoPandas or Arrow. Empty 3D Points and Polygons also lose Z in the C decoder, while the pure-Python fallback drops Z from all three empty geometry types.

Preserve the stored dimensions of empty Points, LineStrings, and Polygons in both paths:

  • The C decoder constructs empty Points and LineStrings from their stored coordinate layout, and empty Polygons from an empty ring with that layout. Empty XY/XYZ sequences use the explicit dimension constructor because the bulk coordinate-copy function in GEOS 3.10–3.11 infers XYZ from an empty buffer. M/ZM sequences retain the flag-aware bulk path on supporting GEOS versions.
  • The pure-Python serializer reads the empty geometry's WKB dimension flag, which also works on Shapely 1.x where _ndim reports 2 for XYZ empties. On GEOS older than 3.9, retain the existing XY encoding for empty Points because those WKB writers cannot represent them. Its decoder constructs typed empty geometries using the stored layout. The fallback continues to reject M/ZM explicitly, as before.
  • Regression tests check exact serialized headers, exact WKB, nested collection members, both directions between the C and fallback paths, and Spark round trips. Remove the existing test workarounds that hid the LineString mismatch. Append the all-empty collection fixture and preserve the original equal-length collection/polygon pair so transform and align=False coverage is retained.

No special case is needed in the Arrow export methods.

How was this patch tested?

Current local serialization results:

Runtime Coverage Result
Python 3.8 / Shapely 1.7.0 / GEOS 3.8.0 Fallback only; C extension unavailable 21 passed, 33 skipped
Python 3.8 / Shapely 1.8.5.post1 / GEOS 3.10.3 C and fallback 75 passed, 31 skipped
Python 3.9 / Shapely 2.0.7 / GEOS 3.11.4 C and fallback 79 passed, 27 skipped
Python 3.10 / Shapely 2.1.2 / GEOS 3.13.1 C and fallback 106 passed

The combined runs use tests/utils/test_geomserde_speedup.py and tests/utils/test_geometry_serde_empty.py. The Shapely 1.7.0 wheel run loads the fallback test file with the native extension unavailable. It checks the actual GEOS 3.8 runtime, including typed empty Point serialization and decode/re-serialize. New regression tests also assert that the unsupported WKB writer is never called.

Before the dimension fixes, the original 103-case suite had 32 failures on GEOS 3.10, 36 on 3.11, and 46 on 3.13. The three added GEOS 3.8 compatibility cases fail against commit 3303296 and pass with the guard. The real Shapely 1.7.0 wheel also reproduced the regression: its empty-Point WKB writer returns empty bytes, causing an IndexError during serialization without the guard.

Version-dependent skips cover M/ZM support, SRID APIs, and Shapely 1.x hiding all-empty collection members. The GEOS 3.8 run additionally skips six requested layouts its runtime cannot preserve and 20 cases requiring the unavailable C extension. Its collection assertions compare members recursively because empty Points cannot be written as WKB; writable leaves require nonempty WKB before comparison. GEOS 3.12 was not run locally.

  • Prior distributed validation on commit 3303296, unchanged C decoder: Python 3.9 / Spark 3.5.0 / GeoPandas 1.0.1 / PyArrow 17.0.0, using the released Sedona 1.9.0 shaded JAR and the rebuilt Python extension: 43 selected tests passed. These include eight new empty-geometry Spark cases, Arrow/WKB exports, all four affected affine-transform groups, and eight binary-operation tests with the original equal-length collection/polygon pair.
  • In the same distributed run, direct checks passed for GeoSeries.to_geopandas(), GeoSeries.to_arrow(), and GeoDataFrame.to_arrow() with 2D/3D empty geometries, all-empty collections, a nonempty 3D point, and a null.
  • All applicable pre-commit hooks and git diff --check passed.

Did this PR include necessary documentation updates?

No public API changes. This corrects dimension preservation in the existing serialization paths.

@jiayuasu jiayuasu changed the title [GH-3329] Preserve empty LineString dimensions during deserialization [GH-3329] Preserve dimensions of empty Points, LineStrings, and Polygons Sep 9, 2026
@jiayuasu jiayuasu added this to the sedona-2.0.0 milestone Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GeoSeries.to_arrow differs for all-empty GeometryCollection members

1 participant