[GH-3329] Preserve dimensions of empty Points, LineStrings, and Polygons - #3335
Open
jiayuasu wants to merge 4 commits into
Open
[GH-3329] Preserve dimensions of empty Points, LineStrings, and Polygons#3335jiayuasu wants to merge 4 commits into
jiayuasu wants to merge 4 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Did you read the Contributor Guide?
Yes.
Is this PR related to a ticket?
Closes #3329. Alternative to #3332 that fixes geometry serialization shared by the conversion methods.
What changes were proposed in this PR?
An empty 2D LineString can acquire a Z dimension when Sedona deserializes it with older GEOS versions. This changes the WKB type codes of an all-empty GeometryCollection during conversion to GeoPandas or Arrow. Empty 3D Points and Polygons also lose Z in the C decoder, while the pure-Python fallback drops Z from all three empty geometry types.
Preserve the stored dimensions of empty Points, LineStrings, and Polygons in both paths:
_ndimreports 2 for XYZ empties. On GEOS older than 3.9, retain the existing XY encoding for empty Points because those WKB writers cannot represent them. Its decoder constructs typed empty geometries using the stored layout. The fallback continues to reject M/ZM explicitly, as before.align=Falsecoverage is retained.No special case is needed in the Arrow export methods.
How was this patch tested?
Current local serialization results:
The combined runs use
tests/utils/test_geomserde_speedup.pyandtests/utils/test_geometry_serde_empty.py. The Shapely 1.7.0 wheel run loads the fallback test file with the native extension unavailable. It checks the actual GEOS 3.8 runtime, including typed empty Point serialization and decode/re-serialize. New regression tests also assert that the unsupported WKB writer is never called.Before the dimension fixes, the original 103-case suite had 32 failures on GEOS 3.10, 36 on 3.11, and 46 on 3.13. The three added GEOS 3.8 compatibility cases fail against commit 3303296 and pass with the guard. The real Shapely 1.7.0 wheel also reproduced the regression: its empty-Point WKB writer returns empty bytes, causing an IndexError during serialization without the guard.
Version-dependent skips cover M/ZM support, SRID APIs, and Shapely 1.x hiding all-empty collection members. The GEOS 3.8 run additionally skips six requested layouts its runtime cannot preserve and 20 cases requiring the unavailable C extension. Its collection assertions compare members recursively because empty Points cannot be written as WKB; writable leaves require nonempty WKB before comparison. GEOS 3.12 was not run locally.
GeoSeries.to_geopandas(),GeoSeries.to_arrow(), andGeoDataFrame.to_arrow()with 2D/3D empty geometries, all-empty collections, a nonempty 3D point, and a null.git diff --checkpassed.Did this PR include necessary documentation updates?
No public API changes. This corrects dimension preservation in the existing serialization paths.