Problem
Sedona's distributed GeoPandas layer uses field metadata to distinguish a known CRS from a geometry column whose CRS must be inferred from its embedded SRID. A locally constructed geometry column with crs=None currently has no CRS metadata, so it is indistinguishable from a raw distributed geometry column with unknown provenance.
Reading .crs for such a metadata-free column performs a distributed ST_SRID(first(...)) aggregation. APIs that inspect both operands' CRS, including binary geometry predicates, can therefore launch two eager Spark actions before returning an otherwise lazy result.
In a local Spark benchmark, comparing two CRS-less local operands submitted four Spark jobs before result materialization. Recording authoritative no-CRS metadata reduced that count to zero. The one-time constructor cost increased by approximately 2.3 ms for GeoSeries and 4.9 ms for GeoDataFrame in a warm one-row benchmark.
Expected behavior
Sedona should represent three distinct states:
- a known CRS stored as CRS metadata;
- a known absence of CRS stored as explicit empty CRS metadata; and
- an unknown raw or legacy distributed geometry column with no CRS metadata, which retains the existing embedded-SRID fallback.
The implementation should:
- record explicit no-CRS metadata for locally owned
GeoSeries and GeoDataFrame geometry columns;
- preserve locally supplied per-column CRS values and synchronize their embedded SRIDs;
- propagate authoritative CRS state through
set_geometry, WKT/WKB and file constructors, copies, selections, and all-null geometry columns;
- preserve public
crs=None without erasing an embedded EWKB SRID;
- leave raw Spark and pandas-on-Spark geometry columns without metadata on the existing SRID-inference path;
- preserve Series names, indexes, active-geometry selection, and laziness; and
- add action-count, metadata propagation, raw-column fallback, and compatibility coverage across supported Spark and GeoPandas versions.
Performance acceptance
- Constructing a binary predicate result from two locally created CRS-less operands should launch no Spark jobs solely for CRS discovery.
- Metadata-free raw distributed operands should continue to infer their embedded SRIDs and warn when they differ.
- The returned predicate and geometry plans should remain distributed and should not add Python execution nodes.
Part of #2230.
Sedona version
2.0.0-SNAPSHOT
API type
Python / distributed GeoPandas API
Problem
Sedona's distributed GeoPandas layer uses field metadata to distinguish a known CRS from a geometry column whose CRS must be inferred from its embedded SRID. A locally constructed geometry column with
crs=Nonecurrently has no CRS metadata, so it is indistinguishable from a raw distributed geometry column with unknown provenance.Reading
.crsfor such a metadata-free column performs a distributedST_SRID(first(...))aggregation. APIs that inspect both operands' CRS, including binary geometry predicates, can therefore launch two eager Spark actions before returning an otherwise lazy result.In a local Spark benchmark, comparing two CRS-less local operands submitted four Spark jobs before result materialization. Recording authoritative no-CRS metadata reduced that count to zero. The one-time constructor cost increased by approximately 2.3 ms for
GeoSeriesand 4.9 ms forGeoDataFramein a warm one-row benchmark.Expected behavior
Sedona should represent three distinct states:
The implementation should:
GeoSeriesandGeoDataFramegeometry columns;set_geometry, WKT/WKB and file constructors, copies, selections, and all-null geometry columns;crs=Nonewithout erasing an embedded EWKB SRID;Performance acceptance
Part of #2230.
Sedona version
2.0.0-SNAPSHOT
API type
Python / distributed GeoPandas API