Repository navigation
geopandas 1.0.0 or pyogrio 0.9.0 doesn't support OGR_GMT output? #3301
Description
geopandas v1.0.0 was released a few hours ago and now our Docs workflow fails to build (https://github.com/GenericMappingTools/pygmt/actions/runs/9559533732/job/26634813440) which is not caught in our "GMT Dev Tests" workflow (why?):
Extension error:
Here is a summary of the problems encountered when running the examples:
Unexpected failing examples:
../examples/get_started/04_table_inputs.py failed leaving traceback:
Traceback (most recent call last):
File "/home/runner/work/pygmt/pygmt/examples/get_started/04_table_inputs.py", line 111, in <module>
fig.plot(data=gdf, style="c0.2c", fill="purple")
File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/pygmt/helpers/decorators.py", line 609, in new_module
return module_func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/pygmt/helpers/decorators.py", line 773, in new_module
return module_func(*bound.args, **bound.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/pygmt/src/plot.py", line 258, in plot
with lib.virtualfile_in(
File "/home/runner/micromamba/envs/pygmt/lib/python3.12/contextlib.py", line 137, in __enter__
return next(self.gen)
^^^^^^^^^^^^^^
File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/pygmt/helpers/tempfile.py", line 164, in tempfile_from_geojson
geojson.to_file(**ogrgmt_kwargs)
File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/geopandas/geodataframe.py", line 1536, in to_file
_to_file(self, filename, driver, schema, index, **kwargs)
File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/geopandas/io/file.py", line 686, in _to_file
_to_file_pyogrio(df, filename, driver, schema, crs, mode, metadata, **kwargs)
File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/geopandas/io/file.py", line 748, in _to_file_pyogrio
pyogrio.write_dataframe(df, filename, driver=driver, metadata=metadata, **kwargs)
File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/pyogrio/geopandas.py", line 456, in write_dataframe
path, driver = _get_write_path_driver(path, driver, append=append)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/pyogrio/raw.py", line 564, in _get_write_path_driver
if not ogr_driver_supports_write(driver):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pyogrio/_ogr.pyx", line 105, in pyogrio._ogr.ogr_driver_supports_write
File "pyogrio/_ogr.pyx", line 287, in pyogrio._ogr._get_driver_metadata_item
pyogrio.errors.DataSourceError: Could not obtain driver: OGR_GMT (check that it was installed correctly into GDAL)
../examples/gallery/lines/roads.py failed leaving traceback:
Traceback (most recent call last):
File "/home/runner/work/pygmt/pygmt/examples/gallery/lines/roads.py", line 19, in <module>
gdf = gpd.read_file(
^^^^^^^^^^^^^^
File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/geopandas/io/file.py", line 294, in _read_file
return _read_file_pyogrio(
^^^^^^^^^^^^^^^^^^^
File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/geopandas/io/file.py", line 547, in _read_file_pyogrio
return pyogrio.read_dataframe(path_or_bytes, bbox=bbox, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/pyogrio/geopandas.py", line 261, in read_dataframe
result = read_func(
^^^^^^^^^^
File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/pyogrio/raw.py", line 196, in read
return ogr_read(
^^^^^^^^^
File "pyogrio/_io.pyx", line 1239, in pyogrio._io.ogr_read
File "pyogrio/_io.pyx", line 219, in pyogrio._io.ogr_open
pyogrio.errors.DataSourceError: No driver registered.
Activity
seisman commentedon Jun 26, 2024
That's weird. I tried to create the same environment locally.
Now I have geopandas 1.0.0, pyogrio 0.9.0 and gdal 3.9.0 installed, but the script works well for me.
seisman commentedon Jun 26, 2024
There is one more failure:
seisman commentedon Jun 26, 2024
This error comes from GDAL https://github.com/OSGeo/gdal/blob/a66c71d0e1e19a5ef9d94b33bef90074173be9c2/gcore/gdaldataset.cpp#L3965C7-L3965C7
weiji14 commentedon Jul 9, 2024
The "GMT Dev Tests" installs
geopandasfrom PyPI, whereas the Docs workflow installsgeopandasfrom conda-forge. Are there differences in GDAL version used? I'm reading https://github.com/geopandas/pyogrio/blob/v0.9.0/docs/source/install.md#troubleshooting-installation-errors, and it says errors likepyogrio.errors.DataSourceError: Could not obtain driver ...may be due to incompatible versions of GDAL in fiona and pyogrio, but we don't have fiona installed anymore 😕Your example at #3301 (comment) works for me too if I run it individually. But when I do
cd docs; make clean; make allto run the sphinx build, it fails the first time, but works if I run it twice. Looks like there's some flaky behaviour going on.seisman commentedon Jul 9, 2024
The "Tests" workflow also has geopandas from conda-forge, but the geopandas tests all pass.
weiji14 commentedon Jul 9, 2024
Found a related issue upstream at geopandas/pyogrio#144. There is supposed to be some logic to register GDAL drivers when pyogrio is imported (geopandas/pyogrio#145), but it is not working properly in Sphinx build for some reason, and causing that
pyogrio.errors.DataSourceError: Could not obtain driver: OGR_GMT (check that it was installed correctly into GDAL)error. Note that the OGR_GMT driver does exist in our GDAL installation, just that the driver isn't loaded properly.Workaround is to manually register the drivers like so:
See also https://github.com/GenericMappingTools/pygmt/pull/3305/files#r1669745089 where this patch is applied.
weiji14 commentedon Jul 10, 2024
Have reported this upstream, and got a super helpful response at geopandas/pyogrio#448 (comment). It seems like there may be some conflict with the order of how rioxarray/rasterio and pyogrio is loading the GDAL drivers.
weiji14 commentedon Dec 2, 2024
Hmm, hitting into this issue again, but this time on
rasterio. Traceback from https://github.com/GenericMappingTools/pygmt/actions/runs/12128509215/job/33815018072#step:7:956 at #3667:Unexpected failing examples (2): ../examples/gallery/maps/tilemaps.py failed leaving traceback: Traceback (most recent call last): File "/home/runner/work/pygmt/pygmt/examples/gallery/maps/tilemaps.py", line 37, in <module> fig.tilemap( File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/pygmt/helpers/decorators.py", line 610, in new_module return module_func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/pygmt/helpers/decorators.py", line 773, in new_module return module_func(*bound.args, **bound.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/pygmt/src/tilemap.py", line 145, in tilemap raster = raster.rio.reproject(dst_crs="OGC:CRS84") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/rioxarray/raster_array.py", line 444, in reproject dst_affine, dst_width, dst_height = _make_dst_affine( ^^^^^^^^^^^^^^^^^ File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/rioxarray/raster_array.py", line 146, in _make_dst_affine dst_affine, dst_width, dst_height = rasterio.warp.calculate_default_transform( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/rasterio/env.py", line 413, in wrapper return f(*args, **kwds) ^^^^^^^^^^^^^^^^ File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/rasterio/warp.py", line 556, in calculate_default_transform dst_affine, dst_width, dst_height = _calculate_default_transform( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "rasterio/_warp.pyx", line 744, in rasterio._warp._calculate_default_transform File "rasterio/_base.pyx", line 221, in rasterio._base.open_dataset File "rasterio/_err.pyx", line 360, in rasterio._err.exc_wrap_pointer SystemError: Unknown GDAL Error. To debug: https://rasterio.readthedocs.io/en/latest/topics/errors.html#debugging-internal-gdal-functions ../examples/gallery/images/rgb_image.py failed leaving traceback: Traceback (most recent call last): File "/home/runner/work/pygmt/pygmt/examples/gallery/images/rgb_image.py", line 24, in <module> with rioxarray.open_rasterio( ^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/rioxarray/_io.py", line 1128, in open_rasterio riods = manager.acquire() ^^^^^^^^^^^^^^^^^ File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/xarray/backends/file_manager.py", line 193, in acquire file, _ = self._acquire_with_cache_info(needs_lock) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/xarray/backends/file_manager.py", line 217, in _acquire_with_cache_info file = self._opener(*self._args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/rasterio/env.py", line 463, in wrapper return f(*args, **kwds) ^^^^^^^^^^^^^^^^ File "/home/runner/micromamba/envs/pygmt/lib/python3.12/site-packages/rasterio/__init__.py", line 356, in open dataset = DatasetReader(path, driver=driver, sharing=sharing, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "rasterio/_base.pyx", line 312, in rasterio._base.DatasetBase.__init__ rasterio.errors.RasterioIOError: No driver registered.Thinking if this has something to do with https://github.com/rasterio/rasterio/releases/tag/1.4.3, possibly due to changes in rasterio/rasterio#3260?