Error when trying to merge & write files from url

[odhondt]

Code Sample, a copy-pastable example if possible

import rioxarray as riox
from rioxarray.merge import merge_arrays
from pystac_client import Client
import planetary_computer
import shapely

shp = shapely.from_wkt("POLYGON ((3.0578210197234057 42.4455659286713, 3.0578210197234057 42.413109506256774, 3.1010384708737035 42.413109506256774, 3.1010384708737035 42.4455659286713, 3.0578210197234057 42.4455659286713))")

catalog = Client.open(
    "https://planetarycomputer.microsoft.com/api/stac/v1",
    modifier=planetary_computer.sign_inplace,
)

search = catalog.search(collections=["cop-dem-glo-30" ], intersects=shp)
items = search.item_collection()

to_merge = []
for item in items:
    url = item.assets["data"].href
    da = riox.open_rasterio(url)
    to_merge.append(da)

dem = merge_arrays(to_merge)
dem.rio.to_raster("/tmp/out.tif")

Problem description

This code was working fine with 0.17.0 and leads to an error in 0.18.2 (RasterioIOError: No such file or directory)

Expected Output

No error and the files are merged and written to disk.

Environment Information

• python -c "import rioxarray; rioxarray.show_versions()"

rioxarray (0.18.2) deps:
  rasterio: 1.4.3
    xarray: 2025.1.1
      GDAL: 3.10.1
      GEOS: 3.13.0
      PROJ: 9.5.1
 PROJ DATA: /root/micromamba/envs/eo_tools/share/proj
 GDAL DATA: /root/micromamba/envs/eo_tools/share/gdal

Other python deps:
     scipy: 1.15.1
    pyproj: 3.7.0

System:
    python: 3.12.8 | packaged by conda-forge | (main, Dec  5 2024, 14:24:40) [GCC 13.3.0]
executable: /root/micromamba/envs/eo_tools/bin/python
   machine: Linux-5.4.0-196-generic-x86_64-with-glibc2.36

Installation method

conda (micromamba)

Conda environment information (if you installed with conda):

Environment (conda list):

$ conda list | grep -E "rasterio|xarray|gdal"

  gdal                              3.10.1          py312hc55c449_2          conda-forge
  libgdal-core                      3.10.1          h3359108_2               conda-forge
  libgdal-jp2openjpeg               3.10.1          ha1d2769_2               conda-forge
  rasterio                          1.4.3           py312h8cae83d_0          conda-forge
  rioxarray                         0.18.2          pyhd8ed1ab_0             conda-forge
  xarray                            2025.1.1        pyhd8ed1ab_0             conda-forge

Details about conda and system ( conda info ):

$ conda info

       libmamba version : 2.0.5  
     micromamba version : 2.0.5  
           curl version : libcurl/8.11.1 OpenSSL/3.4.0 zlib/1.3.1  zstd/1.5.6 libssh2/1.11.1 nghttp2/1.64.0  
     libarchive version : libarchive 3.7.7 zlib/1.3.1 bz2lib/1.0.8  libzstd/1.5.6. 
       envs directories : /root/micromamba/envs  
          package cache : /root/micromamba/pkgs  
                          /root/.mamba/pkgs. 
            environment : eo_tools (active)  
           env location : /root/micromamba/envs/eo_tools  
      user config files : /root/.mambarc  
 populated config files :  
       virtual packages : __unix=0=0  
                          __linux=5.4.0=0  
                          __glibc=2.36=0  
                          __archspec=1=x86_64_v3  
               channels : https://conda.anaconda.org/conda-forge/linux-64  
                          https://conda.anaconda.org/conda-forge/noarch  
       base environment : /root/micromamba  
               platform : linux-64  

[snowman2]

Thank you for the update. Mind providing the full error traceback in text format?

[odhondt]

Sure:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~/micromamba/envs/eo_tools/lib/python3.12/site-packages/xarray/backends/file_manager.py:211, in CachingFileManager._acquire_with_cache_info(self, needs_lock)
    210 try:
--> 211     file = self._cache[self._key]
    212 except KeyError:

File ~/micromamba/envs/eo_tools/lib/python3.12/site-packages/xarray/backends/lru_cache.py:56, in LRUCache.__getitem__(self, key)
     55 with self._lock:
---> 56     value = self._cache[key]
     57     self._cache.move_to_end(key)

KeyError: [<function open at 0x7f53260091c0>, ('/vsimem/20b2ec9f-efab-4d98-8b6d-b79c19601c17/20b2ec9f-efab-4d98-8b6d-b79c19601c17.tif',), 'r', (('sharing', False),), 'caacac69-5276-4da5-b8a2-e2fe26b58118']

During handling of the above exception, another exception occurred:

CPLE_OpenFailedError                      Traceback (most recent call last)
File rasterio/_base.pyx:310, in rasterio._base.DatasetBase.__init__()

File rasterio/_base.pyx:221, in rasterio._base.open_dataset()

File rasterio/_err.pyx:359, in rasterio._err.exc_wrap_pointer()

CPLE_OpenFailedError: No such file or directory

During handling of the above exception, another exception occurred:

RasterioIOError                           Traceback (most recent call last)
Cell In[1], line 24
     21     to_merge.append(da)
     23 dem = merge_arrays(to_merge)
---> 24 dem.rio.to_raster("/tmp/out.tif")

File ~/micromamba/envs/eo_tools/lib/python3.12/site-packages/rioxarray/raster_array.py:1156, in RasterArray.to_raster(self, raster_path, driver, dtype, tags, windowed, recalc_transform, lock, compute, **profile_kwargs)
   1153 # get the output profile from the rasterio object
   1154 # if opened with xarray.open_rasterio()
   1155 try:
-> 1156     out_profile = self._manager.acquire().profile  # type: ignore
   1157 except AttributeError:
   1158     out_profile = {}

File ~/micromamba/envs/eo_tools/lib/python3.12/site-packages/xarray/backends/file_manager.py:193, in CachingFileManager.acquire(self, needs_lock)
    178 def acquire(self, needs_lock=True):
    179     """Acquire a file object from the manager.
    180 
    181     A new file is only opened if it has expired from the
   (...)
    191         An open file object, as returned by ``opener(*args, **kwargs)``.
    192     """
--> 193     file, _ = self._acquire_with_cache_info(needs_lock)
    194     return file

File ~/micromamba/envs/eo_tools/lib/python3.12/site-packages/xarray/backends/file_manager.py:217, in CachingFileManager._acquire_with_cache_info(self, needs_lock)
    215     kwargs = kwargs.copy()
    216     kwargs["mode"] = self._mode
--> 217 file = self._opener(*self._args, **kwargs)
    218 if self._mode == "w":
    219     # ensure file doesn't get overridden when opened again
    220     self._mode = "a"

File ~/micromamba/envs/eo_tools/lib/python3.12/site-packages/rasterio/env.py:463, in ensure_env_with_credentials.<locals>.wrapper(*args, **kwds)
    460     session = DummySession()
    462 with env_ctor(session=session):
--> 463     return f(*args, **kwds)

File ~/micromamba/envs/eo_tools/lib/python3.12/site-packages/rasterio/__init__.py:356, in open(fp, mode, driver, width, height, count, crs, transform, dtype, nodata, sharing, opener, **kwargs)
    353     path = _parse_path(raw_dataset_path)
    355 if mode == "r":
--> 356     dataset = DatasetReader(path, driver=driver, sharing=sharing, **kwargs)
    357 elif mode == "r+":
    358     dataset = get_writer_for_path(path, driver=driver)(
    359         path, mode, driver=driver, sharing=sharing, **kwargs
    360     )

File rasterio/_base.pyx:312, in rasterio._base.DatasetBase.__init__()

RasterioIOError: No such file or directory