Skip to content

Accept dictionaries for store argument #2164

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion src/zarr/api/asynchronous.py
Original file line number Diff line number Diff line change
Expand Up @@ -497,8 +497,18 @@ async def open_group(

Parameters
----------
store : Store or string, optional
store : Store, string, or mapping, optional
Store or path to directory in file system or name of zip file.

Strings are interpreted as paths on the local file system
and used as the ``root`` argument to :class:`zarr.store.LocalStore`.

Dictionaries are used as the ``store_dict`` argument in
:class:`zarr.store.MemoryStore``.

By default (``store=None``) a new :class:`zarr.store.MemoryStore`
is created.

mode : {'r', 'r+', 'a', 'w', 'w-'}, optional
Persistence mode: 'r' means read only (must exist); 'r+' means
read/write (must exist); 'a' means read/write (create if doesn't
Expand Down
4 changes: 2 additions & 2 deletions src/zarr/core/buffer/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -195,8 +195,8 @@ def from_buffer(cls, buffer: Buffer) -> Self:
-------
A new buffer representing the content of the input buffer

Note
----
Notes
-----
Subclasses of `Buffer` must override this method to implement
more optimal conversions that avoid copies where possible
"""
Expand Down
4 changes: 2 additions & 2 deletions src/zarr/core/buffer/cpu.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,8 +73,8 @@ def from_buffer(cls, buffer: core.Buffer) -> Self:
-------
A new buffer representing the content of the input buffer

Note
----
Notes
-----
Subclasses of `Buffer` must override this method to implement
more optimal conversions that avoid copies where possible
"""
Expand Down
12 changes: 6 additions & 6 deletions src/zarr/core/buffer/gpu.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ class Buffer(core.Buffer):
array-like instance can be copied/converted to a regular Numpy array
(host memory).

Note
----
Notes
-----
This buffer is untyped, so all indexing and sizes are in bytes.

Parameters
Expand Down Expand Up @@ -123,8 +123,8 @@ class NDBuffer(core.NDBuffer):
ndarray-like instance can be copied/converted to a regular Numpy array
(host memory).

Note
----
Notes
-----
The two buffer classes Buffer and NDBuffer are very similar. In fact, Buffer
is a special case of NDBuffer where dim=1, stride=1, and dtype="b". However,
in order to use Python's type system to differentiate between the contiguous
Expand Down Expand Up @@ -193,8 +193,8 @@ def from_numpy_array(cls, array_like: npt.ArrayLike) -> Self:
def as_numpy_array(self) -> npt.NDArray[Any]:
"""Returns the buffer as a NumPy array (host memory).

Warning
-------
Warnings
--------
Might have to copy data, consider using `.as_ndarray_like()` instead.

Returns
Expand Down
6 changes: 5 additions & 1 deletion src/zarr/store/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ def __eq__(self, other: Any) -> bool:
return False


StoreLike = Store | StorePath | Path | str
StoreLike = Store | StorePath | Path | str | dict[str, Buffer]


async def make_store_path(
Expand All @@ -94,6 +94,10 @@ async def make_store_path(
return StorePath(await LocalStore.open(root=store_like, mode=mode or "r"))
elif isinstance(store_like, str):
return StorePath(await LocalStore.open(root=Path(store_like), mode=mode or "r"))
elif isinstance(store_like, dict):
# We deliberate only consider dict[str, Buffer] here, and not arbitrary mutable mappings.
# By only allowing dictionaries, which are in-memory, we know that MemoryStore appropriate.
return StorePath(await MemoryStore.open(store_dict=store_like, mode=mode))
raise TypeError


Expand Down
12 changes: 12 additions & 0 deletions tests/v3/test_group.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@
import numpy as np
import pytest

import zarr.api.asynchronous
from zarr import Array, AsyncArray, AsyncGroup, Group
from zarr.api.synchronous import open_group
from zarr.core.buffer import default_buffer_prototype
from zarr.core.common import ZarrFormat
from zarr.core.group import GroupMetadata
Expand Down Expand Up @@ -819,3 +821,13 @@ async def test_require_array(store: LocalStore | MemoryStore, zarr_format: ZarrF
_ = await root.create_group("bar")
with pytest.raises(TypeError, match="Incompatible object"):
await root.require_array("bar", shape=(10,), dtype="int8")


async def test_open_mutable_mapping():
group = await zarr.api.asynchronous.open_group(store={}, mode="w")
assert isinstance(group.store_path.store, MemoryStore)


def test_open_mutable_mapping_sync():
group = open_group(store={}, mode="w")
assert isinstance(group.store_path.store, MemoryStore)