Unexpected time resampling leads to extremely large dask graphs in default installation #2111
Open
2 tasks done
Labels
bug
Something isn't working
Setup Information
Description
When inputting dask datasets, setting
freq
in standardized indices (SPI and SPEI) could lead to unexpected extremely large graphs, which chokes the following computations. However, when the user usefreq=None
, this could be avoided.Besides, this issue only occurs when
flox
is not installed (which is an optional package). So by default the user does not install this package (but would be installed in the development environment) and may input the samefreq
as the input dataset to accidentally trigger this large graph layers instead of usingfreq=None
or installing flox to avoid this. Also, it would raise a UserWarning that is confusing to the user as the user may have complied with non-chunk time dimensions (UserWarning: The input data is chunked on time dimension and must be fully rechunked to run
fiton groups . Beware, this operation can significantly increase the number of tasks dask has to handle.
).Steps To Reproduce
Please note that this only occurs when user uses the default installation (i.e. without flox installed)
Additional context
Using
freq=None
or having flox installed will not lead to this issue but I guess this is not intuitive to users.Contribution
Code of Conduct
The text was updated successfully, but these errors were encountered: