-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update open_mfdataset()
to avoid data vars dim concatenation
#143
Conversation
Codecov Report
@@ Coverage Diff @@
## main #143 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 7 7
Lines 345 346 +1
=========================================
+ Hits 345 346 +1
Continue to review full report at Codecov.
|
Test script # flake8:noqa F401
import os
import xarray as xr
import xcdat
dir = "/p/user_pub/PCMDIobs/PCMDIobs2/atmos/3hr/pr/TRMM-3B43v-7/gn/v20200707"
file_pattern = "pr_3hr_TRMM-3B43v-7_BE_gn_v20200707_1998*.nc"
files = os.path.join(dir, file_pattern)
#%%
# Test using `xr.open_mfdataset()`
ds_xr1 = xr.open_mfdataset(files)
pr_global1 = ds_xr1.xcdat.spatial_avg(
axis=["lat", "lon"], data_var="pr"
).pr # raises error
#%%
ds_xr2 = xr.open_mfdataset(files, data_vars="minimal")
pr_global2 = ds_xr2.xcdat.spatial_avg(axis=["lat", "lon"], data_var="pr").pr # no error
#%%
# Test using `xcdat.open_mfdataset()` and `data_vars="minimal"`
ds_xcdat = xcdat.open_mfdataset(files)
pr_global3 = ds_xcdat.xcdat.spatial_avg(axis=["lat", "lon"]).pr # no error |
0706b47
to
cf434ae
Compare
Hey Steve, here's the related xarray issue with the |
cf434ae
to
11af299
Compare
- Update docstrings for `open_dataset()` and `open_mfdataset()`
74b2a81
to
1d4bd98
Compare
Description
This PR updates
xcdat.open_mfdataset()
with a default option fordata_vars="minimal
to avoid data vars dim concatenation (which breaks a reduction in spatial averaging).The default
xr.open_mfdataset()
behavior is to concat dims across data vars. Addingdata_vars="minimal"
stops this from happening. Notice below that thetime
dimension is no longer added to dims oflat_bnds
andlon_bnds
when passing this kwarg.In #136, the
time
dimension was being added tolat_bnds
andlon_bnds
.As a result, the
lat
andlon
dims were being reduced down, leaving just thetime
dimension. This would produce the errorKeyError: 'Check weights DataArray includes lat dimension.'
https://github.com/XCDAT/xcdat/blob/ca9dee4390cad2ad9856f9959e75981575cc69d7/xcdat/spatial_avg.py#L592-L596
More info on xarray.open_mfdataset().
Checklist
If applicable: