Skip to content

datetime interpolation doesn't work #2667

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
dcherian opened this issue Jan 11, 2019 · 4 comments · Fixed by #2668
Closed

datetime interpolation doesn't work #2667

dcherian opened this issue Jan 11, 2019 · 4 comments · Fixed by #2668
Labels

Comments

@dcherian
Copy link
Contributor

Code Sample, a copy-pastable example if possible

This code doesn't work anymore on master.

a = xr.DataArray(np.arange(21).reshape(3, 7), 
                 dims=['x', 'time'], 
                 coords={'x': [1, 2, 3], 'time': pd.date_range('01-01-2001', periods=7, freq='D')})
xi = xr.DataArray(np.linspace(1, 3, 50),  
                  dims=['time'], 
                  coords={'time': pd.date_range('01-01-2001', periods=50, freq='H')})
a.interp(x=xi, time=xi.time)

Problem description

The above code now raises the error

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-26-dda3a6d5725b> in <module>
      6                   dims=['time'],
      7                   coords={'time': pd.date_range('01-01-2001', periods=50, freq='H')})
----> 8 a.interp(x=xi, time=xi.time)

~/work/python/xarray/xarray/core/dataarray.py in interp(self, coords, method, assume_sorted, kwargs, **coords_kwargs)
   1032         ds = self._to_temp_dataset().interp(
   1033             coords, method=method, kwargs=kwargs, assume_sorted=assume_sorted,
-> 1034             **coords_kwargs)
   1035         return self._from_temp_dataset(ds)
   1036 

~/work/python/xarray/xarray/core/dataset.py in interp(self, coords, method, assume_sorted, kwargs, **coords_kwargs)
   2008                         in indexers.items() if k in var.dims}
   2009                     variables[name] = missing.interp(
-> 2010                         var, var_indexers, method, **kwargs)
   2011                 elif all(d not in indexers for d in var.dims):
   2012                     # keep unrelated object array

~/work/python/xarray/xarray/core/missing.py in interp(var, indexes_coords, method, **kwargs)
    468     new_dims = broadcast_dims + list(destination[0].dims)
    469     interped = interp_func(var.transpose(*original_dims).data,
--> 470                            x, destination, method, kwargs)
    471 
    472     result = Variable(new_dims, interped, attrs=var.attrs)

~/work/python/xarray/xarray/core/missing.py in interp_func(var, x, new_x, method, kwargs)
    535                              new_axis=new_axis, drop_axis=drop_axis)
    536 
--> 537     return _interpnd(var, x, new_x, func, kwargs)
    538 
    539 

~/work/python/xarray/xarray/core/missing.py in _interpnd(var, x, new_x, func, kwargs)
    558     var = var.transpose(range(-len(x), var.ndim - len(x)))
    559     # stack new_x to 1 vector, with reshape
--> 560     xi = np.stack([x1.values.ravel() for x1 in new_x], axis=-1)
    561     rslt = func(x, var, xi, **kwargs)
    562     # move back the interpolation axes to the last position

~/work/python/xarray/xarray/core/missing.py in <listcomp>(.0)
    558     var = var.transpose(range(-len(x), var.ndim - len(x)))
    559     # stack new_x to 1 vector, with reshape
--> 560     xi = np.stack([x1.values.ravel() for x1 in new_x], axis=-1)
    561     rslt = func(x, var, xi, **kwargs)
    562     # move back the interpolation axes to the last position

AttributeError: 'numpy.ndarray' object has no attribute 'values'

I think the issue is this line which returns a numpy array instead of a Variable. This was added in the coarsen PR (cc @fujiisoup)

return np.where(array.isnull(), np.nan, array.astype(dtype))

@dcherian dcherian added the bug label Jan 11, 2019
@fujiisoup
Copy link
Member

Oops. This is definitely my bug. This line should return a Variable. I will send a fix.

@fujiisoup
Copy link
Member

Is there a good way to convert a timedelta array including NaT to a float array with nan?
array.astype(float) makes NaT a certain large value.

In [1]: import numpy as np                                                      

In [2]: import pandas as pd                                                     

In [3]: time = np.array(pd.date_range('15/12/1999', periods=11))                                                                       

In [4]: time[8: 11] = np.nan 

In [5]: time                                                                    
Out[5]: 
array(['1999-12-15T00:00:00.000000000', '1999-12-16T00:00:00.000000000',
       '1999-12-17T00:00:00.000000000', '1999-12-18T00:00:00.000000000',
       '1999-12-19T00:00:00.000000000', '1999-12-20T00:00:00.000000000',
       '1999-12-21T00:00:00.000000000', '1999-12-22T00:00:00.000000000',
                                 'NaT',                           'NaT',
                                 'NaT'], dtype='datetime64[ns]')

In [6]: time.astype(float)                                                      
Out[6]: 
array([ 9.45216000e+17,  9.45302400e+17,  9.45388800e+17,  9.45475200e+17,
        9.45561600e+17,  9.45648000e+17,  9.45734400e+17,  9.45820800e+17,
       -9.22337204e+18, -9.22337204e+18, -9.22337204e+18])

In [7]: (time - np.min(time)).astype(float)                                     
Out[7]: 
array([ 0.00000000e+00,  8.64000000e+13,  1.72800000e+14,  2.59200000e+14,
        3.45600000e+14,  4.32000000e+14,  5.18400000e+14,  6.04800000e+14,
       -9.22337204e+18, -9.22337204e+18, -9.22337204e+18])

@shoyer
Copy link
Member

shoyer commented Jan 11, 2019

I think the best we can do is to is use masking on the result, e.g., with np.where.

@fujiisoup
Copy link
Member

Thanks.

Then, it would be probably nice if datetime_to_numeric only accepts np.ndarray and da.array not Variable or DataArray, and move this function to duck_array_ops.
I'll send a fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants