Question: xr.fulllike (often) fails when other is chunked and fillvalue is non-scalar I've been running into some issues when using , when my is a chunked dask array, and the is a numpy array. Now, I just checked, mentions only scalar in the signature. However, this is a very convenient way to get all the coordinates and dimensions attached to an array like this, so it feels like desirable functionality. And as I mention below, both numpy and dask function similary, taking much more than just scalars. This results in an error: Expected is a DataArray with the dimensions and coords of , and the numpy array of as its data. The issue lies here: Calling with the given number of chunks results in it trying to to apply the for every individual chunk. As one would expect, if I set to the size of a single chunk it doesn't error: It does fail on a similarly chunked dask array (since it's applying it for every chunk): The most obvious solution would be to force it down the route, since all the values already exist in memory anyway. So maybe another type check does the trick. However, accepts quite a variety of arguments for the fill value (scalars, numpy arrays, lists, tuples, ranges). The dask docs mention only a scalar in the signature for: As does However, in all cases, they still broadcast automatically... So kind of undefined behavior of a blocked ? <details<summaryOutput of </summaryINSTALLED VERSIONS commit: None python: 3.7.6 (default, Jan 7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 158 Stepping 9, GenuineIntel byteorder: little LCALL: None LANG: en LOCALE: None.None libhdf5: 1.10.5 libnetcdf: 4.7.3 xarray: 0.15.1 pandas: 0.25.3 numpy: 1.17.5 scipy: 1.3.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: 2.4.0 cftime: 1.0.4.2 nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.2 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.9.2 distributed: 2.10.0 matplotlib: 3.1.2 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 46.1.3.post20200325 pip: 20.0.2 conda: None pytest: 5.3.4 IPython: 7.13.0 sphinx: 2.3.1 </detailsxr.fulllike (often) fails when other is chunked and fillvalue is non-scalar I've been running into some issues when using , when my is a chunked dask array, and the is a numpy array. Now, I just checked, mentions only scalar in the signature. However, this is a very convenient way to get all the coordinates and dimensions attached to an array like this, so it feels like desirable functionality. And as I mention below, both numpy and dask function similary, taking much more than just scalars. This results in an error: Expected is a DataArray with the dimensions and coords of , and the numpy array of as its data. The issue lies here: Calling with the given number of chunks results in it trying to to apply the for every individual chunk. As one would expect, if I set to the size of a single chunk it doesn't error: It does fail on a similarly chunked dask array (since it's applying it for every chunk): The most obvious solution would be to force it down the route, since all the values already exist in memory anyway. So maybe another type check does the trick. However, accepts quite a variety of arguments for the fill value (scalars, numpy arrays, lists, tuples, ranges). The dask docs mention only a scalar in the signature for: As does However, in all cases, they still broadcast automatically... So kind of undefined behavior of a blocked ? <details<summaryOutput of </summaryINSTALLED VERSIONS commit: None python: 3.7.6 (default, Jan 7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 158 Stepping 9, GenuineIntel byteorder: little LCALL: None LANG: en LOCALE: None.None libhdf5: 1.10.5 libnetcdf: 4.7.3 xarray: 0.15.1 pandas: 0.25.3 numpy: 1.17.5 scipy: 1.3.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: 2.4.0 cftime: 1.0.4.2 nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.2 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.9.2 distributed: 2.10.0 matplotlib: 3.1.2 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 46.1.3.post20200325 pip: 20.0.2 conda: None pytest: 5.3.4 IPython: 7.13.0 sphinx: 2.3.1 </details

from .dataset import Dataset     from .variable import Variable <ins>    if not is_scalar(fill_value):        raise ValueError(f"fill_value must be scalar. Received {fill_value} instead.")</ins>     if isinstance(other, Dataset):         data_vars = {             k: _full_like_variable(v, fill_value, dtype)


Question: DataArray.integrate has a 'dim' arg, but Dataset.integrate has a 'coord' arg This is just a minor gripe but I think it should be fixed. The API syntax is inconsistent: It should definitely be - IMO it doesn't make sense to integrate or differentiate over a dim because a dim by definition has no information about the distance between grid points. I think because the distinction between dims and coords is one of the things that new users have to learn about, we should be strict to not confuse up the meanings in the documentation/API. The discussion on the original PR , so I think this was just an small oversight. The only question is whether it requires a deprecation cycle?

return self._from_temp_dataset(ds)      def integrate(        self, dim: Union[Hashable, Sequence[Hashable]], datetime_unit: str = None        self,        coord: Union[Hashable, Sequence[Hashable]] = None,        datetime_unit: str = None,        *,        dim: Union[Hashable, Sequence[Hashable]] = None,     ) -> "DataArray":        """ integrate the array with the trapezoidal rule.        """Integrate along the given coordinate using the trapezoidal rule.          .. note::            This feature is limited to simple cartesian geometry, i.e. dim            This feature is limited to simple cartesian geometry, i.e. coord             must be one dimensional.          Parameters         ----------        coord: hashable, or a sequence of hashable            Coordinate(s) used for the integration.         dim : hashable, or sequence of hashable             Coordinate(s) used for the integration.        datetime_unit : {"Y", "M", "W", "D", "h", "m", "s", "ms", "us", "ns", \                         "ps", "fs", "as"}, optional            Can be used to specify the unit if datetime coordinate is used.        datetime_unit: {'Y', 'M', 'W', 'D', 'h', 'm', 's', 'ms', 'us', 'ns', \                        'ps', 'fs', 'as'}, optional          Returns         -------


Question: DataArray.integrate has a 'dim' arg, but Dataset.integrate has a 'coord' arg This is just a minor gripe but I think it should be fixed. The API syntax is inconsistent: It should definitely be - IMO it doesn't make sense to integrate or differentiate over a dim because a dim by definition has no information about the distance between grid points. I think because the distinction between dims and coords is one of the things that new users have to learn about, we should be strict to not confuse up the meanings in the documentation/API. The discussion on the original PR , so I think this was just an small oversight. The only question is whether it requires a deprecation cycle?

See also         --------<ins>        Dataset.integrate</ins>         numpy.trapz: corresponding numpy function          Examples


Question: DataArray.integrate has a 'dim' arg, but Dataset.integrate has a 'coord' arg This is just a minor gripe but I think it should be fixed. The API syntax is inconsistent: It should definitely be - IMO it doesn't make sense to integrate or differentiate over a dim because a dim by definition has no information about the distance between grid points. I think because the distinction between dims and coords is one of the things that new users have to learn about, we should be strict to not confuse up the meanings in the documentation/API. The discussion on the original PR , so I think this was just an small oversight. The only question is whether it requires a deprecation cycle?

array([5.4, 6.6, 7.8])         Dimensions without coordinates: y         """        ds = self._to_temp_dataset().integrate(dim, datetime_unit)        if dim is not None and coord is not None:            raise ValueError(                "Cannot pass both 'dim' and 'coord'. Please pass only 'coord' instead."            )        if dim is not None and coord is None:            coord = dim            msg = (                "The `dim` keyword argument to `DataArray.integrate` is "                "being replaced with `coord`, for consistency with "                "`Dataset.integrate`. Please pass `coord` instead."                " `dim` will be removed in version 0.19.0."            )            warnings.warn(msg, FutureWarning, stacklevel=2)        ds = self._to_temp_dataset().integrate(coord, datetime_unit)         return self._from_temp_dataset(ds)      def unify_chunks(self) -> "DataArray":diff --git a/xarray/core/dataset.py b/xarray/core/dataset.py-- a/xarray/core/dataset.py++ b/xarray/core/dataset.py


Question: DataArray.integrate has a 'dim' arg, but Dataset.integrate has a 'coord' arg This is just a minor gripe but I think it should be fixed. The API syntax is inconsistent: It should definitely be - IMO it doesn't make sense to integrate or differentiate over a dim because a dim by definition has no information about the distance between grid points. I think because the distinction between dims and coords is one of the things that new users have to learn about, we should be strict to not confuse up the meanings in the documentation/API. The discussion on the original PR , so I think this was just an small oversight. The only question is whether it requires a deprecation cycle?

variables[k] = v         return self._replace(variables)     def integrate(self, coord, datetime_unit=None):        """ integrate the array with the trapezoidal rule.    def integrate(        self, coord: Union[Hashable, Sequence[Hashable]], datetime_unit: str = None    ) -> "Dataset":        """Integrate along the given coordinate using the trapezoidal rule.          .. note::             This feature is limited to simple cartesian geometry, i.e. coord


Question: DataArray.integrate has a 'dim' arg, but Dataset.integrate has a 'coord' arg This is just a minor gripe but I think it should be fixed. The API syntax is inconsistent: It should definitely be - IMO it doesn't make sense to integrate or differentiate over a dim because a dim by definition has no information about the distance between grid points. I think because the distinction between dims and coords is one of the things that new users have to learn about, we should be strict to not confuse up the meanings in the documentation/API. The discussion on the original PR , so I think this was just an small oversight. The only question is whether it requires a deprecation cycle?

Parameters         ----------        coord: str, or sequence of str        coord: hashable, or a sequence of hashable             Coordinate(s) used for the integration.        datetime_unit : {"Y", "M", "W", "D", "h", "m", "s", "ms", "us", "ns", \                         "ps", "fs", "as"}, optional            Can be specify the unit if datetime coordinate is used.        datetime_unit: {'Y', 'M', 'W', 'D', 'h', 'm', 's', 'ms', 'us', 'ns', \                        'ps', 'fs', 'as'}, optional            Specify the unit if datetime coordinate is used.          Returns         -------


Question: [bug] when passing boolean weights to weighted mean <!-- A short summary of the issue, if appropriate --<!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --Returns Passing a boolean array as weights to the weighted mean returns the wrong result because the are not properly normalized (in this case). Internally the is calculated as i.e. the dot product of two boolean arrays. This yields: We'll need to convert it to int or float: which is correct <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.7.6 (default, Mar 23 2020, 23:03:20) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.3.0-51-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.15.1 pandas: 1.0.3 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.1.1.2 nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.3 cfgrib: None iris: None bottleneck: None dask: 2.16.0 distributed: 2.16.0 matplotlib: 3.2.1 cartopy: 0.17.0 seaborn: None numbagg: None setuptools: 46.1.3.post20200325 pip: 20.1 conda: None pytest: 5.4.1 IPython: 7.13.0 sphinx: 3.0.3 </details

# we need to mask data values that are nan; else the weights are wrong         mask = da.notnull()         sum_of_weights = self._reduce(mask, self.weights, dim=dim, skipna=False)        # bool -> int, because ``xr.dot([True, True], [True, True])`` -> True        # (and not 2); GH4074        if self.weights.dtype == bool:            sum_of_weights = self._reduce(                mask, self.weights.astype(int), dim=dim, skipna=False            )        else:            sum_of_weights = self._reduce(mask, self.weights, dim=dim, skipna=False)          # 0-weights are not valid         valid_weights = sum_of_weights != 0.0


Question: tounstackeddataset broken for single-dim variables <!-- A short summary of the issue, if appropriate --A working roundtrip. I need to stack a bunch of variables and later unstack them again, however this doesn't work if the variables only have a single dimension. <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.7.3 (default, Mar 27 2019, 22:11:17) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.15.0-96-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: enGB.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.2 xarray: 0.15.1 pandas: 1.0.3 numpy: 1.17.3 scipy: 1.3.1 netCDF4: 1.4.2 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: None cftime: 1.0.4.2 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.10.1 distributed: 2.10.0 matplotlib: 3.1.1 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 41.0.0 pip: 19.0.3 conda: 4.8.3 pytest: 5.3.5 IPython: 7.9.0 sphinx: None </details

# pull variables out of datarray         data_dict = {}         for k in variables:            data_dict[k] = self.sel({variable_dim: k}).squeeze(drop=True)            data_dict[k] = self.sel({variable_dim: k}, drop=True).squeeze(drop=True)          # unstacked dataset         return Dataset(data_dict)


Question: Pre-expand data and attributes in DataArray/Variable HTML repr? Given that a major purpose for plotting an array is to look at data or attributes, I wonder if we should expand these sections by default? I worry that clicking on icons to expand sections may not be easy to discover This would also be consistent with the text repr, which shows these sections by default (the Dataset repr is already consistent by default between text and HTML already) Currently the HTML repr for DataArray/Variable looks like this: ! To see array data, you have to click on the ! icon: ! (thanks to for making this a little bit more manageably sized in !) There's also a really nice repr for nested dask arrays: !

def array_section(obj):     # "unique" id to expand/collapse the section     data_id = "section-" + str(uuid.uuid4())    collapsed = ""    collapsed = "checked"     variable = getattr(obj, "variable", obj)     preview = escape(inline_variable_array_repr(variable, max_width=70))     data_repr = short_data_repr_html(obj)


Question: Stack + toarray before toxarray is much faster that a simple toxarray I was seeing some slow performance around on MultiIndexed series, and found that unstacking one of the dimensions before running , and then restacking with was ~30x faster. This time difference is consistent with larger data sizes. To reproduce: Create a series with a MultiIndex, ensuring the MultiIndex isn't a simple product: Two approaches for getting this into xarray; 1 - Simple : This takes 536 ms 2 - unstack in pandas first, and then use to do the equivalent of a restack: This takes 17.3 ms To confirm these are identical: A default operation is much slower than a (potentially) equivalent operation that's not the default. I need to look more at what's causing the issues. I think it's to do with the , but I'm unclear why it's so much faster in the alternative route, and whether there's a fix that we can make to make the default path fast. <detailsINSTALLED VERSIONS commit: None python: python-bits: 64 OS: Linux OS-release: 4.9.93-linuxkit-aufs machine: x8664 processor: x8664 byteorder: little LCALL: enUS.UTF-8 LANG: enUS.utf8 LOCALE: None.None xarray: 0.10.9 pandas: 0.23.4 numpy: 1.15.2 scipy: 1.1.0 netCDF4: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: None distributed: None matplotlib: 2.2.3 cartopy: 0.16.0 seaborn: 0.9.0 setuptools: 40.4.3 pip: 18.0 conda: None pytest: 3.8.1 IPython: 5.8.0 sphinx: None </detailstoxarray() result is incorrect when one of multi-index levels is not sorted <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: toxarray() sorts multi-index level values and returns result for the sorted values but it doesn't sort levels or expects levels to be sorted resulting in completely incorrect order of data for the displayed coordinates. What you expected to happen: Should account for the order of levels in the original index. Minimal Complete Verifiable Example: Anything else we need to know?: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.7-100.fc30.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: enGB.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.3 xarray: 0.15.1 pandas: 1.0.5 numpy: 1.19.0 scipy: 1.5.0 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.1.3 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.19.0 distributed: 2.19.0 matplotlib: 3.2.2 cartopy: None seaborn: 0.10.1 numbagg: installed setuptools: 46.3.0.post20200513 pip: 20.1 conda: None pytest: 5.4.3 IPython: 7.15.0 sphinx: None </details

import numpy as npimport pandas as pdfrom . import parameterizedclass MultiIndexSeries:    def setup(self, dtype, subset):        data = np.random.rand(100000).astype(dtype)        index = pd.MultiIndex.from_product(            [                list("abcdefhijk"),                list("abcdefhijk"),                pd.date_range(start="2000-01-01", periods=1000, freq="B"),            ]        )        series = pd.Series(data, index)        if subset:            series = series[::3]        self.series = series    @parameterized(["dtype", "subset"], ([int, float], [True, False]))    def time_to_xarray(self, dtype, subset):        self.series.to_xarray()diff --git a/xarray/core/dataset.py b/xarray/core/dataset.py-- a/xarray/core/dataset.py++ b/xarray/core/dataset.py


Question: Stack + toarray before toxarray is much faster that a simple toxarray I was seeing some slow performance around on MultiIndexed series, and found that unstacking one of the dimensions before running , and then restacking with was ~30x faster. This time difference is consistent with larger data sizes. To reproduce: Create a series with a MultiIndex, ensuring the MultiIndex isn't a simple product: Two approaches for getting this into xarray; 1 - Simple : This takes 536 ms 2 - unstack in pandas first, and then use to do the equivalent of a restack: This takes 17.3 ms To confirm these are identical: A default operation is much slower than a (potentially) equivalent operation that's not the default. I need to look more at what's causing the issues. I think it's to do with the , but I'm unclear why it's so much faster in the alternative route, and whether there's a fix that we can make to make the default path fast. <detailsINSTALLED VERSIONS commit: None python: python-bits: 64 OS: Linux OS-release: 4.9.93-linuxkit-aufs machine: x8664 processor: x8664 byteorder: little LCALL: enUS.UTF-8 LANG: enUS.utf8 LOCALE: None.None xarray: 0.10.9 pandas: 0.23.4 numpy: 1.15.2 scipy: 1.1.0 netCDF4: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: None distributed: None matplotlib: 2.2.3 cartopy: 0.16.0 seaborn: 0.9.0 setuptools: 40.4.3 pip: 18.0 conda: None pytest: 3.8.1 IPython: 5.8.0 sphinx: None </detailstoxarray() result is incorrect when one of multi-index levels is not sorted <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: toxarray() sorts multi-index level values and returns result for the sorted values but it doesn't sort levels or expects levels to be sorted resulting in completely incorrect order of data for the displayed coordinates. What you expected to happen: Should account for the order of levels in the original index. Minimal Complete Verifiable Example: Anything else we need to know?: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.7-100.fc30.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: enGB.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.3 xarray: 0.15.1 pandas: 1.0.5 numpy: 1.19.0 scipy: 1.5.0 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.1.3 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.19.0 distributed: 2.19.0 matplotlib: 3.2.2 cartopy: None seaborn: 0.10.1 numbagg: installed setuptools: 46.3.0.post20200513 pip: 20.1 conda: None pytest: 5.4.3 IPython: 7.15.0 sphinx: None </details

return self._to_dataframe(self.dims)      def _set_sparse_data_from_dataframe(        self, dataframe: pd.DataFrame, dims: tuple        self, idx: pd.Index, arrays: List[Tuple[Hashable, np.ndarray]], dims: tuple     ) -> None:         from sparse import COO         idx = dataframe.index         if isinstance(idx, pd.MultiIndex):             coords = np.stack([np.asarray(code) for code in idx.codes], axis=0)             is_sorted = idx.is_lexsorted()


Question: Stack + toarray before toxarray is much faster that a simple toxarray I was seeing some slow performance around on MultiIndexed series, and found that unstacking one of the dimensions before running , and then restacking with was ~30x faster. This time difference is consistent with larger data sizes. To reproduce: Create a series with a MultiIndex, ensuring the MultiIndex isn't a simple product: Two approaches for getting this into xarray; 1 - Simple : This takes 536 ms 2 - unstack in pandas first, and then use to do the equivalent of a restack: This takes 17.3 ms To confirm these are identical: A default operation is much slower than a (potentially) equivalent operation that's not the default. I need to look more at what's causing the issues. I think it's to do with the , but I'm unclear why it's so much faster in the alternative route, and whether there's a fix that we can make to make the default path fast. <detailsINSTALLED VERSIONS commit: None python: python-bits: 64 OS: Linux OS-release: 4.9.93-linuxkit-aufs machine: x8664 processor: x8664 byteorder: little LCALL: enUS.UTF-8 LANG: enUS.utf8 LOCALE: None.None xarray: 0.10.9 pandas: 0.23.4 numpy: 1.15.2 scipy: 1.1.0 netCDF4: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: None distributed: None matplotlib: 2.2.3 cartopy: 0.16.0 seaborn: 0.9.0 setuptools: 40.4.3 pip: 18.0 conda: None pytest: 3.8.1 IPython: 5.8.0 sphinx: None </detailstoxarray() result is incorrect when one of multi-index levels is not sorted <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: toxarray() sorts multi-index level values and returns result for the sorted values but it doesn't sort levels or expects levels to be sorted resulting in completely incorrect order of data for the displayed coordinates. What you expected to happen: Should account for the order of levels in the original index. Minimal Complete Verifiable Example: Anything else we need to know?: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.7-100.fc30.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: enGB.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.3 xarray: 0.15.1 pandas: 1.0.5 numpy: 1.19.0 scipy: 1.5.0 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.1.3 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.19.0 distributed: 2.19.0 matplotlib: 3.2.2 cartopy: None seaborn: 0.10.1 numbagg: installed setuptools: 46.3.0.post20200513 pip: 20.1 conda: None pytest: 5.4.3 IPython: 7.15.0 sphinx: None </details

is_sorted = True             shape = (idx.size,)         for name, series in dataframe.items():            # Cast to a NumPy array first, in case the Series is a pandas            # Extension array (which doesn't have a valid NumPy dtype)            values = np.asarray(series)        for name, values in arrays:             # In virtually all real use cases, the sparse array will now have             # missing values and needs a fill_value. For consistency, don't             # special case the rare exceptions (e.g., dtype=int without a


Question: Stack + toarray before toxarray is much faster that a simple toxarray I was seeing some slow performance around on MultiIndexed series, and found that unstacking one of the dimensions before running , and then restacking with was ~30x faster. This time difference is consistent with larger data sizes. To reproduce: Create a series with a MultiIndex, ensuring the MultiIndex isn't a simple product: Two approaches for getting this into xarray; 1 - Simple : This takes 536 ms 2 - unstack in pandas first, and then use to do the equivalent of a restack: This takes 17.3 ms To confirm these are identical: A default operation is much slower than a (potentially) equivalent operation that's not the default. I need to look more at what's causing the issues. I think it's to do with the , but I'm unclear why it's so much faster in the alternative route, and whether there's a fix that we can make to make the default path fast. <detailsINSTALLED VERSIONS commit: None python: python-bits: 64 OS: Linux OS-release: 4.9.93-linuxkit-aufs machine: x8664 processor: x8664 byteorder: little LCALL: enUS.UTF-8 LANG: enUS.utf8 LOCALE: None.None xarray: 0.10.9 pandas: 0.23.4 numpy: 1.15.2 scipy: 1.1.0 netCDF4: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: None distributed: None matplotlib: 2.2.3 cartopy: 0.16.0 seaborn: 0.9.0 setuptools: 40.4.3 pip: 18.0 conda: None pytest: 3.8.1 IPython: 5.8.0 sphinx: None </detailstoxarray() result is incorrect when one of multi-index levels is not sorted <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: toxarray() sorts multi-index level values and returns result for the sorted values but it doesn't sort levels or expects levels to be sorted resulting in completely incorrect order of data for the displayed coordinates. What you expected to happen: Should account for the order of levels in the original index. Minimal Complete Verifiable Example: Anything else we need to know?: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.7-100.fc30.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: enGB.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.3 xarray: 0.15.1 pandas: 1.0.5 numpy: 1.19.0 scipy: 1.5.0 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.1.3 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.19.0 distributed: 2.19.0 matplotlib: 3.2.2 cartopy: None seaborn: 0.10.1 numbagg: installed setuptools: 46.3.0.post20200513 pip: 20.1 conda: None pytest: 5.4.3 IPython: 7.15.0 sphinx: None </details

self[name] = (dims, data)      def _set_numpy_data_from_dataframe(        self, dataframe: pd.DataFrame, dims: tuple        self, idx: pd.Index, arrays: List[Tuple[Hashable, np.ndarray]], dims: tuple     ) -> None:        idx = dataframe.index        if isinstance(idx, pd.MultiIndex):            # expand the DataFrame to include the product of all levels            full_idx = pd.MultiIndex.from_product(idx.levels, names=idx.names)            dataframe = dataframe.reindex(full_idx)            shape = tuple(lev.size for lev in idx.levels)        else:            shape = (idx.size,)        for name, series in dataframe.items():            data = np.asarray(series).reshape(shape)        if not isinstance(idx, pd.MultiIndex):            for name, values in arrays:                self[name] = (dims, values)            return        shape = tuple(lev.size for lev in idx.levels)        indexer = tuple(idx.codes)        # We already verified that the MultiIndex has all unique values, so        # there are missing values if and only if the size of output arrays is        # larger that the index.        missing_values = np.prod(shape) > idx.shape[0]        for name, values in arrays:            # NumPy indexing is much faster than using DataFrame.reindex() to            # fill in missing values:            # https://stackoverflow.com/a/35049899/809705            if missing_values:                dtype, fill_value = dtypes.maybe_promote(values.dtype)                data = np.full(shape, fill_value, dtype)            else:                # If there are no missing values, keep the existing dtype                # instead of promoting to support NA, e.g., keep integer                # columns as integers.                # TODO: consider removing this special case, which doesn't                # exist for sparse=True.                data = np.zeros(shape, values.dtype)            data[indexer] = values             self[name] = (dims, data)      @classmethod


Question: Stack + toarray before toxarray is much faster that a simple toxarray I was seeing some slow performance around on MultiIndexed series, and found that unstacking one of the dimensions before running , and then restacking with was ~30x faster. This time difference is consistent with larger data sizes. To reproduce: Create a series with a MultiIndex, ensuring the MultiIndex isn't a simple product: Two approaches for getting this into xarray; 1 - Simple : This takes 536 ms 2 - unstack in pandas first, and then use to do the equivalent of a restack: This takes 17.3 ms To confirm these are identical: A default operation is much slower than a (potentially) equivalent operation that's not the default. I need to look more at what's causing the issues. I think it's to do with the , but I'm unclear why it's so much faster in the alternative route, and whether there's a fix that we can make to make the default path fast. <detailsINSTALLED VERSIONS commit: None python: python-bits: 64 OS: Linux OS-release: 4.9.93-linuxkit-aufs machine: x8664 processor: x8664 byteorder: little LCALL: enUS.UTF-8 LANG: enUS.utf8 LOCALE: None.None xarray: 0.10.9 pandas: 0.23.4 numpy: 1.15.2 scipy: 1.1.0 netCDF4: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: None distributed: None matplotlib: 2.2.3 cartopy: 0.16.0 seaborn: 0.9.0 setuptools: 40.4.3 pip: 18.0 conda: None pytest: 3.8.1 IPython: 5.8.0 sphinx: None </detailstoxarray() result is incorrect when one of multi-index levels is not sorted <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: toxarray() sorts multi-index level values and returns result for the sorted values but it doesn't sort levels or expects levels to be sorted resulting in completely incorrect order of data for the displayed coordinates. What you expected to happen: Should account for the order of levels in the original index. Minimal Complete Verifiable Example: Anything else we need to know?: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.7-100.fc30.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: enGB.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.3 xarray: 0.15.1 pandas: 1.0.5 numpy: 1.19.0 scipy: 1.5.0 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.1.3 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.19.0 distributed: 2.19.0 matplotlib: 3.2.2 cartopy: None seaborn: 0.10.1 numbagg: installed setuptools: 46.3.0.post20200513 pip: 20.1 conda: None pytest: 5.4.3 IPython: 7.15.0 sphinx: None </details

if not dataframe.columns.is_unique:             raise ValueError("cannot convert DataFrame with non-unique columns")         idx, dataframe = remove_unused_levels_categories(dataframe.index, dataframe)        idx = remove_unused_levels_categories(dataframe.index)        if isinstance(idx, pd.MultiIndex) and not idx.is_unique:            raise ValueError(                "cannot convert a DataFrame with a non-unique MultiIndex into xarray"            )        # Cast to a NumPy array first, in case the Series is a pandas Extension        # array (which doesn't have a valid NumPy dtype)        # TODO: allow users to control how this casting happens, e.g., by        # forwarding arguments to pandas.Series.to_numpy?        arrays = [(k, np.asarray(v)) for k, v in dataframe.items()]         obj = cls()          if isinstance(idx, pd.MultiIndex):


Question: Stack + toarray before toxarray is much faster that a simple toxarray I was seeing some slow performance around on MultiIndexed series, and found that unstacking one of the dimensions before running , and then restacking with was ~30x faster. This time difference is consistent with larger data sizes. To reproduce: Create a series with a MultiIndex, ensuring the MultiIndex isn't a simple product: Two approaches for getting this into xarray; 1 - Simple : This takes 536 ms 2 - unstack in pandas first, and then use to do the equivalent of a restack: This takes 17.3 ms To confirm these are identical: A default operation is much slower than a (potentially) equivalent operation that's not the default. I need to look more at what's causing the issues. I think it's to do with the , but I'm unclear why it's so much faster in the alternative route, and whether there's a fix that we can make to make the default path fast. <detailsINSTALLED VERSIONS commit: None python: python-bits: 64 OS: Linux OS-release: 4.9.93-linuxkit-aufs machine: x8664 processor: x8664 byteorder: little LCALL: enUS.UTF-8 LANG: enUS.utf8 LOCALE: None.None xarray: 0.10.9 pandas: 0.23.4 numpy: 1.15.2 scipy: 1.1.0 netCDF4: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: None distributed: None matplotlib: 2.2.3 cartopy: 0.16.0 seaborn: 0.9.0 setuptools: 40.4.3 pip: 18.0 conda: None pytest: 3.8.1 IPython: 5.8.0 sphinx: None </detailstoxarray() result is incorrect when one of multi-index levels is not sorted <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: toxarray() sorts multi-index level values and returns result for the sorted values but it doesn't sort levels or expects levels to be sorted resulting in completely incorrect order of data for the displayed coordinates. What you expected to happen: Should account for the order of levels in the original index. Minimal Complete Verifiable Example: Anything else we need to know?: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.7-100.fc30.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: enGB.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.3 xarray: 0.15.1 pandas: 1.0.5 numpy: 1.19.0 scipy: 1.5.0 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.1.3 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.19.0 distributed: 2.19.0 matplotlib: 3.2.2 cartopy: None seaborn: 0.10.1 numbagg: installed setuptools: 46.3.0.post20200513 pip: 20.1 conda: None pytest: 5.4.3 IPython: 7.15.0 sphinx: None </details

obj[index_name] = (dims, idx)          if sparse:            obj._set_sparse_data_from_dataframe(dataframe, dims)            obj._set_sparse_data_from_dataframe(idx, arrays, dims)         else:            obj._set_numpy_data_from_dataframe(dataframe, dims)            obj._set_numpy_data_from_dataframe(idx, arrays, dims)         return obj      def to_dask_dataframe(self, dim_order=None, set_index=False):diff --git a/xarray/core/indexes.py b/xarray/core/indexes.py-- a/xarray/core/indexes.py++ b/xarray/core/indexes.py


Question: Stack + toarray before toxarray is much faster that a simple toxarray I was seeing some slow performance around on MultiIndexed series, and found that unstacking one of the dimensions before running , and then restacking with was ~30x faster. This time difference is consistent with larger data sizes. To reproduce: Create a series with a MultiIndex, ensuring the MultiIndex isn't a simple product: Two approaches for getting this into xarray; 1 - Simple : This takes 536 ms 2 - unstack in pandas first, and then use to do the equivalent of a restack: This takes 17.3 ms To confirm these are identical: A default operation is much slower than a (potentially) equivalent operation that's not the default. I need to look more at what's causing the issues. I think it's to do with the , but I'm unclear why it's so much faster in the alternative route, and whether there's a fix that we can make to make the default path fast. <detailsINSTALLED VERSIONS commit: None python: python-bits: 64 OS: Linux OS-release: 4.9.93-linuxkit-aufs machine: x8664 processor: x8664 byteorder: little LCALL: enUS.UTF-8 LANG: enUS.utf8 LOCALE: None.None xarray: 0.10.9 pandas: 0.23.4 numpy: 1.15.2 scipy: 1.1.0 netCDF4: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: None distributed: None matplotlib: 2.2.3 cartopy: 0.16.0 seaborn: 0.9.0 setuptools: 40.4.3 pip: 18.0 conda: None pytest: 3.8.1 IPython: 5.8.0 sphinx: None </detailstoxarray() result is incorrect when one of multi-index levels is not sorted <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: toxarray() sorts multi-index level values and returns result for the sorted values but it doesn't sort levels or expects levels to be sorted resulting in completely incorrect order of data for the displayed coordinates. What you expected to happen: Should account for the order of levels in the original index. Minimal Complete Verifiable Example: Anything else we need to know?: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.7-100.fc30.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: enGB.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.3 xarray: 0.15.1 pandas: 1.0.5 numpy: 1.19.0 scipy: 1.5.0 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.1.3 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.19.0 distributed: 2.19.0 matplotlib: 3.2.2 cartopy: None seaborn: 0.10.1 numbagg: installed setuptools: 46.3.0.post20200513 pip: 20.1 conda: None pytest: 5.4.3 IPython: 7.15.0 sphinx: None </details

from .variable import Variable  def remove_unused_levels_categories(index, dataframe=None):def remove_unused_levels_categories(index: pd.Index) -> pd.Index:     """     Remove unused levels from MultiIndex and unused categories from CategoricalIndex     """


Question: Stack + toarray before toxarray is much faster that a simple toxarray I was seeing some slow performance around on MultiIndexed series, and found that unstacking one of the dimensions before running , and then restacking with was ~30x faster. This time difference is consistent with larger data sizes. To reproduce: Create a series with a MultiIndex, ensuring the MultiIndex isn't a simple product: Two approaches for getting this into xarray; 1 - Simple : This takes 536 ms 2 - unstack in pandas first, and then use to do the equivalent of a restack: This takes 17.3 ms To confirm these are identical: A default operation is much slower than a (potentially) equivalent operation that's not the default. I need to look more at what's causing the issues. I think it's to do with the , but I'm unclear why it's so much faster in the alternative route, and whether there's a fix that we can make to make the default path fast. <detailsINSTALLED VERSIONS commit: None python: python-bits: 64 OS: Linux OS-release: 4.9.93-linuxkit-aufs machine: x8664 processor: x8664 byteorder: little LCALL: enUS.UTF-8 LANG: enUS.utf8 LOCALE: None.None xarray: 0.10.9 pandas: 0.23.4 numpy: 1.15.2 scipy: 1.1.0 netCDF4: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: None distributed: None matplotlib: 2.2.3 cartopy: 0.16.0 seaborn: 0.9.0 setuptools: 40.4.3 pip: 18.0 conda: None pytest: 3.8.1 IPython: 5.8.0 sphinx: None </detailstoxarray() result is incorrect when one of multi-index levels is not sorted <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: toxarray() sorts multi-index level values and returns result for the sorted values but it doesn't sort levels or expects levels to be sorted resulting in completely incorrect order of data for the displayed coordinates. What you expected to happen: Should account for the order of levels in the original index. Minimal Complete Verifiable Example: Anything else we need to know?: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.7-100.fc30.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: enGB.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.3 xarray: 0.15.1 pandas: 1.0.5 numpy: 1.19.0 scipy: 1.5.0 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.1.3 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.19.0 distributed: 2.19.0 matplotlib: 3.2.2 cartopy: None seaborn: 0.10.1 numbagg: installed setuptools: 46.3.0.post20200513 pip: 20.1 conda: None pytest: 5.4.3 IPython: 7.15.0 sphinx: None </details

else:                     level = level[index.codes[i]]                 levels.append(level)            # TODO: calling from_array() reorders MultiIndex levels. It would            # be best to avoid this, if possible, e.g., by using            # MultiIndex.remove_unused_levels() (which does not reorder) on the            # part of the MultiIndex that is not categorical, or by fixing this            # upstream in pandas.             index = pd.MultiIndex.from_arrays(levels, names=index.names)     elif isinstance(index, pd.CategoricalIndex):         index = index.remove_unused_categories()    if dataframe is None:        return index    dataframe = dataframe.set_index(index)    return dataframe.index, dataframe    return index   class Indexes(collections.abc.Mapping):


Question: Feature request: show units in dataset overview Here's a hypothetical dataset: It would be really nice if the units of the coordinates and of the data variables were shown in the repr, for example as:

return inline_dask_repr(var.data)     elif isinstance(var._data, sparse_array_type):         return inline_sparse_repr(var.data)<ins>    elif hasattr(var._data, "_repr_inline_"):        return var._data._repr_inline_(max_width)</ins>     elif hasattr(var._data, "__array_function__"):         return maybe_truncate(repr(var._data).replace("\n", " "), max_width)     else:


Question: sum: mincount is not available for reduction with more than one dimensions Is your feature request related to a problem? Please describe. with errors when passing more than one dim: Describe the solution you'd like The logic to calculate the number of valid elements is here: I think this can be fixed by replacing with Additional context Potentially relevant for

"""     xarray version of pandas.core.nanops._maybe_null_out     """    if hasattr(axis, "__len__"):  # if tuple or list        raise ValueError(            "min_count is not available for reduction with more than one dimensions."        )      if axis is not None and getattr(result, "ndim", False):        null_mask = (mask.shape[axis] - mask.sum(axis) - min_count) < 0        null_mask = (np.take(mask.shape, axis).prod() - mask.sum(axis) - min_count) < 0         if null_mask.any():             dtype, fill_value = dtypes.maybe_promote(result.dtype)             result = result.astype(dtype)


Question: concat changes variable order A "Minimal, Complete and Verifiable Example" will make it much easier for maintainers to help you: Case 1: Creation of Dataset without Coordinates yields (assumed correct) output of: Case 2: Creation of Dataset with Coordinates yields (assumed false) output of: changes the dimension order for as well as to an alphanumerically sorted representation. should not change the dimension order in any case. <detailsINSTALLED VERSIONS commit: None python: 3.7.1 (default, Nov 13 2018, 18:33:04) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.12.14-lp150.12.48-default machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: deDE.UTF-8 LOCALE: deDE.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.2 xarray: 0.11.3 pandas: 0.24.1 numpy: 1.16.1 scipy: 1.2.0 netCDF4: 1.4.2 pydap: None h5netcdf: 0.6.2 h5py: 2.9.0 Nio: None zarr: None cftime: 1.0.3.4 PseudonetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: None distributed: None matplotlib: 3.0.2 cartopy: 0.17.0 seaborn: None setuptools: 40.8.0 pip: 19.0.2 conda: None pytest: 4.2.0 IPython: 7.2.0 sphinx: None </details[BUG] inverts coordinates order <!-- A short summary of the issue, if appropriate --Following the issue Merging two datasets using inverts the coordinates order. The concat function should not invert the coordinates but maintain the original order. <details<summaryINSTALLED VERSIONS </summarycommit: None python: 3.6.8 (default, May 7 2019, 14:58:50) [GCC 8.3.0] python-bits: 64 OS: Linux OS-release: 4.15.0-88-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: C.UTF-8 LOCALE: enUS.UTF-8 libhdf5: None libnetcdf: None xarray: 0.15.1 pandas: 1.0.3 numpy: 1.18.2 scipy: None netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.2.0 cartopy: None seaborn: None numbagg: None setuptools: 46.1.3 pip: 9.0.1 conda: None pytest: None IPython: 7.13.0 sphinx: 2.4.3</details

combined = concat_vars(vars, dim, positions)             assert isinstance(combined, Variable)             result_vars[k] = combined<ins>        elif k in result_vars:            # preserves original variable order            result_vars[k] = result_vars.pop(k)</ins>      result = Dataset(result_vars, attrs=result_attrs)     absent_coord_names = coord_names - set(result.variables)


Question: (mincount=1) errors for integer data <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: returns a if has an integer dtype. Of course mincount is not necessary for integer data as it cannot contain . What you expected to happen: should be ignored Minimal Complete Verifiable Example: Anything else we need to know?: Full traceback <details</detailsEnvironment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: python: 3.7.3 (default, Jul 1 2019, 21:52:21) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.0-42-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.15.2.dev64+g2542a63f pandas: 0.25.3 numpy: 1.17.3 scipy: 1.3.1 netCDF4: 1.5.4 pydap: installed h5netcdf: 0.7.4 h5py: 2.10.0 Nio: None zarr: 2.3.2 cftime: 1.0.4.2 nctime_axis: None PseudoNetCDF: installed rasterio: 1.1.0 cfgrib: 0.9.5.4 iris: None bottleneck: 1.2.1 dask: 2.6.0 distributed: 2.6.0 matplotlib: 3.3.1 cartopy: 0.18.0 seaborn: 0.9.0 numbagg: None pint: None setuptools: 49.6.0.post20200814 pip: 19.3.1 conda: None pytest: 5.2.2 IPython: 7.17.0 sphinx: None </details

nanname = "nan" + name             func = getattr(nanops, nanname)         else:<ins>            if name in ["sum", "prod"]:                kwargs.pop("min_count", None)</ins>             func = _dask_or_eager_func(name, dask_module=dask_module)          try:


Question: (mincount=1) errors for integer data <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: returns a if has an integer dtype. Of course mincount is not necessary for integer data as it cannot contain . What you expected to happen: should be ignored Minimal Complete Verifiable Example: Anything else we need to know?: Full traceback <details</detailsEnvironment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: python: 3.7.3 (default, Jul 1 2019, 21:52:21) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.0-42-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.15.2.dev64+g2542a63f pandas: 0.25.3 numpy: 1.17.3 scipy: 1.3.1 netCDF4: 1.5.4 pydap: installed h5netcdf: 0.7.4 h5py: 2.10.0 Nio: None zarr: 2.3.2 cftime: 1.0.4.2 nctime_axis: None PseudoNetCDF: installed rasterio: 1.1.0 cfgrib: 0.9.5.4 iris: None bottleneck: 1.2.1 dask: 2.6.0 distributed: 2.6.0 matplotlib: 3.3.1 cartopy: 0.18.0 seaborn: 0.9.0 numbagg: None pint: None setuptools: 49.6.0.post20200814 pip: 19.3.1 conda: None pytest: 5.2.2 IPython: 7.17.0 sphinx: None </details

median.numeric_only = True prod = _create_nan_agg_method("prod") prod.numeric_only = Truesum.available_min_count = Trueprod.available_min_count = True cumprod_1d = _create_nan_agg_method("cumprod") cumprod_1d.numeric_only = True cumsum_1d = _create_nan_agg_method("cumsum")


Question: todataframe/toseries fails when one out of more than one dims are stacked / multiindex When a dataarray has one multiindex dimension, as produced by stack, and has other dimesnions as well, toseries fails to create an combined multiindex. I would expect a series/dataframe with a multiindex with names a,b,c. Instead I get <detailslib/python2.7/site- in isnanew(obj) 115 # hack (for now) because MI registers as ndarray 116 elif isinstance(obj, ABCMultiIndex): --117 raise NotImplementedError("isna is not defined for MultiIndex") 118 elif isinstance(obj, (ABCSeries, np.ndarray, ABCIndexClass, 119 ABCExtensionArray)): NotImplementedError: isna is not defined for MultiIndex </detailsOn the other hand, when there is only one dimension, which is stacked, toseries and todataframe work <detailsINSTALLED VERSIONS commit: None python: 2.7.15 (default, May 1 2018, 23:32:55) [GCC 7.2.0] python-bits: 64 OS: Linux OS-release: 3.13.0-48-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: en_GB.UTF-8 LOCALE: None.None libhdf5: 1.8.17 libnetcdf: 4.4.1 xarray: 0.11.3 pandas: 0.23.4 numpy: 1.12.1 scipy: 0.19.1 netCDF4: 1.2.8 pydap: None h5netcdf: None h5py: 2.6.0 Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.0 cyordereddict: None dask: 0.17.3 distributed: 1.21.0 matplotlib: 2.2.2 cartopy: None seaborn: 0.7.1 setuptools: 0.6 pip: 19.0.1 conda: None pytest: 3.0.5 IPython: 5.8.0 sphinx: 1.5.1 </details

cast, ) <ins>import numpy as np</ins> import pandas as pd  from . import formatting, indexing


Question: todataframe/toseries fails when one out of more than one dims are stacked / multiindex When a dataarray has one multiindex dimension, as produced by stack, and has other dimesnions as well, toseries fails to create an combined multiindex. I would expect a series/dataframe with a multiindex with names a,b,c. Instead I get <detailslib/python2.7/site- in isnanew(obj) 115 # hack (for now) because MI registers as ndarray 116 elif isinstance(obj, ABCMultiIndex): --117 raise NotImplementedError("isna is not defined for MultiIndex") 118 elif isinstance(obj, (ABCSeries, np.ndarray, ABCIndexClass, 119 ABCExtensionArray)): NotImplementedError: isna is not defined for MultiIndex </detailsOn the other hand, when there is only one dimension, which is stacked, toseries and todataframe work <detailsINSTALLED VERSIONS commit: None python: 2.7.15 (default, May 1 2018, 23:32:55) [GCC 7.2.0] python-bits: 64 OS: Linux OS-release: 3.13.0-48-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: en_GB.UTF-8 LOCALE: None.None libhdf5: 1.8.17 libnetcdf: 4.4.1 xarray: 0.11.3 pandas: 0.23.4 numpy: 1.12.1 scipy: 0.19.1 netCDF4: 1.2.8 pydap: None h5netcdf: None h5py: 2.6.0 Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.0 cyordereddict: None dask: 0.17.3 distributed: 1.21.0 matplotlib: 2.2.2 cartopy: None seaborn: 0.7.1 setuptools: 0.6 pip: 19.0.1 conda: None pytest: 3.0.5 IPython: 5.8.0 sphinx: 1.5.1 </details

return self._data.get_index(dim)  # type: ignore         else:             indexes = [self._data.get_index(k) for k in ordered_dims]  # type: ignore            names = list(ordered_dims)            return pd.MultiIndex.from_product(indexes, names=names)            # compute the sizes of the repeat and tile for the cartesian product            # (taken from pandas.core.reshape.util)            index_lengths = np.fromiter(                (len(index) for index in indexes), dtype=np.intp            )            cumprod_lengths = np.cumproduct(index_lengths)            if cumprod_lengths[-1] != 0:                # sizes of the repeats                repeat_counts = cumprod_lengths[-1] / cumprod_lengths            else:                # if any factor is empty, the cartesian product is empty                repeat_counts = np.zeros_like(cumprod_lengths)            # sizes of the tiles            tile_counts = np.roll(cumprod_lengths, 1)            tile_counts[0] = 1            # loop over the indexes            # for each MultiIndex or Index compute the cartesian product of the codes            code_list = []            level_list = []            names = []            for i, index in enumerate(indexes):                if isinstance(index, pd.MultiIndex):                    codes, levels = index.codes, index.levels                else:                    code, level = pd.factorize(index)                    codes = [code]                    levels = [level]                # compute the cartesian product                code_list += [                    np.tile(np.repeat(code, repeat_counts[i]), tile_counts[i])                    for code in codes                ]                level_list += levels                names += index.names            return pd.MultiIndex(level_list, code_list, names=names)      def update(self, other: Mapping[Hashable, Any]) -> None:         other_vars = getattr(other, "variables", other)


Question: causes chunked dask DataArray to evalute its values eagerly What happened: Used to update a chunked dask DataArray, but the DataArray is no longer chunked after the update. What you expected to happen: The chunked DataArray should still be chunked after the update Minimal Complete Verifiable Example: Environment: <details<summaryOutput of <ttxr.show_versions()</tt</summary</detailsDataset constructor with DataArray triggers computation Is it intentional that creating a Dataset with a DataArray and dimension names for a single variable causes computation of that variable? In other words, why does cause the dask array to compute? A longer example:

if isinstance(obj, Variable):         obj = obj.copy(deep=False)     elif isinstance(obj, tuple):<ins>        if isinstance(obj[1], DataArray):            # TODO: change into TypeError            warnings.warn(                (                    "Using a DataArray object to construct a variable is"                    " ambiguous, please extract the data using the .data property."                    " This will raise a TypeError in 0.19.0."                ),                DeprecationWarning,            )</ins>         try:             obj = Variable(*obj)         except (TypeError, ValueError) as error:


Question: merge(combineattrs='override') does not copy attrs but instead references attrs from the first object <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: After a merge, an attribute value change in the merged product is reflected in the first source. What you expected to happen: After a merge, the attrs of the merged product should be able to be changed without having any effect on the sources. Minimal Complete Verifiable Example: Anything else we need to know?: I believe the issue is with the line for combineattrs == "override": . This should be changed to , like it is for the other combineattrs cases. Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.6.12 (default, Sep 15 2020, 12:49:50) [GCC 4.8.5 (Red Hat 4.8.5-37)] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.6.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: None libnetcdf: None xarray: 0.16.1 pandas: 1.1.4 numpy: 1.19.4 scipy: 1.5.3 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.5.0 cftime: None nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.30.0 distributed: 2.30.0 matplotlib: 3.3.2 cartopy: None seaborn: None numbagg: None pint: None setuptools: 50.3.2 pip: 20.2.4 conda: None pytest: None IPython: None sphinx: 3.3.0 </details

if combine_attrs == "drop":         return {}     elif combine_attrs == "override":        return variable_attrs[0]        return dict(variable_attrs[0])     elif combine_attrs == "no_conflicts":         result = dict(variable_attrs[0])         for attrs in variable_attrs[1:]:


Question: astype method lost its order parameter What happened: I upgraded from xarray 0.15.1 to 0.16.2 and the method seems to have lost the parameter. What you expected to happen: I was expecting to get the same result as with xarray 0.15.1: Anything else we need to know?: Looking at the documentation it seems it disappeared between 0.16.0 and 0.16.1. The documentation at still has this snippet (which was identical to the documentation from at ) while seems to lack that part.

"matplotlib": ("https://matplotlib.org", None),     "dask": ("https://docs.dask.org/en/latest", None),     "cftime": ("https://unidata.github.io/cftime", None),    "sparse": ("https://sparse.pydata.org/en/latest/", None), }  diff --git a/xarray/core/common.py b/xarray/core/common.py-- a/xarray/core/common.py++ b/xarray/core/common.py


Question: astype method lost its order parameter What happened: I upgraded from xarray 0.15.1 to 0.16.2 and the method seems to have lost the parameter. What you expected to happen: I was expecting to get the same result as with xarray 0.15.1: Anything else we need to know?: Looking at the documentation it seems it disappeared between 0.16.0 and 0.16.1. The documentation at still has this snippet (which was identical to the documentation from at ) while seems to lack that part.

dask="allowed",         )     def astype(self, dtype, casting="unsafe", copy=True, keep_attrs=True):    def astype(        self: T,        dtype,        *,        order=None,        casting=None,        subok=None,        copy=None,        keep_attrs=True,    ) -> T:         """         Copy of the xarray object, with data cast to a specified type.         Leaves coordinate dtype unchanged.


Question: astype method lost its order parameter What happened: I upgraded from xarray 0.15.1 to 0.16.2 and the method seems to have lost the parameter. What you expected to happen: I was expecting to get the same result as with xarray 0.15.1: Anything else we need to know?: Looking at the documentation it seems it disappeared between 0.16.0 and 0.16.1. The documentation at still has this snippet (which was identical to the documentation from at ) while seems to lack that part.

----------         dtype : str or dtype             Typecode or data-type to which the array is cast.        order : {'C', 'F', 'A', 'K'}, optional            Controls the memory layout order of the result. ‘C’ means C order,            ‘F’ means Fortran order, ‘A’ means ‘F’ order if all the arrays are            Fortran contiguous, ‘C’ order otherwise, and ‘K’ means as close to            the order the array elements appear in memory as possible.         casting : {'no', 'equiv', 'safe', 'same_kind', 'unsafe'}, optional            Controls what kind of data casting may occur. Defaults to 'unsafe'            for backwards compatibility.            Controls what kind of data casting may occur.              * 'no' means the data types should not be cast at all.             * 'equiv' means only byte-order changes are allowed.             * 'safe' means only casts which can preserve values are allowed.             * 'same_kind' means only safe casts or casts within a kind,                like float64 to float32, are allowed.              like float64 to float32, are allowed.             * 'unsafe' means any data conversions may be done.        subok : bool, optional            If True, then sub-classes will be passed-through, otherwise the            returned array will be forced to be a base-class array.         copy : bool, optional             By default, astype always returns a newly allocated array. If this             is set to False and the `dtype` requirement is satisfied, the input


Question: astype method lost its order parameter What happened: I upgraded from xarray 0.15.1 to 0.16.2 and the method seems to have lost the parameter. What you expected to happen: I was expecting to get the same result as with xarray 0.15.1: Anything else we need to know?: Looking at the documentation it seems it disappeared between 0.16.0 and 0.16.1. The documentation at still has this snippet (which was identical to the documentation from at ) while seems to lack that part.

out : same as object             New object with data cast to the specified type.         Notes        -----        The ``order``, ``casting``, ``subok`` and ``copy`` arguments are only passed        through to the ``astype`` method of the underlying array when a value        different than ``None`` is supplied.        Make sure to only supply these arguments if the underlying array class        supports them.         See also         --------        np.ndarray.astype        numpy.ndarray.astype         dask.array.Array.astype        sparse.COO.astype         """         from .computation import apply_ufunc         kwargs = dict(order=order, casting=casting, subok=subok, copy=copy)        kwargs = {k: v for k, v in kwargs.items() if v is not None}         return apply_ufunc(             duck_array_ops.astype,             self,            kwargs=dict(dtype=dtype, casting=casting, copy=copy),            dtype,            kwargs=kwargs,             keep_attrs=keep_attrs,             dask="allowed",         )diff --git a/xarray/core/duck_array_ops.py b/xarray/core/duck_array_ops.py-- a/xarray/core/duck_array_ops.py++ b/xarray/core/duck_array_ops.py


Question: astype method lost its order parameter What happened: I upgraded from xarray 0.15.1 to 0.16.2 and the method seems to have lost the parameter. What you expected to happen: I was expecting to get the same result as with xarray 0.15.1: Anything else we need to know?: Looking at the documentation it seems it disappeared between 0.16.0 and 0.16.1. The documentation at still has this snippet (which was identical to the documentation from at ) while seems to lack that part.

)  def astype(data, **kwargs):def astype(data, dtype, **kwargs):     try:         import sparse     except ImportError:


Question: astype method lost its order parameter What happened: I upgraded from xarray 0.15.1 to 0.16.2 and the method seems to have lost the parameter. What you expected to happen: I was expecting to get the same result as with xarray 0.15.1: Anything else we need to know?: Looking at the documentation it seems it disappeared between 0.16.0 and 0.16.1. The documentation at still has this snippet (which was identical to the documentation from at ) while seems to lack that part.

)         kwargs.pop("casting")     return data.astype(**kwargs)    return data.astype(dtype, **kwargs)   def asarray(data, xp=np):diff --git a/xarray/core/variable.py b/xarray/core/variable.py-- a/xarray/core/variable.py++ b/xarray/core/variable.py


Question: astype method lost its order parameter What happened: I upgraded from xarray 0.15.1 to 0.16.2 and the method seems to have lost the parameter. What you expected to happen: I was expecting to get the same result as with xarray 0.15.1: Anything else we need to know?: Looking at the documentation it seems it disappeared between 0.16.0 and 0.16.1. The documentation at still has this snippet (which was identical to the documentation from at ) while seems to lack that part.

)         self._data = data     def astype(self, dtype, casting="unsafe", copy=True, keep_attrs=True):    def astype(        self: VariableType,        dtype,        *,        order=None,        casting=None,        subok=None,        copy=None,        keep_attrs=True,    ) -> VariableType:         """         Copy of the Variable object, with data cast to a specified type.          Parameters         ----------         dtype : str or dtype             Typecode or data-type to which the array is cast.            Typecode or data-type to which the array is cast.        order : {'C', 'F', 'A', 'K'}, optional            Controls the memory layout order of the result. ‘C’ means C order,            ‘F’ means Fortran order, ‘A’ means ‘F’ order if all the arrays are            Fortran contiguous, ‘C’ order otherwise, and ‘K’ means as close to            the order the array elements appear in memory as possible.         casting : {'no', 'equiv', 'safe', 'same_kind', 'unsafe'}, optional             Controls what kind of data casting may occur. Defaults to 'unsafe'             for backwards compatibility.             * 'no' means the data types should not be cast at all.             * 'equiv' means only byte-order changes are allowed.             * 'safe' means only casts which can preserve values are allowed.             * 'same_kind' means only safe casts or casts within a kind,                 like float64 to float32, are allowed.             * 'unsafe' means any data conversions may be done.            Controls what kind of data casting may occur.            * 'no' means the data types should not be cast at all.            * 'equiv' means only byte-order changes are allowed.            * 'safe' means only casts which can preserve values are allowed.            * 'same_kind' means only safe casts or casts within a kind,              like float64 to float32, are allowed.            * 'unsafe' means any data conversions may be done.        subok : bool, optional            If True, then sub-classes will be passed-through, otherwise the            returned array will be forced to be a base-class array.         copy : bool, optional             By default, astype always returns a newly allocated array. If this             is set to False and the `dtype` requirement is satisfied, the input             array is returned instead of a copy.            By default, astype always returns a newly allocated array. If this            is set to False and the `dtype` requirement is satisfied, the input            array is returned instead of a copy.         keep_attrs : bool, optional             By default, astype keeps attributes. Set to False to remove             attributes in the returned object.


Question: astype method lost its order parameter What happened: I upgraded from xarray 0.15.1 to 0.16.2 and the method seems to have lost the parameter. What you expected to happen: I was expecting to get the same result as with xarray 0.15.1: Anything else we need to know?: Looking at the documentation it seems it disappeared between 0.16.0 and 0.16.1. The documentation at still has this snippet (which was identical to the documentation from at ) while seems to lack that part.

out : same as object             New object with data cast to the specified type.         Notes        -----        The ``order``, ``casting``, ``subok`` and ``copy`` arguments are only passed        through to the ``astype`` method of the underlying array when a value        different than ``None`` is supplied.        Make sure to only supply these arguments if the underlying array class        supports them.         See also         --------        np.ndarray.astype        numpy.ndarray.astype         dask.array.Array.astype        sparse.COO.astype         """         from .computation import apply_ufunc         kwargs = dict(order=order, casting=casting, subok=subok, copy=copy)        kwargs = {k: v for k, v in kwargs.items() if v is not None}         return apply_ufunc(             duck_array_ops.astype,             self,            kwargs=dict(dtype=dtype, casting=casting, copy=copy),            dtype,            kwargs=kwargs,             keep_attrs=keep_attrs,             dask="allowed",         )


Question: Millisecond precision is lost on datetime64 during IO roundtrip <!-- A short summary of the issue, if appropriate --I have millisecond-resolution time data as a coordinate on a DataArray. That data loses precision when round-tripping through disk. <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): -- Unzip the data. It will result in a pickle file. output: <!-- this should explain why the current behavior is a problem and why the expected output is a better solution --As you can see, I lose millisecond precision in this data. (The same happens when I use millisecond in the encoding). <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.7.6 (default, Jan 7 2020, 22:05:27) [Clang 9.0.1 ] python-bits: 64 OS: Darwin OS-release: 19.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: None.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.7.3 xarray: 0.15.1 pandas: 1.0.1 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: 0.8.0 h5py: 2.10.0 Nio: None zarr: None cftime: 1.0.4.2 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.11.0 distributed: 2.14.0 matplotlib: 3.1.3 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 45.2.0.post20200209 pip: 20.0.2 conda: None pytest: 5.3.5 IPython: 7.12.0 sphinx: 2.4.3 </details

_STANDARD_CALENDARS = {"standard", "gregorian", "proleptic_gregorian"}  _NS_PER_TIME_DELTA = {<ins>    "ns": 1,</ins>     "us": int(1e3),     "ms": int(1e6),     "s": int(1e9),


Question: Millisecond precision is lost on datetime64 during IO roundtrip <!-- A short summary of the issue, if appropriate --I have millisecond-resolution time data as a coordinate on a DataArray. That data loses precision when round-tripping through disk. <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): -- Unzip the data. It will result in a pickle file. output: <!-- this should explain why the current behavior is a problem and why the expected output is a better solution --As you can see, I lose millisecond precision in this data. (The same happens when I use millisecond in the encoding). <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.7.6 (default, Jan 7 2020, 22:05:27) [Clang 9.0.1 ] python-bits: 64 OS: Darwin OS-release: 19.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: None.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.7.3 xarray: 0.15.1 pandas: 1.0.1 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: 0.8.0 h5py: 2.10.0 Nio: None zarr: None cftime: 1.0.4.2 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.11.0 distributed: 2.14.0 matplotlib: 3.1.3 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 45.2.0.post20200209 pip: 20.0.2 conda: None pytest: 5.3.5 IPython: 7.12.0 sphinx: 2.4.3 </details

}  TIME_UNITS = frozenset(    ["days", "hours", "minutes", "seconds", "milliseconds", "microseconds"]    [        "days",        "hours",        "minutes",        "seconds",        "milliseconds",        "microseconds",        "nanoseconds",    ] )


Question: Millisecond precision is lost on datetime64 during IO roundtrip <!-- A short summary of the issue, if appropriate --I have millisecond-resolution time data as a coordinate on a DataArray. That data loses precision when round-tripping through disk. <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): -- Unzip the data. It will result in a pickle file. output: <!-- this should explain why the current behavior is a problem and why the expected output is a better solution --As you can see, I lose millisecond precision in this data. (The same happens when I use millisecond in the encoding). <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.7.6 (default, Jan 7 2020, 22:05:27) [Clang 9.0.1 ] python-bits: 64 OS: Darwin OS-release: 19.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: None.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.7.3 xarray: 0.15.1 pandas: 1.0.1 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: 0.8.0 h5py: 2.10.0 Nio: None zarr: None cftime: 1.0.4.2 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.11.0 distributed: 2.14.0 matplotlib: 3.1.3 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 45.2.0.post20200209 pip: 20.0.2 conda: None pytest: 5.3.5 IPython: 7.12.0 sphinx: 2.4.3 </details

if not units.endswith("s"):         units = "%ss" % units     return {<ins>        "nanoseconds": "ns",</ins>         "microseconds": "us",         "milliseconds": "ms",         "seconds": "s",


Question: Millisecond precision is lost on datetime64 during IO roundtrip <!-- A short summary of the issue, if appropriate --I have millisecond-resolution time data as a coordinate on a DataArray. That data loses precision when round-tripping through disk. <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): -- Unzip the data. It will result in a pickle file. output: <!-- this should explain why the current behavior is a problem and why the expected output is a better solution --As you can see, I lose millisecond precision in this data. (The same happens when I use millisecond in the encoding). <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.7.6 (default, Jan 7 2020, 22:05:27) [Clang 9.0.1 ] python-bits: 64 OS: Darwin OS-release: 19.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: None.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.7.3 xarray: 0.15.1 pandas: 1.0.1 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: 0.8.0 h5py: 2.10.0 Nio: None zarr: None cftime: 1.0.4.2 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.11.0 distributed: 2.14.0 matplotlib: 3.1.3 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 45.2.0.post20200209 pip: 20.0.2 conda: None pytest: 5.3.5 IPython: 7.12.0 sphinx: 2.4.3 </details

# strings, in which case we fall back to using cftime         raise OutOfBoundsDatetime     # fixes: https://github.com/pydata/pandas/issues/14068    # these lines check if the the lowest or the highest value in dates    # cause an OutOfBoundsDatetime (Overflow) error    with warnings.catch_warnings():        warnings.filterwarnings("ignore", "invalid value encountered", RuntimeWarning)        pd.to_timedelta(flat_num_dates.min(), delta) + ref_date        pd.to_timedelta(flat_num_dates.max(), delta) + ref_date    # Cast input dates to integers of nanoseconds because `pd.to_datetime`    # works much faster when dealing with integers    # make _NS_PER_TIME_DELTA an array to ensure type upcasting    flat_num_dates_ns_int = (        flat_num_dates.astype(np.float64) * _NS_PER_TIME_DELTA[delta]    ).astype(np.int64)    # To avoid integer overflow when converting to nanosecond units for integer    # dtypes smaller than np.int64 cast all integer-dtype arrays to np.int64    # (GH 2002).    if flat_num_dates.dtype.kind == "i":        flat_num_dates = flat_num_dates.astype(np.int64)     # Cast input ordinals to integers of nanoseconds because pd.to_timedelta    # works much faster when dealing with integers (GH 1399).    flat_num_dates_ns_int = (flat_num_dates * _NS_PER_TIME_DELTA[delta]).astype(        np.int64    )    # Use pd.to_timedelta to safely cast integer values to timedeltas,    # and add those to a Timestamp to safely produce a DatetimeIndex.  This    # ensures that we do not encounter integer overflow at any point in the    # process without raising OutOfBoundsDatetime.     return (pd.to_timedelta(flat_num_dates_ns_int, "ns") + ref_date).values


Question: Millisecond precision is lost on datetime64 during IO roundtrip <!-- A short summary of the issue, if appropriate --I have millisecond-resolution time data as a coordinate on a DataArray. That data loses precision when round-tripping through disk. <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): -- Unzip the data. It will result in a pickle file. output: <!-- this should explain why the current behavior is a problem and why the expected output is a better solution --As you can see, I lose millisecond precision in this data. (The same happens when I use millisecond in the encoding). <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.7.6 (default, Jan 7 2020, 22:05:27) [Clang 9.0.1 ] python-bits: 64 OS: Darwin OS-release: 19.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: None.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.7.3 xarray: 0.15.1 pandas: 1.0.1 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: 0.8.0 h5py: 2.10.0 Nio: None zarr: None cftime: 1.0.4.2 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.11.0 distributed: 2.14.0 matplotlib: 3.1.3 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 45.2.0.post20200209 pip: 20.0.2 conda: None pytest: 5.3.5 IPython: 7.12.0 sphinx: 2.4.3 </details

def _infer_time_units_from_diff(unique_timedeltas):    for time_unit in ["days", "hours", "minutes", "seconds"]:    # Note that the modulus operator was only implemented for np.timedelta64    # arrays as of NumPy version 1.16.0.  Once our minimum version of NumPy    # supported is greater than or equal to this we will no longer need to cast    # unique_timedeltas to a TimedeltaIndex.  In the meantime, however, the    # modulus operator works for TimedeltaIndex objects.    unique_deltas_as_index = pd.TimedeltaIndex(unique_timedeltas)    for time_unit in [        "days",        "hours",        "minutes",        "seconds",        "milliseconds",        "microseconds",        "nanoseconds",    ]:         delta_ns = _NS_PER_TIME_DELTA[_netcdf_to_numpy_timeunit(time_unit)]         unit_delta = np.timedelta64(delta_ns, "ns")        diffs = unique_timedeltas / unit_delta        if np.all(diffs == diffs.astype(int)):        if np.all(unique_deltas_as_index % unit_delta == np.timedelta64(0, "ns")):             return time_unit     return "seconds"


Question: Millisecond precision is lost on datetime64 during IO roundtrip <!-- A short summary of the issue, if appropriate --I have millisecond-resolution time data as a coordinate on a DataArray. That data loses precision when round-tripping through disk. <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): -- Unzip the data. It will result in a pickle file. output: <!-- this should explain why the current behavior is a problem and why the expected output is a better solution --As you can see, I lose millisecond precision in this data. (The same happens when I use millisecond in the encoding). <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.7.6 (default, Jan 7 2020, 22:05:27) [Clang 9.0.1 ] python-bits: 64 OS: Darwin OS-release: 19.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: None.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.7.3 xarray: 0.15.1 pandas: 1.0.1 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: 0.8.0 h5py: 2.10.0 Nio: None zarr: None cftime: 1.0.4.2 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.11.0 distributed: 2.14.0 matplotlib: 3.1.3 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 45.2.0.post20200209 pip: 20.0.2 conda: None pytest: 5.3.5 IPython: 7.12.0 sphinx: 2.4.3 </details

# Wrap the dates in a DatetimeIndex to do the subtraction to ensure         # an OverflowError is raised if the ref_date is too far away from         # dates to be encoded (GH 2272).        num = (pd.DatetimeIndex(dates.ravel()) - ref_date) / time_delta        dates_as_index = pd.DatetimeIndex(dates.ravel())        time_deltas = dates_as_index - ref_date        # Use floor division if time_delta evenly divides all differences        # to preserve integer dtype if possible (GH 4045).        if np.all(time_deltas % time_delta == np.timedelta64(0, "ns")):            num = time_deltas // time_delta        else:            num = time_deltas / time_delta         num = num.values.reshape(dates.shape)      except (OutOfBoundsDatetime, OverflowError):


Question: not preserving attributes <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: Using on a DataArray with attributes results in a new DataArray without attributes. What you expected to happen: Attributes should be preserved or at least there should be a choice (e.g. pass kwargs to so can be passed). Minimal Complete Verifiable Example: Anything else we need to know?: Apart from loosing attributes the dtype is not conserved. In the example the resulting DataArray has dtype np.int64 instead of np.int8. As far as I can see this might not be an xarray but a numpy problem. Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.7.8 (default, Jul 31 2020, 02:25:08) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 4.14.11--generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.16.0 pandas: 1.1.2 numpy: 1.19.1 scipy: 1.5.2 netCDF4: 1.5.4 pydap: None h5netcdf: 0.8.1 h5py: 2.10.0 Nio: None zarr: 2.4.0 cftime: 1.2.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.5 cfgrib: None iris: None bottleneck: None dask: 2.25.0 distributed: 2.25.0 matplotlib: 3.3.1 cartopy: 0.18.0 seaborn: None numbagg: None pint: None setuptools: 49.6.0.post20200814 pip: 20.2.3 conda: None pytest: 6.0.1 IPython: 7.18.1 sphinx: 3.2.1 () drops attributes <!-- A short summary of the issue, if appropriate --<!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --<!-- this should explain why the current behavior is a problem and why the expected output is a better solution --I would expect the attributes to remain in the data array. <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.0-33-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: None libnetcdf: None xarray: 0.15.1 pandas: 1.0.4 numpy: 1.18.4 scipy: 1.4.1 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.4 cfgrib: None iris: None bottleneck: None dask: 2.16.0 distributed: None matplotlib: 3.2.1 cartopy: None seaborn: None numbagg: None setuptools: 46.2.0 pip: 20.1 conda: None pytest: None IPython: 7.14.0 sphinx: 3.0.4 </details

return result.transpose(*all_dims, missing_dims="ignore")  def where(cond, x, y):def where(cond, x, y, keep_attrs=None):     """Return elements from `x` or `y` depending on `cond`.      Performs xarray-like broadcasting across input arguments.


Question: not preserving attributes <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: Using on a DataArray with attributes results in a new DataArray without attributes. What you expected to happen: Attributes should be preserved or at least there should be a choice (e.g. pass kwargs to so can be passed). Minimal Complete Verifiable Example: Anything else we need to know?: Apart from loosing attributes the dtype is not conserved. In the example the resulting DataArray has dtype np.int64 instead of np.int8. As far as I can see this might not be an xarray but a numpy problem. Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.7.8 (default, Jul 31 2020, 02:25:08) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 4.14.11--generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.16.0 pandas: 1.1.2 numpy: 1.19.1 scipy: 1.5.2 netCDF4: 1.5.4 pydap: None h5netcdf: 0.8.1 h5py: 2.10.0 Nio: None zarr: 2.4.0 cftime: 1.2.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.5 cfgrib: None iris: None bottleneck: None dask: 2.25.0 distributed: 2.25.0 matplotlib: 3.3.1 cartopy: 0.18.0 seaborn: None numbagg: None pint: None setuptools: 49.6.0.post20200814 pip: 20.2.3 conda: None pytest: 6.0.1 IPython: 7.18.1 sphinx: 3.2.1 () drops attributes <!-- A short summary of the issue, if appropriate --<!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --<!-- this should explain why the current behavior is a problem and why the expected output is a better solution --I would expect the attributes to remain in the data array. <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.0-33-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: None libnetcdf: None xarray: 0.15.1 pandas: 1.0.4 numpy: 1.18.4 scipy: 1.4.1 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.4 cfgrib: None iris: None bottleneck: None dask: 2.16.0 distributed: None matplotlib: 3.2.1 cartopy: None seaborn: None numbagg: None setuptools: 46.2.0 pip: 20.1 conda: None pytest: None IPython: 7.14.0 sphinx: 3.0.4 </details

values to choose from where `cond` is True     y : scalar, array, Variable, DataArray or Dataset         values to choose from where `cond` is False<ins>    keep_attrs : bool or str or callable, optional        How to treat attrs. If True, keep the attrs of `x`.</ins>      Returns     -------


Question: not preserving attributes <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: Using on a DataArray with attributes results in a new DataArray without attributes. What you expected to happen: Attributes should be preserved or at least there should be a choice (e.g. pass kwargs to so can be passed). Minimal Complete Verifiable Example: Anything else we need to know?: Apart from loosing attributes the dtype is not conserved. In the example the resulting DataArray has dtype np.int64 instead of np.int8. As far as I can see this might not be an xarray but a numpy problem. Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.7.8 (default, Jul 31 2020, 02:25:08) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 4.14.11--generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.16.0 pandas: 1.1.2 numpy: 1.19.1 scipy: 1.5.2 netCDF4: 1.5.4 pydap: None h5netcdf: 0.8.1 h5py: 2.10.0 Nio: None zarr: 2.4.0 cftime: 1.2.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.5 cfgrib: None iris: None bottleneck: None dask: 2.25.0 distributed: 2.25.0 matplotlib: 3.3.1 cartopy: 0.18.0 seaborn: None numbagg: None pint: None setuptools: 49.6.0.post20200814 pip: 20.2.3 conda: None pytest: 6.0.1 IPython: 7.18.1 sphinx: 3.2.1 () drops attributes <!-- A short summary of the issue, if appropriate --<!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --<!-- this should explain why the current behavior is a problem and why the expected output is a better solution --I would expect the attributes to remain in the data array. <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.0-33-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: None libnetcdf: None xarray: 0.15.1 pandas: 1.0.4 numpy: 1.18.4 scipy: 1.4.1 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.4 cfgrib: None iris: None bottleneck: None dask: 2.16.0 distributed: None matplotlib: 3.2.1 cartopy: None seaborn: None numbagg: None setuptools: 46.2.0 pip: 20.1 conda: None pytest: None IPython: 7.14.0 sphinx: 3.0.4 </details

Dataset.where, DataArray.where :         equivalent methods     """<ins>    if keep_attrs is None:        keep_attrs = _get_keep_attrs(default=False)    if keep_attrs is True:        # keep the attributes of x, the second parameter, by default to        # be consistent with the `where` method of `DataArray` and `Dataset`        keep_attrs = lambda attrs, context: attrs[1]</ins>     # alignment for three arguments is complicated, so don't support it yet     return apply_ufunc(         duck_array_ops.where,