Question: from .options import OPTIONS, _get_keep_attrs from .pycompat import dask_array_type from .rolling_exp import RollingExpfrom .utils import Frozen, either_dict_or_kwargsfrom .utils import Frozen, either_dict_or_kwargs, is_scalar  # Used as a sentinel value to indicate a all dimensions ALL_DIMS = ...

<details<summaryOutput of </summaryINSTALLED VERSIONS commit: None python: 3.7.6 (default, Jan 7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 158 Stepping 9, GenuineIntel byteorder: little LCALL: None LANG: en LOCALE: None.None libhdf5: 1.10.5 libnetcdf: 4.7.3 xarray: 0.15.1 pandas: 0.25.3 numpy: 1.17.5 scipy: 1.3.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: 2.4.0 cftime: 1.0.4.2 nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.2 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.9.2 distributed: 2.10.0 matplotlib: 3.1.2 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 46.1.3.post20200325 pip: 20.0.2 conda: None pytest: 5.3.4 IPython: 7.13.0 sphinx: 2.3.1 </detailsxr.fulllike (often) fails when other is chunked and fillvalue is non-scalar I've been running into some issues when using , when my is a chunked dask array, and the is a numpy array. Now, I just checked, mentions only scalar in the signature. However, this is a very convenient way to get all the coordinates and dimensions attached to an array like this, so it feels like desirable functionality. And as I mention below, both numpy and dask function similary, taking much more than just scalars. This results in an error: Expected is a DataArray with the dimensions and coords of , and the numpy array of as its data.


Question: from .options import OPTIONS, _get_keep_attrs from .pycompat import dask_array_type from .rolling_exp import RollingExpfrom .utils import Frozen, either_dict_or_kwargsfrom .utils import Frozen, either_dict_or_kwargs, is_scalar  # Used as a sentinel value to indicate a all dimensions ALL_DIMS = ...

The issue lies here: Calling with the given number of chunks results in it trying to to apply the for every individual chunk. As one would expect, if I set to the size of a single chunk it doesn't error: It does fail on a similarly chunked dask array (since it's applying it for every chunk): The most obvious solution would be to force it down the route, since all the values already exist in memory anyway. So maybe another type check does the trick. However, accepts quite a variety of arguments for the fill value (scalars, numpy arrays, lists, tuples, ranges). The dask docs mention only a scalar in the signature for: As does However, in all cases, they still broadcast automatically... So kind of undefined behavior of a blocked ?


Question: from .options import OPTIONS, _get_keep_attrs from .pycompat import dask_array_type from .rolling_exp import RollingExpfrom .utils import Frozen, either_dict_or_kwargsfrom .utils import Frozen, either_dict_or_kwargs, is_scalar  # Used as a sentinel value to indicate a all dimensions ALL_DIMS = ...

<details<summaryOutput of </summaryINSTALLED VERSIONS commit: None python: 3.7.6 (default, Jan 7 2020, 21:48:41) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 158 Stepping 9, GenuineIntel byteorder: little LCALL: None LANG: en LOCALE: None.None libhdf5: 1.10.5 libnetcdf: 4.7.3 xarray: 0.15.1 pandas: 0.25.3 numpy: 1.17.5 scipy: 1.3.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: 2.4.0 cftime: 1.0.4.2 nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.2 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.9.2 distributed: 2.10.0 matplotlib: 3.1.2 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 46.1.3.post20200325 pip: 20.0.2 conda: None pytest: 5.3.4 IPython: 7.13.0 sphinx: 2.3.1 </details


Question: from .dataset import Dataset     from .variable import Variable <ins>    if not is_scalar(fill_value):        raise ValueError(f"fill_value must be scalar. Received {fill_value} instead.")</ins>     if isinstance(other, Dataset):         data_vars = {             k: _full_like_variable(v, fill_value, dtype)

xr.fulllike (often) fails when other is chunked and fillvalue is non-scalar I've been running into some issues when using , when my is a chunked dask array, and the is a numpy array. Now, I just checked, mentions only scalar in the signature. However, this is a very convenient way to get all the coordinates and dimensions attached to an array like this, so it feels like desirable functionality. And as I mention below, both numpy and dask function similary, taking much more than just scalars. This results in an error: Expected is a DataArray with the dimensions and coords of , and the numpy array of as its data. The issue lies here: Calling with the given number of chunks results in it trying to to apply the for every individual chunk. As one would expect, if I set to the size of a single chunk it doesn't error: It does fail on a similarly chunked dask array (since it's applying it for every chunk): The most obvious solution would be to force it down the route, since all the values already exist in memory anyway. So maybe another type check does the trick. However, accepts quite a variety of arguments for the fill value (scalars, numpy arrays, lists, tuples, ranges). The dask docs mention only a scalar in the signature for: As does However, in all cases, they still broadcast automatically... So kind of undefined behavior of a blocked ?


Question: from .dataset import Dataset     from .variable import Variable <ins>    if not is_scalar(fill_value):        raise ValueError(f"fill_value must be scalar. Received {fill_value} instead.")</ins>     if isinstance(other, Dataset):         data_vars = {             k: _full_like_variable(v, fill_value, dtype)


Question: from .dataset import Dataset     from .variable import Variable <ins>    if not is_scalar(fill_value):        raise ValueError(f"fill_value must be scalar. Received {fill_value} instead.")</ins>     if isinstance(other, Dataset):         data_vars = {             k: _full_like_variable(v, fill_value, dtype)


Question: from .dataset import Dataset     from .variable import Variable <ins>    if not is_scalar(fill_value):        raise ValueError(f"fill_value must be scalar. Received {fill_value} instead.")</ins>     if isinstance(other, Dataset):         data_vars = {             k: _full_like_variable(v, fill_value, dtype)


Question: return self._from_temp_dataset(ds)      def integrate(        self, dim: Union[Hashable, Sequence[Hashable]], datetime_unit: str = None        self,        coord: Union[Hashable, Sequence[Hashable]] = None,        datetime_unit: str = None,        *,        dim: Union[Hashable, Sequence[Hashable]] = None,     ) -> "DataArray":        """ integrate the array with the trapezoidal rule.        """Integrate along the given coordinate using the trapezoidal rule.          .. note::            This feature is limited to simple cartesian geometry, i.e. dim            This feature is limited to simple cartesian geometry, i.e. coord             must be one dimensional.          Parameters         ----------        coord: hashable, or a sequence of hashable            Coordinate(s) used for the integration.         dim : hashable, or sequence of hashable             Coordinate(s) used for the integration.        datetime_unit : {"Y", "M", "W", "D", "h", "m", "s", "ms", "us", "ns", \                         "ps", "fs", "as"}, optional            Can be used to specify the unit if datetime coordinate is used.        datetime_unit: {'Y', 'M', 'W', 'D', 'h', 'm', 's', 'ms', 'us', 'ns', \                        'ps', 'fs', 'as'}, optional          Returns         -------

DataArray.integrate has a 'dim' arg, but Dataset.integrate has a 'coord' arg This is just a minor gripe but I think it should be fixed. The API syntax is inconsistent: It should definitely be - IMO it doesn't make sense to integrate or differentiate over a dim because a dim by definition has no information about the distance between grid points. I think because the distinction between dims and coords is one of the things that new users have to learn about, we should be strict to not confuse up the meanings in the documentation/API. The discussion on the original PR , so I think this was just an small oversight. The only question is whether it requires a deprecation cycle?


Question: See also         --------<ins>        Dataset.integrate</ins>         numpy.trapz: corresponding numpy function          Examples


Question: array([5.4, 6.6, 7.8])         Dimensions without coordinates: y         """        ds = self._to_temp_dataset().integrate(dim, datetime_unit)        if dim is not None and coord is not None:            raise ValueError(                "Cannot pass both 'dim' and 'coord'. Please pass only 'coord' instead."            )        if dim is not None and coord is None:            coord = dim            msg = (                "The `dim` keyword argument to `DataArray.integrate` is "                "being replaced with `coord`, for consistency with "                "`Dataset.integrate`. Please pass `coord` instead."                " `dim` will be removed in version 0.19.0."            )            warnings.warn(msg, FutureWarning, stacklevel=2)        ds = self._to_temp_dataset().integrate(coord, datetime_unit)         return self._from_temp_dataset(ds)      def unify_chunks(self) -> "DataArray":diff --git a/xarray/core/dataset.py b/xarray/core/dataset.py-- a/xarray/core/dataset.py++ b/xarray/core/dataset.py


Question: variables[k] = v         return self._replace(variables)     def integrate(self, coord, datetime_unit=None):        """ integrate the array with the trapezoidal rule.    def integrate(        self, coord: Union[Hashable, Sequence[Hashable]], datetime_unit: str = None    ) -> "Dataset":        """Integrate along the given coordinate using the trapezoidal rule.          .. note::             This feature is limited to simple cartesian geometry, i.e. coord


Question: Parameters         ----------        coord: str, or sequence of str        coord: hashable, or a sequence of hashable             Coordinate(s) used for the integration.        datetime_unit : {"Y", "M", "W", "D", "h", "m", "s", "ms", "us", "ns", \                         "ps", "fs", "as"}, optional            Can be specify the unit if datetime coordinate is used.        datetime_unit: {'Y', 'M', 'W', 'D', 'h', 'm', 's', 'ms', 'us', 'ns', \                        'ps', 'fs', 'as'}, optional            Specify the unit if datetime coordinate is used.          Returns         -------


Question: # we need to mask data values that are nan; else the weights are wrong         mask = da.notnull()         sum_of_weights = self._reduce(mask, self.weights, dim=dim, skipna=False)        # bool -> int, because ``xr.dot([True, True], [True, True])`` -> True        # (and not 2); GH4074        if self.weights.dtype == bool:            sum_of_weights = self._reduce(                mask, self.weights.astype(int), dim=dim, skipna=False            )        else:            sum_of_weights = self._reduce(mask, self.weights, dim=dim, skipna=False)          # 0-weights are not valid         valid_weights = sum_of_weights != 0.0

[bug] when passing boolean weights to weighted mean <!-- A short summary of the issue, if appropriate --<!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --Returns Passing a boolean array as weights to the weighted mean returns the wrong result because the are not properly normalized (in this case). Internally the is calculated as i.e. the dot product of two boolean arrays. This yields: We'll need to convert it to int or float: which is correct <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.7.6 (default, Mar 23 2020, 23:03:20) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.3.0-51-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.15.1 pandas: 1.0.3 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.1.1.2 nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.3 cfgrib: None iris: None bottleneck: None dask: 2.16.0 distributed: 2.16.0 matplotlib: 3.2.1 cartopy: 0.17.0 seaborn: None numbagg: None setuptools: 46.1.3.post20200325 pip: 20.1 conda: None pytest: 5.4.1 IPython: 7.13.0 sphinx: 3.0.3 </details


Question: # pull variables out of datarray         data_dict = {}         for k in variables:            data_dict[k] = self.sel({variable_dim: k}).squeeze(drop=True)            data_dict[k] = self.sel({variable_dim: k}, drop=True).squeeze(drop=True)          # unstacked dataset         return Dataset(data_dict)

tounstackeddataset broken for single-dim variables <!-- A short summary of the issue, if appropriate --A working roundtrip. I need to stack a bunch of variables and later unstack them again, however this doesn't work if the variables only have a single dimension. <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.7.3 (default, Mar 27 2019, 22:11:17) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.15.0-96-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: enGB.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.2 xarray: 0.15.1 pandas: 1.0.3 numpy: 1.17.3 scipy: 1.3.1 netCDF4: 1.4.2 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: None cftime: 1.0.4.2 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.10.1 distributed: 2.10.0 matplotlib: 3.1.1 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 41.0.0 pip: 19.0.3 conda: 4.8.3 pytest: 5.3.5 IPython: 7.9.0 sphinx: None </details


Question: def array_section(obj):     # "unique" id to expand/collapse the section     data_id = "section-" + str(uuid.uuid4())    collapsed = ""    collapsed = "checked"     variable = getattr(obj, "variable", obj)     preview = escape(inline_variable_array_repr(variable, max_width=70))     data_repr = short_data_repr_html(obj)

Pre-expand data and attributes in DataArray/Variable HTML repr? Given that a major purpose for plotting an array is to look at data or attributes, I wonder if we should expand these sections by default? I worry that clicking on icons to expand sections may not be easy to discover This would also be consistent with the text repr, which shows these sections by default (the Dataset repr is already consistent by default between text and HTML already) Currently the HTML repr for DataArray/Variable looks like this: ! To see array data, you have to click on the ! icon: ! (thanks to for making this a little bit more manageably sized in !) There's also a really nice repr for nested dask arrays: !


Question: import numpy as npimport pandas as pdfrom . import parameterizedclass MultiIndexSeries:    def setup(self, dtype, subset):        data = np.random.rand(100000).astype(dtype)        index = pd.MultiIndex.from_product(            [                list("abcdefhijk"),                list("abcdefhijk"),                pd.date_range(start="2000-01-01", periods=1000, freq="B"),            ]        )        series = pd.Series(data, index)        if subset:            series = series[::3]        self.series = series    @parameterized(["dtype", "subset"], ([int, float], [True, False]))    def time_to_xarray(self, dtype, subset):        self.series.to_xarray()diff --git a/xarray/core/dataset.py b/xarray/core/dataset.py-- a/xarray/core/dataset.py++ b/xarray/core/dataset.py

Stack + toarray before toxarray is much faster that a simple toxarray I was seeing some slow performance around on MultiIndexed series, and found that unstacking one of the dimensions before running , and then restacking with was ~30x faster. This time difference is consistent with larger data sizes. To reproduce: Create a series with a MultiIndex, ensuring the MultiIndex isn't a simple product: Two approaches for getting this into xarray; 1 - Simple : This takes 536 ms 2 - unstack in pandas first, and then use to do the equivalent of a restack: This takes 17.3 ms To confirm these are identical: A default operation is much slower than a (potentially) equivalent operation that's not the default. I need to look more at what's causing the issues. I think it's to do with the , but I'm unclear why it's so much faster in the alternative route, and whether there's a fix that we can make to make the default path fast. <detailsINSTALLED VERSIONS commit: None python: python-bits: 64 OS: Linux OS-release: 4.9.93-linuxkit-aufs machine: x8664 processor: x8664 byteorder: little LCALL: enUS.UTF-8 LANG: enUS.utf8 LOCALE: None.None xarray: 0.10.9 pandas: 0.23.4 numpy: 1.15.2 scipy: 1.1.0 netCDF4: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: None distributed: None matplotlib: 2.2.3 cartopy: 0.16.0 seaborn: 0.9.0 setuptools: 40.4.3 pip: 18.0 conda: None pytest: 3.8.1 IPython: 5.8.0 sphinx: None </detailstoxarray() result is incorrect when one of multi-index levels is not sorted <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted.


Question: import numpy as npimport pandas as pdfrom . import parameterizedclass MultiIndexSeries:    def setup(self, dtype, subset):        data = np.random.rand(100000).astype(dtype)        index = pd.MultiIndex.from_product(            [                list("abcdefhijk"),                list("abcdefhijk"),                pd.date_range(start="2000-01-01", periods=1000, freq="B"),            ]        )        series = pd.Series(data, index)        if subset:            series = series[::3]        self.series = series    @parameterized(["dtype", "subset"], ([int, float], [True, False]))    def time_to_xarray(self, dtype, subset):        self.series.to_xarray()diff --git a/xarray/core/dataset.py b/xarray/core/dataset.py-- a/xarray/core/dataset.py++ b/xarray/core/dataset.py

See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: toxarray() sorts multi-index level values and returns result for the sorted values but it doesn't sort levels or expects levels to be sorted resulting in completely incorrect order of data for the displayed coordinates. What you expected to happen: Should account for the order of levels in the original index. Minimal Complete Verifiable Example: Anything else we need to know?: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.7-100.fc30.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enGB.UTF-8 LOCALE: enGB.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.3 xarray: 0.15.1 pandas: 1.0.5 numpy: 1.19.0 scipy: 1.5.0 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.1.3 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.19.0 distributed: 2.19.0 matplotlib: 3.2.2 cartopy: None seaborn: 0.10.1 numbagg: installed setuptools: 46.3.0.post20200513 pip: 20.1 conda: None pytest: 5.4.3 IPython: 7.15.0 sphinx: None </details


Question: return self._to_dataframe(self.dims)      def _set_sparse_data_from_dataframe(        self, dataframe: pd.DataFrame, dims: tuple        self, idx: pd.Index, arrays: List[Tuple[Hashable, np.ndarray]], dims: tuple     ) -> None:         from sparse import COO         idx = dataframe.index         if isinstance(idx, pd.MultiIndex):             coords = np.stack([np.asarray(code) for code in idx.codes], axis=0)             is_sorted = idx.is_lexsorted()


Question: return self._to_dataframe(self.dims)      def _set_sparse_data_from_dataframe(        self, dataframe: pd.DataFrame, dims: tuple        self, idx: pd.Index, arrays: List[Tuple[Hashable, np.ndarray]], dims: tuple     ) -> None:         from sparse import COO         idx = dataframe.index         if isinstance(idx, pd.MultiIndex):             coords = np.stack([np.asarray(code) for code in idx.codes], axis=0)             is_sorted = idx.is_lexsorted()


Question: is_sorted = True             shape = (idx.size,)         for name, series in dataframe.items():            # Cast to a NumPy array first, in case the Series is a pandas            # Extension array (which doesn't have a valid NumPy dtype)            values = np.asarray(series)        for name, values in arrays:             # In virtually all real use cases, the sparse array will now have             # missing values and needs a fill_value. For consistency, don't             # special case the rare exceptions (e.g., dtype=int without a


Question: is_sorted = True             shape = (idx.size,)         for name, series in dataframe.items():            # Cast to a NumPy array first, in case the Series is a pandas            # Extension array (which doesn't have a valid NumPy dtype)            values = np.asarray(series)        for name, values in arrays:             # In virtually all real use cases, the sparse array will now have             # missing values and needs a fill_value. For consistency, don't             # special case the rare exceptions (e.g., dtype=int without a


Question: self[name] = (dims, data)      def _set_numpy_data_from_dataframe(        self, dataframe: pd.DataFrame, dims: tuple        self, idx: pd.Index, arrays: List[Tuple[Hashable, np.ndarray]], dims: tuple     ) -> None:        idx = dataframe.index        if isinstance(idx, pd.MultiIndex):            # expand the DataFrame to include the product of all levels            full_idx = pd.MultiIndex.from_product(idx.levels, names=idx.names)            dataframe = dataframe.reindex(full_idx)            shape = tuple(lev.size for lev in idx.levels)        else:            shape = (idx.size,)        for name, series in dataframe.items():            data = np.asarray(series).reshape(shape)        if not isinstance(idx, pd.MultiIndex):            for name, values in arrays:                self[name] = (dims, values)            return        shape = tuple(lev.size for lev in idx.levels)        indexer = tuple(idx.codes)        # We already verified that the MultiIndex has all unique values, so        # there are missing values if and only if the size of output arrays is        # larger that the index.        missing_values = np.prod(shape) > idx.shape[0]        for name, values in arrays:            # NumPy indexing is much faster than using DataFrame.reindex() to            # fill in missing values:            # https://stackoverflow.com/a/35049899/809705            if missing_values:                dtype, fill_value = dtypes.maybe_promote(values.dtype)                data = np.full(shape, fill_value, dtype)            else:                # If there are no missing values, keep the existing dtype                # instead of promoting to support NA, e.g., keep integer                # columns as integers.                # TODO: consider removing this special case, which doesn't                # exist for sparse=True.                data = np.zeros(shape, values.dtype)            data[indexer] = values             self[name] = (dims, data)      @classmethod


Question: self[name] = (dims, data)      def _set_numpy_data_from_dataframe(        self, dataframe: pd.DataFrame, dims: tuple        self, idx: pd.Index, arrays: List[Tuple[Hashable, np.ndarray]], dims: tuple     ) -> None:        idx = dataframe.index        if isinstance(idx, pd.MultiIndex):            # expand the DataFrame to include the product of all levels            full_idx = pd.MultiIndex.from_product(idx.levels, names=idx.names)            dataframe = dataframe.reindex(full_idx)            shape = tuple(lev.size for lev in idx.levels)        else:            shape = (idx.size,)        for name, series in dataframe.items():            data = np.asarray(series).reshape(shape)        if not isinstance(idx, pd.MultiIndex):            for name, values in arrays:                self[name] = (dims, values)            return        shape = tuple(lev.size for lev in idx.levels)        indexer = tuple(idx.codes)        # We already verified that the MultiIndex has all unique values, so        # there are missing values if and only if the size of output arrays is        # larger that the index.        missing_values = np.prod(shape) > idx.shape[0]        for name, values in arrays:            # NumPy indexing is much faster than using DataFrame.reindex() to            # fill in missing values:            # https://stackoverflow.com/a/35049899/809705            if missing_values:                dtype, fill_value = dtypes.maybe_promote(values.dtype)                data = np.full(shape, fill_value, dtype)            else:                # If there are no missing values, keep the existing dtype                # instead of promoting to support NA, e.g., keep integer                # columns as integers.                # TODO: consider removing this special case, which doesn't                # exist for sparse=True.                data = np.zeros(shape, values.dtype)            data[indexer] = values             self[name] = (dims, data)      @classmethod


Question: if not dataframe.columns.is_unique:             raise ValueError("cannot convert DataFrame with non-unique columns")         idx, dataframe = remove_unused_levels_categories(dataframe.index, dataframe)        idx = remove_unused_levels_categories(dataframe.index)        if isinstance(idx, pd.MultiIndex) and not idx.is_unique:            raise ValueError(                "cannot convert a DataFrame with a non-unique MultiIndex into xarray"            )        # Cast to a NumPy array first, in case the Series is a pandas Extension        # array (which doesn't have a valid NumPy dtype)        # TODO: allow users to control how this casting happens, e.g., by        # forwarding arguments to pandas.Series.to_numpy?        arrays = [(k, np.asarray(v)) for k, v in dataframe.items()]         obj = cls()          if isinstance(idx, pd.MultiIndex):


Question: if not dataframe.columns.is_unique:             raise ValueError("cannot convert DataFrame with non-unique columns")         idx, dataframe = remove_unused_levels_categories(dataframe.index, dataframe)        idx = remove_unused_levels_categories(dataframe.index)        if isinstance(idx, pd.MultiIndex) and not idx.is_unique:            raise ValueError(                "cannot convert a DataFrame with a non-unique MultiIndex into xarray"            )        # Cast to a NumPy array first, in case the Series is a pandas Extension        # array (which doesn't have a valid NumPy dtype)        # TODO: allow users to control how this casting happens, e.g., by        # forwarding arguments to pandas.Series.to_numpy?        arrays = [(k, np.asarray(v)) for k, v in dataframe.items()]         obj = cls()          if isinstance(idx, pd.MultiIndex):


Question: obj[index_name] = (dims, idx)          if sparse:            obj._set_sparse_data_from_dataframe(dataframe, dims)            obj._set_sparse_data_from_dataframe(idx, arrays, dims)         else:            obj._set_numpy_data_from_dataframe(dataframe, dims)            obj._set_numpy_data_from_dataframe(idx, arrays, dims)         return obj      def to_dask_dataframe(self, dim_order=None, set_index=False):diff --git a/xarray/core/indexes.py b/xarray/core/indexes.py-- a/xarray/core/indexes.py++ b/xarray/core/indexes.py


Question: obj[index_name] = (dims, idx)          if sparse:            obj._set_sparse_data_from_dataframe(dataframe, dims)            obj._set_sparse_data_from_dataframe(idx, arrays, dims)         else:            obj._set_numpy_data_from_dataframe(dataframe, dims)            obj._set_numpy_data_from_dataframe(idx, arrays, dims)         return obj      def to_dask_dataframe(self, dim_order=None, set_index=False):diff --git a/xarray/core/indexes.py b/xarray/core/indexes.py-- a/xarray/core/indexes.py++ b/xarray/core/indexes.py


Question: from .variable import Variable  def remove_unused_levels_categories(index, dataframe=None):def remove_unused_levels_categories(index: pd.Index) -> pd.Index:     """     Remove unused levels from MultiIndex and unused categories from CategoricalIndex     """


Question: from .variable import Variable  def remove_unused_levels_categories(index, dataframe=None):def remove_unused_levels_categories(index: pd.Index) -> pd.Index:     """     Remove unused levels from MultiIndex and unused categories from CategoricalIndex     """


Question: else:                     level = level[index.codes[i]]                 levels.append(level)            # TODO: calling from_array() reorders MultiIndex levels. It would            # be best to avoid this, if possible, e.g., by using            # MultiIndex.remove_unused_levels() (which does not reorder) on the            # part of the MultiIndex that is not categorical, or by fixing this            # upstream in pandas.             index = pd.MultiIndex.from_arrays(levels, names=index.names)     elif isinstance(index, pd.CategoricalIndex):         index = index.remove_unused_categories()    if dataframe is None:        return index    dataframe = dataframe.set_index(index)    return dataframe.index, dataframe    return index   class Indexes(collections.abc.Mapping):


Question: else:                     level = level[index.codes[i]]                 levels.append(level)            # TODO: calling from_array() reorders MultiIndex levels. It would            # be best to avoid this, if possible, e.g., by using            # MultiIndex.remove_unused_levels() (which does not reorder) on the            # part of the MultiIndex that is not categorical, or by fixing this            # upstream in pandas.             index = pd.MultiIndex.from_arrays(levels, names=index.names)     elif isinstance(index, pd.CategoricalIndex):         index = index.remove_unused_categories()    if dataframe is None:        return index    dataframe = dataframe.set_index(index)    return dataframe.index, dataframe    return index   class Indexes(collections.abc.Mapping):


Question: return inline_dask_repr(var.data)     elif isinstance(var._data, sparse_array_type):         return inline_sparse_repr(var.data)<ins>    elif hasattr(var._data, "_repr_inline_"):        return var._data._repr_inline_(max_width)</ins>     elif hasattr(var._data, "__array_function__"):         return maybe_truncate(repr(var._data).replace("\n", " "), max_width)     else:

Feature request: show units in dataset overview Here's a hypothetical dataset: It would be really nice if the units of the coordinates and of the data variables were shown in the repr, for example as:


Question: """     xarray version of pandas.core.nanops._maybe_null_out     """    if hasattr(axis, "__len__"):  # if tuple or list        raise ValueError(            "min_count is not available for reduction with more than one dimensions."        )      if axis is not None and getattr(result, "ndim", False):        null_mask = (mask.shape[axis] - mask.sum(axis) - min_count) < 0        null_mask = (np.take(mask.shape, axis).prod() - mask.sum(axis) - min_count) < 0         if null_mask.any():             dtype, fill_value = dtypes.maybe_promote(result.dtype)             result = result.astype(dtype)

sum: mincount is not available for reduction with more than one dimensions Is your feature request related to a problem? Please describe. with errors when passing more than one dim: Describe the solution you'd like The logic to calculate the number of valid elements is here: I think this can be fixed by replacing with Additional context Potentially relevant for


Question: combined = concat_vars(vars, dim, positions)             assert isinstance(combined, Variable)             result_vars[k] = combined<ins>        elif k in result_vars:            # preserves original variable order            result_vars[k] = result_vars.pop(k)</ins>      result = Dataset(result_vars, attrs=result_attrs)     absent_coord_names = coord_names - set(result.variables)

concat changes variable order A "Minimal, Complete and Verifiable Example" will make it much easier for maintainers to help you: Case 1: Creation of Dataset without Coordinates yields (assumed correct) output of: Case 2: Creation of Dataset with Coordinates yields (assumed false) output of: changes the dimension order for as well as to an alphanumerically sorted representation. should not change the dimension order in any case. <detailsINSTALLED VERSIONS commit: None python: 3.7.1 (default, Nov 13 2018, 18:33:04) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.12.14-lp150.12.48-default machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: deDE.UTF-8 LOCALE: deDE.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.2 xarray: 0.11.3 pandas: 0.24.1 numpy: 1.16.1 scipy: 1.2.0 netCDF4: 1.4.2 pydap: None h5netcdf: 0.6.2 h5py: 2.9.0 Nio: None zarr: None cftime: 1.0.3.4 PseudonetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.1 cyordereddict: None dask: None distributed: None matplotlib: 3.0.2 cartopy: 0.17.0 seaborn: None setuptools: 40.8.0 pip: 19.0.2 conda: None pytest: 4.2.0 IPython: 7.2.0 sphinx: None </details[BUG] inverts coordinates order <!-- A short summary of the issue, if appropriate --Following the issue Merging two datasets using inverts the coordinates order. The concat function should not invert the coordinates but maintain the original order.


Question: combined = concat_vars(vars, dim, positions)             assert isinstance(combined, Variable)             result_vars[k] = combined<ins>        elif k in result_vars:            # preserves original variable order            result_vars[k] = result_vars.pop(k)</ins>      result = Dataset(result_vars, attrs=result_attrs)     absent_coord_names = coord_names - set(result.variables)

<details<summaryINSTALLED VERSIONS </summarycommit: None python: 3.6.8 (default, May 7 2019, 14:58:50) [GCC 8.3.0] python-bits: 64 OS: Linux OS-release: 4.15.0-88-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: C.UTF-8 LOCALE: enUS.UTF-8 libhdf5: None libnetcdf: None xarray: 0.15.1 pandas: 1.0.3 numpy: 1.18.2 scipy: None netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.2.0 cartopy: None seaborn: None numbagg: None setuptools: 46.1.3 pip: 9.0.1 conda: None pytest: None IPython: 7.13.0 sphinx: 2.4.3</details


Question: nanname = "nan" + name             func = getattr(nanops, nanname)         else:<ins>            if name in ["sum", "prod"]:                kwargs.pop("min_count", None)</ins>             func = _dask_or_eager_func(name, dask_module=dask_module)          try:

(mincount=1) errors for integer data <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: returns a if has an integer dtype. Of course mincount is not necessary for integer data as it cannot contain . What you expected to happen: should be ignored Minimal Complete Verifiable Example: Anything else we need to know?


Question: nanname = "nan" + name             func = getattr(nanops, nanname)         else:<ins>            if name in ["sum", "prod"]:                kwargs.pop("min_count", None)</ins>             func = _dask_or_eager_func(name, dask_module=dask_module)          try:

: Full traceback <details</detailsEnvironment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: python: 3.7.3 (default, Jul 1 2019, 21:52:21) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.0-42-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.15.2.dev64+g2542a63f pandas: 0.25.3 numpy: 1.17.3 scipy: 1.3.1 netCDF4: 1.5.4 pydap: installed h5netcdf: 0.7.4 h5py: 2.10.0 Nio: None zarr: 2.3.2 cftime: 1.0.4.2 nctime_axis: None PseudoNetCDF: installed rasterio: 1.1.0 cfgrib: 0.9.5.4 iris: None bottleneck: 1.2.1 dask: 2.6.0 distributed: 2.6.0 matplotlib: 3.3.1 cartopy: 0.18.0 seaborn: 0.9.0 numbagg: None pint: None setuptools: 49.6.0.post20200814 pip: 19.3.1 conda: None pytest: 5.2.2 IPython: 7.17.0 sphinx: None </details


Question: median.numeric_only = True prod = _create_nan_agg_method("prod") prod.numeric_only = Truesum.available_min_count = Trueprod.available_min_count = True cumprod_1d = _create_nan_agg_method("cumprod") cumprod_1d.numeric_only = True cumsum_1d = _create_nan_agg_method("cumsum")


Question: median.numeric_only = True prod = _create_nan_agg_method("prod") prod.numeric_only = Truesum.available_min_count = Trueprod.available_min_count = True cumprod_1d = _create_nan_agg_method("cumprod") cumprod_1d.numeric_only = True cumsum_1d = _create_nan_agg_method("cumsum")


Question: cast, ) <ins>import numpy as np</ins> import pandas as pd  from . import formatting, indexing

todataframe/toseries fails when one out of more than one dims are stacked / multiindex When a dataarray has one multiindex dimension, as produced by stack, and has other dimesnions as well, toseries fails to create an combined multiindex. I would expect a series/dataframe with a multiindex with names a,b,c. Instead I get <detailslib/python2.7/site- in isnanew(obj) 115 # hack (for now) because MI registers as ndarray 116 elif isinstance(obj, ABCMultiIndex): --117 raise NotImplementedError("isna is not defined for MultiIndex") 118 elif isinstance(obj, (ABCSeries, np.ndarray, ABCIndexClass, 119 ABCExtensionArray)): NotImplementedError: isna is not defined for MultiIndex </detailsOn the other hand, when there is only one dimension, which is stacked, toseries and todataframe work <detailsINSTALLED VERSIONS commit: None python: 2.7.15 (default, May 1 2018, 23:32:55) [GCC 7.2.0] python-bits: 64 OS: Linux OS-release: 3.13.0-48-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: en_GB.UTF-8 LOCALE: None.None libhdf5: 1.8.17 libnetcdf: 4.4.1 xarray: 0.11.3 pandas: 0.23.4 numpy: 1.12.1 scipy: 0.19.1 netCDF4: 1.2.8 pydap: None h5netcdf: None h5py: 2.6.0 Nio: None zarr: None cftime: None PseudonetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.0 cyordereddict: None dask: 0.17.3 distributed: 1.21.0 matplotlib: 2.2.2 cartopy: None seaborn: 0.7.1 setuptools: 0.6 pip: 19.0.1 conda: None pytest: 3.0.5 IPython: 5.8.0 sphinx: 1.5.1 </details


Question: return self._data.get_index(dim)  # type: ignore         else:             indexes = [self._data.get_index(k) for k in ordered_dims]  # type: ignore            names = list(ordered_dims)            return pd.MultiIndex.from_product(indexes, names=names)            # compute the sizes of the repeat and tile for the cartesian product            # (taken from pandas.core.reshape.util)            index_lengths = np.fromiter(                (len(index) for index in indexes), dtype=np.intp            )            cumprod_lengths = np.cumproduct(index_lengths)            if cumprod_lengths[-1] != 0:                # sizes of the repeats                repeat_counts = cumprod_lengths[-1] / cumprod_lengths            else:                # if any factor is empty, the cartesian product is empty                repeat_counts = np.zeros_like(cumprod_lengths)            # sizes of the tiles            tile_counts = np.roll(cumprod_lengths, 1)            tile_counts[0] = 1            # loop over the indexes            # for each MultiIndex or Index compute the cartesian product of the codes            code_list = []            level_list = []            names = []            for i, index in enumerate(indexes):                if isinstance(index, pd.MultiIndex):                    codes, levels = index.codes, index.levels                else:                    code, level = pd.factorize(index)                    codes = [code]                    levels = [level]                # compute the cartesian product                code_list += [                    np.tile(np.repeat(code, repeat_counts[i]), tile_counts[i])                    for code in codes                ]                level_list += levels                names += index.names            return pd.MultiIndex(level_list, code_list, names=names)      def update(self, other: Mapping[Hashable, Any]) -> None:         other_vars = getattr(other, "variables", other)


Question: if isinstance(obj, Variable):         obj = obj.copy(deep=False)     elif isinstance(obj, tuple):<ins>        if isinstance(obj[1], DataArray):            # TODO: change into TypeError            warnings.warn(                (                    "Using a DataArray object to construct a variable is"                    " ambiguous, please extract the data using the .data property."                    " This will raise a TypeError in 0.19.0."                ),                DeprecationWarning,            )</ins>         try:             obj = Variable(*obj)         except (TypeError, ValueError) as error:

causes chunked dask DataArray to evalute its values eagerly What happened: Used to update a chunked dask DataArray, but the DataArray is no longer chunked after the update. What you expected to happen: The chunked DataArray should still be chunked after the update Minimal Complete Verifiable Example: Environment: <details<summaryOutput of <ttxr.show_versions()</tt</summary</detailsDataset constructor with DataArray triggers computation Is it intentional that creating a Dataset with a DataArray and dimension names for a single variable causes computation of that variable? In other words, why does cause the dask array to compute? A longer example:


Question: if combine_attrs == "drop":         return {}     elif combine_attrs == "override":        return variable_attrs[0]        return dict(variable_attrs[0])     elif combine_attrs == "no_conflicts":         result = dict(variable_attrs[0])         for attrs in variable_attrs[1:]:

merge(combineattrs='override') does not copy attrs but instead references attrs from the first object <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: After a merge, an attribute value change in the merged product is reflected in the first source. What you expected to happen: After a merge, the attrs of the merged product should be able to be changed without having any effect on the sources. Minimal Complete Verifiable Example: Anything else we need to know?: I believe the issue is with the line for combineattrs == "override": . This should be changed to , like it is for the other combineattrs cases.


Question: if combine_attrs == "drop":         return {}     elif combine_attrs == "override":        return variable_attrs[0]        return dict(variable_attrs[0])     elif combine_attrs == "no_conflicts":         result = dict(variable_attrs[0])         for attrs in variable_attrs[1:]:

Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.6.12 (default, Sep 15 2020, 12:49:50) [GCC 4.8.5 (Red Hat 4.8.5-37)] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.6.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: None libnetcdf: None xarray: 0.16.1 pandas: 1.1.4 numpy: 1.19.4 scipy: 1.5.3 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.5.0 cftime: None nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.30.0 distributed: 2.30.0 matplotlib: 3.3.2 cartopy: None seaborn: None numbagg: None pint: None setuptools: 50.3.2 pip: 20.2.4 conda: None pytest: None IPython: None sphinx: 3.3.0 </details


Question: "matplotlib": ("https://matplotlib.org", None),     "dask": ("https://docs.dask.org/en/latest", None),     "cftime": ("https://unidata.github.io/cftime", None),    "sparse": ("https://sparse.pydata.org/en/latest/", None), }  diff --git a/xarray/core/common.py b/xarray/core/common.py-- a/xarray/core/common.py++ b/xarray/core/common.py

astype method lost its order parameter What happened: I upgraded from xarray 0.15.1 to 0.16.2 and the method seems to have lost the parameter. What you expected to happen: I was expecting to get the same result as with xarray 0.15.1: Anything else we need to know?: Looking at the documentation it seems it disappeared between 0.16.0 and 0.16.1. The documentation at still has this snippet (which was identical to the documentation from at ) while seems to lack that part.


Question: dask="allowed",         )     def astype(self, dtype, casting="unsafe", copy=True, keep_attrs=True):    def astype(        self: T,        dtype,        *,        order=None,        casting=None,        subok=None,        copy=None,        keep_attrs=True,    ) -> T:         """         Copy of the xarray object, with data cast to a specified type.         Leaves coordinate dtype unchanged.


Question: ----------         dtype : str or dtype             Typecode or data-type to which the array is cast.        order : {'C', 'F', 'A', 'K'}, optional            Controls the memory layout order of the result. ‘C’ means C order,            ‘F’ means Fortran order, ‘A’ means ‘F’ order if all the arrays are            Fortran contiguous, ‘C’ order otherwise, and ‘K’ means as close to            the order the array elements appear in memory as possible.         casting : {'no', 'equiv', 'safe', 'same_kind', 'unsafe'}, optional            Controls what kind of data casting may occur. Defaults to 'unsafe'            for backwards compatibility.            Controls what kind of data casting may occur.              * 'no' means the data types should not be cast at all.             * 'equiv' means only byte-order changes are allowed.             * 'safe' means only casts which can preserve values are allowed.             * 'same_kind' means only safe casts or casts within a kind,                like float64 to float32, are allowed.              like float64 to float32, are allowed.             * 'unsafe' means any data conversions may be done.        subok : bool, optional            If True, then sub-classes will be passed-through, otherwise the            returned array will be forced to be a base-class array.         copy : bool, optional             By default, astype always returns a newly allocated array. If this             is set to False and the `dtype` requirement is satisfied, the input


Question: out : same as object             New object with data cast to the specified type.         Notes        -----        The ``order``, ``casting``, ``subok`` and ``copy`` arguments are only passed        through to the ``astype`` method of the underlying array when a value        different than ``None`` is supplied.        Make sure to only supply these arguments if the underlying array class        supports them.         See also         --------        np.ndarray.astype        numpy.ndarray.astype         dask.array.Array.astype        sparse.COO.astype         """         from .computation import apply_ufunc         kwargs = dict(order=order, casting=casting, subok=subok, copy=copy)        kwargs = {k: v for k, v in kwargs.items() if v is not None}         return apply_ufunc(             duck_array_ops.astype,             self,            kwargs=dict(dtype=dtype, casting=casting, copy=copy),            dtype,            kwargs=kwargs,             keep_attrs=keep_attrs,             dask="allowed",         )diff --git a/xarray/core/duck_array_ops.py b/xarray/core/duck_array_ops.py-- a/xarray/core/duck_array_ops.py++ b/xarray/core/duck_array_ops.py


Question: )  def astype(data, **kwargs):def astype(data, dtype, **kwargs):     try:         import sparse     except ImportError:


Question: )         kwargs.pop("casting")     return data.astype(**kwargs)    return data.astype(dtype, **kwargs)   def asarray(data, xp=np):diff --git a/xarray/core/variable.py b/xarray/core/variable.py-- a/xarray/core/variable.py++ b/xarray/core/variable.py


Question: )         self._data = data     def astype(self, dtype, casting="unsafe", copy=True, keep_attrs=True):    def astype(        self: VariableType,        dtype,        *,        order=None,        casting=None,        subok=None,        copy=None,        keep_attrs=True,    ) -> VariableType:         """         Copy of the Variable object, with data cast to a specified type.          Parameters         ----------         dtype : str or dtype             Typecode or data-type to which the array is cast.            Typecode or data-type to which the array is cast.        order : {'C', 'F', 'A', 'K'}, optional            Controls the memory layout order of the result. ‘C’ means C order,            ‘F’ means Fortran order, ‘A’ means ‘F’ order if all the arrays are            Fortran contiguous, ‘C’ order otherwise, and ‘K’ means as close to            the order the array elements appear in memory as possible.         casting : {'no', 'equiv', 'safe', 'same_kind', 'unsafe'}, optional             Controls what kind of data casting may occur. Defaults to 'unsafe'             for backwards compatibility.             * 'no' means the data types should not be cast at all.             * 'equiv' means only byte-order changes are allowed.             * 'safe' means only casts which can preserve values are allowed.             * 'same_kind' means only safe casts or casts within a kind,                 like float64 to float32, are allowed.             * 'unsafe' means any data conversions may be done.            Controls what kind of data casting may occur.            * 'no' means the data types should not be cast at all.            * 'equiv' means only byte-order changes are allowed.            * 'safe' means only casts which can preserve values are allowed.            * 'same_kind' means only safe casts or casts within a kind,              like float64 to float32, are allowed.            * 'unsafe' means any data conversions may be done.        subok : bool, optional            If True, then sub-classes will be passed-through, otherwise the            returned array will be forced to be a base-class array.         copy : bool, optional             By default, astype always returns a newly allocated array. If this             is set to False and the `dtype` requirement is satisfied, the input             array is returned instead of a copy.            By default, astype always returns a newly allocated array. If this            is set to False and the `dtype` requirement is satisfied, the input            array is returned instead of a copy.         keep_attrs : bool, optional             By default, astype keeps attributes. Set to False to remove             attributes in the returned object.


Question: out : same as object             New object with data cast to the specified type.         Notes        -----        The ``order``, ``casting``, ``subok`` and ``copy`` arguments are only passed        through to the ``astype`` method of the underlying array when a value        different than ``None`` is supplied.        Make sure to only supply these arguments if the underlying array class        supports them.         See also         --------        np.ndarray.astype        numpy.ndarray.astype         dask.array.Array.astype        sparse.COO.astype         """         from .computation import apply_ufunc         kwargs = dict(order=order, casting=casting, subok=subok, copy=copy)        kwargs = {k: v for k, v in kwargs.items() if v is not None}         return apply_ufunc(             duck_array_ops.astype,             self,            kwargs=dict(dtype=dtype, casting=casting, copy=copy),            dtype,            kwargs=kwargs,             keep_attrs=keep_attrs,             dask="allowed",         )


Question: _STANDARD_CALENDARS = {"standard", "gregorian", "proleptic_gregorian"}  _NS_PER_TIME_DELTA = {<ins>    "ns": 1,</ins>     "us": int(1e3),     "ms": int(1e6),     "s": int(1e9),

Millisecond precision is lost on datetime64 during IO roundtrip <!-- A short summary of the issue, if appropriate --I have millisecond-resolution time data as a coordinate on a DataArray. That data loses precision when round-tripping through disk. <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): -- Unzip the data. It will result in a pickle file. output: <!-- this should explain why the current behavior is a problem and why the expected output is a better solution --As you can see, I lose millisecond precision in this data. (The same happens when I use millisecond in the encoding). <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.7.6 (default, Jan 7 2020, 22:05:27) [Clang 9.0.1 ] python-bits: 64 OS: Darwin OS-release: 19.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: None.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.7.3 xarray: 0.15.1 pandas: 1.0.1 numpy: 1.18.1 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: 0.8.0 h5py: 2.10.0 Nio: None zarr: None cftime: 1.0.4.2 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2.11.0 distributed: 2.14.0 matplotlib: 3.1.3 cartopy: None seaborn: 0.10.0 numbagg: None setuptools: 45.2.0.post20200209 pip: 20.0.2 conda: None pytest: 5.3.5 IPython: 7.12.0 sphinx: 2.4.3 </details


Question: }  TIME_UNITS = frozenset(    ["days", "hours", "minutes", "seconds", "milliseconds", "microseconds"]    [        "days",        "hours",        "minutes",        "seconds",        "milliseconds",        "microseconds",        "nanoseconds",    ] )


Question: if not units.endswith("s"):         units = "%ss" % units     return {<ins>        "nanoseconds": "ns",</ins>         "microseconds": "us",         "milliseconds": "ms",         "seconds": "s",


Question: # strings, in which case we fall back to using cftime         raise OutOfBoundsDatetime     # fixes: https://github.com/pydata/pandas/issues/14068    # these lines check if the the lowest or the highest value in dates    # cause an OutOfBoundsDatetime (Overflow) error    with warnings.catch_warnings():        warnings.filterwarnings("ignore", "invalid value encountered", RuntimeWarning)        pd.to_timedelta(flat_num_dates.min(), delta) + ref_date        pd.to_timedelta(flat_num_dates.max(), delta) + ref_date    # Cast input dates to integers of nanoseconds because `pd.to_datetime`    # works much faster when dealing with integers    # make _NS_PER_TIME_DELTA an array to ensure type upcasting    flat_num_dates_ns_int = (        flat_num_dates.astype(np.float64) * _NS_PER_TIME_DELTA[delta]    ).astype(np.int64)    # To avoid integer overflow when converting to nanosecond units for integer    # dtypes smaller than np.int64 cast all integer-dtype arrays to np.int64    # (GH 2002).    if flat_num_dates.dtype.kind == "i":        flat_num_dates = flat_num_dates.astype(np.int64)     # Cast input ordinals to integers of nanoseconds because pd.to_timedelta    # works much faster when dealing with integers (GH 1399).    flat_num_dates_ns_int = (flat_num_dates * _NS_PER_TIME_DELTA[delta]).astype(        np.int64    )    # Use pd.to_timedelta to safely cast integer values to timedeltas,    # and add those to a Timestamp to safely produce a DatetimeIndex.  This    # ensures that we do not encounter integer overflow at any point in the    # process without raising OutOfBoundsDatetime.     return (pd.to_timedelta(flat_num_dates_ns_int, "ns") + ref_date).values


Question: def _infer_time_units_from_diff(unique_timedeltas):    for time_unit in ["days", "hours", "minutes", "seconds"]:    # Note that the modulus operator was only implemented for np.timedelta64    # arrays as of NumPy version 1.16.0.  Once our minimum version of NumPy    # supported is greater than or equal to this we will no longer need to cast    # unique_timedeltas to a TimedeltaIndex.  In the meantime, however, the    # modulus operator works for TimedeltaIndex objects.    unique_deltas_as_index = pd.TimedeltaIndex(unique_timedeltas)    for time_unit in [        "days",        "hours",        "minutes",        "seconds",        "milliseconds",        "microseconds",        "nanoseconds",    ]:         delta_ns = _NS_PER_TIME_DELTA[_netcdf_to_numpy_timeunit(time_unit)]         unit_delta = np.timedelta64(delta_ns, "ns")        diffs = unique_timedeltas / unit_delta        if np.all(diffs == diffs.astype(int)):        if np.all(unique_deltas_as_index % unit_delta == np.timedelta64(0, "ns")):             return time_unit     return "seconds"


Question: # Wrap the dates in a DatetimeIndex to do the subtraction to ensure         # an OverflowError is raised if the ref_date is too far away from         # dates to be encoded (GH 2272).        num = (pd.DatetimeIndex(dates.ravel()) - ref_date) / time_delta        dates_as_index = pd.DatetimeIndex(dates.ravel())        time_deltas = dates_as_index - ref_date        # Use floor division if time_delta evenly divides all differences        # to preserve integer dtype if possible (GH 4045).        if np.all(time_deltas % time_delta == np.timedelta64(0, "ns")):            num = time_deltas // time_delta        else:            num = time_deltas / time_delta         num = num.values.reshape(dates.shape)      except (OutOfBoundsDatetime, OverflowError):


Question: return result.transpose(*all_dims, missing_dims="ignore")  def where(cond, x, y):def where(cond, x, y, keep_attrs=None):     """Return elements from `x` or `y` depending on `cond`.      Performs xarray-like broadcasting across input arguments.

not preserving attributes <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: Using on a DataArray with attributes results in a new DataArray without attributes. What you expected to happen: Attributes should be preserved or at least there should be a choice (e.g. pass kwargs to so can be passed). Minimal Complete Verifiable Example: Anything else we need to know?: Apart from loosing attributes the dtype is not conserved. In the example the resulting DataArray has dtype np.int64 instead of np.int8. As far as I can see this might not be an xarray but a numpy problem.


Question: return result.transpose(*all_dims, missing_dims="ignore")  def where(cond, x, y):def where(cond, x, y, keep_attrs=None):     """Return elements from `x` or `y` depending on `cond`.      Performs xarray-like broadcasting across input arguments.

Environment: <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.7.8 (default, Jul 31 2020, 02:25:08) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 4.14.11--generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.6 libnetcdf: 4.7.4 xarray: 0.16.0 pandas: 1.1.2 numpy: 1.19.1 scipy: 1.5.2 netCDF4: 1.5.4 pydap: None h5netcdf: 0.8.1 h5py: 2.10.0 Nio: None zarr: 2.4.0 cftime: 1.2.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.5 cfgrib: None iris: None bottleneck: None dask: 2.25.0 distributed: 2.25.0 matplotlib: 3.3.1 cartopy: 0.18.0 seaborn: None numbagg: None pint: None setuptools: 49.6.0.post20200814 pip: 20.2.3 conda: None pytest: 6.0.1 IPython: 7.18.1 sphinx: 3.2.1 () drops attributes <!-- A short summary of the issue, if appropriate --<!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --<!-- this should explain why the current behavior is a problem and why the expected output is a better solution --I would expect the attributes to remain in the data array.


Question: return result.transpose(*all_dims, missing_dims="ignore")  def where(cond, x, y):def where(cond, x, y, keep_attrs=None):     """Return elements from `x` or `y` depending on `cond`.      Performs xarray-like broadcasting across input arguments.

<details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.4.0-33-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: None libnetcdf: None xarray: 0.15.1 pandas: 1.0.4 numpy: 1.18.4 scipy: 1.4.1 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nctimeaxis: None PseudoNetCDF: None rasterio: 1.1.4 cfgrib: None iris: None bottleneck: None dask: 2.16.0 distributed: None matplotlib: 3.2.1 cartopy: None seaborn: None numbagg: None setuptools: 46.2.0 pip: 20.1 conda: None pytest: None IPython: 7.14.0 sphinx: 3.0.4 </details


Question: values to choose from where `cond` is True     y : scalar, array, Variable, DataArray or Dataset         values to choose from where `cond` is False<ins>    keep_attrs : bool or str or callable, optional        How to treat attrs. If True, keep the attrs of `x`.</ins>      Returns     -------


Question: values to choose from where `cond` is True     y : scalar, array, Variable, DataArray or Dataset         values to choose from where `cond` is False<ins>    keep_attrs : bool or str or callable, optional        How to treat attrs. If True, keep the attrs of `x`.</ins>      Returns     -------


Question: values to choose from where `cond` is True     y : scalar, array, Variable, DataArray or Dataset         values to choose from where `cond` is False<ins>    keep_attrs : bool or str or callable, optional        How to treat attrs. If True, keep the attrs of `x`.</ins>      Returns     -------


Question: Dataset.where, DataArray.where :         equivalent methods     """<ins>    if keep_attrs is None:        keep_attrs = _get_keep_attrs(default=False)    if keep_attrs is True:        # keep the attributes of x, the second parameter, by default to        # be consistent with the `where` method of `DataArray` and `Dataset`        keep_attrs = lambda attrs, context: attrs[1]</ins>     # alignment for three arguments is complicated, so don't support it yet     return apply_ufunc(         duck_array_ops.where,


Question: Dataset.where, DataArray.where :         equivalent methods     """<ins>    if keep_attrs is None:        keep_attrs = _get_keep_attrs(default=False)    if keep_attrs is True:        # keep the attributes of x, the second parameter, by default to        # be consistent with the `where` method of `DataArray` and `Dataset`        keep_attrs = lambda attrs, context: attrs[1]</ins>     # alignment for three arguments is complicated, so don't support it yet     return apply_ufunc(         duck_array_ops.where,


Question: Dataset.where, DataArray.where :         equivalent methods     """<ins>    if keep_attrs is None:        keep_attrs = _get_keep_attrs(default=False)    if keep_attrs is True:        # keep the attributes of x, the second parameter, by default to        # be consistent with the `where` method of `DataArray` and `Dataset`        keep_attrs = lambda attrs, context: attrs[1]</ins>     # alignment for three arguments is complicated, so don't support it yet     return apply_ufunc(         duck_array_ops.where,


Question: join="exact",         dataset_join="exact",         dask="allowed",<ins>        keep_attrs=keep_attrs,</ins>     )


Question: join="exact",         dataset_join="exact",         dask="allowed",<ins>        keep_attrs=keep_attrs,</ins>     )


Question: join="exact",         dataset_join="exact",         dask="allowed",<ins>        keep_attrs=keep_attrs,</ins>     )


Question: # expand the indexer so we can handle Ellipsis             labels = indexing.expanded_indexer(key, self.data_array.ndim)             key = dict(zip(self.data_array.dims, labels))        return self.data_array.sel(**key)        return self.data_array.sel(key)      def __setitem__(self, key, value) -> None:         if not utils.is_dict_like(key):

Naming a dimension "method" throws error when calling ".loc" The name of the dimension should be irrelevant. The error message suggests that at some point the are given to another method in unsanitized form. Edit: Updated to xarray 0.12 from conda-forge channel. The bug is still present. <detailsINSTALLED VERSIONS commit: None python: 3.6.8 (default, Dec 30 2018, 01:22:34) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.18.0-16-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.1 xarray: 0.12.0 pandas: 0.24.2 numpy: 1.16.2 scipy: 1.2.1 netCDF4: 1.4.2 pydap: None h5netcdf: None h5py: 2.9.0 Nio: None zarr: None cftime: 1.0.3.4 nctimeaxis: None PseudonetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.1 dask: None distributed: None matplotlib: 3.0.3 cartopy: None seaborn: None setuptools: 40.8.0 pip: 19.0.3 conda: 4.6.8 pytest: None IPython: 7.3.0 sphinx: 1.8.5 </detailsNaming a dimension "method" throws error when calling ".loc" The name of the dimension should be irrelevant. The error message suggests that at some point the are given to another method in unsanitized form. Edit: Updated to xarray 0.12 from conda-forge channel. The bug is still present.


Question: # expand the indexer so we can handle Ellipsis             labels = indexing.expanded_indexer(key, self.data_array.ndim)             key = dict(zip(self.data_array.dims, labels))        return self.data_array.sel(**key)        return self.data_array.sel(key)      def __setitem__(self, key, value) -> None:         if not utils.is_dict_like(key):

<detailsINSTALLED VERSIONS commit: None python: 3.6.8 (default, Dec 30 2018, 01:22:34) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 4.18.0-16-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: enUS.UTF-8 libhdf5: 1.10.4 libnetcdf: 4.6.1 xarray: 0.12.0 pandas: 0.24.2 numpy: 1.16.2 scipy: 1.2.1 netCDF4: 1.4.2 pydap: None h5netcdf: None h5py: 2.9.0 Nio: None zarr: None cftime: 1.0.3.4 nctimeaxis: None PseudonetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.2.1 dask: None distributed: None matplotlib: 3.0.3 cartopy: None seaborn: None setuptools: 40.8.0 pip: 19.0.3 conda: 4.6.8 pytest: None IPython: 7.3.0 sphinx: 1.8.5 </details


Question: return col_width  def _mapping_repr(mapping, title, summarizer, col_width=None):def _mapping_repr(mapping, title, summarizer, col_width=None, max_rows=None):     if col_width is None:         col_width = _calculate_col_width(mapping)    if max_rows is None:        max_rows = OPTIONS["display_max_rows"]     summary = [f"{title}:"]     if mapping:        summary += [summarizer(k, v, col_width) for k, v in mapping.items()]        if len(mapping) > max_rows:            first_rows = max_rows // 2 + max_rows % 2            items = list(mapping.items())            summary += [summarizer(k, v, col_width) for k, v in items[:first_rows]]            if max_rows > 1:                last_rows = max_rows // 2                summary += [pretty_print("    ...", col_width) + " ..."]                summary += [summarizer(k, v, col_width) for k, v in items[-last_rows:]]        else:            summary += [summarizer(k, v, col_width) for k, v in mapping.items()]     else:         summary += [EMPTY_REPR]     return "\n".join(summary)diff --git a/xarray/core/options.py b/xarray/core/options.py-- a/xarray/core/options.py++ b/xarray/core/options.py

Limit number of data variables shown in repr <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: xarray feels very unresponsive when using datasets with 2000 data variables because it has to print all the 2000 variables everytime you print something to console. What you expected to happen: xarray should limit the number of variables printed to console. Maximum maybe 25? Same idea probably apply to dimensions, coordinates and attributes as well, pandas only shows 2 for reference, the first and last variables. Minimal Complete Verifiable Example: Anything else we need to know?: Out of scope brainstorming: Though printing 2000 variables is probably madness for most people it is kind of nice to show all variables because you sometimes want to know what happened to a few other variables as well. Is there already an easy and fast way to create subgroup of the dataset, so we don' have to rely on the dataset printing everything to the console everytime?


Question: return col_width  def _mapping_repr(mapping, title, summarizer, col_width=None):def _mapping_repr(mapping, title, summarizer, col_width=None, max_rows=None):     if col_width is None:         col_width = _calculate_col_width(mapping)    if max_rows is None:        max_rows = OPTIONS["display_max_rows"]     summary = [f"{title}:"]     if mapping:        summary += [summarizer(k, v, col_width) for k, v in mapping.items()]        if len(mapping) > max_rows:            first_rows = max_rows // 2 + max_rows % 2            items = list(mapping.items())            summary += [summarizer(k, v, col_width) for k, v in items[:first_rows]]            if max_rows > 1:                last_rows = max_rows // 2                summary += [pretty_print("    ...", col_width) + " ..."]                summary += [summarizer(k, v, col_width) for k, v in items[-last_rows:]]        else:            summary += [summarizer(k, v, col_width) for k, v in mapping.items()]     else:         summary += [EMPTY_REPR]     return "\n".join(summary)diff --git a/xarray/core/options.py b/xarray/core/options.py-- a/xarray/core/options.py++ b/xarray/core/options.py

Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryxr.showversions() INSTALLED VERSIONS commit: None python: 3.8.5 (default, Sep 3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 libhdf5: 1.10.4 libnetcdf: None xarray: 0.16.2 pandas: 1.1.5 numpy: 1.17.5 scipy: 1.4.1 netCDF4: None pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: None cftime: None nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2020.12.0 distributed: 2020.12.0 matplotlib: 3.3.2 cartopy: None seaborn: 0.11.1 numbagg: None pint: None setuptools: 51.0.0.post20201207 pip: 20.3.3 conda: 4.9.2 pytest: 6.2.1 IPython: 7.19.0 sphinx: 3.4.0 </details


Question: import warnings DISPLAY_WIDTH = "display_width" ARITHMETIC_JOIN = "arithmetic_join"CMAP_DIVERGENT = "cmap_divergent"CMAP_SEQUENTIAL = "cmap_sequential"DISPLAY_MAX_ROWS = "display_max_rows"DISPLAY_STYLE = "display_style"DISPLAY_WIDTH = "display_width" ENABLE_CFTIMEINDEX = "enable_cftimeindex" FILE_CACHE_MAXSIZE = "file_cache_maxsize"WARN_FOR_UNCLOSED_FILES = "warn_for_unclosed_files"CMAP_SEQUENTIAL = "cmap_sequential"CMAP_DIVERGENT = "cmap_divergent" KEEP_ATTRS = "keep_attrs"DISPLAY_STYLE = "display_style"WARN_FOR_UNCLOSED_FILES = "warn_for_unclosed_files"   OPTIONS = {    DISPLAY_WIDTH: 80,     ARITHMETIC_JOIN: "inner",    CMAP_DIVERGENT: "RdBu_r",    CMAP_SEQUENTIAL: "viridis",    DISPLAY_MAX_ROWS: 12,    DISPLAY_STYLE: "html",    DISPLAY_WIDTH: 80,     ENABLE_CFTIMEINDEX: True,     FILE_CACHE_MAXSIZE: 128,    WARN_FOR_UNCLOSED_FILES: False,    CMAP_SEQUENTIAL: "viridis",    CMAP_DIVERGENT: "RdBu_r",     KEEP_ATTRS: "default",    DISPLAY_STYLE: "html",    WARN_FOR_UNCLOSED_FILES: False, }  _JOIN_OPTIONS = frozenset(["inner", "outer", "left", "right", "exact"])


Question: import warnings DISPLAY_WIDTH = "display_width" ARITHMETIC_JOIN = "arithmetic_join"CMAP_DIVERGENT = "cmap_divergent"CMAP_SEQUENTIAL = "cmap_sequential"DISPLAY_MAX_ROWS = "display_max_rows"DISPLAY_STYLE = "display_style"DISPLAY_WIDTH = "display_width" ENABLE_CFTIMEINDEX = "enable_cftimeindex" FILE_CACHE_MAXSIZE = "file_cache_maxsize"WARN_FOR_UNCLOSED_FILES = "warn_for_unclosed_files"CMAP_SEQUENTIAL = "cmap_sequential"CMAP_DIVERGENT = "cmap_divergent" KEEP_ATTRS = "keep_attrs"DISPLAY_STYLE = "display_style"WARN_FOR_UNCLOSED_FILES = "warn_for_unclosed_files"   OPTIONS = {    DISPLAY_WIDTH: 80,     ARITHMETIC_JOIN: "inner",    CMAP_DIVERGENT: "RdBu_r",    CMAP_SEQUENTIAL: "viridis",    DISPLAY_MAX_ROWS: 12,    DISPLAY_STYLE: "html",    DISPLAY_WIDTH: 80,     ENABLE_CFTIMEINDEX: True,     FILE_CACHE_MAXSIZE: 128,    WARN_FOR_UNCLOSED_FILES: False,    CMAP_SEQUENTIAL: "viridis",    CMAP_DIVERGENT: "RdBu_r",     KEEP_ATTRS: "default",    DISPLAY_STYLE: "html",    WARN_FOR_UNCLOSED_FILES: False, }  _JOIN_OPTIONS = frozenset(["inner", "outer", "left", "right", "exact"])


Question: _VALIDATORS = {    DISPLAY_WIDTH: _positive_integer,     ARITHMETIC_JOIN: _JOIN_OPTIONS.__contains__,    DISPLAY_MAX_ROWS: _positive_integer,    DISPLAY_STYLE: _DISPLAY_OPTIONS.__contains__,    DISPLAY_WIDTH: _positive_integer,     ENABLE_CFTIMEINDEX: lambda value: isinstance(value, bool),     FILE_CACHE_MAXSIZE: _positive_integer,    WARN_FOR_UNCLOSED_FILES: lambda value: isinstance(value, bool),     KEEP_ATTRS: lambda choice: choice in [True, False, "default"],    DISPLAY_STYLE: _DISPLAY_OPTIONS.__contains__,    WARN_FOR_UNCLOSED_FILES: lambda value: isinstance(value, bool), }


Question: _VALIDATORS = {    DISPLAY_WIDTH: _positive_integer,     ARITHMETIC_JOIN: _JOIN_OPTIONS.__contains__,    DISPLAY_MAX_ROWS: _positive_integer,    DISPLAY_STYLE: _DISPLAY_OPTIONS.__contains__,    DISPLAY_WIDTH: _positive_integer,     ENABLE_CFTIMEINDEX: lambda value: isinstance(value, bool),     FILE_CACHE_MAXSIZE: _positive_integer,    WARN_FOR_UNCLOSED_FILES: lambda value: isinstance(value, bool),     KEEP_ATTRS: lambda choice: choice in [True, False, "default"],    DISPLAY_STYLE: _DISPLAY_OPTIONS.__contains__,    WARN_FOR_UNCLOSED_FILES: lambda value: isinstance(value, bool), }


Question: _SETTERS = {    FILE_CACHE_MAXSIZE: _set_file_cache_maxsize,     ENABLE_CFTIMEINDEX: _warn_on_setting_enable_cftimeindex,    FILE_CACHE_MAXSIZE: _set_file_cache_maxsize, }


Question: _SETTERS = {    FILE_CACHE_MAXSIZE: _set_file_cache_maxsize,     ENABLE_CFTIMEINDEX: _warn_on_setting_enable_cftimeindex,    FILE_CACHE_MAXSIZE: _set_file_cache_maxsize, }


Question: return other + self.as_timedelta()  <ins>class Millisecond(BaseCFTimeOffset):    _freq = "L"    def as_timedelta(self):        return timedelta(milliseconds=self.n)    def __apply__(self, other):        return other + self.as_timedelta()class Microsecond(BaseCFTimeOffset):    _freq = "U"    def as_timedelta(self):        return timedelta(microseconds=self.n)    def __apply__(self, other):        return other + self.as_timedelta()</ins> _FREQUENCIES = {     "A": YearEnd,     "AS": YearBegin,

CFTime offsets missing for milli- and micro-seconds <!-- A short summary of the issue, if appropriate --The smallest cftime offset defined in is "second" (S), but the precision of cftime objects goes down to the millisecond (L) and microsecond (U). They should be easily . PR adds a that supports the two, but they are currently untested as cannot generate an index. <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): --<!-- this should explain why the current behavior is a problem and why the expected output is a better solution --An error gets raised : . <details<summaryOutput of <ttxr.showversions()</tt</summary<!-- Paste the output here xr.showversions() here --INSTALLED VERSIONS commit: None python: 3.8.2 (default, Apr 24 2020, 08:20:52) [GCC 7.3.0] python-bits: 64 OS: Linux OS-release: 5.6.13-arch1-1 machine: x8664 processor: byteorder: little LCALL: None LANG: frCA.utf8 LOCALE: frCA.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.7.4 xarray: 0.15.2.dev9+g6378a711.


Question: return other + self.as_timedelta()  <ins>class Millisecond(BaseCFTimeOffset):    _freq = "L"    def as_timedelta(self):        return timedelta(milliseconds=self.n)    def __apply__(self, other):        return other + self.as_timedelta()class Microsecond(BaseCFTimeOffset):    _freq = "U"    def as_timedelta(self):        return timedelta(microseconds=self.n)    def __apply__(self, other):        return other + self.as_timedelta()</ins> _FREQUENCIES = {     "A": YearEnd,     "AS": YearBegin,

pandas: 1.0.3 numpy: 1.18.4 scipy: 1.4.1 netCDF4: 1.5.3 pydap: None h5netcdf: None h5py: 2.10.0 Nio: None zarr: None cftime: 1.1.1.2 nctime_axis: 1.2.0 PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2.16.0 distributed: 2.16.0 matplotlib: 3.2.1 cartopy: None seaborn: None numbagg: None pint: 0.11 setuptools: 46.1.3.post20200325 pip: 20.0.2 conda: None pytest: 5.4.1 IPython: 7.13.0 sphinx: 3.0.2 </details


Question: "T": Minute,     "min": Minute,     "S": Second,<ins>    "L": Millisecond,    "ms": Millisecond,    "U": Microsecond,    "us": Microsecond,</ins>     "AS-JAN": partial(YearBegin, month=1),     "AS-FEB": partial(YearBegin, month=2),     "AS-MAR": partial(YearBegin, month=3),


Question: "T": Minute,     "min": Minute,     "S": Second,<ins>    "L": Millisecond,    "ms": Millisecond,    "U": Microsecond,    "us": Microsecond,</ins>     "AS-JAN": partial(YearBegin, month=1),     "AS-FEB": partial(YearBegin, month=2),     "AS-MAR": partial(YearBegin, month=3),


Question: `ISO-8601 format <https://en.wikipedia.org/wiki/ISO_8601>`_.     - It supports many, but not all, frequencies supported by       ``pandas.date_range``.  For example it does not currently support any of      the business-related, semi-monthly, or sub-second frequencies.      the business-related or semi-monthly frequencies.     - Compound sub-monthly frequencies are not supported, e.g. '1H1min', as       these can easily be written in terms of the finest common resolution,       e.g. '61min'.


Question: `ISO-8601 format <https://en.wikipedia.org/wiki/ISO_8601>`_.     - It supports many, but not all, frequencies supported by       ``pandas.date_range``.  For example it does not currently support any of      the business-related, semi-monthly, or sub-second frequencies.      the business-related or semi-monthly frequencies.     - Compound sub-monthly frequencies are not supported, e.g. '1H1min', as       these can easily be written in terms of the finest common resolution,       e.g. '61min'.


Question: +--------+--------------------------+     | S      | Second frequency         |     +--------+--------------------------+    | L, ms  | Millisecond frequency    |    +--------+--------------------------+    | U, us  | Microsecond frequency    |    +--------+--------------------------+      Any multiples of the following anchored offsets are also supported. diff --git a/xarray/coding/times.py b/xarray/coding/times.py-- a/xarray/coding/times.py++ b/xarray/coding/times.py


Question: +--------+--------------------------+     | S      | Second frequency         |     +--------+--------------------------+    | L, ms  | Millisecond frequency    |    +--------+--------------------------+    | U, us  | Microsecond frequency    |    +--------+--------------------------+      Any multiples of the following anchored offsets are also supported. diff --git a/xarray/coding/times.py b/xarray/coding/times.py-- a/xarray/coding/times.py++ b/xarray/coding/times.py


Question: import re import warningsfrom datetime import datetimefrom datetime import datetime, timedelta from distutils.version import LooseVersion from functools import partial


Question: import re import warningsfrom datetime import datetimefrom datetime import datetime, timedelta from distutils.version import LooseVersion from functools import partial


Question: "D": int(1e9) * 60 * 60 * 24, } <ins>_US_PER_TIME_DELTA = {    "microseconds": 1,    "milliseconds": 1_000,    "seconds": 1_000_000,    "minutes": 60 * 1_000_000,    "hours": 60 * 60 * 1_000_000,    "days": 24 * 60 * 60 * 1_000_000,}_NETCDF_TIME_UNITS_CFTIME = [    "days",    "hours",    "minutes",    "seconds",    "milliseconds",    "microseconds",]_NETCDF_TIME_UNITS_NUMPY = _NETCDF_TIME_UNITS_CFTIME + ["nanoseconds"]</ins> TIME_UNITS = frozenset(     [         "days",


Question: "D": int(1e9) * 60 * 60 * 24, } <ins>_US_PER_TIME_DELTA = {    "microseconds": 1,    "milliseconds": 1_000,    "seconds": 1_000_000,    "minutes": 60 * 1_000_000,    "hours": 60 * 60 * 1_000_000,    "days": 24 * 60 * 60 * 1_000_000,}_NETCDF_TIME_UNITS_CFTIME = [    "days",    "hours",    "minutes",    "seconds",    "milliseconds",    "microseconds",]_NETCDF_TIME_UNITS_NUMPY = _NETCDF_TIME_UNITS_CFTIME + ["nanoseconds"]</ins> TIME_UNITS = frozenset(     [         "days",


Question: if calendar in _STANDARD_CALENDARS:                     dates = cftime_to_nptime(dates)     elif use_cftime:        dates = _decode_datetime_with_cftime(            flat_num_dates.astype(float), units, calendar        )        dates = _decode_datetime_with_cftime(flat_num_dates, units, calendar)     else:         dates = _decode_datetime_with_pandas(flat_num_dates, units, calendar)


Question: if calendar in _STANDARD_CALENDARS:                     dates = cftime_to_nptime(dates)     elif use_cftime:        dates = _decode_datetime_with_cftime(            flat_num_dates.astype(float), units, calendar        )        dates = _decode_datetime_with_cftime(flat_num_dates, units, calendar)     else:         dates = _decode_datetime_with_pandas(flat_num_dates, units, calendar)


Question: return result.reshape(num_timedeltas.shape)  def _unit_timedelta_cftime(units):    return timedelta(microseconds=_US_PER_TIME_DELTA[units])def _unit_timedelta_numpy(units):    numpy_units = _netcdf_to_numpy_timeunit(units)    return np.timedelta64(_NS_PER_TIME_DELTA[numpy_units], "ns") def _infer_time_units_from_diff(unique_timedeltas):    # Note that the modulus operator was only implemented for np.timedelta64    # arrays as of NumPy version 1.16.0.  Once our minimum version of NumPy    # supported is greater than or equal to this we will no longer need to cast    # unique_timedeltas to a TimedeltaIndex.  In the meantime, however, the    # modulus operator works for TimedeltaIndex objects.    unique_deltas_as_index = pd.TimedeltaIndex(unique_timedeltas)    for time_unit in [        "days",        "hours",        "minutes",        "seconds",        "milliseconds",        "microseconds",        "nanoseconds",    ]:        delta_ns = _NS_PER_TIME_DELTA[_netcdf_to_numpy_timeunit(time_unit)]        unit_delta = np.timedelta64(delta_ns, "ns")        if np.all(unique_deltas_as_index % unit_delta == np.timedelta64(0, "ns")):    if unique_timedeltas.dtype == np.dtype("O"):        time_units = _NETCDF_TIME_UNITS_CFTIME        unit_timedelta = _unit_timedelta_cftime        zero_timedelta = timedelta(microseconds=0)        timedeltas = unique_timedeltas    else:        time_units = _NETCDF_TIME_UNITS_NUMPY        unit_timedelta = _unit_timedelta_numpy        zero_timedelta = np.timedelta64(0, "ns")        # Note that the modulus operator was only implemented for np.timedelta64        # arrays as of NumPy version 1.16.0.  Once our minimum version of NumPy        # supported is greater than or equal to this we will no longer need to cast        # unique_timedeltas to a TimedeltaIndex.  In the meantime, however, the        # modulus operator works for TimedeltaIndex objects.        timedeltas = pd.TimedeltaIndex(unique_timedeltas)    for time_unit in time_units:        if np.all(timedeltas % unit_timedelta(time_unit) == zero_timedelta):             return time_unit     return "seconds"


Question: return result.reshape(num_timedeltas.shape)  def _unit_timedelta_cftime(units):    return timedelta(microseconds=_US_PER_TIME_DELTA[units])def _unit_timedelta_numpy(units):    numpy_units = _netcdf_to_numpy_timeunit(units)    return np.timedelta64(_NS_PER_TIME_DELTA[numpy_units], "ns") def _infer_time_units_from_diff(unique_timedeltas):    # Note that the modulus operator was only implemented for np.timedelta64    # arrays as of NumPy version 1.16.0.  Once our minimum version of NumPy    # supported is greater than or equal to this we will no longer need to cast    # unique_timedeltas to a TimedeltaIndex.  In the meantime, however, the    # modulus operator works for TimedeltaIndex objects.    unique_deltas_as_index = pd.TimedeltaIndex(unique_timedeltas)    for time_unit in [        "days",        "hours",        "minutes",        "seconds",        "milliseconds",        "microseconds",        "nanoseconds",    ]:        delta_ns = _NS_PER_TIME_DELTA[_netcdf_to_numpy_timeunit(time_unit)]        unit_delta = np.timedelta64(delta_ns, "ns")        if np.all(unique_deltas_as_index % unit_delta == np.timedelta64(0, "ns")):    if unique_timedeltas.dtype == np.dtype("O"):        time_units = _NETCDF_TIME_UNITS_CFTIME        unit_timedelta = _unit_timedelta_cftime        zero_timedelta = timedelta(microseconds=0)        timedeltas = unique_timedeltas    else:        time_units = _NETCDF_TIME_UNITS_NUMPY        unit_timedelta = _unit_timedelta_numpy        zero_timedelta = np.timedelta64(0, "ns")        # Note that the modulus operator was only implemented for np.timedelta64        # arrays as of NumPy version 1.16.0.  Once our minimum version of NumPy        # supported is greater than or equal to this we will no longer need to cast        # unique_timedeltas to a TimedeltaIndex.  In the meantime, however, the        # modulus operator works for TimedeltaIndex objects.        timedeltas = pd.TimedeltaIndex(unique_timedeltas)    for time_unit in time_units:        if np.all(timedeltas % unit_timedelta(time_unit) == zero_timedelta):             return time_unit     return "seconds"


Question: reference_date = dates[0] if len(dates) > 0 else "1970-01-01"         reference_date = format_cftime_datetime(reference_date)     unique_timedeltas = np.unique(np.diff(dates))<del>    if unique_timedeltas.dtype == np.dtype("O"):        # Convert to np.timedelta64 objects using pandas to work around a        # NumPy casting bug: https://github.com/numpy/numpy/issues/11096        unique_timedeltas = to_timedelta_unboxed(unique_timedeltas)</del>     units = _infer_time_units_from_diff(unique_timedeltas)     return f"{units} since {reference_date}"


Question: reference_date = dates[0] if len(dates) > 0 else "1970-01-01"         reference_date = format_cftime_datetime(reference_date)     unique_timedeltas = np.unique(np.diff(dates))<del>    if unique_timedeltas.dtype == np.dtype("O"):        # Convert to np.timedelta64 objects using pandas to work around a        # NumPy casting bug: https://github.com/numpy/numpy/issues/11096        unique_timedeltas = to_timedelta_unboxed(unique_timedeltas)</del>     units = _infer_time_units_from_diff(unique_timedeltas)     return f"{units} since {reference_date}"


Question: # unstacked dataset         return Dataset(data_dict)     def transpose(self, *dims: Hashable, transpose_coords: bool = True) -> "DataArray":    def transpose(        self,        *dims: Hashable,        transpose_coords: bool = True,        missing_dims: str = "raise",    ) -> "DataArray":         """Return a new DataArray object with transposed dimensions.          Parameters

DataArray transpose inconsistent with Dataset Ellipsis usage This works: This doesn't (subset air): The error message is a bit inaccurate too since I do have Ellipsis included; might be related to two calls of:


Question: dimensions to this order.         transpose_coords : bool, default: True             If True, also transpose the coordinates of this DataArray.<ins>        missing_dims : {"raise", "warn", "ignore"}, default: "raise"            What to do if dimensions that should be selected from are not present in the            DataArray:            - "raise": raise an exception            - "warning": raise a warning, and ignore the missing dimensions            - "ignore": ignore the missing dimensions</ins>          Returns         -------


Question: Dataset.transpose         """         if dims:            dims = tuple(utils.infix_dims(dims, self.dims))            dims = tuple(utils.infix_dims(dims, self.dims, missing_dims))         variable = self.variable.transpose(*dims)         if transpose_coords:             coords: Dict[Hashable, Variable] = {}diff --git a/xarray/core/utils.py b/xarray/core/utils.py-- a/xarray/core/utils.py++ b/xarray/core/utils.py


Question: return len(self._data) - num_hidden  def infix_dims(dims_supplied: Collection, dims_all: Collection) -> Iterator:def infix_dims(    dims_supplied: Collection, dims_all: Collection, missing_dims: str = "raise") -> Iterator:     """    Resolves a supplied list containing an ellispsis representing other items, to    Resolves a supplied list containing an ellipsis representing other items, to     a generator with the 'realized' list of all items     """     if ... in dims_supplied:         if len(set(dims_all)) != len(dims_all):             raise ValueError("Cannot use ellipsis with repeated dims")        if len([d for d in dims_supplied if d == ...]) > 1:        if list(dims_supplied).count(...) > 1:             raise ValueError("More than one ellipsis supplied")         other_dims = [d for d in dims_all if d not in dims_supplied]        for d in dims_supplied:            if d == ...:        existing_dims = drop_missing_dims(dims_supplied, dims_all, missing_dims)        for d in existing_dims:            if d is ...:                 yield from other_dims             else:                 yield d     else:        if set(dims_supplied) ^ set(dims_all):        existing_dims = drop_missing_dims(dims_supplied, dims_all, missing_dims)        if set(existing_dims) ^ set(dims_all):             raise ValueError(                 f"{dims_supplied} must be a permuted list of {dims_all}, unless `...` is included"             )        yield from dims_supplied        yield from existing_dims   def get_temp_dimname(dims: Container[Hashable], new_dim: Hashable) -> Hashable:


Question: invalid = indexers.keys() - set(dims)         if invalid:             raise ValueError(                f"dimensions {invalid} do not exist. Expected one or more of {dims}"                f"Dimensions {invalid} do not exist. Expected one or more of {dims}"             )          return indexers


Question: invalid = indexers.keys() - set(dims)         if invalid:             warnings.warn(                f"dimensions {invalid} do not exist. Expected one or more of {dims}"                f"Dimensions {invalid} do not exist. Expected one or more of {dims}"             )         for key in invalid:             indexers.pop(key)


Question: )  <ins>def drop_missing_dims(    supplied_dims: Collection, dims: Collection, missing_dims: str) -> Collection:    """Depending on the setting of missing_dims, drop any dimensions from supplied_dims that    are not present in dims.    Parameters    ----------    supplied_dims : dict    dims : sequence    missing_dims : {"raise", "warn", "ignore"}    """    if missing_dims == "raise":        supplied_dims_set = set(val for val in supplied_dims if val is not ...)        invalid = supplied_dims_set - set(dims)        if invalid:            raise ValueError(                f"Dimensions {invalid} do not exist. Expected one or more of {dims}"            )        return supplied_dims    elif missing_dims == "warn":        invalid = set(supplied_dims) - set(dims)        if invalid:            warnings.warn(                f"Dimensions {invalid} do not exist. Expected one or more of {dims}"            )        return [val for val in supplied_dims if val in dims or val is ...]    elif missing_dims == "ignore":        return [val for val in supplied_dims if val in dims or val is ...]    else:        raise ValueError(            f"Unrecognised option {missing_dims} for missing_dims argument"        )</ins> class UncachedAccessor:     """Acts like a property, but on both classes and class instances


Question: add_offset = pop_to(attrs, encoding, "add_offset", name=name)             dtype = _choose_float_dtype(data.dtype, "add_offset" in attrs)             if np.ndim(scale_factor) > 0:                scale_factor = scale_factor.item()                scale_factor = np.asarray(scale_factor).item()             if np.ndim(add_offset) > 0:                add_offset = add_offset.item()                add_offset = np.asarray(add_offset).item()             transform = partial(                 _scale_offset_decoding,                 scale_factor=scale_factor,

Decodecf fails when scalefactor is a length-1 list Some datasets I work with have and encoded as length-1 lists. The following code worked as of Xarray 0.16.1 In 0.16.2 (just released) and current master, it fails with this error I'm very confused, because this feels quite similar to , and I thought it was resolved . However, the behavior is different with . That works fine--no error. How might I end up with a dataset with as a python list? It happens when I open a netcdf file using the engine (documented by in ) and then write it to zarr. The numpy array gets encoded as a list in the zarr json metadata. This problem would go away if we could resolve the discrepancies between the two engines' treatment of scalar attributes.


Question: ds = self._to_temp_dataset().drop_sel(labels, errors=errors)         return self._from_temp_dataset(ds)     def drop_isel(self, indexers=None, **indexers_kwargs):        """Drop index positions from this DataArray.        Parameters        ----------        indexers : mapping of hashable to Any            Index locations to drop        **indexers_kwargs : {dim: position, ...}, optional            The keyword arguments form of ``dim`` and ``positions``        Returns        -------        dropped : DataArray        Raises        ------        IndexError        """        dataset = self._to_temp_dataset()        dataset = dataset.drop_isel(indexers=indexers, **indexers_kwargs)        return self._from_temp_dataset(dataset)     def dropna(         self, dim: Hashable, how: str = "any", thresh: int = None     ) -> "DataArray":diff --git a/xarray/core/dataset.py b/xarray/core/dataset.py-- a/xarray/core/dataset.py++ b/xarray/core/dataset.py

dropsel indices in dimension that doesn't have coordinates? <!-- Please do a quick search of existing issues to make sure that this has not been asked before. --Is your feature request related to a problem? Please describe. I am trying to drop particular indices from a dimension that doesn't have coordinates. Following: , but leaving out the coordinate labels: gives me an error. Describe the solution you'd like I would think should exist and work in analogy to as does to . Describe alternatives you've considered As far as I know, I could either create coordinates especially to in order to drop, or rebuild a new dataset. Both are not congenial. (I'd be grateful to know if there is actually a straightforward way to do this I've overlooked.


Question: labels_for_dim = [labels_for_dim]             labels_for_dim = np.asarray(labels_for_dim)             try:                index = self.indexes[dim]                index = self.get_index(dim)             except KeyError:                 raise ValueError("dimension %r does not have coordinate labels" % dim)             new_index = index.drop(labels_for_dim, errors=errors)             ds = ds.loc[{dim: new_index}]         return ds     def drop_isel(self, indexers=None, **indexers_kwargs):        """Drop index positions from this Dataset.        Parameters        ----------        indexers : mapping of hashable to Any            Index locations to drop        **indexers_kwargs : {dim: position, ...}, optional            The keyword arguments form of ``dim`` and ``positions``        Returns        -------        dropped : Dataset        Raises        ------        IndexError        Examples        --------        >>> data = np.arange(6).reshape(2, 3)        >>> labels = ["a", "b", "c"]        >>> ds = xr.Dataset({"A": (["x", "y"], data), "y": labels})        >>> ds        <xarray.Dataset>        Dimensions:  (x: 2, y: 3)        Coordinates:          * y        (y) <U1 'a' 'b' 'c'        Dimensions without coordinates: x        Data variables:            A        (x, y) int64 0 1 2 3 4 5        >>> ds.drop_isel(y=[0, 2])        <xarray.Dataset>        Dimensions:  (x: 2, y: 1)        Coordinates:          * y        (y) <U1 'b'        Dimensions without coordinates: x        Data variables:            A        (x, y) int64 1 4        >>> ds.drop_isel(y=1)        <xarray.Dataset>        Dimensions:  (x: 2, y: 2)        Coordinates:          * y        (y) <U1 'a' 'c'        Dimensions without coordinates: x        Data variables:            A        (x, y) int64 0 2 3 5        """        indexers = either_dict_or_kwargs(indexers, indexers_kwargs, "drop")        ds = self        dimension_index = {}        for dim, pos_for_dim in indexers.items():            # Don't cast to set, as it would harm performance when labels            # is a large numpy array            if utils.is_scalar(pos_for_dim):                pos_for_dim = [pos_for_dim]            pos_for_dim = np.asarray(pos_for_dim)            index = self.get_index(dim)            new_index = index.delete(pos_for_dim)            dimension_index[dim] = new_index        ds = ds.loc[dimension_index]        return ds     def drop_dims(         self, drop_dims: Union[Hashable, Iterable[Hashable]], *, errors: str = "raise"     ) -> "Dataset":