Question: Very poor html repr performance on large multi-indexes <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: We have catestrophic performance on the html repr of some long multi-indexed data arrays. Here's a case of it taking 12s. Minimal Complete Verifiable Example: Anything else we need to know?: I thought we'd fixed some issues here: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.8.10 (default, May 9 2021, 13:21:55) [Clang 12.0.5 (clang-1205.0.22.9)] python-bits: 64 OS: Darwin OS-release: 20.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: None LOCALE: ('enUS', 'UTF-8') libhdf5: None libnetcdf: None xarray: 0.18.2 pandas: 1.2.4 numpy: 1.20.3 scipy: 1.6.3 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.8.3 cftime: 1.4.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.2.3 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2021.06.1 distributed: 2021.06.1 matplotlib: 3.4.2 cartopy: None seaborn: 0.11.1 numbagg: 0.2.1 pint: None setuptools: 56.0.0 pip: 21.1.2 conda: None pytest: 6.2.4 IPython: 7.24.0 sphinx: 4.0.1 </details

from contextlib import suppress from dataclasses import dataclass, field from datetime import timedelta<ins>from html import escape</ins> from typing import (     TYPE_CHECKING,     Any,


Question: Very poor html repr performance on large multi-indexes <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: We have catestrophic performance on the html repr of some long multi-indexed data arrays. Here's a case of it taking 12s. Minimal Complete Verifiable Example: Anything else we need to know?: I thought we'd fixed some issues here: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.8.10 (default, May 9 2021, 13:21:55) [Clang 12.0.5 (clang-1205.0.22.9)] python-bits: 64 OS: Darwin OS-release: 20.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: None LOCALE: ('enUS', 'UTF-8') libhdf5: None libnetcdf: None xarray: 0.18.2 pandas: 1.2.4 numpy: 1.20.3 scipy: 1.6.3 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.8.3 cftime: 1.4.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.2.3 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2021.06.1 distributed: 2021.06.1 matplotlib: 3.4.2 cartopy: None seaborn: 0.11.1 numbagg: 0.2.1 pint: None setuptools: 56.0.0 pip: 21.1.2 conda: None pytest: 6.2.4 IPython: 7.24.0 sphinx: 4.0.1 </details

from . import duck_array_ops, nputils, utils from .npcompat import DTypeLike<ins>from .options import OPTIONS</ins> from .pycompat import dask_version, integer_types, is_duck_dask_array, sparse_array_type from .types import T_Xarray from .utils import either_dict_or_kwargs, get_valid_numpy_dtype


Question: Very poor html repr performance on large multi-indexes <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: We have catestrophic performance on the html repr of some long multi-indexed data arrays. Here's a case of it taking 12s. Minimal Complete Verifiable Example: Anything else we need to know?: I thought we'd fixed some issues here: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.8.10 (default, May 9 2021, 13:21:55) [Clang 12.0.5 (clang-1205.0.22.9)] python-bits: 64 OS: Darwin OS-release: 20.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: None LOCALE: ('enUS', 'UTF-8') libhdf5: None libnetcdf: None xarray: 0.18.2 pandas: 1.2.4 numpy: 1.20.3 scipy: 1.6.3 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.8.3 cftime: 1.4.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.2.3 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2021.06.1 distributed: 2021.06.1 matplotlib: 3.4.2 cartopy: None seaborn: 0.11.1 numbagg: 0.2.1 pint: None setuptools: 56.0.0 pip: 21.1.2 conda: None pytest: 6.2.4 IPython: 7.24.0 sphinx: 4.0.1 </details

)             return f"{type(self).__name__}{props}"     def _repr_inline_(self, max_width) -> str:        # special implementation to speed-up the repr for big multi-indexes    def _get_array_subset(self) -> np.ndarray:        # used to speed-up the repr for big multi-indexes        threshold = max(100, OPTIONS["display_values_threshold"] + 2)        if self.size > threshold:            pos = threshold // 2            indices = np.concatenate([np.arange(0, pos), np.arange(-pos, 0)])            subset = self[OuterIndexer((indices,))]        else:            subset = self        return np.asarray(subset)    def _repr_inline_(self, max_width: int) -> str:        from .formatting import format_array_flat         if self.level is None:             return "MultiIndex"         else:            from .formatting import format_array_flat            return format_array_flat(self._get_array_subset(), max_width)             if self.size > 100 and max_width < self.size:                n_values = max_width                indices = np.concatenate(                    [np.arange(0, n_values), np.arange(-n_values, 0)]                )                subset = self[OuterIndexer((indices,))]            else:                subset = self    def _repr_html_(self) -> str:        from .formatting import short_numpy_repr             return format_array_flat(np.asarray(subset), max_width)        array_repr = short_numpy_repr(self._get_array_subset())        return f"
{escape(array_repr)}
"      def copy(self, deep: bool = True) -> "PandasMultiIndexingAdapter":         # see PandasIndexingAdapter.copydiff --git a/xarray/core/options.py b/xarray/core/options.py-- a/xarray/core/options.py++ b/xarray/core/options.py


Question: Very poor html repr performance on large multi-indexes <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: We have catestrophic performance on the html repr of some long multi-indexed data arrays. Here's a case of it taking 12s. Minimal Complete Verifiable Example: Anything else we need to know?: I thought we'd fixed some issues here: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.8.10 (default, May 9 2021, 13:21:55) [Clang 12.0.5 (clang-1205.0.22.9)] python-bits: 64 OS: Darwin OS-release: 20.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: None LOCALE: ('enUS', 'UTF-8') libhdf5: None libnetcdf: None xarray: 0.18.2 pandas: 1.2.4 numpy: 1.20.3 scipy: 1.6.3 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.8.3 cftime: 1.4.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.2.3 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2021.06.1 distributed: 2021.06.1 matplotlib: 3.4.2 cartopy: None seaborn: 0.11.1 numbagg: 0.2.1 pint: None setuptools: 56.0.0 pip: 21.1.2 conda: None pytest: 6.2.4 IPython: 7.24.0 sphinx: 4.0.1 </details

cmap_divergent: Union[str, "Colormap"]     cmap_sequential: Union[str, "Colormap"]     display_max_rows: int<ins>    display_values_threshold: int</ins>     display_style: Literal["text", "html"]     display_width: int     display_expand_attrs: Literal["default", True, False]


Question: Very poor html repr performance on large multi-indexes <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: We have catestrophic performance on the html repr of some long multi-indexed data arrays. Here's a case of it taking 12s. Minimal Complete Verifiable Example: Anything else we need to know?: I thought we'd fixed some issues here: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.8.10 (default, May 9 2021, 13:21:55) [Clang 12.0.5 (clang-1205.0.22.9)] python-bits: 64 OS: Darwin OS-release: 20.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: None LOCALE: ('enUS', 'UTF-8') libhdf5: None libnetcdf: None xarray: 0.18.2 pandas: 1.2.4 numpy: 1.20.3 scipy: 1.6.3 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.8.3 cftime: 1.4.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.2.3 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2021.06.1 distributed: 2021.06.1 matplotlib: 3.4.2 cartopy: None seaborn: 0.11.1 numbagg: 0.2.1 pint: None setuptools: 56.0.0 pip: 21.1.2 conda: None pytest: 6.2.4 IPython: 7.24.0 sphinx: 4.0.1 </details

"cmap_divergent": "RdBu_r",     "cmap_sequential": "viridis",     "display_max_rows": 12,<ins>    "display_values_threshold": 200,</ins>     "display_style": "html",     "display_width": 80,     "display_expand_attrs": "default",


Question: Very poor html repr performance on large multi-indexes <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: We have catestrophic performance on the html repr of some long multi-indexed data arrays. Here's a case of it taking 12s. Minimal Complete Verifiable Example: Anything else we need to know?: I thought we'd fixed some issues here: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.8.10 (default, May 9 2021, 13:21:55) [Clang 12.0.5 (clang-1205.0.22.9)] python-bits: 64 OS: Darwin OS-release: 20.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: None LOCALE: ('enUS', 'UTF-8') libhdf5: None libnetcdf: None xarray: 0.18.2 pandas: 1.2.4 numpy: 1.20.3 scipy: 1.6.3 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.8.3 cftime: 1.4.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.2.3 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2021.06.1 distributed: 2021.06.1 matplotlib: 3.4.2 cartopy: None seaborn: 0.11.1 numbagg: 0.2.1 pint: None setuptools: 56.0.0 pip: 21.1.2 conda: None pytest: 6.2.4 IPython: 7.24.0 sphinx: 4.0.1 </details

_VALIDATORS = {     "arithmetic_join": _JOIN_OPTIONS.__contains__,     "display_max_rows": _positive_integer,<ins>    "display_values_threshold": _positive_integer,</ins>     "display_style": _DISPLAY_OPTIONS.__contains__,     "display_width": _positive_integer,     "display_expand_attrs": lambda choice: choice in [True, False, "default"],


Question: Very poor html repr performance on large multi-indexes <!-- Please include a self-contained copy-pastable example that generates the issue if possible. Please be concise with code posted. See guidelines below on how to provide a good bug report: Craft Minimal Bug Reports: Minimal Complete Verifiable Examples: Bug reports that follow these guidelines are easier to diagnose, and so are often handled much more quickly. --What happened: We have catestrophic performance on the html repr of some long multi-indexed data arrays. Here's a case of it taking 12s. Minimal Complete Verifiable Example: Anything else we need to know?: I thought we'd fixed some issues here: Environment: <details<summaryOutput of <ttxr.showversions()</tt</summaryINSTALLED VERSIONS commit: None python: 3.8.10 (default, May 9 2021, 13:21:55) [Clang 12.0.5 (clang-1205.0.22.9)] python-bits: 64 OS: Darwin OS-release: 20.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: None LOCALE: ('enUS', 'UTF-8') libhdf5: None libnetcdf: None xarray: 0.18.2 pandas: 1.2.4 numpy: 1.20.3 scipy: 1.6.3 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.8.3 cftime: 1.4.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.2.3 cfgrib: None iris: None bottleneck: 1.3.2 dask: 2021.06.1 distributed: 2021.06.1 matplotlib: 3.4.2 cartopy: None seaborn: 0.11.1 numbagg: 0.2.1 pint: None setuptools: 56.0.0 pip: 21.1.2 conda: None pytest: 6.2.4 IPython: 7.24.0 sphinx: 4.0.1 </details

* ``default`` : to expand unless over a pre-defined limit     display_max_rows : int, default: 12         Maximum display rows.<ins>    display_values_threshold : int, default: 200        Total number of array elements which trigger summarization rather        than full repr for variable data views (numpy arrays).</ins>     display_style : {"text", "html"}, default: "html"         Display style to use in jupyter for xarray objects.     display_width : int, default: 80


Question: with scalar as second argument fails with keepattrs=True fails with The workaround is to pass No response No response No response No response xarray 2022.3.0

"""     if keep_attrs is None:         keep_attrs = _get_keep_attrs(default=False)     if keep_attrs is True:         # keep the attributes of x, the second parameter, by default to         # be consistent with the `where` method of `DataArray` and `Dataset`        keep_attrs = lambda attrs, context: attrs[1]        keep_attrs = lambda attrs, context: getattr(x, "attrs", {})      # alignment for three arguments is complicated, so don't support it yet     return apply_ufunc(


Question: xr.polyval first arg requires name attribute I have some polynomial coefficients and want to evaluate them at some values using . As described in the docstring/docu I created a 1D coordinate DataArray and pass it to but it raises a KeyError (see example). I expected that the polynomial would be evaluated at the given points. No response I assume that the "standard" workflow is to obtain the argument from an existing DataArrays coordinate, where the name would be correctly set already. However, that is not clear from the description, and also prevents my "manual" workflow. It could be that the problem will be solved by replacing the coord DataArray argument by an explicit Index in the future. <detailsINSTALLED VERSIONS commit: None python: 3.9.10 (main, Mar 15 2022, 15:56:56) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.49.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4 xarray: 2022.3.0 pandas: 1.4.2 numpy: 1.22.3 scipy: None netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.1 cartopy: 0.20.2 seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None setuptools: 58.1.0 pip: 22.0.4 conda: None pytest: None IPython: 8.2.0 sphinx: None </details

import numpy as npimport xarray as xrfrom . import parameterized, randn, requires_daskNDEGS = (2, 5, 20)NX = (10**2, 10**6)class Polyval:    def setup(self, *args, **kwargs):        self.xs = {nx: xr.DataArray(randn((nx,)), dims="x", name="x") for nx in NX}        self.coeffs = {            ndeg: xr.DataArray(                randn((ndeg,)), dims="degree", coords={"degree": np.arange(ndeg)}            )            for ndeg in NDEGS        }    @parameterized(["nx", "ndeg"], [NX, NDEGS])    def time_polyval(self, nx, ndeg):        x = self.xs[nx]        c = self.coeffs[ndeg]        xr.polyval(x, c).compute()    @parameterized(["nx", "ndeg"], [NX, NDEGS])    def peakmem_polyval(self, nx, ndeg):        x = self.xs[nx]        c = self.coeffs[ndeg]        xr.polyval(x, c).compute()class PolyvalDask(Polyval):    def setup(self, *args, **kwargs):        requires_dask()        super().setup(*args, **kwargs)        self.xs = {k: v.chunk({"x": 10000}) for k, v in self.xs.items()}diff --git a/xarray/core/computation.py b/xarray/core/computation.py-- a/xarray/core/computation.py++ b/xarray/core/computation.py


Question: xr.polyval first arg requires name attribute I have some polynomial coefficients and want to evaluate them at some values using . As described in the docstring/docu I created a 1D coordinate DataArray and pass it to but it raises a KeyError (see example). I expected that the polynomial would be evaluated at the given points. No response I assume that the "standard" workflow is to obtain the argument from an existing DataArrays coordinate, where the name would be correctly set already. However, that is not clear from the description, and also prevents my "manual" workflow. It could be that the problem will be solved by replacing the coord DataArray argument by an explicit Index in the future. <detailsINSTALLED VERSIONS commit: None python: 3.9.10 (main, Mar 15 2022, 15:56:56) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.49.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4 xarray: 2022.3.0 pandas: 1.4.2 numpy: 1.22.3 scipy: None netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.1 cartopy: 0.20.2 seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None setuptools: 58.1.0 pip: 22.0.4 conda: None pytest: None IPython: 8.2.0 sphinx: None </details

Iterable,     Mapping,     Sequence,    overload, )  import numpy as np  from . import dtypes, duck_array_ops, utils from .alignment import align, deep_alignfrom .common import zeros_likefrom .duck_array_ops import datetime_to_numeric from .indexes import Index, filter_indexes_from_coords from .merge import merge_attrs, merge_coordinates_without_align from .options import OPTIONS, _get_keep_attrs


Question: xr.polyval first arg requires name attribute I have some polynomial coefficients and want to evaluate them at some values using . As described in the docstring/docu I created a 1D coordinate DataArray and pass it to but it raises a KeyError (see example). I expected that the polynomial would be evaluated at the given points. No response I assume that the "standard" workflow is to obtain the argument from an existing DataArrays coordinate, where the name would be correctly set already. However, that is not clear from the description, and also prevents my "manual" workflow. It could be that the problem will be solved by replacing the coord DataArray argument by an explicit Index in the future. <detailsINSTALLED VERSIONS commit: None python: 3.9.10 (main, Mar 15 2022, 15:56:56) [GCC 7.5.0] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.49.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4 xarray: 2022.3.0 pandas: 1.4.2 numpy: 1.22.3 scipy: None netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.1 cartopy: 0.20.2 seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None setuptools: 58.1.0 pip: 22.0.4 conda: None pytest: None IPython: 8.2.0 sphinx: None </details

)  def polyval(coord, coeffs, degree_dim="degree"):@overloaddef polyval(coord: DataArray, coeffs: DataArray, degree_dim: Hashable) -> DataArray:    ...@overloaddef polyval(coord: T_Xarray, coeffs: Dataset, degree_dim: Hashable) -> Dataset:    ...@overloaddef polyval(coord: Dataset, coeffs: T_Xarray, degree_dim: Hashable) -> Dataset:    ...def polyval(    coord: T_Xarray, coeffs: T_Xarray, degree_dim: Hashable = "degree") -> T_Xarray:     """Evaluate a polynomial at specific values      Parameters     ----------    coord : DataArray        The 1D coordinate along which to evaluate the polynomial.    coeffs : DataArray        Coefficients of the polynomials.    degree_dim : str, default: "degree"    coord : DataArray or Dataset        Values at which to evaluate the polynomial.    coeffs : DataArray or Dataset        Coefficients of the polynomial.    degree_dim : Hashable, default: "degree"         Name of the polynomial degree dimension in `coeffs`.     Returns    -------    DataArray or Dataset        Evaluated polynomial.     See Also     --------     xarray.DataArray.polyfit    numpy.polyval    numpy.polynomial.polynomial.polyval     """    from .dataarray import DataArray    from .missing import get_clean_interp_index     x = get_clean_interp_index(coord, coord.name, strict=False)    if degree_dim not in coeffs._indexes:        raise ValueError(            f"Dimension `{degree_dim}` should be a coordinate variable with labels."        )    if not np.issubdtype(coeffs[degree_dim].dtype, int):        raise ValueError(            f"Dimension `{degree_dim}` should be of integer dtype. Received {coeffs[degree_dim].dtype} instead."        )    max_deg = coeffs[degree_dim].max().item()    coeffs = coeffs.reindex(        {degree_dim: np.arange(max_deg + 1)}, fill_value=0, copy=False    )    coord = _ensure_numeric(coord)    # using Horner's method    # https://en.wikipedia.org/wiki/Horner%27s_method    res = coeffs.isel({degree_dim: max_deg}, drop=True) + zeros_like(coord)    for deg in range(max_deg - 1, -1, -1):        res *= coord        res += coeffs.isel({degree_dim: deg}, drop=True)     deg_coord = coeffs[degree_dim]    return res     lhs = DataArray(        np.vander(x, int(deg_coord.max()) + 1),        dims=(coord.name, degree_dim),        coords={coord.name: coord, degree_dim: np.arange(deg_coord.max() + 1)[::-1]},    )    return (lhs * coeffs).sum(degree_dim)def _ensure_numeric(data: T_Xarray) -> T_Xarray:    """Converts all datetime64 variables to float64    Parameters    ----------    data : DataArray or Dataset        Variables with possible datetime dtypes.    Returns    -------    DataArray or Dataset        Variables with datetime64 dtypes converted to float64.    """    from .dataset import Dataset    def to_floatable(x: DataArray) -> DataArray:        if x.dtype.kind in "mM":            return x.copy(                data=datetime_to_numeric(                    x.data,                    offset=np.datetime64("1970-01-01"),                    datetime_unit="ns",                ),            )        return x    if isinstance(data, Dataset):        return data.map(to_floatable)    else:        return to_floatable(data)   def _calc_idxminmax(


Question: xarray improperly decodes times from a NetCDF when it is a uint improperly decodes times from a NetCDF when it is a . The generates a NetCDF file with the right time ('goodtime') and the wrong time ('time') (use ) to be properly decoded (see ). [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response <detailsINSTALLED VERSIONS commit: None python: 3.8.13 (default, Mar 25 2022, 06:04:10) [GCC 10.3.0] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.62.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.10.6 libnetcdf: 4.8.0 xarray: 2022.3.0 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.7.0 netCDF4: 1.5.7 pydap: None h5netcdf: 1.0.0 h5py: 3.3.0 Nio: None zarr: 2.11.3 cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: 0.9.10.1 iris: None bottleneck: 1.3.4 dask: 2022.04.1 distributed: 2022.4.1 matplotlib: 3.5.1 cartopy: None seaborn: 0.11.2 numbagg: None fsspec: 2021.06.1 cupy: None pint: 0.19.1 sparse: None setuptools: 62.1.0 pip: 22.0.4 conda: 4.12.0 pytest: 7.1.1 IPython: 8.2.0 sphinx: 4.5.0 </details

pd.to_timedelta(flat_num_dates.max(), delta) + ref_date      # To avoid integer overflow when converting to nanosecond units for integer    # dtypes smaller than np.int64 cast all integer-dtype arrays to np.int64    # (GH 2002).    if flat_num_dates.dtype.kind == "i":    # dtypes smaller than np.int64 cast all integer and unsigned integer dtype    # arrays to np.int64 (GH 2002, GH 6589).  Note this is safe even in the case    # of np.uint64 values, because any np.uint64 value that would lead to    # overflow when converting to np.int64 would not be representable with a    # timedelta64 value, and therefore would raise an error in the lines above.    if flat_num_dates.dtype.kind in "iu":         flat_num_dates = flat_num_dates.astype(np.int64)      # Cast input ordinals to integers of nanoseconds because pd.to_timedelta


Question: with timedelta64 coordinates produces wrong results I'm not sure if this is a bug or an expected breaking change, but I'm not able to reproduce the results generated by using a timedelta64 coordinate. The results are correct in , whereas they are wrong in the latest unreleased version (, ). Both the stable and latest functions should return the same results. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response <detailsINSTALLED VERSIONS commit: None python: 3.10.4 (main, Mar 24 2022, 17:43:32) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: None LOCALE: (None, 'UTF-8') libhdf5: None libnetcdf: None xarray: 2022.3.0 or 2022.3.1.dev102+g6bb2b855 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.8.0 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: None nctime_axis: None PseudoNetCDF: None rasterio: 1.2.10 cfgrib: None iris: None bottleneck: None dask: 2022.05.0 distributed: 2022.5.0 matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.2.0 pip: 22.1 conda: None pytest: 7.1.2 IPython: None sphinx: None </details

from .dataset import Dataset      def to_floatable(x: DataArray) -> DataArray:        if x.dtype.kind in "mM":        if x.dtype.kind == "M":            # datetimes             return x.copy(                 data=datetime_to_numeric(                     x.data,


Question: with timedelta64 coordinates produces wrong results I'm not sure if this is a bug or an expected breaking change, but I'm not able to reproduce the results generated by using a timedelta64 coordinate. The results are correct in , whereas they are wrong in the latest unreleased version (, ). Both the stable and latest functions should return the same results. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response <detailsINSTALLED VERSIONS commit: None python: 3.10.4 (main, Mar 24 2022, 17:43:32) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.4.0 machine: x8664 processor: i386 byteorder: little LCALL: None LANG: None LOCALE: (None, 'UTF-8') libhdf5: None libnetcdf: None xarray: 2022.3.0 or 2022.3.1.dev102+g6bb2b855 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.8.0 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: None nctime_axis: None PseudoNetCDF: None rasterio: 1.2.10 cfgrib: None iris: None bottleneck: None dask: 2022.05.0 distributed: 2022.5.0 matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.2.0 pip: 22.1 conda: None pytest: 7.1.2 IPython: None sphinx: None </details

datetime_unit="ns",                 ),             )<ins>        elif x.dtype.kind == "m":            # timedeltas            return x.astype(float)</ins>         return x      if isinstance(data, Dataset):


Question: returns objects with different dimension order I noticed that the dimension order of the object returned by the latest (, unreleased) is different compared to . For example, the following code returns different results. Is this the expected behaviour? If yes, is it worth mentioning this change in what's new/breaking changes? cc:

# using Horner's method     # https://en.wikipedia.org/wiki/Horner%27s_method    res = coeffs.isel({degree_dim: max_deg}, drop=True) + zeros_like(coord)    res = zeros_like(coord) + coeffs.isel({degree_dim: max_deg}, drop=True)     for deg in range(max_deg - 1, -1, -1):         res *= coord         res += coeffs.isel({degree_dim: deg}, drop=True)


Question: Accessing chunks on zarr backed xarray seems to load entire array into memory When running the following example it appears the entire dataset is loaded into memory when accessing the attribute: According to accessing the chunks attribute should simply inspect the attribute on the underlying DataArrays. No response No response <detailsINSTALLED VERSIONS commit: None python: 3.9.12 (main, Mar 24 2022, 23:24:38) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.2.0 machine: arm64 processor: arm byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: None libnetcdf: None xarray: 2022.3.0 pandas: 1.4.2 numpy: 1.21.2 scipy: 1.8.0 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.8.1 cftime: None nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.4 dask: 2022.04.0 distributed: 2022.4.0 matplotlib: 3.4.3 cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.0.0 pip: 22.0.4 conda: None pytest: 7.1.1 IPython: 8.2.0 sphinx: None </details

chunks: dict[Any, tuple[int, ...]] = {}     for v in variables:        if hasattr(v.data, "chunks"):        if hasattr(v._data, "chunks"):             for dim, c in v.chunksizes.items():                 if dim in chunks and c != chunks[dim]:                     raise ValueError(


Question: "center" kwarg ignored when manually iterating over DataArrayRolling <div type='discussions-op-text'<supOriginally posted by ckingdon95 June 29, 2022</supHello, I am trying to manually iterate over a DataArrayRolling object, as described in the documentation. I am confused why the following two code chunks do not produce the same sequence of values. I would like to be able to manually iterate over a DataArrayRolling object, and still be given center-justified windows. Is there a way to do this? This returns the following values, as expected: Whereas when I do it manually, it is not equivalent: This returns Is this an issue with the window iterator? If it is not an issue, then is there a way for me to get the center-justified windows in the manual iteration? </div

# TODO legacy attribute         self.window_labels = self.obj[self.dim[0]]     def __iter__(self) -> Iterator[tuple[RollingKey, DataArray]]:    def __iter__(self) -> Iterator[tuple[DataArray, DataArray]]:         if self.ndim > 1:             raise ValueError("__iter__ is only supported for 1d-rolling")        stops = np.arange(1, len(self.window_labels) + 1)        starts = stops - int(self.window[0])        starts[: int(self.window[0])] = 0        dim0 = self.dim[0]        window0 = int(self.window[0])        offset = (window0 + 1) // 2 if self.center[0] else 1        stops = np.arange(offset, self.obj.sizes[dim0] + offset)        starts = stops - window0        starts[: window0 - offset] = 0         for (label, start, stop) in zip(self.window_labels, starts, stops):            window = self.obj.isel({self.dim[0]: slice(start, stop)})            window = self.obj.isel({dim0: slice(start, stop)})             counts = window.count(dim=self.dim[0])            counts = window.count(dim=dim0)             window = window.where(counts >= self.min_periods)              yield (label, window)


Question: Dropping a MultiIndex variable raises an error after explicit indexes refactor With the latest released version of Xarray, it is possible to delete all variables corresponding to a MultiIndex by simply deleting the name of the MultiIndex. After the explicit indexes refactor (i.e,. using the "main" development branch) this now raises error about how this would "corrupt" index state. This comes up when using and and possibly some other methods. This is not hard to work around, but we may want to consider this bug a blocker for the next Xarray release. I found the issue surfaced in several projects when attempting to use the new version of Xarray inside Google's codebase. CC in case you have any thoughts to share. For now, we should preserve the behavior of deleting the variables corresponding to MultiIndex levels, but should issue a deprecation warning encouraging users to explicitly delete everything. No response <detailsINSTALLED VERSIONS commit: python: 3.10.4 (main, Mar 24 2022, 17:42:03) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.4.0 machine: arm64 processor: arm byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.1 libnetcdf: 4.8.1 xarray: 0.18.3.dev137+g96c56836 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.8.0 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2022.04.1 distributed: 2022.4.1 matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.1.0 pip: 22.0.4 conda: None pytest: 7.1.1 IPython: 8.2.0 sphinx: None </details

from __future__ import annotations <ins>import warnings</ins> from contextlib import contextmanager from typing import TYPE_CHECKING, Any, Hashable, Iterator, Mapping, Sequence, cast


Question: Dropping a MultiIndex variable raises an error after explicit indexes refactor With the latest released version of Xarray, it is possible to delete all variables corresponding to a MultiIndex by simply deleting the name of the MultiIndex. After the explicit indexes refactor (i.e,. using the "main" development branch) this now raises error about how this would "corrupt" index state. This comes up when using and and possibly some other methods. This is not hard to work around, but we may want to consider this bug a blocker for the next Xarray release. I found the issue surfaced in several projects when attempting to use the new version of Xarray inside Google's codebase. CC in case you have any thoughts to share. For now, we should preserve the behavior of deleting the variables corresponding to MultiIndex levels, but should issue a deprecation warning encouraging users to explicitly delete everything. No response <detailsINSTALLED VERSIONS commit: python: 3.10.4 (main, Mar 24 2022, 17:42:03) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.4.0 machine: arm64 processor: arm byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.1 libnetcdf: 4.8.1 xarray: 0.18.3.dev137+g96c56836 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.8.0 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2022.04.1 distributed: 2022.4.1 matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.1.0 pip: 22.0.4 conda: None pytest: 7.1.1 IPython: 8.2.0 sphinx: None </details

import pandas as pd  from . import formattingfrom .indexes import Index, Indexes, assert_no_index_corruptedfrom .indexes import Index, Indexes, PandasMultiIndex, assert_no_index_corrupted from .merge import merge_coordinates_without_align, merge_coords from .utils import Frozen, ReprObject from .variable import Variable, calculate_dimensions


Question: Dropping a MultiIndex variable raises an error after explicit indexes refactor With the latest released version of Xarray, it is possible to delete all variables corresponding to a MultiIndex by simply deleting the name of the MultiIndex. After the explicit indexes refactor (i.e,. using the "main" development branch) this now raises error about how this would "corrupt" index state. This comes up when using and and possibly some other methods. This is not hard to work around, but we may want to consider this bug a blocker for the next Xarray release. I found the issue surfaced in several projects when attempting to use the new version of Xarray inside Google's codebase. CC in case you have any thoughts to share. For now, we should preserve the behavior of deleting the variables corresponding to MultiIndex levels, but should issue a deprecation warning encouraging users to explicitly delete everything. No response <detailsINSTALLED VERSIONS commit: python: 3.10.4 (main, Mar 24 2022, 17:42:03) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.4.0 machine: arm64 processor: arm byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.1 libnetcdf: 4.8.1 xarray: 0.18.3.dev137+g96c56836 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.8.0 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2022.04.1 distributed: 2022.4.1 matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.1.0 pip: 22.0.4 conda: None pytest: 7.1.1 IPython: 8.2.0 sphinx: None </details

def _update_coords(self, coords, indexes):         raise NotImplementedError() <ins>    def _maybe_drop_multiindex_coords(self, coords):        raise NotImplementedError()</ins>     def __iter__(self) -> Iterator[Hashable]:         # needs to be in the same order as the dataset variables         for k in self.variables:


Question: Dropping a MultiIndex variable raises an error after explicit indexes refactor With the latest released version of Xarray, it is possible to delete all variables corresponding to a MultiIndex by simply deleting the name of the MultiIndex. After the explicit indexes refactor (i.e,. using the "main" development branch) this now raises error about how this would "corrupt" index state. This comes up when using and and possibly some other methods. This is not hard to work around, but we may want to consider this bug a blocker for the next Xarray release. I found the issue surfaced in several projects when attempting to use the new version of Xarray inside Google's codebase. CC in case you have any thoughts to share. For now, we should preserve the behavior of deleting the variables corresponding to MultiIndex levels, but should issue a deprecation warning encouraging users to explicitly delete everything. No response <detailsINSTALLED VERSIONS commit: python: 3.10.4 (main, Mar 24 2022, 17:42:03) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.4.0 machine: arm64 processor: arm byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.1 libnetcdf: 4.8.1 xarray: 0.18.3.dev137+g96c56836 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.8.0 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2022.04.1 distributed: 2022.4.1 matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.1.0 pip: 22.0.4 conda: None pytest: 7.1.1 IPython: 8.2.0 sphinx: None </details

def update(self, other: Mapping[Any, Any]) -> None:         other_vars = getattr(other, "variables", other)<ins>        self._maybe_drop_multiindex_coords(set(other_vars))</ins>         coords, indexes = merge_coords(             [self.variables, other_vars], priority_arg=1, indexes=self.xindexes         )


Question: Dropping a MultiIndex variable raises an error after explicit indexes refactor With the latest released version of Xarray, it is possible to delete all variables corresponding to a MultiIndex by simply deleting the name of the MultiIndex. After the explicit indexes refactor (i.e,. using the "main" development branch) this now raises error about how this would "corrupt" index state. This comes up when using and and possibly some other methods. This is not hard to work around, but we may want to consider this bug a blocker for the next Xarray release. I found the issue surfaced in several projects when attempting to use the new version of Xarray inside Google's codebase. CC in case you have any thoughts to share. For now, we should preserve the behavior of deleting the variables corresponding to MultiIndex levels, but should issue a deprecation warning encouraging users to explicitly delete everything. No response <detailsINSTALLED VERSIONS commit: python: 3.10.4 (main, Mar 24 2022, 17:42:03) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.4.0 machine: arm64 processor: arm byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.1 libnetcdf: 4.8.1 xarray: 0.18.3.dev137+g96c56836 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.8.0 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2022.04.1 distributed: 2022.4.1 matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.1.0 pip: 22.0.4 conda: None pytest: 7.1.1 IPython: 8.2.0 sphinx: None </details

original_indexes.update(indexes)         self._data._indexes = original_indexes <ins>    def _maybe_drop_multiindex_coords(self, coords: set[Hashable]) -> None:        """Drops variables in coords, and any associated variables as well."""        assert self._data.xindexes is not None        variables, indexes = drop_coords(            coords, self._data._variables, self._data.xindexes        )        self._data._variables = variables        self._data._indexes = indexes</ins>     def __delitem__(self, key: Hashable) -> None:         if key in self:             del self._data[key]


Question: Dropping a MultiIndex variable raises an error after explicit indexes refactor With the latest released version of Xarray, it is possible to delete all variables corresponding to a MultiIndex by simply deleting the name of the MultiIndex. After the explicit indexes refactor (i.e,. using the "main" development branch) this now raises error about how this would "corrupt" index state. This comes up when using and and possibly some other methods. This is not hard to work around, but we may want to consider this bug a blocker for the next Xarray release. I found the issue surfaced in several projects when attempting to use the new version of Xarray inside Google's codebase. CC in case you have any thoughts to share. For now, we should preserve the behavior of deleting the variables corresponding to MultiIndex levels, but should issue a deprecation warning encouraging users to explicitly delete everything. No response <detailsINSTALLED VERSIONS commit: python: 3.10.4 (main, Mar 24 2022, 17:42:03) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.4.0 machine: arm64 processor: arm byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.1 libnetcdf: 4.8.1 xarray: 0.18.3.dev137+g96c56836 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.8.0 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2022.04.1 distributed: 2022.4.1 matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.1.0 pip: 22.0.4 conda: None pytest: 7.1.1 IPython: 8.2.0 sphinx: None </details

original_indexes.update(indexes)         self._data._indexes = original_indexes <ins>    def _maybe_drop_multiindex_coords(self, coords: set[Hashable]) -> None:        """Drops variables in coords, and any associated variables as well."""        variables, indexes = drop_coords(            coords, self._data._coords, self._data.xindexes        )        self._data._coords = variables        self._data._indexes = indexes</ins>     @property     def variables(self):         return Frozen(self._data._coords)


Question: Dropping a MultiIndex variable raises an error after explicit indexes refactor With the latest released version of Xarray, it is possible to delete all variables corresponding to a MultiIndex by simply deleting the name of the MultiIndex. After the explicit indexes refactor (i.e,. using the "main" development branch) this now raises error about how this would "corrupt" index state. This comes up when using and and possibly some other methods. This is not hard to work around, but we may want to consider this bug a blocker for the next Xarray release. I found the issue surfaced in several projects when attempting to use the new version of Xarray inside Google's codebase. CC in case you have any thoughts to share. For now, we should preserve the behavior of deleting the variables corresponding to MultiIndex levels, but should issue a deprecation warning encouraging users to explicitly delete everything. No response <detailsINSTALLED VERSIONS commit: python: 3.10.4 (main, Mar 24 2022, 17:42:03) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.4.0 machine: arm64 processor: arm byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.1 libnetcdf: 4.8.1 xarray: 0.18.3.dev137+g96c56836 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.8.0 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2022.04.1 distributed: 2022.4.1 matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.1.0 pip: 22.0.4 conda: None pytest: 7.1.1 IPython: 8.2.0 sphinx: None </details

return self._data._ipython_key_completions_()  def drop_coords(    coords_to_drop: set[Hashable], variables, indexes: Indexes) -> tuple[dict, dict]:    """Drop index variables associated with variables in coords_to_drop."""    # Only warn when we're dropping the dimension with the multi-indexed coordinate    # If asked to drop a subset of the levels in a multi-index, we raise an error    # later but skip the warning here.    new_variables = dict(variables.copy())    new_indexes = dict(indexes.copy())    for key in coords_to_drop & set(indexes):        maybe_midx = indexes[key]        idx_coord_names = set(indexes.get_all_coords(key))        if (            isinstance(maybe_midx, PandasMultiIndex)            and key == maybe_midx.dim            and (idx_coord_names - coords_to_drop)        ):            warnings.warn(                f"Updating MultiIndexed coordinate {key!r} would corrupt indices for "                f"other variables: {list(maybe_midx.index.names)!r}. "                f"This will raise an error in the future. Use `.drop_vars({idx_coord_names!r})` before "                "assigning new coordinate values.",                DeprecationWarning,                stacklevel=4,            )            for k in idx_coord_names:                del new_variables[k]                del new_indexes[k]    return new_variables, new_indexes def assert_coordinate_consistent(     obj: DataArray | Dataset, coords: Mapping[Any, Variable] ) -> None:diff --git a/xarray/core/dataset.py b/xarray/core/dataset.py-- a/xarray/core/dataset.py++ b/xarray/core/dataset.py


Question: Dropping a MultiIndex variable raises an error after explicit indexes refactor With the latest released version of Xarray, it is possible to delete all variables corresponding to a MultiIndex by simply deleting the name of the MultiIndex. After the explicit indexes refactor (i.e,. using the "main" development branch) this now raises error about how this would "corrupt" index state. This comes up when using and and possibly some other methods. This is not hard to work around, but we may want to consider this bug a blocker for the next Xarray release. I found the issue surfaced in several projects when attempting to use the new version of Xarray inside Google's codebase. CC in case you have any thoughts to share. For now, we should preserve the behavior of deleting the variables corresponding to MultiIndex levels, but should issue a deprecation warning encouraging users to explicitly delete everything. No response <detailsINSTALLED VERSIONS commit: python: 3.10.4 (main, Mar 24 2022, 17:42:03) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.4.0 machine: arm64 processor: arm byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.1 libnetcdf: 4.8.1 xarray: 0.18.3.dev137+g96c56836 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.8.0 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2022.04.1 distributed: 2022.4.1 matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.1.0 pip: 22.0.4 conda: None pytest: 7.1.1 IPython: 8.2.0 sphinx: None </details

data = self.copy()         # do all calculations first...         results: CoercibleMapping = data._calc_assign_results(variables)        data.coords._maybe_drop_multiindex_coords(set(results.keys()))         # ... and then assign         data.update(results)         return datadiff --git a/xarray/core/indexes.py b/xarray/core/indexes.py-- a/xarray/core/indexes.py++ b/xarray/core/indexes.py


Question: Dropping a MultiIndex variable raises an error after explicit indexes refactor With the latest released version of Xarray, it is possible to delete all variables corresponding to a MultiIndex by simply deleting the name of the MultiIndex. After the explicit indexes refactor (i.e,. using the "main" development branch) this now raises error about how this would "corrupt" index state. This comes up when using and and possibly some other methods. This is not hard to work around, but we may want to consider this bug a blocker for the next Xarray release. I found the issue surfaced in several projects when attempting to use the new version of Xarray inside Google's codebase. CC in case you have any thoughts to share. For now, we should preserve the behavior of deleting the variables corresponding to MultiIndex levels, but should issue a deprecation warning encouraging users to explicitly delete everything. No response <detailsINSTALLED VERSIONS commit: python: 3.10.4 (main, Mar 24 2022, 17:42:03) [Clang 12.0.1 ] python-bits: 64 OS: Darwin OS-release: 21.4.0 machine: arm64 processor: arm byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.1 libnetcdf: 4.8.1 xarray: 0.18.3.dev137+g96c56836 pandas: 1.4.2 numpy: 1.22.3 scipy: 1.8.0 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: 2.11.3 cftime: 1.6.0 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: 2022.04.1 distributed: 2022.4.1 matplotlib: None cartopy: None seaborn: None numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: None setuptools: 62.1.0 pip: 22.0.4 conda: None pytest: 7.1.1 IPython: 8.2.0 sphinx: None </details

return Frozen(self._dims) <ins>    def copy(self):        return type(self)(dict(self._indexes), dict(self._variables))</ins>     def get_unique(self) -> list[T_PandasOrXarrayIndex]:         """Return a list of unique indexes, preserving order."""


Question: Use pytorch as backend for xarrays I would be interested in using pytorch as a backend for xarrays - because: a) pytorch is very similar to numpy - so the conceptual overhead is small b) [most helpful] enable having a GPU as the underlying hardware for compute - which would provide non-trivial speed up c) it would allow seamless integration with deep-learning algorithms and techniques Any thoughts on what the interest for such a feature might be ? I would be open to implementing parts of it - so any suggestions on where I could start ? Thanks

if name in ["sum", "prod"]:                 kwargs.pop("min_count", None)             func = getattr(np, name)            if hasattr(values, "__array_namespace__"):                xp = values.__array_namespace__()                func = getattr(xp, name)            else:                func = getattr(np, name)          try:             with warnings.catch_warnings():diff --git a/xarray/core/indexing.py b/xarray/core/indexing.py-- a/xarray/core/indexing.py++ b/xarray/core/indexing.py


Question: Use pytorch as backend for xarrays I would be interested in using pytorch as a backend for xarrays - because: a) pytorch is very similar to numpy - so the conceptual overhead is small b) [most helpful] enable having a GPU as the underlying hardware for compute - which would provide non-trivial speed up c) it would allow seamless integration with deep-learning algorithms and techniques Any thoughts on what the interest for such a feature might be ? I would be open to implementing parts of it - so any suggestions on where I could start ? Thanks

return DaskIndexingAdapter(array)     if hasattr(array, "__array_function__"):         return NdArrayLikeIndexingAdapter(array)<ins>    if hasattr(array, "__array_namespace__"):        return ArrayApiIndexingAdapter(array)</ins>      raise TypeError(f"Invalid array type: {type(array)}")


Question: Use pytorch as backend for xarrays I would be interested in using pytorch as a backend for xarrays - because: a) pytorch is very similar to numpy - so the conceptual overhead is small b) [most helpful] enable having a GPU as the underlying hardware for compute - which would provide non-trivial speed up c) it would allow seamless integration with deep-learning algorithms and techniques Any thoughts on what the interest for such a feature might be ? I would be open to implementing parts of it - so any suggestions on where I could start ? Thanks

self.array = array  class ArrayApiIndexingAdapter(ExplicitlyIndexedNDArrayMixin):    """Wrap an array API array to use explicit indexing."""    __slots__ = ("array",)    def __init__(self, array):        if not hasattr(array, "__array_namespace__"):            raise TypeError(                "ArrayApiIndexingAdapter must wrap an object that "                "implements the __array_namespace__ protocol"            )        self.array = array    def __getitem__(self, key):        if isinstance(key, BasicIndexer):            return self.array[key.tuple]        elif isinstance(key, OuterIndexer):            # manual orthogonal indexing (implemented like DaskIndexingAdapter)            key = key.tuple            value = self.array            for axis, subkey in reversed(list(enumerate(key))):                value = value[(slice(None),) * axis + (subkey, Ellipsis)]            return value        else:            if isinstance(key, VectorizedIndexer):                raise TypeError("Vectorized indexing is not supported")            else:                raise TypeError(f"Unrecognized indexer: {key}")    def __setitem__(self, key, value):        if isinstance(key, (BasicIndexer, OuterIndexer)):            self.array[key.tuple] = value        else:            if isinstance(key, VectorizedIndexer):                raise TypeError("Vectorized indexing is not supported")            else:                raise TypeError(f"Unrecognized indexer: {key}")    def transpose(self, order):        xp = self.array.__array_namespace__()        return xp.permute_dims(self.array, order) class DaskIndexingAdapter(ExplicitlyIndexedNDArrayMixin):     """Wrap a dask array to support explicit indexing.""" diff --git a/xarray/core/utils.py b/xarray/core/utils.py-- a/xarray/core/utils.py++ b/xarray/core/utils.py


Question: Use pytorch as backend for xarrays I would be interested in using pytorch as a backend for xarrays - because: a) pytorch is very similar to numpy - so the conceptual overhead is small b) [most helpful] enable having a GPU as the underlying hardware for compute - which would provide non-trivial speed up c) it would allow seamless integration with deep-learning algorithms and techniques Any thoughts on what the interest for such a feature might be ? I would be open to implementing parts of it - so any suggestions on where I could start ? Thanks

hasattr(value, "ndim")         and hasattr(value, "shape")         and hasattr(value, "dtype")        and hasattr(value, "__array_function__")        and hasattr(value, "__array_ufunc__")        and (            (hasattr(value, "__array_function__") and hasattr(value, "__array_ufunc__"))            or hasattr(value, "__array_namespace__")        )     )


Question: Use pytorch as backend for xarrays I would be interested in using pytorch as a backend for xarrays - because: a) pytorch is very similar to numpy - so the conceptual overhead is small b) [most helpful] enable having a GPU as the underlying hardware for compute - which would provide non-trivial speed up c) it would allow seamless integration with deep-learning algorithms and techniques Any thoughts on what the interest for such a feature might be ? I would be open to implementing parts of it - so any suggestions on where I could start ? Thanks

or not (             isinstance(value, (Iterable,) + NON_NUMPY_SUPPORTED_ARRAY_TYPES)             or hasattr(value, "__array_function__")            or hasattr(value, "__array_namespace__")         )     ) diff --git a/xarray/core/variable.py b/xarray/core/variable.py-- a/xarray/core/variable.py++ b/xarray/core/variable.py


Question: Use pytorch as backend for xarrays I would be interested in using pytorch as a backend for xarrays - because: a) pytorch is very similar to numpy - so the conceptual overhead is small b) [most helpful] enable having a GPU as the underlying hardware for compute - which would provide non-trivial speed up c) it would allow seamless integration with deep-learning algorithms and techniques Any thoughts on what the interest for such a feature might be ? I would be open to implementing parts of it - so any suggestions on where I could start ? Thanks

else:             data = np.asarray(data)     if not isinstance(data, np.ndarray) and hasattr(data, "__array_function__"):    if not isinstance(data, np.ndarray) and (        hasattr(data, "__array_function__") or hasattr(data, "__array_namespace__")    ):         return data      # validate whether the data is valid data types.


Question: RuntimeError when formatting sparse-backed DataArray in f-string On upgrading from xarray 2022.3.0 to 2022.6.0, f-string formatting of sparse-backed DataArray raises an exception. Code does not error, or A breaking change is listed in the section of the docs.[X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. Along with the versions below, I have confirmed the error occurs with both sparse 0.12 and sparse 0.13. <detailsINSTALLED VERSIONS commit: None python: 3.10.4 (main, Jun 29 2022, 12:14:53) [GCC 11.2.0] python-bits: 64 OS: Linux OS-release: 5.15.0-41-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enCA.UTF-8 LOCALE: ('enCA', 'UTF-8') libhdf5: 1.10.7 libnetcdf: 4.8.1 xarray: 2022.6.0 pandas: 1.4.2 numpy: 1.22.4 scipy: 1.8.0 netCDF4: 1.5.8 pydap: None h5netcdf: 0.12.0 h5py: 3.6.0 Nio: None zarr: None cftime: 1.5.2 nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: 1.3.2 dask: 2022.01.0+dfsg distributed: 2022.01.0+ds.1 matplotlib: 3.5.1 cartopy: 0.20.2 seaborn: 0.11.2 numbagg: None fsspec: 2022.01.0 cupy: None pint: 0.18 sparse: 0.13.0 flox: None numpy_groupies: None setuptools: 62.1.0 pip: 22.0.2 conda: None pytest: 6.2.5 IPython: 7.31.1 sphinx: 4.5.0 </details

return f"
{escape(repr(self))}
"         return formatting_html.array_repr(self)     def __format__(self: Any, format_spec: str) -> str:        # we use numpy: scalars will print fine and arrays will raise        return self.values.__format__(format_spec)    def __format__(self: Any, format_spec: str = "") -> str:        if format_spec != "":            if self.shape == ():                # Scalar values might be ok use format_spec with instead of repr:                return self.data.__format__(format_spec)            else:                # TODO: If it's an array the formatting.array_repr(self) should                # take format_spec as an input. If we'd only use self.data we                # lose all the information about coords for example which is                # important information:                raise NotImplementedError(                    "Using format_spec is only supported"                    f" when shape is (). Got shape = {self.shape}."                )        else:            return self.__repr__()      def _iter(self: Any) -> Iterator[Any]:         for n in range(len(self)):


Question: Testing DataArray equality using built-in '==' operator leads to mutilated dictionary In previous versions of xarray, testing numerical equivalence of two DataArrays was possible using the built-in operator '==' and without side affects. Now in version 2022.6.0, when one DataArray lacks an attribute that the other DataArray has, the DataArray with the attribute is mutilated during comparison leading to an empty attrs dictionary. DataArray1 == DataArray2 should not have side affects. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response <detailsINSTALLED VERSIONS commit: None python: 3.9.13 (tags/v3.9.13:, May 17 2022, 16:36:42) [MSC v.1929 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 85 Stepping 4, GenuineIntel byteorder: little LCALL: None LANG: None LOCALE: ('EnglishUnited States', '1252') libhdf5: None libnetcdf: None xarray: 2022.6.0 pandas: 1.4.3 numpy: 1.23.1 scipy: 1.9.0 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.2 cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None flox: None numpygroupies: None setuptools: 63.2.0 pip: 22.2.1 conda: None pytest: 7.1.2 IPython: 8.4.0 sphinx: None </details

if obj_idx is not None:                     for name, var in self.aligned_index_vars[key].items():                         new_indexes[name] = aligned_idx                        new_variables[name] = var                        new_variables[name] = var.copy()              objects[i + 1] = obj._overwrite_indexes(new_indexes, new_variables)


Question: Testing DataArray equality using built-in '==' operator leads to mutilated dictionary In previous versions of xarray, testing numerical equivalence of two DataArrays was possible using the built-in operator '==' and without side affects. Now in version 2022.6.0, when one DataArray lacks an attribute that the other DataArray has, the DataArray with the attribute is mutilated during comparison leading to an empty attrs dictionary. DataArray1 == DataArray2 should not have side affects. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response <detailsINSTALLED VERSIONS commit: None python: 3.9.13 (tags/v3.9.13:, May 17 2022, 16:36:42) [MSC v.1929 64 bit (AMD64)] python-bits: 64 OS: Windows OS-release: 10 machine: AMD64 processor: Intel64 Family 6 Model 85 Stepping 4, GenuineIntel byteorder: little LCALL: None LANG: None LOCALE: ('EnglishUnited States', '1252') libhdf5: None libnetcdf: None xarray: 2022.6.0 pandas: 1.4.3 numpy: 1.23.1 scipy: 1.9.0 netCDF4: None pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: None nctimeaxis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.2 cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None flox: None numpygroupies: None setuptools: 63.2.0 pip: 22.2.1 conda: None pytest: 7.1.2 IPython: 8.4.0 sphinx: None </details

if obj_idx is not None:                 for name, var in index_vars.items():                     new_indexes[name] = aligned_idx                    new_variables[name] = var                    new_variables[name] = var.copy()          return new_indexes, new_variables


Question: Cannot open NetCDF file if dimension with time coordinate has length 0 ( when decoding CF datetime) If a data set has a zero-sized coordinate that is a time index, reading fails. A is triggered when xarray tries to decode the array, as shown below:

def _decode_datetime_with_cftime(num_dates, units, calendar):     if cftime is None:         raise ModuleNotFoundError("No module named 'cftime'")    return np.asarray(        cftime.num2date(num_dates, units, calendar, only_use_cftime_datetimes=True)    )    if num_dates.size > 0:        return np.asarray(            cftime.num2date(num_dates, units, calendar, only_use_cftime_datetimes=True)        )    else:        return np.array([], dtype=object)   def _decode_datetime_with_pandas(flat_num_dates, units, calendar):


Question: Cannot open NetCDF file if dimension with time coordinate has length 0 ( when decoding CF datetime) If a data set has a zero-sized coordinate that is a time index, reading fails. A is triggered when xarray tries to decode the array, as shown below:

with warnings.catch_warnings():         warnings.filterwarnings("ignore", "invalid value encountered", RuntimeWarning)        pd.to_timedelta(flat_num_dates.min(), delta) + ref_date        pd.to_timedelta(flat_num_dates.max(), delta) + ref_date        if flat_num_dates.size > 0:            # avoid size 0 datetimes GH1329            pd.to_timedelta(flat_num_dates.min(), delta) + ref_date            pd.to_timedelta(flat_num_dates.max(), delta) + ref_date      # To avoid integer overflow when converting to nanosecond units for integer     # dtypes smaller than np.int64 cast all integer and unsigned integer dtype


Question: Alignment of dataset with MultiIndex fails after applying After applying the function to a dataset with a Multiindex, a lot of functions related to indexing are broken. For example, it is not possible to apply to itself anymore. The error is raised in the alignment module. It seems that the function does not find indexes that belong to the same dimension. I expected the alignment to be functional and that these basic functions work. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response <detailsINSTALLED VERSIONS commit: None python: 3.10.5 (main, Jun 14 2022, 07:04:59) [GCC 10.3.0] python-bits: 64 OS: Linux OS-release: 5.15.0-41-generic machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.1 libnetcdf: 4.8.1 xarray: 2022.6.0 pandas: 1.4.2 numpy: 1.21.6 scipy: 1.8.1 netCDF4: 1.6.0 pydap: None h5netcdf: None h5py: 3.6.0 Nio: None zarr: None cftime: 1.5.1.1 nctimeaxis: None PseudoNetCDF: None rasterio: 1.2.10 cfgrib: None iris: None bottleneck: 1.3.4 dask: 2022.6.1 distributed: 2022.6.1 matplotlib: 3.5.1 cartopy: 0.20.2 seaborn: 0.11.2 numbagg: None fsspec: 2022.3.0 cupy: None pint: None sparse: 0.13.0 flox: None numpygroupies: None setuptools: 61.2.0 pip: 22.1.2 conda: 4.13.0 pytest: 7.1.2 IPython: 7.33.0 sphinx: 5.0.2 /home/fabian/.miniconda3/lib/python3.10/site- UserWarning: Setuptools is replacing distutils. ("Setuptools is replacing distutils.") </details

)      # determine which variables to merge, and then merge them according to compat    variables_to_merge = (coord_names | data_names) - concat_over - dim_names    variables_to_merge = (coord_names | data_names) - concat_over - unlabeled_dims      result_vars = {}     result_indexes = {}


Question: can modify original object This is kind of a convoluted example, but something I ran into. It appears that in certain cases can modify the original object, here the of a data variable that was swapped into being a dimension coordinate variable. I expected it not to modify the original object. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response_ More experiments in . <details</details

indexes: dict[Hashable, Index] = {}         for k, v in self.variables.items():             dims = tuple(dims_dict.get(dim, dim) for dim in v.dims)            var: Variable             if k in result_dims:                 var = v.to_index_variable()                 var.dims = dimsdiff --git a/xarray/core/variable.py b/xarray/core/variable.py-- a/xarray/core/variable.py++ b/xarray/core/variable.py


Question: can modify original object This is kind of a convoluted example, but something I ran into. It appears that in certain cases can modify the original object, here the of a data variable that was swapped into being a dimension coordinate variable. I expected it not to modify the original object. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response_ More experiments in . <details</details

Iterable,     Literal,     Mapping,<ins>    NoReturn,</ins>     Sequence, )


Question: can modify original object This is kind of a convoluted example, but something I ran into. It appears that in certain cases can modify the original object, here the of a data variable that was swapped into being a dimension coordinate variable. I expected it not to modify the original object. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response_ More experiments in . <details</details

def values(self, values):         self.data = values     def to_base_variable(self):    def to_base_variable(self) -> Variable:         """Return this variable as a base xarray.Variable"""         return Variable(            self.dims, self._data, self._attrs, encoding=self._encoding, fastpath=True            self._dims, self._data, self._attrs, encoding=self._encoding, fastpath=True         )      to_variable = utils.alias(to_base_variable, "to_variable")     def to_index_variable(self):    def to_index_variable(self) -> IndexVariable:         """Return this variable as an xarray.IndexVariable"""         return IndexVariable(            self.dims, self._data, self._attrs, encoding=self._encoding, fastpath=True            self._dims, self._data, self._attrs, encoding=self._encoding, fastpath=True         )      to_coord = utils.alias(to_index_variable, "to_coord")     def to_index(self):    def to_index(self) -> pd.Index:         """Convert this variable to a pandas.Index"""         return self.to_index_variable().to_index()


Question: can modify original object This is kind of a convoluted example, but something I ran into. It appears that in certain cases can modify the original object, here the of a data variable that was swapped into being a dimension coordinate variable. I expected it not to modify the original object. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response_ More experiments in . <details</details

def _data_equals(self, other):         return self.to_index().equals(other.to_index())     def to_index_variable(self):    def to_index_variable(self) -> IndexVariable:         """Return this variable as an xarray.IndexVariable"""        return self        return self.copy()      to_coord = utils.alias(to_index_variable, "to_coord")     def to_index(self):    def to_index(self) -> pd.Index:         """Convert this variable to a pandas.Index"""         # n.b. creating a new pandas.Index from an old pandas.Index is         # basically free as pandas.Index objects are immutable


Question: can modify original object This is kind of a convoluted example, but something I ran into. It appears that in certain cases can modify the original object, here the of a data variable that was swapped into being a dimension coordinate variable. I expected it not to modify the original object. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response_ More experiments in . <details</details

return index      @property    def level_names(self):    def level_names(self) -> list[str] | None:         """Return MultiIndex level names or None if this IndexVariable has no         MultiIndex.         """


Question: can modify original object This is kind of a convoluted example, but something I ran into. It appears that in certain cases can modify the original object, here the of a data variable that was swapped into being a dimension coordinate variable. I expected it not to modify the original object. [X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [X] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [X] New issue — a search of GitHub Issues suggests this is not a duplicate. No response_ More experiments in . <details</details

return type(self)(self.dims, index.get_level_values(level))      @property    def name(self):    def name(self) -> Hashable:         return self.dims[0]      @name.setter    def name(self, value):    def name(self, value) -> NoReturn:         raise AttributeError("cannot modify name of IndexVariable in-place")      def _inplace_binary_op(self, other, f):


Question: index refactor: more than on Dataset assumes that everything that is in and not in is a "data variable". However, since the index refactor we can end up with more than which breaks a number of stuff (e.g. the repr). Well it seems this assumption is now wrong. [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [ ] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. The error comes from here Bisected to - which probably does not help too much. <details</details

dim_coords = either_dict_or_kwargs(indexes, indexes_kwargs, "set_index")          new_indexes: dict[Hashable, Index] = {}        new_variables: dict[Hashable, IndexVariable] = {}        maybe_drop_indexes: list[Hashable] = []        drop_variables: list[Hashable] = []        new_variables: dict[Hashable, Variable] = {}        drop_indexes: set[Hashable] = set()        drop_variables: set[Hashable] = set()         replace_dims: dict[Hashable, Hashable] = {}        all_var_names: set[Hashable] = set()          for dim, _var_names in dim_coords.items():             if isinstance(_var_names, str) or not isinstance(_var_names, Sequence):


Question: index refactor: more than on Dataset assumes that everything that is in and not in is a "data variable". However, since the index refactor we can end up with more than which breaks a number of stuff (e.g. the repr). Well it seems this assumption is now wrong. [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [ ] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. The error comes from here Bisected to - which probably does not help too much. <details</details

+ " variable(s) do not exist"                 )             current_coord_names = self.xindexes.get_all_coords(dim, errors="ignore")            all_var_names.update(var_names)            drop_variables.update(var_names)             # drop any pre-existing index involved            maybe_drop_indexes += list(current_coord_names) + var_names            # drop any pre-existing index involved and its corresponding coordinates            index_coord_names = self.xindexes.get_all_coords(dim, errors="ignore")            all_index_coord_names = set(index_coord_names)             for k in var_names:                maybe_drop_indexes += list(                all_index_coord_names.update(                     self.xindexes.get_all_coords(k, errors="ignore")                 )             drop_variables += var_names            drop_indexes.update(all_index_coord_names)            drop_variables.update(all_index_coord_names)              if len(var_names) == 1 and (not append or dim not in self._indexes):                 var_name = var_names[0]


Question: index refactor: more than on Dataset assumes that everything that is in and not in is a "data variable". However, since the index refactor we can end up with more than which breaks a number of stuff (e.g. the repr). Well it seems this assumption is now wrong. [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [ ] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. The error comes from here Bisected to - which probably does not help too much. <details</details

)                 idx = PandasIndex.from_variables({dim: var})                 idx_vars = idx.create_variables({var_name: var})                # trick to preserve coordinate order in this case                if dim in self._coord_names:                    drop_variables.remove(dim)             else:                 if append:                     current_variables = {                        k: self._variables[k] for k in current_coord_names                        k: self._variables[k] for k in index_coord_names                     }                 else:                     current_variables = {}


Question: index refactor: more than on Dataset assumes that everything that is in and not in is a "data variable". However, since the index refactor we can end up with more than which breaks a number of stuff (e.g. the repr). Well it seems this assumption is now wrong. [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [ ] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. The error comes from here Bisected to - which probably does not help too much. <details</details

new_indexes.update({k: idx for k in idx_vars})             new_variables.update(idx_vars)         # re-add deindexed coordinates (convert to base variables)        for k in drop_variables:            if (                k not in new_variables                and k not in all_var_names                and k in self._coord_names            ):                new_variables[k] = self._variables[k].to_base_variable()         indexes_: dict[Any, Index] = {            k: v for k, v in self._indexes.items() if k not in maybe_drop_indexes            k: v for k, v in self._indexes.items() if k not in drop_indexes         }         indexes_.update(new_indexes)


Question: index refactor: more than on Dataset assumes that everything that is in and not in is a "data variable". However, since the index refactor we can end up with more than which breaks a number of stuff (e.g. the repr). Well it seems this assumption is now wrong. [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [ ] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. The error comes from here Bisected to - which probably does not help too much. <details</details

new_dims = [replace_dims.get(d, d) for d in v.dims]                 variables[k] = v._replace(dims=new_dims)         coord_names = self._coord_names - set(drop_variables) | set(new_variables)        coord_names = self._coord_names - drop_variables | set(new_variables)          return self._replace_with_new_dims(             variables, coord_names=coord_names, indexes=indexes_


Question: index refactor: more than on Dataset assumes that everything that is in and not in is a "data variable". However, since the index refactor we can end up with more than which breaks a number of stuff (e.g. the repr). Well it seems this assumption is now wrong. [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [ ] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. The error comes from here Bisected to - which probably does not help too much. <details</details

f"{tuple(invalid_coords)} are not coordinates with an index"             )         drop_indexes: list[Hashable] = []        drop_variables: list[Hashable] = []        replaced_indexes: list[PandasMultiIndex] = []        drop_indexes: set[Hashable] = set()        drop_variables: set[Hashable] = set()        seen: set[Index] = set()         new_indexes: dict[Hashable, Index] = {}        new_variables: dict[Hashable, IndexVariable] = {}        new_variables: dict[Hashable, Variable] = {}        def drop_or_convert(var_names):            if drop:                drop_variables.update(var_names)            else:                base_vars = {                    k: self._variables[k].to_base_variable() for k in var_names                }                new_variables.update(base_vars)          for name in dims_or_levels:             index = self._indexes[name]            drop_indexes += list(self.xindexes.get_all_coords(name))            if isinstance(index, PandasMultiIndex) and name not in self.dims:                # special case for pd.MultiIndex (name is an index level):                # replace by a new index with dropped level(s) instead of just drop the index                if index not in replaced_indexes:                    level_names = index.index.names                    level_vars = {                        k: self._variables[k]                        for k in level_names                        if k not in dims_or_levels                    }                    if level_vars:                        idx = index.keep_levels(level_vars)                        idx_vars = idx.create_variables(level_vars)                        new_indexes.update({k: idx for k in idx_vars})                        new_variables.update(idx_vars)                replaced_indexes.append(index)             if drop:                drop_variables.append(name)            if index in seen:                continue            seen.add(index)            idx_var_names = set(self.xindexes.get_all_coords(name))            drop_indexes.update(idx_var_names)            if isinstance(index, PandasMultiIndex):                # special case for pd.MultiIndex                level_names = index.index.names                keep_level_vars = {                    k: self._variables[k]                    for k in level_names                    if k not in dims_or_levels                }                if index.dim not in dims_or_levels and keep_level_vars:                    # do not drop the multi-index completely                    # instead replace it by a new (multi-)index with dropped level(s)                    idx = index.keep_levels(keep_level_vars)                    idx_vars = idx.create_variables(keep_level_vars)                    new_indexes.update({k: idx for k in idx_vars})                    new_variables.update(idx_vars)                    if not isinstance(idx, PandasMultiIndex):                        # multi-index reduced to single index                        # backward compatibility: unique level coordinate renamed to dimension                        drop_variables.update(keep_level_vars)                    drop_or_convert(                        [k for k in level_names if k not in keep_level_vars]                    )                else:                    # always drop the multi-index dimension variable                    drop_variables.add(index.dim)                    drop_or_convert(level_names)            else:                drop_or_convert(idx_var_names)          indexes = {k: v for k, v in self._indexes.items() if k not in drop_indexes}         indexes.update(new_indexes)


Question: index refactor: more than on Dataset assumes that everything that is in and not in is a "data variable". However, since the index refactor we can end up with more than which breaks a number of stuff (e.g. the repr). Well it seems this assumption is now wrong. [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [ ] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. The error comes from here Bisected to - which probably does not help too much. <details</details

}         variables.update(new_variables)         coord_names = set(new_variables) | self._coord_names        coord_names = self._coord_names - drop_variables         return self._replace(variables, coord_names=coord_names, indexes=indexes)        return self._replace_with_new_dims(            variables, coord_names=coord_names, indexes=indexes        )      def reorder_levels(         self: T_Dataset,diff --git a/xarray/core/indexes.py b/xarray/core/indexes.py-- a/xarray/core/indexes.py++ b/xarray/core/indexes.py


Question: index refactor: more than on Dataset assumes that everything that is in and not in is a "data variable". However, since the index refactor we can end up with more than which breaks a number of stuff (e.g. the repr). Well it seems this assumption is now wrong. [ ] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray. [ ] Complete example — the example is self-contained, including all data and the text of any traceback. , returning the result. [ ] New issue — a search of GitHub Issues suggests this is not a duplicate. The error comes from here Bisected to - which probably does not help too much. <details</details

level_coords_dtype = {k: self.level_coords_dtype[k] for k in index.names}             return self._replace(index, level_coords_dtype=level_coords_dtype)         else:            # backward compatibility: rename the level coordinate to the dimension name             return PandasIndex(                index, self.dim, coord_dtype=self.level_coords_dtype[index.name]                index.rename(self.dim),                self.dim,                coord_dtype=self.level_coords_dtype[index.name],             )      def reorder_levels(


Question: [Bug]: renamevars to dimension coordinate does not create an index We used to make a coordinate a dimension coordinate (instead of ). This results in the coordinate correctly being displayed as a dimension coordinate (with the *) but it does not create an index, such that further operations like fail with a strange . I expect one of two things to be true: does not allow setting dimension coordinates (raises Error and tells you to use setindex) checks for this occasion and sets the index correctly No response No response INSTALLED VERSIONS commit: None python: 3.9.1 (default, Jan 13 2021, 15:21:08) [GCC 4.8.5 (Red Hat 4.8.5-44)] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.49.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4 xarray: 0.20.2 pandas: 1.3.5 numpy: 1.21.5 scipy: 1.7.3 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.5.1.1 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.1 cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None setuptools: 49.2.1 pip: 22.0.2 conda: None pytest: 6.2.5 IPython: 8.0.0 sphinx: None

if utils.is_dict_like(new_name_or_name_dict) or new_name_or_name_dict is None:             # change dims/coords             name_dict = either_dict_or_kwargs(new_name_or_name_dict, names, "rename")            dataset = self._to_temp_dataset().rename(name_dict)            dataset = self._to_temp_dataset()._rename(name_dict)             return self._from_temp_dataset(dataset)         if utils.hashable(new_name_or_name_dict) and names:             # change name + dims/coords            dataset = self._to_temp_dataset().rename(names)            dataset = self._to_temp_dataset()._rename(names)             dataarray = self._from_temp_dataset(dataset)             return dataarray._replace(name=new_name_or_name_dict)         # only change namediff --git a/xarray/core/dataset.py b/xarray/core/dataset.py-- a/xarray/core/dataset.py++ b/xarray/core/dataset.py


Question: [Bug]: renamevars to dimension coordinate does not create an index We used to make a coordinate a dimension coordinate (instead of ). This results in the coordinate correctly being displayed as a dimension coordinate (with the *) but it does not create an index, such that further operations like fail with a strange . I expect one of two things to be true: does not allow setting dimension coordinates (raises Error and tells you to use setindex) checks for this occasion and sets the index correctly No response No response INSTALLED VERSIONS commit: None python: 3.9.1 (default, Jan 13 2021, 15:21:08) [GCC 4.8.5 (Red Hat 4.8.5-44)] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.49.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4 xarray: 0.20.2 pandas: 1.3.5 numpy: 1.21.5 scipy: 1.7.3 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.5.1.1 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.1 cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None setuptools: 49.2.1 pip: 22.0.2 conda: None pytest: 6.2.5 IPython: 8.0.0 sphinx: None

return variables, coord_names, dims, indexes <ins>    def _rename(        self: T_Dataset,        name_dict: Mapping[Any, Hashable] | None = None,        **names: Hashable,    ) -> T_Dataset:        """Also used internally by DataArray so that the warning (if any)        is raised at the right stack level.        """        name_dict = either_dict_or_kwargs(name_dict, names, "rename")        for k in name_dict.keys():            if k not in self and k not in self.dims:                raise ValueError(                    f"cannot rename {k!r} because it is not a "                    "variable or dimension in this dataset"                )            create_dim_coord = False            new_k = name_dict[k]            if k in self.dims and new_k in self._coord_names:                coord_dims = self._variables[name_dict[k]].dims                if coord_dims == (k,):                    create_dim_coord = True            elif k in self._coord_names and new_k in self.dims:                coord_dims = self._variables[k].dims                if coord_dims == (new_k,):                    create_dim_coord = True            if create_dim_coord:                warnings.warn(                    f"rename {k!r} to {name_dict[k]!r} does not create an index "                    "anymore. Try using swap_dims instead or use set_index "                    "after rename to create an indexed coordinate.",                    UserWarning,                    stacklevel=3,                )        variables, coord_names, dims, indexes = self._rename_all(            name_dict=name_dict, dims_dict=name_dict        )        return self._replace(variables, coord_names, dims=dims, indexes=indexes)</ins>     def rename(         self: T_Dataset,         name_dict: Mapping[Any, Hashable] | None = None,


Question: [Bug]: renamevars to dimension coordinate does not create an index We used to make a coordinate a dimension coordinate (instead of ). This results in the coordinate correctly being displayed as a dimension coordinate (with the *) but it does not create an index, such that further operations like fail with a strange . I expect one of two things to be true: does not allow setting dimension coordinates (raises Error and tells you to use setindex) checks for this occasion and sets the index correctly No response No response INSTALLED VERSIONS commit: None python: 3.9.1 (default, Jan 13 2021, 15:21:08) [GCC 4.8.5 (Red Hat 4.8.5-44)] python-bits: 64 OS: Linux OS-release: 3.10.0-1160.49.1.el7.x8664 machine: x8664 processor: x8664 byteorder: little LCALL: None LANG: enUS.UTF-8 LOCALE: ('enUS', 'UTF-8') libhdf5: 1.12.0 libnetcdf: 4.7.4 xarray: 0.20.2 pandas: 1.3.5 numpy: 1.21.5 scipy: 1.7.3 netCDF4: 1.5.8 pydap: None h5netcdf: None h5py: None Nio: None zarr: None cftime: 1.5.1.1 nctime_axis: None PseudoNetCDF: None rasterio: None cfgrib: None iris: None bottleneck: None dask: None distributed: None matplotlib: 3.5.1 cartopy: None seaborn: None numbagg: None fsspec: None cupy: None pint: None sparse: None setuptools: 49.2.1 pip: 22.0.2 conda: None pytest: 6.2.5 IPython: 8.0.0 sphinx: None

Dataset.rename_dims         DataArray.rename         """        name_dict = either_dict_or_kwargs(name_dict, names, "rename")        for k in name_dict.keys():            if k not in self and k not in self.dims:                raise ValueError(                    f"cannot rename {k!r} because it is not a "                    "variable or dimension in this dataset"                )        variables, coord_names, dims, indexes = self._rename_all(            name_dict=name_dict, dims_dict=name_dict        )        return self._replace(variables, coord_names, dims=dims, indexes=indexes)        return self._rename(name_dict=name_dict, **names)      def rename_dims(         self: T_Dataset,


Question: Indexes.getunique() TypeError with pandas indexes I also just tested the method that you mentioned and maybe noticed a related issue here, which I'm not sure is wanted / expected. Taking the above dataset , accessing this function results in an error: However, for it works: Originally posted by in

"""Return a list of unique indexes, preserving order."""          unique_indexes: list[T_PandasOrXarrayIndex] = []        seen: set[T_PandasOrXarrayIndex] = set()        seen: set[int] = set()          for index in self._indexes.values():            if index not in seen:            index_id = id(index)            if index_id not in seen:                 unique_indexes.append(index)                seen.add(index)                seen.add(index_id)          return unique_indexes