Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

silence UnitStrippedWarnings #4163

Merged
merged 55 commits into from Jul 2, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
62bb825
globally promote UnitStrippedWarning to errors
keewis Jun 17, 2020
6f7e760
separately test apply_ufunc with units in dims, coords and data
keewis Jun 17, 2020
b5a4902
split the DataArray align test into data, dims and coords tests
keewis Jun 19, 2020
65473d4
use dtypes instead of python types and use a dtype specific fill value
keewis Jun 19, 2020
3b0fdc3
rewrite the dataset align tests
keewis Jun 19, 2020
a4ce5d8
compare with dtypes.NA instead of using np.isnan
keewis Jun 19, 2020
15399b2
mention the issue in the xfail reason
keewis Jun 19, 2020
86bc323
make sure the combine_* variants are properly separated from each other
keewis Jun 19, 2020
5cc076c
improve the test case names
keewis Jun 19, 2020
4b71aa6
note that broadcast uses align
keewis Jun 19, 2020
218bb27
properly separate the test cases for concat
keewis Jun 19, 2020
766638d
always use the same reason when xfailing units in indexes tests
keewis Jun 19, 2020
e5569a4
also check that the replication functions work with dims and units
keewis Jun 19, 2020
4737d41
apply full_like to the data instead of the variable
keewis Jun 19, 2020
b080cb7
check full_like with units in dims, data and coords separately
keewis Jun 19, 2020
b35bb6c
clearly separate the test variants of the merge tests
keewis Jun 19, 2020
a754056
don't use indexes for the dataset where tests
keewis Jun 19, 2020
08cf7be
replace numpy.testing.assert_allclose with assert numpy.allclose
keewis Jun 20, 2020
daceb13
remove a conditional xfail that depends on a very old pint version
keewis Jun 21, 2020
8fdee00
use assert_identical from the local namespace
keewis Jun 22, 2020
72ad5b0
properly separate between the broadcast_like test variants
keewis Jun 22, 2020
c9d1c18
don't accept "data" as an alias of the DataArray's data
keewis Jun 22, 2020
e59bd59
properly separate between the variants of the content manipulation tests
keewis Jun 22, 2020
d91922d
use assert np.allclose(...) instead of np.testing.assert_allclose(...)
keewis Jun 22, 2020
b4f7bb2
don't test units in indexes in the isel tests
keewis Jun 22, 2020
db44dd5
don't use units in indexes for the head / tail / thin tests
keewis Jun 22, 2020
cef0dda
properly separate the variants of more tests
keewis Jun 22, 2020
bdf7145
rewrite the squeeze tests
keewis Jun 22, 2020
dbd5512
use assert_allclose from the module's namespace
keewis Jun 22, 2020
a1539ff
rewrite the copy tests
keewis Jun 22, 2020
a1f3f72
xfail the equal comparison for a pint version lower than 0.14
keewis Jun 22, 2020
80f49d6
try to implement a duckarray friendly assert_array_equal
keewis Jun 22, 2020
372040c
Merge branch 'master' into silence-unit-stripped-warnings
keewis Jun 23, 2020
32da9b9
add tests for not raising an assertion error
keewis Jun 23, 2020
ea14ef3
skip only the dask test if it isn't installed
keewis Jun 23, 2020
21d2e8e
also check using pint if available
keewis Jun 23, 2020
7a597b3
add a duckarray version of np.testing.assert_allclose
keewis Jun 23, 2020
cc4fff0
add both to __all__
keewis Jun 23, 2020
7320d43
make both available in xarray.tests
keewis Jun 23, 2020
c9c0c87
don't inherit from VariableSubtests since that was not written to tes…
keewis Jun 25, 2020
3bcb050
test the constant pad mode along with all other modes
keewis Jun 25, 2020
c341401
remove most pint version checks, now that pint 0.13 has been released
keewis Jun 25, 2020
0f4391b
Merge branch 'master' into silence-unit-stripped-warnings
keewis Jun 25, 2020
a9cc7d4
use conda to install pint
keewis Jun 25, 2020
673cabc
xfail the DataArray comparison test until pint's dev version fixed it
keewis Jun 25, 2020
158c83e
add tests for the pad method of DataArray and Dataset
keewis Jun 26, 2020
c8a6466
add tests for weighted
keewis Jun 26, 2020
d8783e4
update whats-new.rst
keewis Jun 26, 2020
2f5a2a7
Merge branch 'master' into silence-unit-stripped-warnings
keewis Jun 27, 2020
f39dc78
Merge branch 'assert_array_equal-duckarray' into silence-unit-strippe…
keewis Jun 28, 2020
1c827da
replace assert np.allclose(...) with assert_duckarray_allclose(...)
keewis Jun 28, 2020
1ce16ea
fix the dask fallback
keewis Jun 28, 2020
a68048b
xfail the pint tests for now since there's a bug in pint
keewis Jun 28, 2020
34931eb
use utils.is_array_like and utils.is_scalar
keewis Jun 29, 2020
f5ef2f5
Merge branch 'master' into silence-unit-stripped-warnings
keewis Jun 29, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 1 addition & 2 deletions ci/requirements/py36-min-nep18.yml
Expand Up @@ -11,12 +11,11 @@ dependencies:
- msgpack-python=0.6 # remove once distributed is bumped. distributed GH3491
- numpy=1.17
- pandas=0.25
- pint=0.13
- pip
- pytest
- pytest-cov
- pytest-env
- scipy=1.2
- setuptools=41.2
- sparse=0.8
- pip:
- pint==0.13
2 changes: 1 addition & 1 deletion ci/requirements/py36.yml
Expand Up @@ -28,6 +28,7 @@ dependencies:
- numba
- numpy
- pandas
- pint
- pip
- pseudonetcdf
- pydap
Expand All @@ -44,4 +45,3 @@ dependencies:
- zarr
- pip:
- numbagg
- pint
2 changes: 1 addition & 1 deletion ci/requirements/py37-windows.yml
Expand Up @@ -28,6 +28,7 @@ dependencies:
- numba
- numpy
- pandas
- pint
- pip
- pseudonetcdf
- pydap
Expand All @@ -44,4 +45,3 @@ dependencies:
- zarr
- pip:
- numbagg
- pint
2 changes: 1 addition & 1 deletion ci/requirements/py37.yml
Expand Up @@ -28,6 +28,7 @@ dependencies:
- numba
- numpy
- pandas
- pint
- pip
- pseudonetcdf
- pydap
Expand All @@ -44,4 +45,3 @@ dependencies:
- zarr
- pip:
- numbagg
- pint
2 changes: 1 addition & 1 deletion ci/requirements/py38-all-but-dask.yml
Expand Up @@ -25,6 +25,7 @@ dependencies:
- numba
- numpy
- pandas
- pint
- pip
- pseudonetcdf
- pydap
Expand All @@ -41,4 +42,3 @@ dependencies:
- zarr
- pip:
- numbagg
- pint
2 changes: 1 addition & 1 deletion ci/requirements/py38.yml
Expand Up @@ -28,6 +28,7 @@ dependencies:
- numba
- numpy
- pandas
- pint
- pip
- pseudonetcdf
- pydap
Expand All @@ -44,4 +45,3 @@ dependencies:
- zarr
- pip:
- numbagg
- pint
2 changes: 1 addition & 1 deletion doc/whats-new.rst
Expand Up @@ -91,7 +91,7 @@ New Features
- Support dask handling for :py:meth:`DataArray.idxmax`, :py:meth:`DataArray.idxmin`,
:py:meth:`Dataset.idxmax`, :py:meth:`Dataset.idxmin`. (:pull:`3922`, :pull:`4135`)
By `Kai Mühlbauer <https://github.com/kmuehlbauer>`_ and `Pascal Bourgault <https://github.com/aulemahal>`_.
- More support for unit aware arrays with pint (:pull:`3643`, :pull:`3975`)
- More support for unit aware arrays with pint (:pull:`3643`, :pull:`3975`, :pull:`4163`)
By `Justus Magin <https://github.com/keewis>`_.
- Support overriding existing variables in ``to_zarr()`` with ``mode='a'`` even
without ``append_dim``, as long as dimension sizes do not change.
Expand Down
2 changes: 1 addition & 1 deletion xarray/core/common.py
Expand Up @@ -1434,7 +1434,7 @@ def _full_like_variable(other, fill_value, dtype: DTypeLike = None):
other.shape, fill_value, dtype=dtype, chunks=other.data.chunks
)
else:
data = np.full_like(other, fill_value, dtype=dtype)
data = np.full_like(other.data, fill_value, dtype=dtype)
keewis marked this conversation as resolved.
Show resolved Hide resolved

return Variable(dims=other.dims, data=data, attrs=other.attrs)

Expand Down
6 changes: 6 additions & 0 deletions xarray/core/utils.py
Expand Up @@ -247,6 +247,12 @@ def is_list_like(value: Any) -> bool:
return isinstance(value, list) or isinstance(value, tuple)


def is_array_like(value: Any) -> bool:
return (
hasattr(value, "ndim") and hasattr(value, "shape") and hasattr(value, "dtype")
)


def either_dict_or_kwargs(
pos_kwargs: Optional[Mapping[Hashable, T]],
kw_kwargs: Mapping[str, T],
Expand Down
65 changes: 64 additions & 1 deletion xarray/testing.py
Expand Up @@ -11,7 +11,14 @@
from xarray.core.indexes import default_indexes
from xarray.core.variable import IndexVariable, Variable

__all__ = ("assert_allclose", "assert_chunks_equal", "assert_equal", "assert_identical")
__all__ = (
"assert_allclose",
"assert_chunks_equal",
"assert_duckarray_equal",
"assert_duckarray_allclose",
"assert_equal",
"assert_identical",
)


def _decode_string_data(data):
Expand Down Expand Up @@ -148,6 +155,62 @@ def compat_variable(a, b):
raise TypeError("{} not supported by assertion comparison".format(type(a)))


def _format_message(x, y, err_msg, verbose):
diff = x - y
abs_diff = max(abs(diff))
rel_diff = "not implemented"

n_diff = int(np.count_nonzero(diff))
n_total = diff.size

fraction = f"{n_diff} / {n_total}"
percentage = float(n_diff / n_total * 100)

parts = [
"Arrays are not equal",
err_msg,
f"Mismatched elements: {fraction} ({percentage:.0f}%)",
f"Max absolute difference: {abs_diff}",
f"Max relative difference: {rel_diff}",
]
if verbose:
parts += [
f" x: {x!r}",
f" y: {y!r}",
]

return "\n".join(parts)


def assert_duckarray_allclose(
actual, desired, rtol=1e-07, atol=0, err_msg="", verbose=True
):
""" Like `np.testing.assert_allclose`, but for duckarrays. """
__tracebackhide__ = True

allclose = duck_array_ops.allclose_or_equiv(actual, desired, rtol=rtol, atol=atol)
assert allclose, _format_message(actual, desired, err_msg=err_msg, verbose=verbose)


def assert_duckarray_equal(x, y, err_msg="", verbose=True):
""" Like `np.testing.assert_array_equal`, but for duckarrays """
__tracebackhide__ = True

if not utils.is_array_like(x) and not utils.is_scalar(x):
x = np.asarray(x)

if not utils.is_array_like(y) and not utils.is_scalar(y):
y = np.asarray(y)

if (utils.is_array_like(x) and utils.is_scalar(y)) or (
utils.is_scalar(x) and utils.is_array_like(y)
):
equiv = (x == y).all()
dcherian marked this conversation as resolved.
Show resolved Hide resolved
else:
equiv = duck_array_ops.array_equiv(x, y)
assert equiv, _format_message(x, y, err_msg=err_msg, verbose=verbose)


def assert_chunks_equal(a, b):
"""
Assert that chunksizes along chunked dimensions are equal.
Expand Down
4 changes: 4 additions & 0 deletions xarray/tests/__init__.py
Expand Up @@ -16,6 +16,10 @@
from xarray.core.duck_array_ops import allclose_or_equiv # noqa: F401
from xarray.core.indexing import ExplicitlyIndexed
from xarray.core.options import set_options
from xarray.testing import ( # noqa: F401
assert_duckarray_allclose,
assert_duckarray_equal,
)

# import mpl and change the backend before other mpl imports
try:
Expand Down
99 changes: 99 additions & 0 deletions xarray/tests/test_testing.py
@@ -1,7 +1,31 @@
import numpy as np
import pytest

import xarray as xr

from . import has_dask

try:
from dask.array import from_array as dask_from_array
except ImportError:
dask_from_array = lambda x: x

try:
import pint

unit_registry = pint.UnitRegistry(force_ndarray_like=True)

def quantity(x):
return unit_registry.Quantity(x, "m")

has_pint = True
except ImportError:

def quantity(x):
return x

has_pint = False


def test_allclose_regression():
x = xr.DataArray(1.01)
Expand Down Expand Up @@ -30,3 +54,78 @@ def test_allclose_regression():
def test_assert_allclose(obj1, obj2):
with pytest.raises(AssertionError):
xr.testing.assert_allclose(obj1, obj2)


@pytest.mark.filterwarnings("error")
@pytest.mark.parametrize(
"duckarray",
(
pytest.param(np.array, id="numpy"),
pytest.param(
dask_from_array,
id="dask",
marks=pytest.mark.skipif(not has_dask, reason="requires dask"),
),
pytest.param(
quantity,
id="pint",
marks=[
pytest.mark.skipif(not has_pint, reason="requires pint"),
pytest.mark.xfail(
reason="inconsistencies in the return value of pint's implementation of eq"
),
],
),
),
)
@pytest.mark.parametrize(
["obj1", "obj2"],
(
pytest.param([1e-10, 2], [0.0, 2.0], id="both arrays"),
pytest.param([1e-17, 2], 0.0, id="second scalar"),
pytest.param(0.0, [1e-17, 2], id="first scalar"),
),
)
def test_assert_duckarray_equal_failing(duckarray, obj1, obj2):
# TODO: actually check the repr
a = duckarray(obj1)
b = duckarray(obj2)
with pytest.raises(AssertionError):
xr.testing.assert_duckarray_equal(a, b)


@pytest.mark.filterwarnings("error")
@pytest.mark.parametrize(
"duckarray",
(
pytest.param(np.array, id="numpy"),
pytest.param(
dask_from_array,
id="dask",
marks=pytest.mark.skipif(not has_dask, reason="requires dask"),
),
pytest.param(
quantity,
id="pint",
marks=[
pytest.mark.skipif(not has_pint, reason="requires pint"),
pytest.mark.xfail(
reason="inconsistencies in the return value of pint's implementation of eq"
),
],
),
),
)
@pytest.mark.parametrize(
["obj1", "obj2"],
(
pytest.param([0, 2], [0.0, 2.0], id="both arrays"),
pytest.param([0, 0], 0.0, id="second scalar"),
pytest.param(0.0, [0, 0], id="first scalar"),
),
)
def test_assert_duckarray_equal(duckarray, obj1, obj2):
a = duckarray(obj1)
b = duckarray(obj2)

xr.testing.assert_duckarray_equal(a, b)