Expand test coverage #854

keflavich · 2022-12-18T14:27:05Z

And make sure tests are still passing in prep for a new release (0.6.1)

keflavich · 2022-12-18T14:30:04Z

Immediate test failures:

py39-test: exit 2 (0.00 seconds) .tmp/py39-test> '!cov-!noopenfiles:' pytest --open-files --pyargs spectral_cube /home/runner/work/spectral-cube/spectral-cube/docs
.pkg: _exit> python /opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: exit None (0.00 seconds) /home/runner/work/spectral-cube/spectral-cube> python /opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/pyproject_api/_backend.py True setuptools.build_meta pid=1745
  py39-test: FAIL code 2 (25.98=setup[25.58]+cmd[0.39,0.00] seconds)
  evaluation failed :( (26.08 seconds)
Error: Process completed with exit code 2.

No idea what this means

dhomeier · 2022-12-19T17:44:10Z

Actually the !cov-!bla syntax now fails for astropy as well (astropy/astropy#14139), e.g.
https://github.com/astropy/astropy/actions/runs/3733576067/jobs/6334552564
so this was in fact broken with a more recent tox update pulled in with 4.0.13
tox-dev/tox#2747.

keflavich · 2023-03-09T15:31:18Z

https://github.com/radio-astro-tools/spectral-cube/actions/runs/4374807279/jobs/7654700456#step:5:1516 we have real failures in some tests because of unimplemented floor_divide? I hope this just means we need to bump version numbers

codecov-commenter · 2023-03-09T15:45:27Z

Codecov Report

Patch coverage: 80.00% and project coverage change: +2.51 🎉

Comparison is base (d004d79) 77.36% compared to head (dad37a2) 79.88%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #854      +/-   ##
==========================================
+ Coverage   77.36%   79.88%   +2.51%     
==========================================
  Files          24       24              
  Lines        6027     6035       +8     
==========================================
+ Hits         4663     4821     +158     
+ Misses       1364     1214     -150

Impacted Files	Coverage Δ
spectral_cube/lower_dimensional_structures.py	`84.70% <71.42%> (+4.00%)`	⬆️
spectral_cube/base_class.py	`86.23% <100.00%> (+0.12%)`	⬆️

... and 5 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

keflavich · 2023-03-10T01:59:52Z

There's at least one genuine bug here introduced by astropy upstream. This test:
https://github.com/radio-astro-tools/spectral-cube/blob/master/spectral_cube/tests/test_projection.py#L137
fails on dev but not on older versions. The sum p2 = p + p does not retain the beam from p

keflavich · 2023-03-10T02:02:30Z

I don't see any sign of this breaking change in astropy's changelog. The only way I can think to track it down is with bisection, which I'm really bad at and can't do on my personal computer.

keflavich · 2023-03-10T02:15:01Z

Maybe this is our breaking change? astropy/astropy@c6ff27d

keflavich · 2023-03-10T02:17:20Z

No, that seems like it's not the problem; the default is to pass on info except for a few cases that don't apply to these tests.

keflavich · 2023-03-10T02:59:49Z

I had to do some test refactoring to get tests to run locally at all.

keflavich · 2023-03-10T03:02:33Z

right, so, it looks like noviz works, but 'all' doesn't.

keflavich · 2023-03-10T12:26:55Z

OK, dev passes on its own, so it's not dev. It's some interaction between the viz stuff and the dev stuff, but I'm not convinced the viz stuff works at all. Split up the tests further.

keflavich · 2023-03-10T12:43:30Z

So getting rid of the 'all' mode has solved my local woes - I can now test everything in a grid locally. This seems also to have "fixed" tests remotely. But I think this makes no sense, so I'm worried I've done something that I don't understand.

keflavich · 2023-03-10T13:13:58Z

Dev and viz independently cause failures. The dev failures originate somewhere in quantity, but there's no difference in astropy quantity between 5.2.1 and 5.3.dev, so I'm stuck.

keflavich · 2023-03-10T13:25:53Z

I can't test dev locally because:

      Traceback (most recent call last):
        File "/private/var/folders/k_/7qh4l0nn72b7qgq15pkd4hw40000gt/T/pip-build-env-1rq0a8qx/overlay/lib/python3.9/site-packages/setuptools/_distutils/unixccompiler.py", line 185, in _compile
          self.spawn(compiler_so + cc_args + [src, '-o', obj] + extra_postargs)
        File "/private/var/folders/k_/7qh4l0nn72b7qgq15pkd4hw40000gt/T/pip-build-env-1rq0a8qx/overlay/lib/python3.9/site-packages/setuptools/_distutils/ccompiler.py", line 1041, in spawn
          spawn(cmd, dry_run=self.dry_run, **kwargs)
        File "/private/var/folders/k_/7qh4l0nn72b7qgq15pkd4hw40000gt/T/pip-build-env-1rq0a8qx/overlay/lib/python3.9/site-packages/setuptools/_distutils/spawn.py", line 70, in spawn
          raise DistutilsExecError(
      distutils.errors.DistutilsExecError: command '/usr/bin/clang' failed with exit code 1
  ERROR: Failed building wheel for numcodecs
ERROR: Could not build wheels for numcodecs, which is required to install pyproject.toml-based projects

I'm at a loss with this; I've googled and found similar issues, but no fixes - the issues just end with 'oh ok we solved it'

keflavich · 2023-03-10T13:41:21Z

the viz tests are amusing:
https://github.com/radio-astro-tools/spectral-cube/actions/runs/4384619998/jobs/7676312443#step:5:195

../../spectral_cube/tests/test_subcubes.py ..ssssssssssssssssssss        [ 98%]
Fatal Python error: Aborted

I think this is an indication that the tests crash completely as soon as test_visualization.py, the next test code alphabetically, is initiated. I'm guessing the failure happens on the matplotlib import step.

keflavich · 2023-03-10T13:50:11Z

https://github.com/radio-astro-tools/spectral-cube/actions/runs/4384619998/jobs/7676312293#step:5:1534 is pointing to an incorrect error being raised:

spectral-cube/spectral_cube/tests/test_spectral_cube.py

Line 366 in d004d79

with pytest.raises(u.UnitConversionError):

keflavich · 2023-03-10T14:58:33Z

I finally tracked down one error: astropy/astropy#14514.

keflavich · 2023-03-10T22:11:09Z

ok last (?) open issue:
https://github.com/radio-astro-tools/spectral-cube/actions/runs/4387984028/jobs/7685493794#step:5:221

=================================== FAILURES ===================================
___ TestSpectralCube.test_apply_everywhere_floordivide[True-floordiv-value0] ___

self = <spectral_cube.tests.test_spectral_cube.TestSpectralCube object at 0x7fcd29aec5e0>
operation = <built-in function floordiv>, value = <Quantity 0.5 K>
data_advs = PosixPath('/tmp/pytest-of-runner/pytest-0/test_apply_everywhere_floordiv1/advs.fits')
use_dask = True

    @pytest.mark.parametrize(('operation', 'value'),
                             ((operator.div if hasattr(operator,'div') else operator.floordiv, 0.5*u.K),))
    def test_apply_everywhere_floordivide(self, operation, value, data_advs, use_dask):
        c1, d1 = cube_and_raw(data_advs, use_dask=use_dask)
        # floordiv doesn't work, which is why it's NotImplemented
        with pytest.raises(u.UnitConversionError):
>           c1o = c1._apply_everywhere(operation, value)

../../spectral_cube/tests/test_spectral_cube.py:367: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../spectral_cube/utils.py:49: in wrapper
    return function(self, *args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = DaskSpectralCube with shape=(2, 3, 4) and unit=K and chunk size (2, 3, 4):
 n_x:      4  type_x: RA---SIN  unit_x: deg...094 deg:   29.935218 deg
 n_s:      2  type_s: VOPT      unit_s: km / s  range:     -321.[215](https://github.com/radio-astro-tools/spectral-cube/actions/runs/4387984028/jobs/7685493794#step:5:216) km / s:    -319.926 km / s
function = <built-in function floordiv>, check_units = True
args = (<Quantity 0.5 K>,), test_result = <Quantity [[[2.]]]>
new_unit = Unit(dimensionless)

    @warn_slow
    def _apply_everywhere(self, function, *args, check_units=True):
        """
        Return a new cube with ``function`` applied to all pixels
    
        Private because this doesn't have an obvious and easy-to-use API
    
        Parameters
        ----------
        function : function
            An operator that takes the data (self) and any number of additional
            arguments
        check_units : bool
            When doing the initial test before running the full operation,
            should units be included on the 'fake' test quantity?  This is
            specifically added as an option to enable using the subtraction and
            addition operators without checking unit compatibility here because
            they _already_ enforce unit compatibility.
    
        Examples
        --------
        >>> newcube = cube.apply_everywhere(np.add, 0.5*u.Jy)
        """
    
        try:
            if check_units:
                test_result = function(np.ones([1,1,1])*self.unit, *args)
                new_unit = test_result.unit
            else:
                test_result = function(np.ones([1,1,1]), *args)
                new_unit = self.unit
            # First, check that function returns same # of dims?
            assert test_result.ndim == 3,"Output is not 3-dimensional"
        except Exception as ex:
            raise AssertionError("Function could not be applied to a simple "
                                 "cube.  The error was: {0}".format(ex))
    
        # We don't need to convert to a quantity here because the shape check
        data_in = self._get_filled_data(fill=self._fill_value)
>       data = function(data_in, *args)
E       TypeError: operand type(s) all returned NotImplemented from __array_ufunc__(<ufunc 'floor_divide'>, '__call__', dask.array<FilledArrayHandler 017ac2fc-5bf3-4d25-bf15, shape=(2, 3, 4), dtype=>f8, chunksize=(2, 3, 4), chunktype=numpy.ndarray>, <Quantity 0.5 K>): 'Array', 'Quantity'

keflavich · 2023-03-11T02:19:12Z

All of the relevant tests have been stuck yellow for hours? Did I overuse our minutes or something?

keflavich · 2023-03-11T02:19:41Z

no... I tihnk I introduced an infinite loop 🤦

keflavich · 2023-03-11T02:50:04Z

Got dev to go green, finally. Have a yt error:

self = <spectral_cube.tests.test_spectral_cube.TestYt object at 0x2ae7055d5d60>, request = <SubRequest 'setup_method_fixture' for <Function test_yt[False]>>
data_adv = PosixPath('/tmp/pytest-of-adamginsburg/pytest-12/test_yt_False_0/adv.fits'), use_dask = False

    @pytest.fixture(autouse=True)
    def setup_method_fixture(self, request, data_adv, use_dask):
        print("HERE")
        self.cube = SpectralCube.read(data_adv, use_dask=use_dask)
        # Without any special arguments
        print(self.cube)
        print(self.cube.to_yt)
>       self.ytc1 = self.cube.to_yt()

spectral_cube/tests/test_spectral_cube.py:899:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
spectral_cube/spectral_cube.py:2371: in to_yt
    ds = FITSDataset(hdu, nprocs=nprocs,
/orange/adamginsburg/miniconda3/envs/python39/lib/python3.9/site-packages/yt/data_objects/static_output.py:192: in __new__
    obj.__init__(filename, *args, **kwargs)
/orange/adamginsburg/miniconda3/envs/python39/lib/python3.9/site-packages/yt/frontends/fits/data_structures.py:766: in __init__
    super().__init__(
/orange/adamginsburg/miniconda3/envs/python39/lib/python3.9/site-packages/yt/frontends/fits/data_structures.py:384: in __init__
    Dataset.__init__(
/orange/adamginsburg/miniconda3/envs/python39/lib/python3.9/site-packages/yt/data_objects/static_output.py:273: in __init__
    _ds_store.check_ds(self)
/orange/adamginsburg/miniconda3/envs/python39/lib/python3.9/site-packages/yt/utilities/parameter_file_storage.py:135: in check_ds
    hash = ds._hash()
/orange/adamginsburg/miniconda3/envs/python39/lib/python3.9/site-packages/yt/data_objects/static_output.py:391: in _hash
    s = f"{self.basename};{self.current_time};{self.unique_identifier}"
/orange/adamginsburg/miniconda3/envs/python39/lib/python3.9/functools.py:993: in __get__
    val = self.func(instance)
/orange/adamginsburg/miniconda3/envs/python39/lib/python3.9/site-packages/yt/frontends/fits/data_structures.py:439: in unique_identifier
    return super().unique_identifier
/orange/adamginsburg/miniconda3/envs/python39/lib/python3.9/functools.py:993: in __get__
    val = self.func(instance)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = SpectralCubeFITSDataset: /blue/adamginsburg/adamginsburg/repos/spectral-cube/InMemoryFITSFile_d26b1c73bcb149e9a9fc50a3bacda733

    @cached_property
    def unique_identifier(self) -> str:
>       retv = int(os.stat(self.parameter_filename)[ST_CTIME])
E       FileNotFoundError: [Errno 2] No such file or directory: '/blue/adamginsburg/adamginsburg/repos/spectral-cube/InMemoryFITSFile_d26b1c73bcb149e9a9fc50a3bacda733'

/orange/adamginsburg/miniconda3/envs/python39/lib/python3.9/site-packages/yt/data_objects/static_output.py:315: FileNotFoundError

keflavich · 2023-03-11T03:08:38Z

Note that this depends on astropy/reproject#349

setup.cfg

e-koch

@keflavich Looks good. Just some questions on the LDO creation without a WCS and whether that should be allowed or not.

spectral_cube/base_class.py

spectral_cube/tests/test_projection.py

expand test coverage fix test failure for dask trivial whitespace change? try to emulate astropy/astroquery#2626 that last one seems to maybe have broken CI? undoing the CI changes try putting things on one line ? try changing ubuntu-latest to linux based on the versions-manifest, which doesn't mention ubuntu at all try changing 3.10 -> '3.10' instead of the linux change shorten job names so we can see the whole name in the sidebar of gh-actions add a mask setter to solve problem where numpy masked arrays are trying to set the mask attribute but being blocked from doing so drop openfiles b/c it's deprecated (not sure what we're supposed to replace it with yet) make some of the failing tests more extensive and explicit

…ming from (pdb revealed that code that succeeded on the previous line is the direct source of this failure) fix mistake add a new test for "all" add all-dev and remove -openfiles from windoze move dev dependenc installs from tox to setup.cfg whitespace and naming expand test coverage yet again: try dev w/o novis/all separate out 'viz' from 'noviz' fix a typo.... add extras that were all missing into tox.ini make sure basic tests use latest numpy rearrange and make tests more granular still drop 3.7 from grid b/c astropy 5.2.1 doesn't support it see if pvextractor can be included in the "good" viz hack around astropy 14514 hack needs to be hackier b/c _new_view didn't always have finalize= option try to run tests w/dev branch fix syntax fix install url fix name again? try to fix last exception allow for three types of error? this might not be the right approach fix test again remove mask setter - I'm not sure it was actually needed, I think that might have been a side-effect error caused by something else? Pushing this to run tests to check; may neeed to partially revert this if tests fail set ndim fix the naxis defn restore the mask setter, it _is_ needed reproject high_level_wcs fix merged; revert to master in tests fix names of # of dims

keflavich · 2023-03-13T16:14:36Z

ah crud. Everything is not fine.

…ke tests go green again

keflavich · 2023-03-13T16:37:02Z

OK so something changed in astropy-dev in the last 4 days that broke this. I'll bisect this time.

keflavich · 2023-03-13T17:10:33Z

Broken by astropy/astropy#14508. Can't stay ahead of astropy!

keflavich · 2023-03-13T17:51:40Z

In the last commit, I was trying to replicate this line:
https://github.com/larrybradley/jdaviz/blob/2ad11c120a7317c01c0ee6957c19c494a54e7428/tox.ini#L62

what did I get wrong?

… go green for <24h b/c the breaking change is <8h old, but the intent is to make builds faster change order of commands oh, I just had stupidly pasted in setup.cfg too

keflavich mentioned this pull request Dec 18, 2022

New version planned soon? #853

Closed

keflavich force-pushed the dec2022 branch from cb03404 to fa9cd77 Compare March 9, 2023 12:57

keflavich mentioned this pull request Mar 10, 2023

Add a mask-setter property #869

Closed

keflavich requested a review from e-koch March 11, 2023 03:08

keflavich commented Mar 11, 2023

View reviewed changes

setup.cfg Show resolved Hide resolved

e-koch reviewed Mar 11, 2023

View reviewed changes

spectral_cube/base_class.py Show resolved Hide resolved

spectral_cube/tests/test_projection.py Outdated Show resolved Hide resolved

keflavich added 2 commits March 13, 2023 11:51

keflavich force-pushed the dec2022 branch from 4bfa0ff to 1abb821 Compare March 13, 2023 15:52

revert to my branch, even though it was merged, to see if that can ma…

6951e56

…ke tests go green again

try using astropy nightly wheels. This might inadvertently make tests…

ef841c0

… go green for <24h b/c the breaking change is <8h old, but the intent is to make builds faster change order of commands oh, I just had stupidly pasted in setup.cfg too

keflavich force-pushed the dec2022 branch from 08ed797 to ef841c0 Compare March 13, 2023 18:09

keflavich mentioned this pull request Mar 13, 2023

Cube mosaicing: Expand capabilities substantially #868

Open

dhomeier mentioned this pull request Mar 13, 2023

Revert "Fixed incorrect string representation of floats in Cards" astropy/astropy#14524

Closed

put some sticky tape over the check engine light.

d0b15f9

e-koch approved these changes Mar 13, 2023

View reviewed changes

keflavich merged commit 8747c37 into radio-astro-tools:master Mar 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Expand test coverage #854

Expand test coverage #854

keflavich commented Dec 18, 2022

keflavich commented Dec 18, 2022

dhomeier commented Dec 19, 2022 •

edited

keflavich commented Mar 9, 2023

codecov-commenter commented Mar 9, 2023 •

edited

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 11, 2023

keflavich commented Mar 11, 2023

keflavich commented Mar 11, 2023

keflavich commented Mar 11, 2023

e-koch left a comment

keflavich commented Mar 13, 2023

keflavich commented Mar 13, 2023

keflavich commented Mar 13, 2023

keflavich commented Mar 13, 2023

Expand test coverage #854

Expand test coverage #854

Conversation

keflavich commented Dec 18, 2022

keflavich commented Dec 18, 2022

dhomeier commented Dec 19, 2022 • edited

keflavich commented Mar 9, 2023

codecov-commenter commented Mar 9, 2023 • edited

Codecov Report

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 10, 2023

keflavich commented Mar 11, 2023

keflavich commented Mar 11, 2023

keflavich commented Mar 11, 2023

keflavich commented Mar 11, 2023

e-koch left a comment

Choose a reason for hiding this comment

keflavich commented Mar 13, 2023

keflavich commented Mar 13, 2023

keflavich commented Mar 13, 2023

keflavich commented Mar 13, 2023

dhomeier commented Dec 19, 2022 •

edited

codecov-commenter commented Mar 9, 2023 •

edited