Partial Fix: Change order of arguments for adding auxiliary data for 3.0 release (Issue #3811) #4532

talagayev · 2024-03-26T14:48:20Z

Partially Fixes #3811

Partially, since currently i modified the order of add_auxiliary, but retained aux_data as optional, since some of the pytests, which would have only aux_spec would display errors in pytests and since it is already multiple files, that were modified i thought that it makes sense to first change the order and then make it mandatory if desired.

Changes made in this Pull Request:

Change order of arguments of aux_data and aux_spec in package/MDAnalysis/coordinates/base.py.
Adjusted the Order of those arguments in testsuite/MDAnalysisTests/auxiliary and testsuite/MDAnalysisTests/coordinates tests that used add_auxiliary
Changed the order of add_auxiliary in package/MDAnalysis/coordinates/base.py in copy in classes class ReaderBase and class SingleFrameReaderBase that both used this function
Changed the order of add_auxiliary in the docs of package/MDAnalysis/auxiliary/EDR.py and package/MDAnalysis/coordinates/base.py

I am not sure if these are all of the docs where add_auxiliary is used, i checked the package and didn't find any other cases of usage of add_auxiliary in docs, except for the ones that i modified

PR Checklist

Tests?
Docs?
CHANGELOG updated?
Issue raised/referenced?

Developers certificate of origin

I certify that this contribution is covered by the LGPLv2.1+ license as defined in our LICENSE and adheres to the Developer Certificate of Origin.

📚 Documentation preview 📚: https://mdanalysis--4532.org.readthedocs.build/en/4532/

…, due to pytests without auxdata

…lass

pep8speaks · 2024-03-26T14:48:31Z

Hello @talagayev! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file package/MDAnalysis/auxiliary/EDR.py:

Line 147:59: W291 trailing whitespace

In the file testsuite/MDAnalysisTests/coordinates/base.py:

Line 216:80: E501 line too long (106 > 79 characters)
Line 217:80: E501 line too long (109 > 79 characters)

Comment last updated at 2024-03-29 21:57:00 UTC

github-actions · 2024-03-26T14:52:14Z

Linter Bot Results:

Hi @talagayev! Thanks for making this PR. We linted your code and found the following:

Some issues were found with the formatting of your code.

Code Location	Outcome
main package	⚠️ Possible failure
testsuite	⚠️ Possible failure

Please have a look at the darker-main-code and darker-test-code steps here for more details: https://github.com/MDAnalysis/mdanalysis/actions/runs/8438378464/job/23110406304

Please note: The black linter is purely informational, you can safely ignore these outcomes if there are no flake8 failures!

codecov · 2024-03-26T15:00:23Z

Codecov Report

Attention: Patch coverage is 33.33333% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 93.63%. Comparing base (f5335b4) to head (6ad9ba0).
Report is 5 commits behind head on develop.

❗ Current head 6ad9ba0 differs from pull request most recent head 0da1436. Consider uploading reports for the commit 0da1436 to get more accurate results

Files	Patch %	Lines
package/MDAnalysis/coordinates/base.py	50.00%	1 Missing ⚠️
package/MDAnalysis/coordinates/memory.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #4532      +/-   ##
===========================================
- Coverage    93.65%   93.63%   -0.03%     
===========================================
  Files          168      180      +12     
  Lines        21215    22294    +1079     
  Branches      3908     3908              
===========================================
+ Hits         19869    20875    +1006     
- Misses         888      961      +73     
  Partials       458      458

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

talagayev · 2024-03-26T15:46:04Z

I checked again which tests would fail if aux_data would be set as mandatory, those would be test_edr.py and test_xvg.py delivering the following error message once in test_edr.py and twice in test_xvg.py

with the error mesage being:

______________ TestEDRReader.test_raise_error_no_auxdata_provided ______________
self = <MDAnalysisTests.auxiliary.test_edr.TestEDRReader object at 0x779471f340b0>
ref = <MDAnalysisTests.auxiliary.test_edr.EDRReference object at 0x77947516a990>
ref_universe = <Universe with 33876 atoms>

`def test_raise_error_no_auxdata_provided(self, ref, ref_universe):
    with pytest.raises(ValueError, match="No input `auxdata`"):
        ref_universe.trajectory.add_auxiliary()
         TypeError: ProtoReader.add_auxiliary() missing 1 required positional argument: 'auxdata'`

all three of the errors would rely on the missing of the required auxdata argument

talagayev · 2024-03-26T16:47:40Z

Quick follow-up on the reason of the errors during test_edr.py and twice in test_xvg.py if aux_data is set as mandatory.

The error in them stands from testsuite/MDAnalysisTests/auxiliary/base.py that has a specific test:

def test_raise_error_no_auxdata_provided(self, ref, ref_universe):

which expects an ValueError, which would occur in the variant where the aux_data is set as optional in add_auxiliary and has a condition:
if auxdata is None: raise ValueError

But gets instead an TypeError currently, now that it is missing aux_data as an mandatory argument. Thus the tests can be adjusted to check for an TypeError instead, which removes the errors in the tests, but then i am not sure if the condition:
if auxdata is None: raise ValueError should also be removed from package/MDAnalysis/coordinates/base.py

IAlibay · 2024-03-26T17:33:23Z

@hmacdope you suggested folks work on this issue, however I'm not sure this can be tackled right now, i.e. this looks like a 3.0 breaking change? (might be missing something)

hmacdope · 2024-03-26T23:06:27Z

@hmacdope you suggested folks work on this issue, however I'm not sure this can be tackled right now, i.e. this looks like a 3.0 breaking change? (might be missing something)

@IAlibay its not mergeable right now, but we could make the change and leave it open for 3.0 and do another PR for a deprecation warning. Would that be OK?

IAlibay · 2024-03-26T23:26:41Z

:/ I think I understand your plan @hmacdope

I'll say this in two parts:

If this is GSoC related (sorry I am not keeping track of contributors), then there's plenty of precedent where we've approved but not merged a PR (for various reasons), so I would advise not seeing any decisions here as affecting contributor efforts. i.e. "symbolic merge" is a legit strategy
(let's move this discussion to an issue or elsewhere) I think it might be ok to "hold" PRs but we might need to come up with a policy for it. We're at least one (if not two) minor versions away, so roughly 6+ months given our current release schedule. It might be hard to do right by contributors if we don't have a good merge/release plan that everyone sticks to.

In the case of aux, it's infrequently touched enough that code conflicts might not be an issue - not sure what @BFedder's remaining plans were.

orbeckst

This PR looks really good and if we could, I'd merge it. For GSOC purposes I'd count the PR.

@hmacdope if you want to count it, please add an approving review.

As per discussion, this has to go into 3.0.0 so I am blocking it.

orbeckst · 2024-03-29T03:43:48Z

package/CHANGELOG

@@ -34,6 +34,7 @@ Fixes
 * Fix deploy action to use the correct version of the pypi upload action.

 Enhancements
+ * Change order of arguments for adding auxiliary data (Issue #3811)


This is a Change so it should go there.

Additionally, it's a breaking change (see @IAlibay 's comments) so it won't go into 2.8.0, which is unfortunate.

I'd suggest to make this clear, start a new section

??/??/?? talagayev * 3.0.0 Changes: ....

orbeckst · 2024-03-29T03:45:18Z

package/MDAnalysis/coordinates/base.py

@@ -1701,7 +1700,7 @@ def copy(self):

        new.ts = self.ts.copy()
        for auxname, auxread in self._auxs.items():
-            new.add_auxiliary(auxname, auxread.copy())
+            new.add_auxiliary(auxread.copy(), auxname)


We'll eventually want this covered by tests. Not sure why it wasn't originally covered.

Ah yes, from what i've seen in the tests there is test_copy_with_auxiliary_pyedr() in test_copying.py, which does call the copy function from the ReaderBase class, but there is no case of testing the copy function of the SingleFrameReaderBase class

orbeckst · 2024-03-29T03:45:36Z

package/MDAnalysis/coordinates/memory.py

@@ -451,7 +451,7 @@ def copy(self):
        new[self.ts.frame]

        for auxname, auxread in self._auxs.items():
-            new.add_auxiliary(auxname, auxread.copy())
+            new.add_auxiliary(auxread.copy(), auxname)


tests... not your fault, though

talagayev · 2024-03-29T16:49:24Z

Should it stay right now as it is, with aux_data optional and be changed to mandatory later on in a follow-up PR?

orbeckst · 2024-03-29T20:35:30Z

Given that this PR has to wait for 3.0, it would make sense to implement the complete change to fully address #3811, i.e., make aux_data mandatory and aux_spec optional.

orbeckst · 2024-03-29T20:36:51Z

I am not quite sure how we will implement deprecation/change warnings. That's a separate issue/PR, though.

orbeckst · 2024-03-29T20:38:28Z

Btw, @talagayev please also click the checkbox for Developer Certificate of Origin in the issue. Without it, we will not be able to ever merge the contribution.

talagayev · 2024-03-29T20:39:31Z

Given that this PR has to wait for 3.0, it would make sense to implement the complete change to fully address #3811, i.e., make aux_data mandatory and aux_spec optional.

Sounds good, will adjust it to make aux_data mandatory and leave aux_spec optional

talagayev · 2024-03-29T20:40:03Z

Btw, @talagayev please also click the checkbox for Developer Certificate of Origin in the issue. Without it, we will not be able to ever merge the contribution.

Done

…ge to mandatory

orbeckst · 2024-03-30T17:11:03Z

@fiona-naughton I've assigned you to keep an eye on this PR, given that you invented the aux system. We can't merge it until 3.0 so there's not much to do right now, just being aware of it. Thank you!

hmacdope · 2024-04-01T11:25:28Z

More than happy for this to count for GSOC.

BFedder · 2024-04-01T15:40:22Z

:/ I think I understand your plan @hmacdope

I'll say this in two parts:

* If this is GSoC related (sorry I am not keeping track of contributors), then there's plenty of precedent where we've approved but not merged a PR (for various reasons), so I would advise not seeing any decisions here as affecting contributor efforts. i.e. "symbolic merge" is a legit strategy

* (let's move this discussion to an issue or elsewhere) I think it might be ok to "hold" PRs but we might need to come up with a policy for it. We're at least one (if not two) minor versions away, so roughly 6+ months given our current release schedule. It might be hard to do right by contributors if we don't have a good merge/release plan that everyone sticks to.

In the case of aux, it's infrequently touched enough that code conflicts might not be an issue - not sure what @BFedder's remaining plans were.

I still hope to get a NumPy reader out at some point, but haven't been able to yet for a variety of reasons. I'll keep these changes here in mind if I do find the time before 3.0. In the meantime, these changes here look good to me.

talagayev added 13 commits March 26, 2024 13:46

Changed auxdata&aux_spec positions, while still leaving both optional…

2c2396c

…, due to pytests without auxdata

changed add_auxiliary order in test_edr.py and removed TODO mentions

fa5d964

changed add_auxiliary order in test_xvg.py and removed TODO mentions

d1f666a

changed order of add_auxiliary in coordinates base.py

1820d12

changed add_auxiliary order in test_gro.py

e65771f

changed add_auxiliary order in test_copying.py

87c63bd

adjusted add_auxiliary order in EDR.py docs

87a0da2

fixed add_auxiliary order in copy function

57bf0f4

fixed add_auxiliary order in copy function of SingleFrameReaderBase c…

6dbb284

…lass

changed order of add_auxiliary in docs in __init__.py

5d4ed18

changed add_auxiliary order in copy function of class MemoryReader

d7147e6

Added name and enhancment to changelog

ddd0ebd

Update AUTHORS list

1c6315c

github-actions bot added the Component-Readers label Mar 26, 2024

fixed add_auxiliary argument order in test_chemfiles.py

6ad9ba0

orbeckst requested changes Mar 29, 2024

View reviewed changes

IAlibay mentioned this pull request Mar 29, 2024

Merge 3.0 pending PRs #4539

Open

1 task

talagayev added 2 commits March 29, 2024 22:55

Changed auxdata to mandatory

8224244

Changed test_raise_error_no_auxdata_provided to TypeError due to chan…

0da1436

…ge to mandatory

orbeckst assigned fiona-naughton Mar 30, 2024

hmacdope approved these changes Apr 1, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Partial Fix: Change order of arguments for adding auxiliary data for 3.0 release (Issue #3811) #4532

Partial Fix: Change order of arguments for adding auxiliary data for 3.0 release (Issue #3811) #4532

talagayev commented Mar 26, 2024 •

edited

pep8speaks commented Mar 26, 2024 •

edited

github-actions bot commented Mar 26, 2024 •

edited

codecov bot commented Mar 26, 2024 •

edited

talagayev commented Mar 26, 2024

talagayev commented Mar 26, 2024

IAlibay commented Mar 26, 2024

hmacdope commented Mar 26, 2024

IAlibay commented Mar 26, 2024

orbeckst left a comment

orbeckst Mar 29, 2024

orbeckst Mar 29, 2024

talagayev Mar 29, 2024

orbeckst Mar 29, 2024

talagayev commented Mar 29, 2024

orbeckst commented Mar 29, 2024

orbeckst commented Mar 29, 2024

orbeckst commented Mar 29, 2024

talagayev commented Mar 29, 2024

talagayev commented Mar 29, 2024

orbeckst commented Mar 30, 2024

hmacdope commented Apr 1, 2024

BFedder commented Apr 1, 2024

Partial Fix: Change order of arguments for adding auxiliary data for 3.0 release (Issue #3811) #4532

Are you sure you want to change the base?

Partial Fix: Change order of arguments for adding auxiliary data for 3.0 release (Issue #3811) #4532

Conversation

talagayev commented Mar 26, 2024 • edited

PR Checklist

Developers certificate of origin

pep8speaks commented Mar 26, 2024 • edited

Comment last updated at 2024-03-29 21:57:00 UTC

github-actions bot commented Mar 26, 2024 • edited

Linter Bot Results:

codecov bot commented Mar 26, 2024 • edited

Codecov Report

talagayev commented Mar 26, 2024

talagayev commented Mar 26, 2024

IAlibay commented Mar 26, 2024

hmacdope commented Mar 26, 2024

IAlibay commented Mar 26, 2024

orbeckst left a comment

Choose a reason for hiding this comment

orbeckst Mar 29, 2024

Choose a reason for hiding this comment

orbeckst Mar 29, 2024

Choose a reason for hiding this comment

talagayev Mar 29, 2024

Choose a reason for hiding this comment

orbeckst Mar 29, 2024

Choose a reason for hiding this comment

talagayev commented Mar 29, 2024

orbeckst commented Mar 29, 2024

orbeckst commented Mar 29, 2024

orbeckst commented Mar 29, 2024

talagayev commented Mar 29, 2024

talagayev commented Mar 29, 2024

orbeckst commented Mar 30, 2024

hmacdope commented Apr 1, 2024

BFedder commented Apr 1, 2024

talagayev commented Mar 26, 2024 •

edited

pep8speaks commented Mar 26, 2024 •

edited

github-actions bot commented Mar 26, 2024 •

edited

codecov bot commented Mar 26, 2024 •

edited