UnitTests

The unit tests and the test data are bundled together in the package MDAnalyisTests. In order to run the tests, this package must be installed in addition to MDAnalysis.

The tests also rely on the pytest and numpy packages, and require both to run.

Users

Install MDAnalysisTests via pip

pip install --upgrade MDAnalysisTests

or via conda

conda install -c conda-forge MDAnalysisTests

or download the tar file, unpack, and run python setup.py install.

Run the tests by invoking (--disable-pytest-warnings suppresses a large number of harmless warnings)

pytest --disable-pytest-warnings --pyargs MDAnalysisTests

(See Plugins for how to use the pytest-xdist plugin to run tests in parallel on a multicore machine.)

All tests should pass (i.e. no FAIL, ERROR); SKIPPED or XFAIL are ok. For anything that fails or gives an error ask on the user mailing list or raise an issue.

Developers

We are borrowing some of NumPy's testing frame work; thus, numpy must be installed for the tests to run at all.

pytest

It is recommended that you run the tests from the git source repository, which are located in the testsuite/MDAnalysisTests directory:

cd testsuite/MDAnalysisTests
pytest  --disable-pytest-warnings

Note: We use the --disable-pytest-warnings when the whole testsuite is running since we have a lot of false positives when we warn users about missing topology attributes. When running single tests or only single modules consider running the tests with warnings enabled. This allows you to see if you trigger any un-caught deprecation warnings or other warnings in libraries we use.

Running the tests serially can take some time, depending on the performance of your computer. (You can speed this up by running tests in parallel using pytest-xdist - explained in the plugin section)

To run specific tests just specify the path to the test file:

pytest path_to/MDAnalysisTests/analysis/test_align.py

Note: You have to replace path_to with the actual path to where the code is.

Specific test classes inside test files, and even specific test methods, can also be specified:

# Test the entire TestContactMatrix class
pytest path_to/MDAnalysisTests/analysis/test_analysis.py::TestContactMatrix


# Test only test_sparse in the TestContactMatrix class
pytest path_to/MDAnalysisTests/analysis/test_analysis.py::TestContactMatrix::test_sparse

This is very useful when you add a new test and want to check if it passes.

Note on testing Cython functions

Sometimes test failures in compiled functions wrapped in Cython can be obscured by a segfault or other memory error in the compiled layer. If you suspect this may be the case try re-running the tests using a protected memory environment, eg using Valgrind.

Plugins

• pytest-xdist - This can be used to run the tests in parallel.

  pip install pytest-xdist
  pytest --disable-pytest-warnings --numprocesses 4
  

  You can try increasing the number of processes to speed up the test run depending on you machine.

• pytest-cov This can be used to generate the coverage report locally.

  pip install pytest-cov
  pytest --cov=MDAnalysis
  

  Note: You can use the --numprocesses flag with the above command too. This will print the coverage statistic for every module in MDAnalysis at the end of a run. To get detailed line by
  line statistics you can add the --cov-report=html flag. This will create a htmlcov folder (in the directory you run the command from) and there will be an index.html file in this folder, open this file in your browser and you will be able to see overall statistics and detailed line coverage for each file.

pylint

We use pylint for checking source code formatting. Install pylint with conda or pip.

Run the tests with our configuration with

pylint --rcfile=package/.pylintrc package/MDAnalysis
pylint --rcfile=package/.pylintrc testsuite/MDAnalysisTests

This should finish without warnings or errors and show a message such as

-------------------------------------------------------------------
Your code has been rated at 10.00/10

Building docs

We also test that our documentation builds without problems. You will need to have the following packages installed:

• sphinx (install with conda or pip)
• sphinx_sitemap (might require a pip install, pip install sphinx_sitemap as it has not been available on conda-forge)

cd package && python setup.py build_sphinx

Look at the generated html pages in build/sphinx/html/index.html.

Writing test cases

The tests are in a separate package, together with any data files required for running the tests (see Issue 87 for details). Whenever you add a new feature to the code you should also add a test case (ideally, in the same git commit so that the code and the test case are treated as one unit).

Add a test for

• new functionality
• fixed issues (typically named test_IssueXX or referencing the issue in the doc string (to avoid regression)
• anything you think worthwhile – the more the better!

The SciPy testing guidelines are a good howto for writing test cases.

Requirements for tests in MDAnalysisTests

General

• The unit tests use pytest. See the examples in the MDAnalysisTests package.
• Tests must follow the Style Guide: Tests
• Tests are organized by top-level module. Each file containing tests must start with test_ by convention. Tests itself also have to follow the appropriate naming conventions. See the docs above or the source.

Banned code constructs

• Relative import statements are banned from unit testing modules (see Issue #189 for details)
• using os.chdir() is banned because it can break the tests in really weird ways (see Issue #556): use the with tmpdir.as_cwd() context manager of the tmpdir fixture (see Temporary files and directories in the Style Guide for Tests).

Test data

• Test input data is stored in MDAnalysisTests/data.
• Keep files small if possible; for trajectories 10 frames or less are sufficient.
• Add the file name of test data files to MDAnalysisTests/datafiles.py (see the code for details).
• Add the file(s) or a glob pattern to the package_data in setup.py; otherwise the file will not be included in the python package.
• If you use data from a published paper then add a reference to this wiki page and the doc string in MDAnalysisTests/__init__.py – see Data.

Data

The simulation data used in tests are all released under the same license as MDAnalysis or are in the Public Domain (such as PDBs from the Protein Databank). An incomplete list of sources:

• from Beckstein et al. (2009) (adk.psf,adk_dims.dcd)
  • adk_dims Trajectory of a macromolecular transition of the enzyme adenylate kinase between a closed and an open conformation. The simulation was run in CHARMM c35a1.
• unpublished simulations (O. Beckstein)
  • adk_oplsaa Ten frames from the first 1 ns of a equilibrium trajectory of AdK in water with Na+ counter ions. The OPLS/AA forcefield is used with the TIP4P water model. The simulation was run with Gromacs 4.0.2.
• contributions from developers and users
• Protein Databank

References

• O. Beckstein, E.J. Denning, J.R. Perilla and T.B. Woolf, Zipping and Unzipping of Adenylate Kinase: Atomistic Insights into the Ensemble of Open-Closed Transitions. J Mol Biol 394 (2009), 160--176, doi:10.1016/j.jmb.2009.09.009

Changes with releases

The way we organized the unit tests changed between releases. The procedure for the current release is detailed at the very top of the page. The following list is for historical reference and in case you ever want to go back to a previous release.

1. release 0.17.0: the testsuite was overhauled to remove dependency from nose and move to pytest. See issue #884 for more details. pytest is the way to run the tests and mda_nosetests has been removed.
2. since 0.11.0: the testing subsystem was overhauled to allow the use of plugins external to nose. We also no longer use numpy's test() wrapper. mda_nosetests is now the preferred way to run the tests from the command-line in a mostly backward-compatible way with the usage of nosetests. Most numpy-specific arguments to test() are now deprecated in favor of nose flags.
3. since 0.7.5: tests and data are together in package MDAnalysisTests. See Issue 87 for details.
4. release 0.7.4: tests are in MDAnalysis and data is in MDAnalysisTestData (for MDAnalysis == 0.7.4). To install MDAnalysisTestData download the MDAnalysisTestData-0.7.4.tar.gz from the Download section or try easy_install http://mdanalysis.googlecode.com/files/MDAnalysisTestData-0.7.4.tar.gz
5. release 0.6.1 to 0.7.3: tests and data were included with MDAnalysis
6. release 0.4 to 0.6.0: no tests included