Skip to content

Latest commit

 

History

History
168 lines (107 loc) · 6.88 KB

mark.rst

File metadata and controls

168 lines (107 loc) · 6.88 KB
Raw

Marking test functions with attributes

By using the pytest.mark helper you can easily set metadata on your test functions. There are some builtin markers, for example:

  • skip <skip> - always skip a test function
  • skipif <skipif> - skip a test function if a certain condition is met
  • xfail <xfail> - produce an "expected failure" outcome if a certain condition is met
  • parametrize <parametrizemark> to perform multiple calls to the same test function.

It's easy to create custom markers or to apply markers to whole test classes or modules. See mark examples for examples which also serve as documentation.

Note

Marks can only be applied to tests, having no effect on fixtures <fixtures>.

Raising errors on unknown marks

Unknown marks applied with the @pytest.mark.name_of_the_mark decorator will always emit a warning, in order to avoid silently doing something surprising due to mis-typed names. You can disable the warning for custom marks by registering them in pytest.ini like this:

[pytest]
markers =
    slow
    serial

When the --strict command-line flag is passed, any unknown marks applied with the @pytest.mark.name_of_the_mark decorator will trigger an error. Marks added by pytest or by a plugin instead of the decorator will not trigger the warning or this error. Test suites that want to enforce a limited set of markers can add --strict to addopts:

[pytest]
addopts = --strict
markers =
    slow
    serial

Third-party plugins should always register their markers <registering-markers> so that they appear in pytest's help text and do not emit warnings.

Marker revamp and iteration

3.6

pytest's marker implementation traditionally worked by simply updating the __dict__ attribute of functions to cumulatively add markers. As a result, markers would unintentionally be passed along class hierarchies in surprising ways. Further, the API for retrieving them was inconsistent, as markers from parameterization would be stored differently than markers applied using the @pytest.mark decorator and markers added via node.add_marker.

This state of things made it technically next to impossible to use data from markers correctly without having a deep understanding of the internals, leading to subtle and hard to understand bugs in more advanced usages.

Depending on how a marker got declared/changed one would get either a MarkerInfo which might contain markers from sibling classes, MarkDecorators when marks came from parameterization or from a node.add_marker call, discarding prior marks. Also MarkerInfo acts like a single mark, when it in fact represents a merged view on multiple marks with the same name.

On top of that markers were not accessible the same way for modules, classes, and functions/methods. In fact, markers were only accessible in functions, even if they were declared on classes/modules.

A new API to access markers has been introduced in pytest 3.6 in order to solve the problems with the initial design, providing _pytest.nodes.Node.iter_markers method to iterate over markers in a consistent manner and reworking the internals, which solved great deal of problems with the initial design.

Updating code

The old Node.get_marker(name) function is considered deprecated because it returns an internal MarkerInfo object which contains the merged name, *args and **kwargs of all the markers which apply to that node.

In general there are two scenarios on how markers should be handled:

1. Marks overwrite each other. Order matters but you only want to think of your mark as a single item. E.g. log_level('info') at a module level can be overwritten by log_level('debug') for a specific test.

In this case, use Node.get_closest_marker(name):

# replace this:
marker = item.get_marker("log_level")
if marker:
    level = marker.args[0]

# by this:
marker = item.get_closest_marker("log_level")
if marker:
    level = marker.args[0]

2. Marks compose in an additive manner. E.g. skipif(condition) marks mean you just want to evaluate all of them, order doesn't even matter. You probably want to think of your marks as a set here.

In this case iterate over each mark and handle their *args and **kwargs individually.

# replace this
skipif = item.get_marker("skipif")
if skipif:
    for condition in skipif.args:
        # eval condition
        ...

# by this:
for skipif in item.iter_markers("skipif"):
    condition = skipif.args[0]
    # eval condition

If you are unsure or have any questions, please consider opening an issue.

Related issues

Here is a non-exhaustive list of issues fixed by the new implementation:

  • Marks don't pick up nested classes (#199).
  • Markers stain on all related classes (#568).
  • Combining marks - args and kwargs calculation (#2897).
  • request.node.get_marker('name') returns None for markers applied in classes (#902).
  • Marks applied in parametrize are stored as markdecorator (#2400).
  • Fix marker interaction in a backward incompatible way (#1670).
  • Refactor marks to get rid of the current "marks transfer" mechanism (#2363).
  • Introduce FunctionDefinition node, use it in generate_tests (#2522).
  • Remove named marker attributes and collect markers in items (#891).
  • skipif mark from parametrize hides module level skipif mark (#1540).
  • skipif + parametrize not skipping tests (#1296).
  • Marker transfer incompatible with inheritance (#535).

More details can be found in the original PR.

Note

in a future major relase of pytest we will introduce class based markers, at which point markers will no longer be limited to instances of :pyMark