Pytest options – how to skip or run specific tests

In this post, we will see how to use pytest options or parameters to run or skip specific tests.

Pytest options are basically the command line parameters used with pytest to run tests, these options helps while running tests in tolls like jenkins, circle CI or azure-devops environments.

let’s first consider below test methods as our example code & understand each options in detail

# File - test_pyTestOptions.py
# Author - qavbox / qavalidation.com

import pytest


def test_regression():
    print("Test_regression")


def test_sanity():
    print("Test_sanity")


def test_release():
    print("Test_release")


def test_api():
    print("Test_api")

To run this above tests, open command prompt or terminal & navigate to project directory, type

pytest test_pytestOptions.py

This above command will run all the test methods, but will not print the output to console.

To print the output, we have to use -s along with pytest

pytest test_pytestOptions.py -s

To print the console output along with specific test names, we can use the pytest option -v [verbose]

pytest test_pytestOptions.py -sv

Run specific test method

To run specific test method from a specific test file, we can type

pytest test_pytestOptions.py::test_api -sv

This above will run the test method test_api() from file test_pyTestOptions.py

Run tests based on string match

Let’s say you want to run test methods or test classes based on a string match.

Run all test class or test methods whose name matches to the string provided with -k parameter

pytest test_pytestOptions.py -sv -k "release"

This above command will run all test class or test methods whose name matches with “release”

From above test file, test_release() will be running.

Run based on the category match – pytest mark

We can add category name to each test method using pytest.mark

Let’s see below example code

# File - test_pyTestOptions.py
# Author - qavbox / qavalidation.com

import pytest


@pytest.mark.login
def test_regression():
    print("Test_regression")


@pytest.mark.login
def test_sanity():
    print("Test_sanity")


@pytest.mark.login
def test_release():
    print("Test_release")


@pytest.mark.settings
def test_api():
    print("Test_api")
    
    
@pytest.mark.login    
@pytest.mark.settings
def test_api1():
    print("Test_api1")    

To run specific mark or category, we can use the -m parameter

pytest Test_pytestOptions.py -sv -m "login"

While running, you might get error as

Unknown pytest.mark.rel - is this a typo? You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
@pytest.mark.rel

To resolve above error, create a pytest.ini file under root directory and add all the category or marks under this file

Note – after “:” it’s optional, you can just add any description

# file - pytest.ini
# # Author - qavbox / qavalidation.com

[pytest]
markers =
    rel: marks tests as rel (deselect with '-m "not rel"')
    api

Run tests using multiple mark or categories

We can use or and operators and run multiple marks or categories

To run either login or settings related tests

pytest Test_pytestOptions.py -sv -m "login or settings"

All the above tests will be running.

To run tests that has both login & settings

pytest Test_pytestOptions.py -sv -m "login and settings"

This above command will only run method – test_api1()

Exclude or skip tests based on mark

We can use not prefix to the mark to skip specific tests

pytest test_pytestOptions.py -sv -m "not login"

This above code will not run tests with mark login, only settings related tests will be running.

We can use combination of marks with not, means we can include or exclude specific marks at once

pytest test_pytestOptions.py -sv -m "not login and settings"

This above command will only run test method test_api()

Note – you can create different combinations of marks in each test method and run using or and operators to get more understanding.

Skip tests with out pytest options

using @pytest.mark.skip

Using the skip mark in each test method, pytest will skip those tests, let’s see this in action with below example code

# File - test_pyTestOptions.py
# Author - qavbox / qavalidation.com

import pytest


@pytest.mark.login
def test_regression():
    print("Test_regression")


@pytest.mark.login
def test_sanity():
    print("Test_sanity")


@pytest.mark.skip(reason="just testing skip")
def test_release():
    print("Test_release")


@pytest.mark.api
def test_api():
    print("Test_api")

to run the above code, just type

pytest test_pytestOptions.py -sv

This above command will skip the test method – test_release()

Note – reason is optional, but recommended to use, as the analyser will not get confuse why the test skipped, is it intentional or any issue with the run.

Skip the test based on a condition

Pytest provides an option as skipif to use a condition to skip a test, if the condition evaluates to true, then only test will skip else run.

Let’s say, if the os == macos, then skip the test

# File - test_pyTestOptions.py
# Author - qavbox / qavalidation.com

import pytest


@pytest.mark.login
def test_regression():
    print("Test_regression")


@pytest.mark.skipif(os.name == 'posix', reason="do not run on mac os")
def test_sanity():
    print("test_sanity")

Simply type –

pytest test_pytestOptions.py -sv

This above command will run the test method test_regression() if you are running on mac os.

Note – if mac os, then os.name will give the output as “posix”

you can evaluate any condition inside the skipif

Hope this helps!