Featured image of post Building and Publishing a Custom Pytest Plugin

Building and Publishing a Custom Pytest Plugin

Preface

PyTest is a testing framework that is very widely used in the field of test automation today. Automation based on Python scripts usually relies on this framework to manage test scripts and schedule their execution.

What makes Pytest powerful, besides the Fixture feature we introduced earlier, is another very important aspect: its strong ecosystem, namely its very powerful plugin support. The official plugin list alone already contains more than 1,600 plugins

In real-world work, besides enhancing capabilities through Pytest’s rich plugin ecosystem, it is actually very convenient for us to develop our own plugins to meet some of our individualized requirements; this, of course, is also the foundation of why Pytest’s plugin ecosystem can be so flourishing.

In this article, we will use a basic example to introduce how to develop a local Pytest plugin.

How Pytest Plugins Work

Pytest’s plugin mechanism is actually implemented mainly through the hook functions it provides and through command-line extensions.

Hook Functions

Hook functions are a class of special functions provided by Pytest that allow plugins to execute custom code when specific test events occur. Commonly used hook functions include:

  • pytest_addoption: Used to add command-line options.
  • pytest_configure: Performs configuration before the test run.
  • pytest_runtest_setup: Called before each test case is executed.
  • pytest_runtest_teardown: Called after each test case is executed.
  • pytest_sessionfinish: Called when the test session ends.

Command-Line Options

Through the pytest_addoption hook, a plugin can define custom command-line options. This allows users to control the plugin’s behavior when running tests. For example, an option can be added to enable or disable certain features.

The Configuration Object

Within hook functions, a plugin can access Pytest’s configuration object (config), which contains command-line options, the test context, and other configuration information. Through the configuration object, a plugin can obtain the options entered by the user and adjust its behavior accordingly.

Data Storage

A plugin can use the configuration object within hook functions to store and manage data. For example, it can initialize a list in pytest_configure, store data into this list after each test case is executed, and finally output the results in pytest_sessionfinish.

The hookimpl Decorator

In Pytest, hook functions can use the @pytest.hookimpl decorator to control the execution order and behavior of hooks. The main parameters include:

  • tryfirst: If set to True, this hook will run before other hooks of the same kind. It is usually used when specific logic needs to be handled with priority.
  • trylast: The opposite of tryfirst; if set to True, this hook will run after other hooks of the same kind.

In this way, a plugin can precisely control the insertion of custom logic at various stages of the test execution process.

A Hands-On Example

In this article, we will develop a local Pytest plugin that records the execution time of each test case and provides summary statistics after the test run ends.

Environment Setup

Create a new directory for your plugin:

1
2
mkdir pytest_timing_plugin
cd pytest_timing_plugin

Plugin Implementation Code

In the plugin directory, create a file named pytest_timing_plugin.py:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
# pytest_timing_plugin.py

import pytest
import time

def pytest_addoption(parser):
    parser.addoption("--timing", action="store_true", help="Enable timing report")

@pytest.hookimpl(tryfirst=True)
def pytest_configure(config):
    if config.getoption("timing"):
        config._timing_data = []
        config._total_duration = 0.0

@pytest.hookimpl(tryfirst=True)
def pytest_runtest_setup(item):
    if item.config.getoption("timing"):
        item.start_time = time.time()

@pytest.hookimpl(tryfirst=True)
def pytest_runtest_teardown(item):
    if item.config.getoption("timing"):
        duration = time.time() - item.start_time
        item.config._timing_data.append((item.nodeid, duration))
        item.config._total_duration += duration

@pytest.hookimpl(tryfirst=True)
def pytest_sessionfinish(session, exitstatus):
    if session.config.getoption("timing"):
        print("\nTiming Report:")
        for test_id, duration in session.config._timing_data:
            print(f"{test_id}: {duration:.4f} seconds")
        print(f"Total Duration: {session.config._total_duration:.4f} seconds")
  • pytest_addoption: Adds the --timing option to enable timing.
  • pytest_runtest_setup: Records the start time when each test begins.
  • pytest_runtest_teardown: Calculates and records the duration when each test ends, and accumulates it into the total duration.
  • pytest_sessionfinish: Prints the execution time report for each test and the total execution time when the test session ends.

Preparing the Test File

In the same directory, create a test file test_sample.py:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
# test_sample.py

import time

def test_example_1():
    time.sleep(1)
    assert True

def test_example_2():
    time.sleep(2)
    assert True

def test_example_3():
    time.sleep(1.5)
    assert True

Packaging the Plugin

If we want to share the plugin with others, we can package it. Create a setup.py file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
# setup.py

from setuptools import setup

setup(
    name='pytest-timing-plugin',
    version='0.1',
    description='A pytest plugin to measure and report test execution time',
    py_modules=['pytest_timing_plugin'],
    entry_points={
        'pytest11': [
            'timing_plugin = pytest_timing_plugin',
        ],
    },
)

In the plugin directory, run:

1
pip install .

This includes the plugin we just developed in the local Pytest environment.

Running the Tests

Run the tests with the pytest command, passing the custom option at the same time:

1
pytest --timing

The run result:

Publishing the Plugin

And if we want to publicly release the plugin we developed on a wider scale, we can publish it to PyPI for broader dissemination and adoption.

Preparing for Release

Install the relevant publishing tools:

1
pip install setuptools twine
  • setuptools: Used to package Python projects
  • twine: Used to upload packages to PyPI

And upload the code to an open source platform such as Github

Preparing setup.py

Add the following content, including the source code project:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# setup.py

from setuptools import setup

setup(
    name='pytest-timing-plugin',
    version='0.5',  # updated version number
    description='A pytest plugin to measure and report test execution time',
    py_modules=['pytest_timing_plugin'],
    entry_points={
        'pytest11': [
            'timing_plugin = pytest_timing_plugin',
        ],
    },
    author='城下秋草',
    author_email='your.email@example.com',
    url='https://github.com/chengxiaqiucao/pytest-timing-plugin',
    classifiers=[
        'Programming Language :: Python :: 3',
        'Framework :: Pytest',
        'License :: OSI Approved :: MIT License',
    ],
)

Creating README

Create a README.md file describing the plugin’s features and usage.

Building the Distribution Package

In the plugin directory, run the following command to generate the distribution package:

1
python setup.py sdist bdist_wheel

This generates .tar.gz and .whl files in the dist directory.

Publishing to PyPI

Use twine to upload the package to PyPI. First of all, of course, we need to have an account on PyPI.

Then, run the following command:

1
python -m twine upload dist/*

Enter the PyPI API Token as prompted to complete the upload

Once complete, the plugin can be found on the PyPI official site

Verifying the Installation

After a successful upload, you can install the plugin in other environments with the following command:

1
pip install pytest-timing-plugin

Updating the Plugin

If you need to update the plugin, simply update the version number in setup.py, regenerate the distribution package, and then upload it again with twine.


The above is a practical example of Pytest plugin development

Built with Hugo
Theme Stack designed by Jimmy