What Is `pytest.ini` And How To Save Time Using Pytest Config

Posted on Edited on In , Word count in article: 1.4k Reading time ≈ 5 mins.

Staying up-to-date on pytest CLI commands can be painful.

When running tests via the console you have to remember to log the output, runtime environment variables, set timeouts, coverage, parallel executions, requirements and many others.

Programmers are good at logic, not remembering things.

That’s why we made config files. They let you specify the settings once and move on with writing your tests.

So how do you specify config in Pytest?

There are a few ways — pytest.ini, pyproject.toml, tox.ini, setup.config are some of these.

pytest.ini takes priority over other config files, even when empty.

In this article we’ll focus primarily on pytest.ini but you can find the documentation for others on the pytest official site.

This article will give you a fairly comprehensive overview of how to set up and use pytest.ini across your tests, with an example.

We’ll look at how to set up console logs format, timeout, environment variables, test paths, custom markers, minimum versions and others.

After this, you’ll have a good understanding of how to set up config in pytest-ini.

  • What Is pytest.ini And Why Is It Useful?
  • How To Use Pytest.ini — With Examples
    • Quick Repo Overview
    • Where To Store pytest.ini File
  • Using Pytest.ini
    • Logging
    • Output Console Logs To Log File
    • Timeout Slow Tests
    • Define Environment Variables
    • Minimum Version
    • Set Required Plugins
    • Use Custom Markers
  • Conclusion
  • Additional Reading

Let’s get started then?

Link To GitHub Repo

What Is pytest.ini And Why Is It Useful?

Before we get into the meat of the article and how to use pytest-ini, let’s quickly define what is a config file and its use (in the pytest context).

Config files help you define how you want each program or unit test to behave on execution.

Without it, you would need to specify how the test should work every time you run it, often using several CLI commands.

As an example, perhaps you want to see all INFOlogs, or DEBUG . Maybe you want to produce a coverage report on each run.

Or you want to log all runs into a separate log file that you can export later.

Perhaps you want to define common environment variables for all your tests.

Imagine passing CLI commands for each of these.

Not only could you forget some, but it also gets incredibly messy.

Config files are designed to alleviate these complexities and pytest automatically recognises the values defined in these files. Brilliant, isn’t it?

As I mentioned in the introduction there are several types of config files but pytest.ini takes precedence over the others.

How To Use Pytest.ini — With Examples

Let’s get to the fun part. How to use pytest.ini .

Quick Repo Overview

If you navigate to the test repo you can see we have a simple function in core.py that accepts a string as an argument and capitalizes it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
def capitalise(value: str) -> str:  
    """  
    Function to Capitalise a String  
    :param value: Input string, str  
    :return: Capitalised string  
    """  
    if isinstance(value, str):  
        logging.debug(f"Input Value: {value}")  
        cap_value = value.upper()  
        logging.info(f"Capitalised Value: {cap_value}")  
        return cap_value  
    else:  
        logging.error("Incorrect Input Value!!")  
        raise ValueError("Input Value MUST be string")

We’ve defined the Unit Tests under tests/unit/ — where there are 3 tests.

  1. Test the capitalise functionality
  2. Test that a ValueError is raised if for an incorrect input type.
  3. Test if environment variables (dummy) are defined

Where To Store pytest.ini File

We’ve placed the pytest.ini file at the root of the repo here. You can place it within the tests directory.

Using Pytest.ini

In this section, we’ll cover some of the important pytest.ini settings that are commonly used in testing.

Logging

How should your logs look like? How to make logs more readable and less noisy?

Should you print logs for INFO, DEBUG, and ERROR levels?

All of this can be easily defined within pytest.ini and will be applied across all your tests.

For example, by setting our logging levels to DEBUG we can see the execution results below.

Note the use of custom log_date_format.

1
2
3
4
5
[pytest]  
log_cli=true  
log_level=DEBUG  
log_format = %(asctime)s %(levelname)s %(message)s  
log_date_format = %Y-%m-%d %H:%M:%S

pytest-ini-1

Output Console Logs To Log File

Perhaps you want to keep a record of all unit tests that ran as part of a CI/CD pipeline?

Maybe your latest release broke something and you wanna review the test logs.

Or stream them into a search database like Elasticsearch.

Pytest test execution run outputs can be nicely captured into a log file using the log_file command in pytest.ini

1
2
[pytest]  
log_file = logs/pytest-logs.txt

Along with pytest code coverage reports these offer really good accountability and traceability.

Timeout Slow Tests

Its also really easy to specify pytest-timeout in our pytest.ini file.

If you’re not familiar with timeout or hanging tests, this short article on pytest-timeout is an excellent resource.

1
2
[pytest]  
timeout=5

To test this let’s quickly add a delay to our capitalise function

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
def capitalise(value: str) -> str:  
    """  
    Function to Capitalise a String  
    :param value: Input string, str  
    :return: Capitalised string  
    """  
    if isinstance(value, str):  
        logging.debug(f"Input Value: {value}")  
        time.sleep(10)  # ADD Delay - Comment if not needed  
        cap_value = value.upper()  
        logging.info(f"Capitalised Value: {cap_value}")  
        return cap_value  
    else:  
        logging.error("Incorrect Input Value!!")  
        raise ValueError("Input Value MUST be string")

pytest-ini-2

As expected, our test timed out after 5 seconds resulting in failure.

Define Environment Variables

There are a few ways to define runtime environment variables in pytest, namely

  1. Via CLI Commands
  2. Via Pytest.ini
  3. Within the test file

This article on Pytest-environment variables goes into more detail with examples.

You can define runtime environment variables within pytest.ini and share them across tests.

1
2
3
4
[pytest]  
env =  
    ENVIRONMENT=dev  
    ACCOUNT=12345

For example, these 2 environment variables can automatically be accessed like this

pytest-ini-3

Minimum Version

You can also specify the required minimum version of Pytest.

This is particularly useful in the case when you’re testing functionality that’s not compatible with older Pytest versions.

For example, I’ve specified that we need pytest==7.2 and actually installed pytest==7.1.3 .

1
2
[pytest]  
minversion = 7.2

As expected, this will throw an error.

pytest-ini-4

Set Required Plugins

Often it’s useful to include a condition that all tests must have important plugins installed before execution.

For example, pytest-xdist is an important plugin that helps you run tests in parallel, pytest-mock allows you to create and reference mock objects in your code.

Similarly, pytest-asyncio allows you to test asynchronous code. You can set the standards within your pytest.ini file like this

1
2
[pytest]  
required_plugins = pytest-xdist>=3.2.0 pytest-env<=0.8.0

pytest-ini-5

You can accordingly specify any important packages and pytest will only run the tests if those requirements are met.

Use Custom Markers

Custom markers are very handy when you want to run specific tests.

Rather than specify the name of the test in the CLI command you can simply pass the -m marker flag and pytest will only run the tests that have that marker.

For example, we can apply the @pytest.mark.core marker on our Unit test test_capitalise thus marking this as a core functionality test (for example if we just wanna check it still works).

Similarly, you can set markers to mark tests that are purely for local testing and shouldn’t be run as part of your CI/CD pipeline. Or tests that should be run only on Windows instead of Linux/Mac.

We then specify it within our pytest.ini file as below.

1
2
3
4
5
[pytest]  
addopts = --strict-markers  
markers =  
    core: marks tests as core (deselect with '-m "not core"')  
    serial

Running this using the pytest -m core CLI command we get

pytest-ini-6

Conclusion

In this article, we only scratched the surface of pytest config and the various settings you can apply to configure pytest to your company’s needs and policies.

We looked at what a config file is and why it’s so much more helpful than passing a bunch of CLI commands.

We also looked at commonly used settings like environment variables, logging, package version, timeout and custom markers.

There are many config options to play with, and you’ll find information on how to configure them in the pytest documentation.

Overall I hope this article has been helpful.

If you have any ideas for improvement or like me to cover any topics please comment below or send me a message via Twitter, GitHub or Email.

Till the next time… Cheers!

Additional Reading

https://buildmedia.readthedocs.org/media/pdf/pytest/latest/pytest.pdf

https://docs.pytest.org/en/7.1.x/reference/reference.html

https://chromium.googlesource.com/chromiumos/chromite/+/HEAD/pytest.ini