Pytest Config Files - A Practical Guide To Good Config Management

Posted on 2024-01-16 Edited on 2024-11-13 In Beginner , Pytest Configuration Word count in article: 2.5k Reading time ≈ 9 mins.

Have you ever spent more time fiddling with test settings than actually writing tests?

Maybe you’ve been through the Pytest docs and seen a bunch of config (short for Configuration) files like pytest.ini, tox.ini, pyproject.toml, and setup.cfg and wondered what they are and how to use them?

Or perhaps you’re familiar with these files but did you know you can also define Pytest config in files like pyproject.toml and setup.cfg?

Well, you’ve come to the right place.

This article will teach you all you need to know to leverage Pytest config files to enhance the speed and efficiency of your testing process.

Config files are a powerful tool in your Python kit, from building packages to testing modules.

Luckily, Pytest supports all the popular Python config files, making it easy to customize your test environment.

You’ll learn how to customize Pytest behavior using Pytest config files - pytest.ini, pyproject.toml, setup.cfg and tox.ini, so you can even bundle your test and build configurations together.

Learning this means you’ll have more control over your build and test environment, making it easier to write and run tests efficiently.

Excited to learn? Let’s jump right in!

Example Code

What You’ll Learn

In this article, you’ll learn to:

Customize the default Pytest settings for your project.
Set Pytest configuration using different Pytest config files - pytest.ini, tox.ini, pyproject.toml, and setup.cfg.
Learn the above using practical examples.

What Is A Pytest Config File?

Let’s start by actually defining what a config file is.

Config files are files that contain configuration settings for an application or framework.

In Python, there are many types of config files, notably pyproject.toml , setup.cfg and so on.

They allow you to specify the settings for your project, including metadata, dependencies, and development dependencies.

Now Pytest has it’s own default config file called pytest.ini, which takes precedence over other config files.

If you’ve never used pytest.ini before or are new to Pytest, you may find this article on pytest.ini helpful.

Pytest Config files enable you to specify Pytest specific settings (like minimum Pytest version, environment variables, testpath, command line options) or customize the default configuration based on your test requirements.

The beauty of Pytest is that it also supports all the popular Python config files, making it easy to customize your test environment.

Why Use Pytest Config Files?

You might wonder, why should you use a Pytest Config file. Can you not define the settings in the command line or in the test files themselves?

Besides being incredibly tedious to define config for each test (when you have hundreds), there are a few reasons and why Pytest config files are useful:

Consistency across Runs: With a Pytest config file, you can define default Pytest settings centrally, ensuring consistent behavior across different runs. It helps you run your tests in different local environments or CI platforms with peace of mind.
Test Discovery Patterns: If you have special non-standard class and function naming convention, You can define patterns for test discovery, specifying which files, classes, or functions should be considered as tests via a Pytest config file.
Custom Markers: Pytest Markers allow you to label and categorize tests. This can be useful for selectively running or excluding specific groups of tests based on markers. Pytest config files contain marker definitions that can be used across all the tests and provide a centralized way to manage markers.
Fixture Usage: If you’re using fixtures, Pytest config files like conftest.py allow you to easily specify and share fixtures across tests. This reduces the need to repeat fixture definitions in each test file.
Plugin Configuration: Pytest supports a large set of plugins that enhance its functionality. Using Pytest config files, you can enable or disable plugins for specific tests. This makes it easy to manage the overall testing environment.
Custom Command-Line Options: Pytest config files allow you to include custom command-line options, making it easier to pass specific settings without the need for 2-3 lines of Command-Line arguments.
Version Controlling: Pytest config files provide a standardized way to manage testing configurations across different versions of Pytest.
Logging and Reporting: Pytest config files allow to customize logging settings, specifying how the testing information is recorded and displayed. It also helps you control the verbosity of outputs, making it easier to identify and troubleshoot issues during test runs.

The Pytest config file is like a handler for maintaining the overall test environment and can help speed up your tests, by feeding Pytest with the right information.

There’s an endless list of use cases, but having learnt a few let’s move onto something practical and check out how to use Pytest Config Files.

Types of Config Files

Pytest initially supported only the pytest.ini config file.

However due to incredible work by the community, it now supports all the popular Python config files, including pyproject.toml, setup.cfg and even tox.ini.

pytest.ini

pytest.ini is the default Pytest config file that can be used to specify just about any Pytest setting.

It takes precedence over other config files, even when empty.

It allows you to specify minimum Pytest version, configure markers with the markers option to label and categorize tests, configure logging, reporting, and other Pytest-specific settings.

pytest.ini defines its configuration under the [pytest] section. Here is a simple pytest.ini file,

[pytest]
addopts = -v
python_files = test_*.py
python_functions = test_*
markers =
    slow: marks tests as slow (deselect with '-m "not slow"')

We’ll cover some important pytest.ini configurations later in this article.

pyproject.toml

pyproject.toml is the new defacto Python config file for dependency management and packaging. You can declare project metadata, dependencies, and development dependencies.

It allows you to specify Python version compatibility. Below is a simple pyproject.toml file,

[tool.poetry]
name = "your_project_name"
version = "0.1.0"
description = "Your project description"
authors = ["Your Name <your.email@example.com>"]
python = "^3.8"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.black]
line-length = 88
target-version = ["py38"]

[tool.flake8]
max-line-length = 88

You can use tooling like Poetry or Flit to build and package your project using pyproject.toml.

Don’t worry, we’ll shortly see how to define config in each of these Pytest config files.

setup.cfg

setup.cfg is the legacy configuration file for Python packaging.

You can easily define package metadata like name, version, description, and author. It also supports specifying package classifiers and keywords. Following is a simple setup.cfg file:

[metadata]
name = your_project_name
version = 0.1.0
description = Your project description
author = Your Name
author_email = your.email@example.com

[options]
packages = find:

[tool:pytest]
addopts = -v

tox.ini

Tox is a useful tool to automate and standardize testing in different environments. It enables you to test environments and dependencies for testing across different Python versions.

Tox operates via an ini file called tox.ini.

Conveniently, you can also define Pytest configurations in tox.ini.

A simple tox.ini looks like the below,

[tox]
envlist = py36, py37, py38

[testenv]
deps = pytest
commands = pytest

You can learn more about tox in our detailed step-by-step guide on How To Test And Build Python Packages With Pytest, Tox And Poetry.

Now that you’ve had a sneak peak at the different config files, let’s learn how to use them in a practical setting.

Practical Example

First, let’s set up your environment.

Prerequisites

To follow this guide, you should have basic working knowledge of:

Python (I’ve used 3.12 but any recent version should work)
Pytest

Getting Started

Our example repo looks like this:

├── .gitignore
├── README.md
├── pyproject.toml
├── pytest.ini
├── requirements.txt
├── setup.cfg
├── src
│   └── distance_calculator.py
├── tests
│   ├── __init__.py
│   └── test_distance_calculator.py
└── tox.ini

To get started. clone the Github Repo here, or you can create your own repo by creating a folder and running git init to initialize it.

Create a virtual environment and install any requirements (packages) using

1	pip install -r requirements.txt

Example Code

Our example is a straightforward - kilometer to mile convertor.

src/distance_calculator.py

def kilometer_to_mile(k: float) -> float:
    """
    Function to convert Kilometer to Mile
    """
    return k * 0.621371


def mile_to_kilometer(m: float) -> float:
    """
    Function to convert Mile to Kilometer
    """
    return m / 0.621371

Testing The Code

Let’s write a couple of simple tests for our example code.

tests/test_distance_calculator.py

from src.distance_calculator import kilometer_to_mile, mile_to_kilometer

def test_kilometer_to_mile():
    assert kilometer_to_mile(5) == 3.106855

def test_mile_to_kilometer():
    assert mile_to_kilometer(3.106855) == 5

Let’s configure the Pytest settings via different config files.

Note - Ensure to always place the config files in the root of your project directory.

Using `pytest.ini`

Pytest config can be defined in pytest.ini under the [pytest] section`.

pytest.ini

[pytest]
testpaths =
    tests
    integration
addopts = -v -ra -q
log_cli = True
log_cli_level = INFO
log_format = %(asctime)s %(levelname)s %(message)s
log_date_format = %Y-%m-%d %H:%M:%S
minversion = 6.0
filterwarnings = ignore
norecursedirs = docs build

# Specify a custom directory for pytest cache
cache_dir = .pytest_cache

Here we specified some configurations for the test environment.

testpaths: Specify the directories to search for tests.
addopts: Specify additional command line options for Pytest.
log_cli: Enable logging to the console.
log_cli_level: Set the logging level to INFO.
log_format: Set the logging format.
log_date_format: Set the logging date format.
minversion: Specify the minimum version of Pytest required for running the tests.
filterwarnings: Filter out specific warning messages during test execution.
cache_dir: Set the directory for storing cache files.
norecursedirs: Specify directories to avoid when discovering tests.

You can include a lot more configurations so please read through the official docs and this guide to learn more.

Using pyproject.toml

You can also define configurations in pyproject.toml using the tool.pytest.ini_options section.

pyproject.toml

[tool.poetry]
name = "distance_calculator"
version = "0.1.0"
description = "Simple Distance Calculator"
authors = ["ericsalesdeandrade <sdaeric19@gmail.com>"]
readme = "README.md"
python = "^3.11"

[tool.pytest.ini_options]
testpaths = ["tests", "integration"]
addopts = "-v -ra -q"
log_cli = true
log_cli_level = "INFO"
log_format = "%(asctime)s %(levelname)s %(message)s"
log_date_format = "%Y-%m-%d %H:%M:%S"
minversion = "6.0"
filterwarnings = "ignore"
norecursedirs = docs build

# Specify a custom directory for pytest cache
cache_dir = ".pytest_cache"

[tool.poetry.dependencies]
python = "^3.11"
requests = "^2.31.0"

[tool.poetry.group.dev.dependencies]
pytest = "^7.4"
tox = "^4.11.4"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Here we specified pretty much the same config as above using pyproject.toml.

Note - this file makes use of Poetry, a cool Python packaging and dependency management tool.

You can learn how to package your Python modules with Poetry in this detailed guide.

Using setup.cfg

Following is the configuration using setup.cfg. The Pytest config can be defined under the [tool:pytest] section.

setup.cfg

[metadata]
name = distance_calculator
version = attr: my_project.__version__
description = My distance calculator application
author = ericsalesdeandrade
author_email = sdaeric19@gmail.com
long_description = file: README.md
long_description_content_type = text/markdown

[options]
packages = find:
install_requires =
    pytest
    tox
    poetry

dev =
    pytest
    tox

[tool:pytest]
testpaths =
    tests
    integration
addopts = -v -ra -q
log_cli = True
log_cli_level = INFO
log_format = %(asctime)s %(levelname)s %(message)s
log_date_format = %Y-%m-%d %H:%M:%S
minversion = 6.0
filterwarnings = ignore
norecursedirs = docs build

# Specify a custom directory for pytest cache
cache_dir = .pytest_cache

Using tox.ini

Pytest config can also be defined via tox.ini in the [pytest] section.

tox.ini

[tox]
requires =  tox>=4 
envlist = py311
min_version = 3.24.0
skip_missing_interpreters = True
ignore_errors = True

[testenv]
description = run the tests with pytest 
skip_install = true
deps =
    pytest
    -r requirements.txt

commands =
    pytest

[pytest]
testpaths =
    tests
    integration
addopts = -v -ra -q
log_cli = True
log_cli_level = INFO
log_format = %(asctime)s %(levelname)s %(message)s
log_date_format = %Y-%m-%d %H:%M:%S
minversion = 6.0
filterwarnings = ignore
norecursedirs = docs build

# Specify a custom directory for pytest cache
cache_dir = .pytest_cache

If you’ve never used Tox before, here’s a nice read on how to use Tox with Poetry to test and build your Python packages.

Running The Test

Now we have the example code with its tests and configuration.

Let’s run the test,

pytest

We get the below output with pytest.ini:

pytest-config-file-example-result

Note the configfile option in the output. It shows the config file used for the test run.

Output with setup.cfg will look like:

pytest-config-file-example-result

Here is the output with pyproject.toml:

pytest-config-file-example-result

Let’s test it with Tox:

tox

It’ll show the below output:

pytest-config-file-example-result

It doesn’t matter too much which Pytest config file you use, as long as you understand that pytest.ini takes precedence over others.

Besides that, you can use any of the config files to customize your test environment, whichever you prefer.

Best Practices in Config File Management

Managing config files is crucial and highly effective in keeping your project clean and organized.

Here are some best practices when it comes to config file management:

Standardized File Naming: Always use consistent and standardized names for configuration files. For example, pytest.ini, tox.ini, pyproject.toml, and setup.cfg are commonly used names so don’t deviate from these. Following conventions makes it easier for developers to find and understand configuration files.
Location: For visibility and easy maintenance, place the configuration file in the root of your project directory (also the default expectation).
Consider the scope: When using configuration files in Pytest, consider their scopes to use. For example, pyproject.toml and setup.cfg, may serve broader project purposes beyond testing (like build settings) where pytest.ini and tox.ini are specifically for testing-related configurations. Avoid overwhelming the config file with too many settings.
Review and update config files: Review and update the config files regularly as your project evolves, ensuring that your configuration files align with the latest tools and practices.
Automated Testing: Enable automated testing using tools like Tox or Pre-Commit Hooks for your projects. This will help you to catch syntax errors or misconfigurations early in the development process.

Final Thoughts

That’s all I have for you about Pytest config files.

In this article, you learned about various Pytest config files, some of which are popular Python packaging files.

You learnt the importance of why to use Pytest config files these in your testing workflows, and how they help customize the default Pytest settings.

We also went through all 4 supported files using a practical example - pytest.ini, tox.ini, pyproject.toml, and setup.cfg - and saw how you can define your Pytest settings in any of them.

My preference is pytest.ini for Pytest specific settings and pyproject.toml for general project settings.

Lastly, you learned some best practices when working with Pytest config files.

With this in mind, let the world be your oyster and go forth and write some awesome tests!

If you have any ideas for improvement or like me to cover any topics please comment below or send me a message via Twitter, GitHub or Email.

Till the next time… Cheers!

Notes and Additional Readings

Example Code
Pytest Config Documentation
What Is pytest.ini And How To Save Time Using Pytest Config
How To Test And Build Python Packages With Pytest, Tox And Poetry
Command line options and configuration file settings
How To Debug Failing Tests Like A Pro (Use Pytest Verbosity Options)
Ultimate Guide To Pytest Markers And Good Test Management
How To Use Pytest Logging And Print To Console And File (A Comprehensive Guide)
13 Proven Ways To Improve Test Runtime With Pytest

What You’ll Learn

What Is A Pytest Config File?

Why Use Pytest Config Files?

Types of Config Files

pytest.ini

pyproject.toml

setup.cfg

tox.ini

Practical Example

Prerequisites

Getting Started

Example Code

Testing The Code

Using pytest.ini

Using pyproject.toml

Using setup.cfg

Using tox.ini

Running The Test

Best Practices in Config File Management

Final Thoughts

Notes and Additional Readings

Using `pytest.ini`