3 Simple Ways To Omit Subfolders From Python Coverage Reports

Posted on 2024-09-26 Edited on 2024-10-14 In Beginner , Test Coverage Word count in article: 2.3k Reading time ≈ 8 mins.

Do you ever feel like your test coverage reports are cluttered with irrelevant code, making it hard to see what really matters?

Maybe you’ve got a folder full of Django migrations or some legacy code that’s no longer in use.

Sure, your coverage numbers look fine, but do they really reflect true test quality?

This noise can lead to inflated coverage percentages, giving you a false sense of security about your codebase.

Worse yet, it can slow you down when trying to spot untested areas that actually matter.

But don’t worry—there’s an easy solution!

In this article, I’m going to show you how to exclude specific subfolders from your Python coverage reports using coverage.py and pytest-cov.

Whether you’re dealing with auto-generated code, unused libraries, or sections of code you just don’t need to cover, we’ve got you covered (pun intended).

We’ll walk you through step-by-step examples using .coveragerc, pyproject.toml, and even some advanced configuration tricks.

By the end of this guide, you’ll know exactly how to keep your coverage reports clean, accurate, and focused on the code that really counts.

If you’re unfamiliar with how to generate coverage reports with Poetry, don’t worry!

I’ve got you covered with this article on Poetry Test Coverage.

Let’s get started and take your test coverage game to the next level!

What is Code Coverage?

Code coverage measures the percentage of your codebase that is executed when running tests.

It helps identify areas of your code that are tested and, more importantly, those that aren’t.

While high coverage doesn’t guarantee your code is bug-free, it’s a useful metric for ensuring your tests cover a broad range of scenarios.

Coverage reports highlight untested lines or branches, guiding you toward more comprehensive testing.

By keeping your reports focused and relevant, especially by omitting unnecessary subfolders, you can get a clearer picture of your project’s real test coverage.

Why Omit Subfolders From Coverage?

Not every file in your project needs to be included in your test coverage reports.

Some common reasons to exclude subfolders:

Auto-generated code: Files like Django migrations or compiled assets aren’t part of your core logic and don’t need testing.
External libraries: Third-party code bundled with your project isn’t your responsibility to test.
Legacy files: Outdated code that’s no longer maintained can clutter your reports with irrelevant data.

By omitting these subfolders, you can focus on the code that matters most, making your coverage reports more meaningful and actionable.

Generating Coverage Reports - High-Level Overview

Let’s quickly go over the steps needed to generate coverage reports in your Python project.

Get the Example Code from GitHub.

1	$ git clone <repo-url>

Create a Virtual Environment

Start by creating a virtual environment and activating it.

1 2	$ python3 -m venv venv $ source venv/bin/activate

Install pytest and coverage.py

Install the necessary packages, including pytest for testing and coverage.py for coverage reports.

1	$ pip install pytest coverage

or if you’re using requirements.txt:

1	$ pip install -r requirements.txt

Example Source Code

Here’s a simple example to calculate the areas of different shapes:

src/area.py

def area_circle(radius):
    return 3.14159 * radius**2

def area_rectangle(length, width):
    return length * width

def area_triangle(base, height):
    return 0.5 * base * height

def area_square(side):
    return side**2

def area_trapezoid(base1, base2, height):
    return 0.5 * (base1 + base2) * height

def area_parallelogram(base, height):
    return base * height

def area_ellipse(major_axis, minor_axis):
    return 3.14159 * major_axis * minor_axis

Example Test Code

Write some tests for these functions:

tests/test_area.py

from src.area import (
    area_circle,
    area_rectangle,
    area_triangle,
    area_square,
    area_trapezoid,
    area_parallelogram,
    area_ellipse,
)

def test_area_circle():
    assert area_circle(5) == 78.53975

def test_area_rectangle():
    assert area_rectangle(5, 10) == 50

def test_area_triangle():
    assert area_triangle(5, 10) == 25

def test_area_square():
    assert area_square(5) == 25

def test_area_trapezoid():
    assert area_trapezoid(5, 10, 15) == 112.5

def test_area_parallelogram():
    assert area_parallelogram(5, 10) == 50

def test_area_ellipse():
    assert area_ellipse(5, 10) == 157.0795

Running Tests and Generating Coverage Reports

Run the tests using pytest with coverage tracking:

1 2	$ coverage run -m pytest $ coverage report

You should see output like this:

python-coverage-omit-subfolder

Repository Structure

Your repository structure should look something like this:

.
├── .coverage
├── .gitignore
├── .python-version
├── requirements.txt
├── src
│   └── area.py
└── tests
    ├── __init__.py
    └── test_area.py

Now that you have a fully functioning project with test coverage enabled, let’s move on to omitting specific subfolders from your coverage reports.

3 Ways To Omit Subfolders Using coverage.py

When generating test coverage reports, you may want to exclude certain subfolders like auto-generated files, legacy code, or third-party dependencies that you don’t need to cover.

This can help keep your coverage reports clean and relevant.

Let’s explore the different ways to do this.

Configuring .coveragerc

One of the easiest ways to exclude subfolders from coverage is by configuring the .coveragerc file.

This file allows you to set exclusion rules centrally.

Example .coveragerc file:

[run]
omit = 
    */migrations/*
    */legacy_code/*
    */third_party/*

In this example, we’re excluding three subfolders: migrations, legacy_code, and third_party.

Note - Depending on the folder structure, you may need to refine this rule. Using just */migrations/* could unintentionally miss exclusions if migrations are nested deeper in the folder structure. It might need to be adjusted to something like src/*/migrations/* or similar.

Let’s see how this works in practice with a very simple example.

Assume we have some legacy code defined in the src/legacy folder that we want to exclude from coverage reports.

src/legacy/old.py


def add(a, b):
    return a + b


def subtract(a, b):
    return a - b

It’s corresponding test file:

tests/legacy/test_old.py

from src.legacy.old import add


def test_add():
    assert add(1, 2) == 3

Generating our coverage report:

1 2	$ coverage run -m pytest $ coverage report

python-coverage-omit-subfolder

As you can see, the coverage report includes the legacy code in the src/legacy folder. As the subtraction function is not tested, it brings our overall coverage down.

Now let’s exclude the src/legacy folder by adding it to the .coveragerc file.

.coveragerc

[run]
omit = 
    src/legacy/*
    tests/legacy/*

Now, when we generate the coverage report, the legacy code will be excluded:

1 2	$ coverage run -m pytest $ coverage report

python-coverage-omit-subfolder

Similarly you can also configure the pyproject.toml file to exclude subfolders from coverage reports.

Configuring pyproject.toml

If you prefer to manage your project configuration in pyproject.toml, you can also add coverage exclusion rules there.

Example pyproject.toml:

[tool.coverage.run]
omit = [
    "*/migrations/*",
    "*/legacy_code/*",
    "*/third_party/*",
]

Using Command-Line Options

You can also specify exclusion rules directly on the command line when running coverage.

Example command-line usage:

1	$ coverage run --omit=/migrations/,/legacy_code/,/third_party/ -m pytest

Using Environment Variables

Another way to exclude subfolders is by setting environment variables.

Example environment variable usage:

1 2	$ export COVERAGE_OMIT="src/migrations/,src/legacy_code/" $ coverage run -m pytest

Using `# pragma: no cover` for Excluding Specific Code Blocks

You can use # pragma: no cover to exclude specific code blocks from coverage reports.

This is helpful for excluding small sections of code that don’t need to be covered, like debug statements or specialized exception handling.

Example:

def area_rectangle(length, width):
    if length == 0 or width == 0:  # pragma: no cover
        return "Invalid dimensions"
    return length * width

Here, the line and it’s branches are excluded from the coverage report. Handy it’s it?

How To Omit Subfolders Using `pytest-cov` Plugin

If you’re using the pytest-cov plugin to measure test coverage, it can easily inherit settings from coverage.py or be configured directly through pytest.ini.

Here’s how to manage folder exclusions in your test coverage reports using pytest-cov.

You can configure folder exclusions directly in your pytest.ini file.

This configuration tells pytest-cov which directories or files should be omitted from the coverage report.

Example pytest.ini configuration:

pytest.ini

1 2	[pytest] addopts = --cov=src --cov-report=term --cov-report=html --cov-config=.coveragerc

In this example, we’re using --cov-config=.coveragerc to tell pytest-cov to use the exclusion settings from our .coveragerc file.

This means that any omit rules defined in .coveragerc will be applied automatically when generating coverage reports with pytest-cov.

Example .coveragerc:

[run]
omit = 
    src/legacy/*
    src/migrations/*

By using the –cov-config option in pytest.ini, you ensure that the omit rules from .coveragerc are respected.

If you prefer not to manage exclusions in pytest.ini, you can allow pytest-cov to automatically inherit the settings from your .coveragerc or pyproject.toml file.

As long as you have the necessary omit rules defined in either of these files, pytest-cov will respect them without needing any additional configuration.

If you’d prefer not to use a .coveragerc file, you can still configure pytest-cov to exclude specific subfolders directly through command-line options or in your pytest.ini file.

Here’s how to manage folder exclusions without relying on the .coveragerc file.

1 2	[pytest] addopts = --cov=src --cov-report=term --cov-report=html --cov-branch --cov-config=none --cov-omit=src/legacy/,src/migrations/

--cov=src: Tells pytest-cov to measure coverage for the src folder.
--cov-report=term and --cov-report=html: Generate terminal and HTML coverage reports.
--cov-branch: Measures branch coverage as well as line coverage.
--cov-config=none: Specifies that no external config file will be read, but coverage will still run.
--cov-omit=src/legacy/*,src/migrations/*: Specifies the directories you want to omit from the coverage report.

This approach allows you to keep all your configuration centralized in pytest.ini, eliminating the need for an external .coveragerc file.

Common Pitfalls with Coverage and How To Avoid Them

When configuring subfolder exclusions in coverage reports, make sure to avoid these common mistakes.

Omitting Tests From Coverage

Some teams exclude test files from coverage measurement, thinking they aren’t “real code.” This is a bad idea.

Your tests are an essential part of your codebase, and coverage provides valuable insight into whether those tests are complete.

Imagine accidentally duplicating a test without changing its name—coverage would alert you that a test wasn’t being run. Including test files ensures your test suite is complete and accurate.

Incorrectly Specifying Paths

Another common mistake is misconfiguring the paths for exclusion, leading to unintended files still appearing in the report.

This often happens when the directory structure isn’t fully considered, or relative paths are misused.

Ensure that paths to subfolders are correctly specified. Double-check for typos and confirm whether you’re using relative or absolute paths as needed. For example, src/legacy and tests/legacy are relative paths, while */migrations/* is an absolute path.

Over-Matching with Regex Patterns

When using regex to exclude code (e.g., via the exclude_also option), it’s easy to over-match, especially if your patterns are too broad.

This can inadvertently exclude necessary code from your coverage reports.

Be precise with your regex patterns. Test them on smaller sections of code first and avoid using overly broad wildcards like .*. This will prevent the accidental exclusion of other functions.

Not Testing Your Exclusions

Setting up exclusions is one thing—validating that they work as intended is another.

Developers often configure exclusions without checking whether the right folders or files are actually being omitted from the coverage report.

Always review the coverage report after configuring exclusions.

Run the report and inspect whether the intended files are being excluded, and ensure that important parts of your code are still fully covered.

Excluding Tests to Inflate Coverage

Some developers exclude tests to “game” coverage statistics, artificially inflating the coverage percentage.

This is counterproductive and undermines the purpose of testing.

Remember, coverage tells you how complete your tests are, but it doesn’t guarantee bug-free code.

Keep your tests in the coverage reports.

Instead of aiming for arbitrary goals like 90% coverage, include your tests and set a higher target, such as 95%.

Coverage isn’t about hitting a number; it’s about ensuring your tests fully cover the various use cases in your application.

Forgetting to Exclude in CI/CD Pipelines

It’s easy to forget to replicate your local exclusion settings in your CI/CD pipeline.

If exclusions aren’t properly applied in automated environments, you may see different coverage results between local and CI builds.

Ensure that your .coveragerc or other exclusion settings are included in your CI/CD pipeline configuration.

If using GitHub Actions or similar tools, ensure that the exclusion rules are part of the environment setup.

Coverage Changes in Pull Requests (Important)

Tracking coverage percentage changes in pull requests is a critical practice.

When the coverage percentage drops in a PR, it’s an early warning that new code may not be fully tested. Ignoring such drops can lead to untested code making it into production.

Set up your CI/CD pipeline to highlight changes in test coverage for every PR.

Most tools, such as GitHub Actions or Codecov, can track whether the coverage has increased or decreased.

If coverage drops, investigate why. Are there missing test cases? Did you omit important edge cases? Address the gaps by writing new tests before merging the PR.

By avoiding these common pitfalls and following best practices, you can ensure your coverage reports are accurate, meaningful, and truly reflective of your codebase’s quality.

Include your tests, monitor coverage changes in PRs, craft exclusion rules carefully, and validate your configuration for the best results.

Conclusion

I hope this article helped you understand how to clean up your coverage reports by excluding specific subfolders and unnecessary code.

We covered how to use .coveragerc, pyproject.toml, and even command-line options to exclude irrelevant files, ensuring your coverage reports stay focused and meaningful.

We also explored some common pitfalls, such as the risks of omitting test files and over-matching with regex patterns.

Now it’s time to apply what you’ve learned! Implement these practices in your project, track coverage changes in your pull requests, and keep improving your code quality.

If you have any ideas for improvement or like me to cover any topics please message via Twitter, GitHub or Email.

Till the next time… Cheers! Happy testing!

Additional Reading

Example Code - GitHub
How To Measure And Improve Test Coverage With Poetry And Pytest
Coverage.py - Official Documentation
Excluding code from coverage.py
pytest-cov - Official Documentation

What is Code Coverage?

Why Omit Subfolders From Coverage?

Generating Coverage Reports - High-Level Overview

Create a Virtual Environment

Install pytest and coverage.py

Example Source Code

Example Test Code

Running Tests and Generating Coverage Reports

Repository Structure

3 Ways To Omit Subfolders Using coverage.py

Configuring .coveragerc

Configuring pyproject.toml

Using Command-Line Options

Using Environment Variables

Using # pragma: no cover for Excluding Specific Code Blocks

How To Omit Subfolders Using pytest-cov Plugin

Common Pitfalls with Coverage and How To Avoid Them

Omitting Tests From Coverage

Incorrectly Specifying Paths

Over-Matching with Regex Patterns

Not Testing Your Exclusions

Excluding Tests to Inflate Coverage

Forgetting to Exclude in CI/CD Pipelines

Coverage Changes in Pull Requests (Important)

Conclusion

Additional Reading

Using `# pragma: no cover` for Excluding Specific Code Blocks

How To Omit Subfolders Using `pytest-cov` Plugin