Pytest vs Unittest (Honest Review Of The 2 Most Popular Python Testing Frameworks)

Software testing is critical to your development process, ensuring your code works as expected.

As Python continues to gain popularity as a backend and scripting language, choosing the right testing framework becomes increasingly important.

Two prominent options in the Python ecosystem are Unittest and Pytest. Both frameworks provide powerful capabilities for writing and executing tests but differ in approach, features, and community support.

In this article, we delve into the comparison between Unittest vs Pytest, shedding light on their strengths, weaknesses, and distinctive features.

We’ll start by looking at how to test a simple program using both — Unittest and Pytest.

Next, we’ll examine how they handle test discovery, organization, parameterization, fixture management, and test execution, highlighting their syntax, structure, and ease of use differences.

Lastly, we’ll explore the foundations of Unittest and Pytest, discussing their origins, philosophies, and core principles.

By the end of this article, you’ll have a well-rounded understanding of Unittest and Pytest, enabling you to make informed decisions when selecting the most suitable testing framework for your Python projects.

Let’s get into it.

Link To GitHub Repo

Objectives

By the end of this article you should be able to:

• Understand the key differences between the unittest and pytest testing frameworks
• Choose the best Python testing framework for your project, based on needs
• Understand the origins, background and key philosophy behind both — unittest and pytest

Project Set Up

The project has the following structure

Getting Started

To start, clone the repo here, or you can create your own repo by creating a folder and running git init to initialise it.

Prerequisites

In this project, we’ll be using Python 3.11.3. Create a virtual environment and install the requirements (packages) using

1

pip install -r requirements.txt

Test Code Example

Let’s look at some simple test code and how to write unit tests — using Pytest and Unittest .

miles_to_km/core.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

def miles_to_km(miles: float) -> float:
    """
    Converts miles to kilometers
    Args:
        miles (float): Miles to convert to kilometers
    Returns:
        float: Kilometers
        Raises:
            TypeError: If input is not a float or int        
    """
    if isinstance(miles, float) or isinstance(miles, int) or isinstance(miles, str):
        km = miles * 1.60934
        logger.info(f'{miles} miles is {km} km')
        return round(km, 2)
    else:
        logger.error(f'Input must be a float, int or str. Input was {type(miles)}')
        raise TypeError(f'Input must be a float or int. Input was {type(miles)}')


def km_to_miles(km: float) -> float:
    """
    Converts kilometers to miles
    Args:
        km (float): Kilometers to convert to miles 
    Returns:
        float: Miles
        Raises:
            TypeError: If input is not a float or int
    """
    if isinstance(km, float) or isinstance(km, int) or isinstance(km, str):
        miles = km / 1.60934
        logger.info(f'{km} km is {miles} miles')
        return round(miles, 2)
    else:
        logger.error(f'Input must be a float, int or str. Input was {type(km)}')
        raise TypeError(f'Input must be a float or int. Input was {type(km)}')

The above piece of code converts miles to km and vice-versa, with basic type checking.

Testing — Unittest

Now, let’s look at how we can write tests for the above source code using the inbuilt unittest library.

tests/unit/test_miles_to_km_unittest.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

import unittest
import logging
from miles_to_km.core import miles_to_km, km_to_miles

# Get logger
logger = logging.getLogger('__miles_to_km__')
logger.setLevel(logging.INFO)

class Test(unittest.TestCase):
    def test_miles_to_km(self):
        self.assertEqual(miles_to_km(1), 1.61)

    def test_km_to_miles(self):
        self.assertEqual(km_to_miles(1), 0.62)

    def test_miles_to_km_dict_type(self):
        with self.assertRaises(TypeError):
            miles_to_km({'a': 1, 'b': 2})

    def test_miles_to_km_list_type(self):
        with self.assertRaises(TypeError):
            miles_to_km([1,2,3])
            

The above file contains 4 unit tests in the Test class.

1. Test miles to km conversion

2. Test km to miles conversion

3. Test unsupported input datatype- dict

4. Test unsupported input datatype- list

To execute the tests, you can run the below in the terminal

1

python -m unittest tests/unit/test_miles_to_km_unittest.py

The following will be the output.

Outputs

The unittestframework produces 3 possible outputs — OK, FAILED, and ERROR.

• OK: When all the tests pass.

• FAILED: When an AssertionError exception is raised (i.e. the assertion is unsuccessful), the output shows FAILED.

• ERROR: When ALL the test cases did not pass and it raises an exception other than AssertionError.

As in the below example, it produces a Failed result when one test fails.

or an Error if there’s an error in the test

Testing — Pytest

Now, let’s look at how to write tests for the above example code using the pytestframework.

tests/unit/test_miles_to_km_pytest.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

import pytest
import logging
from miles_to_km.core import miles_to_km, km_to_miles

# Get logger
logger = logging.getLogger('__miles_to_km__')
logger.setLevel(logging.INFO)


def test_miles_to_km():
    assert miles_to_km(1) == 1.61

def test_km_to_miles():
    assert km_to_miles(1) == 0.62

def test_miles_to_km_dict_type():
    with pytest.raises(TypeError):
        miles_to_km({'a': 1, 'b': 2})

def test_miles_to_km_list_type():
    with pytest.raises(TypeError):
        mile_to_km([1,2,3])

To execute the tests, you can run the below in the terminal

1

pytest tests/unit/test_miles_to_km_pytest.py

or pytest -s to see live logs.

The following will be the output.

Now the really cool part, is if you run pytest in your terminal, it will also run your unittests.

Just like unittest , pytest give you a Fail message if your assertion fails.

However, it also gives you a Fail message, if there’s an error in the test, which in my opinion is not as clear as unittest .

Comparison Table — Unittest vs Pytest

Now that we’ve seen the code differences, let’s review some other key differences.

About Unittest

The unittest library, also known as the Python Unit Testing Framework, has been a fundamental part of Python’s standard library since version 2.1.

The origins of unittest can be traced back to the early 2000s when the Agile software development movement gained traction, emphasizing the importance of automated testing.

Inspired by similar testing frameworks, such as JUnit for Java, unittest was created to provide a standardized testing framework for Python.

The initial implementation was influenced by the PyUnit framework, which itself was derived from JUnit.

However, unittest was designed to be more Pythonic, incorporating the language’s idioms and conventions.

Guido van Rossum, the creator of Python, played a significant role in defining the design and functionality of unittest.

With its inclusion in the Python standard library, unittest became widely accessible and served as the de facto testing framework for Python developers.

It offered a comprehensive set of tools for test discovery, fixture management, and assertions, enabling developers to write robust and maintainable test suites.

It provides a rich set of classes and methods for structuring tests, running test cases, and generating test reports.

Unittest supports both functional and object-oriented styles of test development, allowing developers to choose the approach that best suits their needs.

Despite its longevity and widespread adoption, some developers found unittest to be overly verbose and lacking certain features.

As a result, alternative testing frameworks, such as Pytest, gained popularity by offering a more concise syntax and additional capabilities.

About Pytest

The pytest library was created by Holger Krekel in 2004, who sought to provide a simpler and more flexible alternative to the built-in unittest framework.

Holger initially developed pytest as an extension to the py.test tool, which was inspired by the testing tool py-trial used in the Twisted project.

Over time, pytest evolved into a standalone testing framework with a focus on simplicity, ease of use, and powerful features.

One of the key motivations behind pytest was to offer a more intuitive and expressive test syntax than the inbuilt unittest framework

It introduced a convention-based approach, where tests are defined as simple Python functions with names starting with “test_”. This allowed for more concise and readable test code.

Another notable aspect of pytest is its extensive plugin ecosystem. Holger designed the framework to be highly extensible, enabling developers to customize and enhance its functionality through plugins.

The community-driven development approach fostered a culture of innovation, resulting in the introduction of new features and improvements.

Today, pytest offers a rich set of features, including advanced assertion capabilities, fixture management, test parametrization, test discovery, and comprehensive reporting.

Its plugin ecosystem provides further extensibility, allowing developers to tailor pytest to their specific needs and integrate seamlessly with other tools and frameworks.

Recommendation - Unittest vs Pytest?

So, after reading this article, you may be wondering — what’s the best testing framework?

There is no one solution that fits all.

In my opinion

Unittest is a good choice for simple testing scenarios, PyTest is more powerful, flexible and extensible, making it a better choice for larger, more complex testing projects.

Ultimately it’s advisable that you try both — to see which better suits your requirements and style.

Conclusion

In this article, you learnt a lot about both unittest vs pytest .

We looked at example code for both and how each of them raise and report errors.

We also looked at a tabular comparison of the key characteristics of how each handles fixtures, parameters, async code, coverage, test data generation, test naming convention and a lot more.

While there is no one solution fits all, this article will help you compare and decide what’s the best framework for your project, based on complexity, your skills and development time.

Equipped with this knowledge, you can now continue to write good unit tests and ensure quality code.

If you have ideas for improvement or like for me to cover anything specific, please send me a message via Twitter, GitHub or Email.

Till the next time… Cheers!

Additional Reading

1. Python Official Documentation - Unittest: This link provides information about the unittest module in Python, which is a unit testing framework included in the standard library.
2. knapsackpro.com - Unittest vs. Pytest: This website discusses the differences between unittest and pytest testing frameworks.
3. Python Pool - Python Unittest vs Pytest: This article on Python Pool compares unittest and pytest frameworks, discussing their features and helping you choose the best one for your testing needs.
4. Delft Stack - Python Unittest vs Pytest: This article on Delft Stack aims to discuss the two most commonly used frameworks for unit testing in Python, unittest and pytest, highlighting their similarities and differences.
5. renzon/pytest-vs-unittest: This GitHub repository provides a comparison between pytest and unittest, offering insights and examples to help you understand the differences between the two frameworks.