How To Mock In Pytest? (A Comprehensive Guide)

Posted on 2023-03-27 Edited on 2024-11-13 In Intermediate , Mocking Word count in article: 2.8k Reading time ≈ 10 mins.

Complex software is made up of several moving parts - Rest APIs, Databases, Cloud services, etc.

However, when writing unit tests, you’re often interested in testing a specific part of your code, not the entire system.

So how do you test your code when it depends on external services? Do you call the REST API or connect to the database in your unit tests?

No, that’s not a good idea.

Calling external systems in your unit tests can be slow, unreliable, and expensive.

The answer lies in Mocking.

Mocking is a technique that allows you to isolate a piece of code being tested from its dependencies so that the test can focus on the code under test in isolation.

In this article, we’ll learn how to use Pytest’s mocking features to simulate parts of your code and external dependencies.

You’ll learn how to mock variables and constants, entire functions and classes, Rest API responses and even AWS services (S3).

Let’s get started then?

Link To GitHub Repo

Objectives

By the end of this tutorial you should be able to:

Understand the importance of mocking.
Understand where to use mocking in your Unit or Integration Tests.
Mock functions, variables, classes, Rest APIs and AWS services.

What Is Mocking?

Mocking is a technique allows you to isolate a piece of code being tested from its dependencies so that the test can focus on the code under test in isolation.

This is achieved by replacing the dependencies with mock objects that simulate the behaviour of the real objects.

Mock objects are typically pre-programmed with specific responses to method calls that the code under test is expected to make.

This allows your test to verify that the code under the test behaves correctly under different circumstances.

Mocking is a valuable technique in unit testing because it helps to isolate bugs, and improve test coverage.

It also allows you to test code that is not yet fully implemented or that depends on components that are not available.

Some popular mocking frameworks include Mockito for Java, unittest.mock and pytest-mockfor Python, and Moq for .NET.

Benefits of Mocking

1. Shorter feedback loop

Imagine, you’re testing a Rest API to get Cat Facts.

Every time you run your test, it calls an external API and depending on the server’s capacity to handle requests, your call may take a few ms or a few seconds.

That’s precious development and execution time wasted.

Mocking helps you to shorten the testing feedback loop by pre-setting what you expect the API to return and you can test quicker.

We go more into detail on this in our article on Python Rest API Unit Testing.

2. Reduce dependency on External Services

In our article on How To Design Your Python Testing Strategy, we discussed the importance of isolating your tests from external dependencies for outgoing command messages.

Perhaps you need to test that your webhook has notified another service, or that your service can query an external database and return results.

Waiting for the DB admin to grant access and the network team to open the firewall, can be a painstakingly slow process.

Mocking DB functionality helps you validate your work with minimum to no external dependency.

This is incredibly useful for unit testing.

We’ll talk more about integration testing and whether mocking or actual connections are necessary later in this article.

Project Set Up

In this project, we’ll define a simple module containing a bunch of sample mocking example code.

We’ll then test each of these with and without mocking to achieve a similar test result.

The pytest-mock plugin provides a mocker fixture and a wrapper around the standard Python mock package.

The project has the following structure

.
├── .gitignore
├── README.md
├── mock_examples
│   ├── __init__.py
│   ├── api.py
│   ├── area.py
│   ├── aws_s3.py
│   ├── core.py
│   ├── file_handler.py
│   ├── person.py
│   └── sleep_function.py
├── requirements.txt
└── tests
    ├── __init__.py
    ├── conftest.py
    ├── test_api.py
    ├── test_area.py
    ├── test_aws_s3.py
    ├── test_file_handler.py
    ├── test_person.py
    └── test_sleep.py

This repo uses

Python 3.12
Pytest

Create a virtual environment and install the required packages.

1	$ pip install -r requirements.txt

Mocking In Pytest

Let’s take a look at a basic example of mocking defined in the pytest-mock documentation.

import os

class UnixFS:
    @staticmethod
    def rm(filename):
        os.remove(filename)
def test_unix_fs(mocker):
    mocker.patch('os.remove')
    UnixFS.rm('file')
    os.remove.assert_called_once_with('file')

This code snippet defined a UnixFS class that contains one static method, removing a file from the disk.

We can test this functionality without actually deleting a file from the disk, using the mocker.patch method to simulate the os.remove functionality.

We then use the assert_called_once_with method to validate that the os.remove method was called.

The standard mocking library also allows you to use the mocker.patch function as a Context Manager and Wrapper.

Although this use is discouraged by thepytest-mock plugin.

Important Note — Mock an item where it is used, not where it came from

For e.g. mock the item in the unit test or where the class is initialised, rather than just defined.

`mocker` Fixture

The pytest-mock plugin provides a mocker fixture that can be used to create mock objects and patch functions.

The mocker fixture is an instance of the MockFixture class, which is a subclass of the unittest.mock module.

It offers the following methods:

mocker.patch - Patch a function or method
mocker.patch.object - Patch a method of an object
mocker.patch.multiple - Patch multiple functions or methods
mocker.patch.dict - Patch a dictionary
mocker.patch.stopall - Stop all patches
mocker.patch.stop - Stop a specific patch

While all of these are useful, we’ll mostly use the mocker.patch method in this article and mocker.patch.object for class instances.

Mock A Constant Or Variable

Let’s look at the simplest form of Mocking - Mocking a constant or variable.

mock_examples/area.py

PI = 3.14159


def area_of_circle(radius: float) -> float:
    """
    Function to calculate area of a circle
    :param radius: Radius of the circle
    :return: Area of the circle
    """
    return PI * radius * radius

The above code snippet defines a constant PI and a function area_of_circle that calculates the area of a circle given the radius.

Let’s write a test.

tests/test_area.py

from mock_examples.area import area_of_circle


def test_area_of_circle():
    """
    Function to test area of circle
    """
    assert area_of_circle(5) == 78.53975

This works. But what if we wanted to test the function with a different value of PI?

We can mock the PI constant to test the function with a different value of PI in the mock_examples/area.py file.

tests/test_area.py

def test_area_of_circle_with_mock(mocker):
    """
    Function to test area of circle with mocked PI value
    """
    mocker.patch("mock_examples.area.PI", 3.0)
    assert area_of_circle(5) == 75.0

In this test, we use the mocker.patch method to replace the value of PI with 3.0.

While this is a trivial example, it demonstrates how you can use mocking to test code that depends on constants or variables.

Mock A Function: Create or Remove file

Let’s consider a file handler with 2 functions - create_file and remove_file that create and remove a file respectively.

mock_examples/file_handler.py

import os

def create_file(filename: str) -> None:
    """
    Function to create a file
    :param filename: Name of the file to create
    :return: None
    """
    with open(f"{filename}", "w") as f:
        f.write("hello")

def remove_file(filename: str) -> None:
    """
    Function to remove a file
    :param filename: Name of the file to remove
    :return: None
    """
    os.remove(filename)

This code has 2 functions. We can test it as follows.

tests/test_file_handler.py

import os
from mock_examples.file_handler import create_file, remove_file


def test_create_file():
    """
    Function to test make file
    """
    create_file(filename="delete_me.txt")
    assert os.path.isfile("delete_me.txt")

def test_remove_file():
    """
    Function to test remove file
    """
    create_file(filename="delete_me.txt")
    remove_file(filename="delete_me.txt")
    assert not os.path.isfile("delete_me.txt")

While this works, it’s kinda risky.

When unit testing, the last thing you want to do is mess with the local file system.

You can eliminate this risk by mocking. Alternatively you can use the in-built tmp_path fixture to manage temporary files when testing.

Let’s see how to mock our file handler methods.

tests/test_file_handler.py

def test_create_file_with_mock(mocker):
    """
    Function to test make file with mock
    """
    filename = "delete_me.txt"

    # Mock the 'open' function call to return a file object.
    mock_file = mocker.mock_open()
    mocker.patch("builtins.open", mock_file)

    # Call the function that creates the file.
    create_file(filename)

    # Assert that the 'open' function was called with the expected arguments.
    mock_file.assert_called_once_with(filename, "w")

    # Assert that the file was written to with the expected text.
    mock_file().write.assert_called_once_with("hello")

The above test mocks the open function call, replacing it with a mock file object. We then assert that the open function was called with the expected arguments and that the file was written to with the expected text.

Similarly, we can mock the os.remove function to test the remove_file function.

tests/test_file_handler.py

def test_remove_file_with_mock(mocker):
    """
    Test the removal of a file using mocking to avoid actual file system operations.
    """
    filename = "delete_me.txt"

    # Mock os.remove to test file deletion without deleting anything
    mock_remove = mocker.patch("os.remove")

    # Mock os.path.isfile to control its return value
    mocker.patch("os.path.isfile", return_value=False)

    # Mock open for the create_file function
    mocker.patch("builtins.open", mocker.mock_open())

    # Simulate file creation and removal
    create_file(filename)
    remove_file(filename)

    # Assert that os.remove was called correctly
    mock_remove.assert_called_once_with(filename)

    # Assert that os.path.isfile returns False, simulating that the file does not exist
    assert not os.path.isfile(filename)

Mock A Function — Sleep

Often your code may contain sleep functionality, for e.g. while waiting for another task to execute or get data.

mock_examples/sleep_function.py

import time

def sleep_for_a_bit(duration: int):
    """
    Function to Sleep for a bit
    :param duration: Duration to sleep for
    """
    time.sleep(duration)

The above function puts the code to sleep for a specified duration.

We can easily write a test that calls this function and wait X seconds for it to execute.

Or… we can mock the sleep functionality so our test runs instantly.

tests/test_sleep.py

import time
from mock_examples.sleep_function import sleep_for_a_bit


def test_sleep_for_a_bit_with_mock(mocker):
    """
    Function to test sleep for a bit with mock
    """
    mocker.patch("mock_examples.sleep_function.time.sleep")
    sleep_for_a_bit(duration=5)
    time.sleep.assert_called_once_with(
        5
    )  # check that time.sleep was called with the correct argument

Mock External Rest API Calls

Probably one of the most interesting and useful cases is learning to mock external APIs.

Imagine you’re building a Rest API that depends on another Rest API to get data. Calling it during tests is not ideal.

Not only could the bill rack up, but it also introduces dependency, unpredictability and latency.

Let’s take a simple Weather API example.

mock_examples/api.py

from typing import Dict
import requests

def get_weather(city: str) -> Dict:
    """
    Function to get weather
    :return: Response from the API
    """
    response = requests.get(f"https://goweather.herokuapp.com/weather/{city}")
    return response.json()

The above code snippet defines a function get_weather that calls an external API to get the weather of a city.

Let’s write a quick test.

tests/test_api.py

from mock_examples.api import get_weather


def test_get_weather():
    """
    Function to test get weather
    """
    response = get_weather(city="London")
    assert type(response) is dict

You can assert the type of the response, status code and so on.

However, this test is so dependent on the external API and may fail if the API is down, changes or rate limits you. Not to mention the costs if you’re calling it frequently.

Let’s mock the API call.

tests/test_api.py

def test_get_weather_mocked(mocker):
    mock_data = {
        "temperature": "+7 °C",
        "wind": "13 km/h",
        "description": "Partly cloudy",
        "forecast": [
            {"day": "1", "temperature": "+10 °C", "wind": "13 km/h"},
            {"day": "2", "temperature": "+6 °C", "wind": "26 km/h"},
            {"day": "3", "temperature": "+15 °C", "wind": "21 km/h"},
        ],
    }

    # Create a mock response object with a .json() method that returns the mock data
    mock_response = mocker.MagicMock()
    mock_response.json.return_value = mock_data

    # Patch 'requests.get' to return the mock response
    mocker.patch("requests.get", return_value=mock_response)

    # Call the function
    result = get_weather(city="London")

    # Assertions to check if the returned data is as expected
    assert result == mock_data
    assert type(result) is dict
    assert result["temperature"] == "+7 °C"

In this test, we create a mock response object that returns the mock data. This is a fantastic way to test your code without actually calling the external API.

Now let’s look at how to mock a method within a class.

Mock A Class

Let’s say we have a simple Person class.

mock_examples/person.py

from typing import Dict


class Person:
    def __init__(self, name: str, age: int = None, address: str = None) -> None:
        self._name = name
        self._age = age
        self._address = address

    @property
    def name(self) -> str:
        return self._name

    @property
    def age(self) -> int:
        return self._age

    @property
    def address(self) -> str:
        return self._address

    def get_person_json(self) -> Dict[str, str]:
        return {"name": self._name, "age": self._age, "address": self._address}

The above code snippet defines a Person class with 3 properties - name, age and address and a method get_person_json that returns a dictionary representation of the person.

Let’s write a test.

tests/test_person.py

import pytest
from mock_examples.person import Person


@pytest.fixture
def person():
    return Person(name="Eric", age=25, address="123 Farmville Rd")


def test_person_properties(person):
    """
    Test individual properties of the Person class.
    """
    assert person.name == "Eric"
    assert person.age == 25
    assert person.address == "123 Farmville Rd"

This test is simple and works. But what if we wanted to mock the get_person_json method of the class instance?

tests/test_person.py

def test_person_class_with_mock(mocker):
    """
    Test the Person class using a mock for the 'get_person_json' method 
    """
    person = Person(name="Eric", age=25, address="123 Farmville Rd")
    mock_response = {"name": "FAKE_NAME", "age": "FAKE_AGE", "address": "FAKE_ADDRESS"}

    # Patch the method
    mocker.patch.object(person, "get_person_json", return_value=mock_response)

    assert person.get_person_json() == mock_response

The above code uses the mocker.patch.object method to patch the get_person_json method of the Person class instance.

Mock AWS Services — E.g. S3

As our last example, let’s look at how to Mock an AWS service, e.g. S3.

We use the moto library to mock AWS services.

mock_examples/aw_s3_.py

from typing import Any
import boto3


def get_my_object(bucket: str, key: str) -> Any:
    """
    Function to get an object from S3
    :param bucket: Bucket name
    :param key: Key name
    :return: Response from S3
    """
    s3_client = boto3.client("s3")
    response = s3_client.get_object(Bucket=bucket, Key=key)
    return response

This simple piece of code gets an object from an S3 bucket, taking bucket_name and key as an argument.

When we run this code in our test, it will attempt to communicate with an actual AWS account and service to get the object.

This can be avoided by simulating the Bucket and Key with Moto.

First, set up Moto and the required fixtures in a conftest.py file.

tests/conftest.py

import os
import pytest
from moto import mock_aws
import boto3


@pytest.fixture(scope="function")
def aws_credentials():
    """Mocked AWS Credentials for moto."""
    os.environ["AWS_ACCESS_KEY_ID"] = "testing"
    os.environ["AWS_SECRET_ACCESS_KEY"] = "testing"
    os.environ["AWS_SECURITY_TOKEN"] = "testing"
    os.environ["AWS_SESSION_TOKEN"] = "testing"
    os.environ["AWS_DEFAULT_REGION"] = "us-east-1"


@pytest.fixture(scope="function")
def aws_s3(aws_credentials):
    with mock_aws():
        yield boto3.client("s3", region_name="us-east-1")

This will set up the AWS credentials and the S3 client for the tests.

tests/test_aws_s3.py

from moto import mock_aws
from mock_examples.aws_s3 import get_my_object


@mock_aws
def test_get_my_object_mocked(aws_s3):
    """
    Function to test get my object
    :param s3: pytest-mock fixture
    :return: None
    """
    # Create a mock S3 bucket.
    aws_s3.create_bucket(Bucket="mock-bucket")

    # Create a mock object in the mock S3 bucket.
    aws_s3.put_object(Bucket="mock-bucket", Key="mock-key", Body="mock-body")

    # Get the mock object from the mock S3 bucket.
    response = get_my_object(bucket="mock-bucket", key="mock-key")
    assert response["Body"].read() == b"mock-body"

We’ve just tested the get_object functionality without making any calls to S3 (and costing ££).

Pretty neat isn’t it?

Running Tests

Let’s finally run the tests to make sure it all works.

$ pytest

pytest-mock

(Ignore the warnings, they are related to an outstanding deprecation issue with Boto3)

When Should You Mock?

So,

Maybe you’re wondering — When should you mock and when should you use real resources?

There is no concrete answer. But here’s a simple rule of thumb.

It’s a good idea to mock when you want to test a single module in isolation and avoid external dependencies.

When you want to test the functionality as a system you need to test the real connections e.g. your API can connect to the database or external API.

However, when testing within the unit, for e.g. object transformation, or logic, it’s advisable to mock external resources that are not part of the module.

High-level recommendation:

Unit Testing — Mock where necessary
Integration Testing — Don’t mock, use real connections

Conclusion

I hope you enjoyed this article and it cleared some of your doubts about mocking.

In conclusion, you learnt about mocking, its benefits and how to use it in various contexts, where appropriate.

You explored (with examples) how to mock constants, variables, functions, classes, Rest APIs and even AWS services like S3.

Lastly, you learnt how to decide if you should mock a service, or use it as it is. If you want to test a single module, in isolation, it’s better to mock it.

If you’re testing a system and its integrations, you’ll need the system to be as representative of the real one as possible.

If you have ideas for improvement or like for me to cover anything specific, please send me a message via Twitter, GitHub or Email.

Till the next time… Cheers!

Additional Reading

This article was inspired by the following content

Link To Example Code Used
https://changhsinlee.com/pytest-mock/
Python REST API Unit Testing for External APIs
Python Testing 101 (How To Decide What To Test)

Objectives

What Is Mocking?

Benefits of Mocking

Project Set Up

Mocking In Pytest

mocker Fixture

Mock A Constant Or Variable

Mock A Function: Create or Remove file

Mock A Function — Sleep

Mock External Rest API Calls

Mock A Class

Mock AWS Services — E.g. S3

Running Tests

When Should You Mock?

Conclusion

Additional Reading

`mocker` Fixture