Introduction to Pytest Mocking - What It Is and Why You Need It

Posted on Edited on In Word count in article: 2.6k Reading time ≈ 9 mins.

Testing is an essential part of software development and delivery.

Writing code without testing it is like delivering a half-baked cake.

We explored various types of production software testing in our article on types of software testing.

However, writing tests can be a tedious and time-consuming task, and you often find yourself repeating the same code over and over again.

That’s where mocking comes in. It fills the gap between Unit and Integration testing.

Mocking is a technique that allows you to simulate the behaviour of certain parts of your code, so you can test your software in isolation.

It involves replacing a real object with a fake one, which can be configured to behave in a certain way.

By doing so, you can test different scenarios without the need to interact with external systems, databases or APIs.

In this article, we’ll learn how to use Pytest’s mocking features to write effective and efficient tests.

We’ll learn how to mock variables and constants, entire functions and classes, Rest API responses and even AWS services (S3).

Let’s get started then?

Link To GitHub Repo

Objectives

By the end of this tutorial you should be able to:

  • Understand the importance of mocking.
  • Understand where to use mocking in your Unit or Integration Tests.
  • Mock your existing or new Unit Tests where necessary.
  • Use thepytest-mock library to mock variables, functions, classes, system operations, Rest API and even AWS S3 Responses.

What Is Mocking?

Mocking is a technique used to isolate a piece of code being tested from its dependencies so that the test can focus on the code under test in isolation.

This is achieved by replacing the dependencies with mock objects that simulate the behaviour of the real objects.

Mock objects are typically pre-programmed with specific responses to method calls that the code under test is expected to make.

This allows the test to verify that the code under the test behaves correctly under different circumstances.

Mocking is a valuable technique in unit testing because it helps to isolate bugs, and improve test coverage.

It also allows you to test code that is not yet fully implemented or that depends on components that are not yet available.

Some popular mocking frameworks include Mockito for Java, unittest.mock and pytest-mockfor Python, and Moq for .NET.

Shorten the feedback loop

Imagine, you’re testing a Rest API to get Cat Facts.

Every time you run your test, it calls an external API and depending on the server’s capacity to handle requests, your call may take a few ms or a few seconds.

That’s precious development and execution time wasted.

Mocking helps you to shorten the testing feedback loop by pre-setting what you expect the API to return and you can test quicker.

We go more into detail on this in our article on Python Rest API Unit Testing.

Reduce dependency

Imagine you’re working on one module and your colleague on another feature, part of the same delivery.

You’ve finished the source code and it’s time to test it, or you developed it in tandem (TDD).

How do you validate and deliver your final work ensuring smooth integration with your colleague’s module?

Mocking the other module’s functionality can be a very useful tool in your toolbox.

Or perhaps you need to test that your API can query an external database and return results.

Waiting for the DB admin to grant access and the network team to open the firewall, can be a painstakingly slow process.

Mocking DB functionality helps you validate your work with minimum to no external dependency.

This is incredibly useful for unit testing.

We’ll talk more about integration testing and whether mocking or actual connections are necessary later in this article.

Mocking In Pytest

Let’s take a look at a basic example of mocking defined in the pytest-mock documentation.

1
2
3
4
5
6
7
8
9
10
import os

class UnixFS:
    @staticmethod
    def rm(filename):
        os.remove(filename)
def test_unix_fs(mocker):
    mocker.patch('os.remove')
    UnixFS.rm('file')
    os.remove.assert_called_once_with('file')

This code snippet defined a UnixFS class that contains one static method, removing a file from the disk.

We can test this functionality without actually deleting a file from the disk, using the mocker.patch method to simulate the os.remove functionality.

We then use the assert_called_once_with method to validate that the os.remove method was called.

The standard mocking library also allows you to use the mocker.path function as a Context Manager and Wrapper.

Although this use is discouraged by thepytest-mock plugin.

Important Note — Mock an item where it is used, not where it came from

For e.g. mock the item in the unit test or where the class is initialised, rather than just defined.

Project Set Up

In this project, we’ll define a simple module containing a bunch of sample mocking example functions and classes.

We’ll then test each of these with and without mocking to achieve a similar test result.

The pytest-mock plugin provides a mocker fixture and a wrapper around the standard Python mock package.

The project has the following structure

pytest-mocking-repo

Mock A Constant Or Variable

Navigating to mock-examples/core.py we’ll take a look at the source code, and then corresponding tests.

core.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
import os
import time
import logging
from typing import Any
import requests
import boto3

logging.basicConfig(level=logging.INFO)

PI = 3.14159

def area_of_circle(radius: float) -> float:
    """
    Function to calculate area of a cicle
    :param radius: Radius of the circle
    :return: Area of the circle
    """
    return PI * radius * radius

In this code, we calculate the area of a circle using a basic formula A = π r², where π is a constant and r is the radius of the circle.

The unit test looks something like this

test_mock_examples.py

1
2
3
4
5
6
def test_area_of_circle():
    """
    Function to test area of circle
    :return: None
    """
    assert area_of_circle(5) == 78.53975

Here we pass a radius of 5 and the function calculates a result.

Now, our value of π is fixed at 3.14159.

Let’s say (for some crazy reason) we wanted to change its value.

Constants are not supposed to change but maybe you’d want to patch an environment variable (like a server endpoint).

We can achieve it like this

test_mock_examples.py

1
2
3
4
5
6
7
8
def test_area_of_circle_with_mock(mocker):
    """
    Function to test area of circle with mock
    :param mocker:
    :return: None
    """
    mocker.patch("mock_examples.core.PI", 3.0)
    assert area_of_circle(5) == 75.0

We patch the value of PI using the mocker.patch function and specify a return value of 3.0.

This will create a mock object for PI with a value of 3.0 and compute a new Area value of 75.0 (A = 3 * 5 * 5).

Mock A Function — Make or Remove file

Make File

If we navigate back to our source code, we have 2 functions — make_file and remove_file .

While this is useful in production, during testing, the creation and deletion of actual files are not necessary.

core.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
def make_file(filename: str) -> None:
    """
    Function to create a file
    :param filename: Name of the file to create
    :return: None
    """
    with open(f"{filename}", "w") as f:
        f.write("hello")

def remove_file(filename: str) -> None:
    """
    Function to remove a file
    :param filename: Name of the file to remove
    :return: None
    """
    os.remove(filename)

This code has 2 functions. We can test it as follows.

test_mock_examples.py

1
2
3
4
5
6
7
def test_make_file():
    """
    Function to test make file
    :return: None
    """
    make_file(filename="delete_me.txt")
    assert os.path.isfile("delete_me.txt")

Running this test will make an actual file on disk. While desirable as part of the overall pipeline, it’s not required to test isolated functionality.

We can mock this as follows.

test_mock_examples.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
def test_make_file_with_mock(mocker):
    """
    Function to test make file with mock
    :param mocker: pytest-mock fixture
    :return: None
    """
    filename = "delete_me.txt"

    # Mock the 'open' function call to return a file object.
    mock_file = mocker.mock_open()
    mocker.patch("builtins.open", mock_file)

    # Call the function that creates the file.
    make_file(filename)

    # Assert that the 'open' function was called with the expected arguments.
    mock_file.assert_called_once_with(filename, "w")

    # Assert that the file was written to with the expected text.
    mock_file().write.assert_called_once_with("hello")

In this example, we use mocker.mock_open() to create a mock file object.

We can then use it to assert that the open function was called with the expected arguments and that the file was written with the expected text.

We then use mocker.patch() to replace the open function call within create_file with our mock file object.

Finally, we use assert_called_once_with to assert that the open function was called with the expected filename and mode and that the write method of the mock file object was called with the expected text.

This approach allows us to test the behaviour of code that creates files without actually creating files on disk.

We can do something similar with remove_file

Remove File

test_mock_examples.py

1
2
3
4
5
6
7
8
def test_remove_file():
    """
    Function to test remove file
    :return: None
    """
    make_file(filename="delete_me.txt")
    remove_file(filename="delete_me.txt")
    assert not os.path.isfile("delete_me.txt")

Mock A Function — Sleep

Often your code may contain sleep functionality, for e.g. while waiting for another task to execute or get data.

core.py

1
2
3
4
5
6
7
def sleep_for_a_bit(duration: int) -> None:
    """
    Function to Sleep for a bit
    :param duration: Duration to sleep for
    :return: None
    """
    time.sleep(duration)

The above function puts the code to sleep for a specified duration.

We can easily write a test that calls this function and wait X seconds for it to execute.

Or… we can mock the sleep functionality so our test runs instantly.

test_mock_examples.py

1
2
3
4
5
6
7
8
9
10
11
def test_sleep_for_a_bit_with_mock(mocker):
    """
    Function to test sleep for a bit with mock
    :param mocker: pytest-mock fixture
    :return: None
    """
    mocker.patch("mock_examples.core.time.sleep")
    sleep_for_a_bit(duration=5)
    time.sleep.assert_called_once_with(
        5
    )  # check that time.sleep was called with the correct argument

Mock External Rest API Calls

As we discussed before, calling external Rest APIs in each Unit Test execution is not a good idea.

Not only could the bill rack up, but it also introduces dependency, unpredictability and latency.

core.py

1
2
3
4
5
6
7
def get_yo_mamma_jokes() -> Any | None:
    """
    Function to get yo mamma jokes from an API
    :return: Response from the API
    """
    response = requests.get("https://api.yomomma.info/")
    return response.json()

This simple call uses the Requests library to get jokes from the Yo Mamma API.

We can test it as follows.

test_mock_examples.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
def test_get_yo_mamma_jokes():
    """
    Function to test get yo mamma jokes
    :return: None
    """
    response = get_yo_mamma_jokes()
    logging.info(response)

def test_get_yo_mamma_jokes_with_mock(mocker):
    """
    Function to test get yo mamma jokes with mock
    :param mocker: pytest-mock fixture
    :return: None
    """
    mock_response = {
        "joke": "Yo mamma so ugly she made One Direction go another direction."
    }
    mocker.patch(
        "mock_examples.core.requests.get"
    ).return_value.json.return_value = mock_response
    response = get_yo_mamma_jokes()
    requests.get.assert_called_once_with(
        "https://api.yomomma.info/"
    )  # check that requests.get was called with the correct URL
    assert (
        response == mock_response
    )  # check that the result is the expected mock response

Above we have 2 tests. The 1st one gets a random Yo Mamma joke and we can assert the response produces a 200 status code.

The second test actually mocks the requests.get functionality to simulate what the API should produce.

We can then assert that the endpoint was called and the response is as expected.

Pretty cool isn’t it?

Mock A Class

Let’s say we have a simple Person class.

core.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
class Person:
    def __init__(self, name: str, age: str | int = None, address: str = None) -> None:
        self.name = name
        self.age = age
        self.address = address

    @property
    def get_name(self) -> str:
        return self.name

    @property
    def get_age(self) -> str:
        return self.age

    @property
    def get_address(self) -> str:
        return self.address

    @property
    def get_person_json(self) -> dict[str, Any]:
        return {"name": self.name, "age": self.age, "address": self.address}

The class takes a name , age and address parameter on initialization.

It returns the various properties including a JSON-type Python Object with the 3 parameters.

test_mock_examples.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
def test_person_class():
    """
    Function to test Person class
    :return: None
    """
    person = Person(name="John", age=30, address="123 Main St")
    assert person.get_name == "John"
    assert person.get_age == 30
    assert person.get_address == "123 Main St"
    assert person.get_person_json == {
        "name": "John",
        "age": 30,
        "address": "123 Main St",
    }

Now, what if we wanted to initialize the Class to always return a specific value?

Perhaps this can be used to test a database driver e.g. Connect to Redshift, Snowflake or another and we don’t want to reinitialize it.

We can test it using the mock functionality.

test_mock_examples.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
def test_person_class_with_mock(mocker):
    """
    Function to test Person class with mock
    :param mocker: pytest-mock fixture
    :return: None
    """
    fake_response = {"name": "FAKE_NAME", "age": "FAKE_AGE", "address": "FAKE_ADDRESS"}
    # Mock the 'Person' class to return a mock object.
    mocker.patch(
        "mock_examples.core.Person.get_person_json", return_value=fake_response
    )

    # Initalize the Person class with fresh data.
    person = Person(name="Eric", age=25, address="123 Farmville Rd")
    actual = person.get_person_json()
    assert actual == fake_response

In this test, we mock the Person class to ALWAYS return the fake response, even if we initialize it with new variables.

This can be useful to control the behaviour of external modules in your Unit tests.

Mock AWS Services — E.g. S3

As our last example, let’s look at how to Mock an AWS service, e.g. S3.

This is incredibly useful when testing our Lambda functions locally where we don’t need it to actually communicate with S3 using a Boto3 command.

We use the moto library to mock AWS services.

core.py

1
2
3
4
5
6
7
8
9
10
def get_my_object(bucket: str, key: str) -> Any | None:
    """
    Function to get an object from S3
    :param bucket: Bucket name
    :param key: Key name
    :return: Response from S3
    """
    s3_client = boto3.client("s3")
    response = s3_client.get_object(Bucket=bucket, Key=key)
    return response

This simple piece of code gets an object from an S3 bucket, taking bucket_name and key as an argument.

When we run this code in our test, it will attempt to communicate with an actual AWS account and service to get the object.

This can be avoided by simulating the Bucket and Key with Moto.

test_mock_examples.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
@mock_s3
def test_get_my_object(s3):
    """
    Function to test get my object
    :param s3: pytest-mock fixture
    :return: None
    """
    # Create a mock S3 bucket.
    s3.create_bucket(Bucket="fake-bucket")

    # Create a mock object in the mock S3 bucket.
    s3.put_object(Bucket="fake-bucket", Key="fake-key", Body="fake-body")

    # Get the mock object from the mock S3 bucket.
    response = get_my_object(bucket="fake-bucket", key="fake-key")
    assert response["Body"].read() == b"fake-body"

We’ve just tested the get_object functionality without making any calls to S3 (and costing ££).

Pretty neat isn’t it?

Should You Always Mock?

So,

Maybe you’re wondering — When should you mock and when should you use real resources?

There is no concrete answer. But here’s my take on it.

Unit testing refers to the testing of an individual resource in isolation, rather than the system as a whole.

That in my opinion is a good rule of thumb.

When you want to test the functionality as a system you need to test the real connections e.g. your API can connect to the database or external API.

However, when testing within the unit, for e.g. object transformation, or logic, it’s advisable to mock external resources that are not part of the module.

High-level recommendation:

  • Unit Testing — Mock where necessary
  • Integration Testing — Don’t mock, use real connections

Conclusion

I hope you enjoyed this article.

In conclusion, you learnt about mocking, its benefits and how to use it in various contexts, where appropriate.

You explored (with examples) how to mock constants, variables, functions, classes, Rest APIs and even AWS services like S3.

Lastly, you learnt how to decide if you should mock a service, or use it as it is. If you want to test a single module, in isolation, it’s better to mock it.

If you’re testing a system and its integrations, you’ll need the system to be as representative of the real one as possible.

If you have ideas for improvement or like for me to cover anything specific, please send me a message via Twitter, GitHub or Email.

Till the next time… Cheers!

Additional Reading

This article was inspired by the following content

https://changhsinlee.com/pytest-mock/
https://www.toptal.com/python/an-introduction-to-mocking-in-python
https://pypi.org/project/pytest-mock/