Python REST API Unit Testing for External APIs
So you’re tasked with building a service that talks to external REST API.
You use the Requests
or similar library, whip up some GET or POST methods and voila! All done.
Easy peasy, right?
Well not so. If you’re doing it for a hobby or quick side project then yes.
But as a professional developer, you know that you’ve to account for all edge cases.
The challenge when dealing with external APIs is that the behaviour is outside your control.
Schema or payload changes, new error codes, and updated speed caps are some problems that may plague you.
A good external API will issue release notes and ensure the new version is backwards compatible (meaning the old version still works).
But it’s not an ideal world, sometimes this may change without notice and break your system.
So how do you safeguard against it? Well, you can’t. You can only control YOUR code. This is where Unit Tests come in handy.
Python REST API unit testing helps engineer your code to gracefully handle most edge cases.
In this article, we’ll talk about
- What Is A REST API?
- Why We Use REST APIs
- Simple REST API — Sample Code
- How To Test External Rest APIs?
- Functionality Test
- Authentication Test
- Error Handling Test
- Schema Test
- Speed and RunTime Tests
- Should You Call The Real API?
- Mock The Response Instead
- Benefits Of Mocking/Patching
- Concerns with Mocking
- Conclusion
We’ll look at real examples of how to implement this. You can find the link to the code on GitHub below.
What Is A REST API?
If you’re wondering or have no idea what a REST API is, a good starting place would be this article on What is a REST API, from IBM.
In brief, a REST API is a form of Application Programming Interface (API) that conforms to the REST architectural style and patterns.
REST stands for Representational State Transfer.
The above IBM article explains the REST Architectural Style in detail.
At a fundamental level, REST APIs are stateless.
There is no state maintained between the client and server, and each client request has to include all the information required by the server to process it.
Why We Use REST APIs
Since the dawn of the internet, sharing data between applications has been of importance.
Without an API, sharing data between tools and applications would be near impossible.
Each service has a custom data model and the use of unified standards (such as JSON, XML) has made data sharing possible.
Some examples may include sharing
- Share health data from your Smart Watch to a Fitness App so you receive personalised recommendations.
- Share marketing, sales or CRM data into an Aggregation tool so you can compare performance.
- Get live weather or maps information from an external provider so you can calculate ETA.
- Get stock price data so you can tell your clients when to buy or sell.
- Get the latest currency exchange values if you’re a currency trading house.
The list is endless and I’m sure you can see the immense value of various applications talking to one another in a standard format.
Simple REST API — Sample Code
OK, enough with the talk. Let’s dive into some sample code.
For this example, we’ll use the Dog API to get some data about dogs.
Why a Dog API? Because every day is a Dog Day :)
This is a simple API that uses an API Key which is sent via email after you register. And it’s totally free (for up to 10,000 requests/month).
Pre-Requisite
- Get an API Key from the Dog API and store it as an Environment Variable.
This API Key needs to be sent as a header when making Requests to the API.
Strangely, even though the documentation says it needs an API Key, the GET requests seem to work without one :/
core.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112import json
import logging
from enum import Enum
import requests
import os
# Set Logging
logging.basicConfig(level=logging.INFO)
class RequestType(Enum):
"""
Enum class for RequestType containing 4 values - GET, POST, PUT, PATCH, DELETE
"""
GET = "GET"
POST = "POST"
PUT = "PUT"
PATCH = "PATCH"
DELETE = "DELETE"
class DogAPI:
def __init__(self):
"""
Function to initialise the Dog API Class
"""
api_key = os.environ.get("API_KEY")
self.headers = {"Content-Type": "application/json",
"x-api-key": api_key}
self.base_url = "https://api.thedogapi.com/v1"
def call_api(self, request_type: str, endpoint: str,
payload: dict | str = None) -> str:
"""
Function to call the API via the Requests Library
:param request_type: Type of Request.
Supported Values - GET, POST, PUT, PATCH, DELETE.
Type - String
:param endpoint: API Endpoint. Type - String
:param payload: API Request Parameters or Query String.
Type - String or Dict
:return: Response. Type - JSON Formatted String
"""
try:
response = ""
if request_type == "GET":
response = requests.get(endpoint, timeout=30,
params=payload)
elif request_type == "POST":
response = requests.post(endpoint, headers=self.headers,
timeout=30, json=payload)
if response.status_code in (200, 201):
return response.json()
elif response.status_code == 401:
return json.dumps({"ERROR": "Authorization Error. "
"Please check API Key"})
response.raise_for_status()
except requests.exceptions.HTTPError as errh:
logging.error(errh)
except requests.exceptions.ConnectionError as errc:
logging.error(errc)
except requests.exceptions.Timeout as errt:
logging.error(errt)
except requests.exceptions.RequestException as err:
logging.error(err)
def list_breeds(self, query_dict: dict) -> str:
"""
Function to List Dog Breeds -
https://docs.thedogapi.com/api-reference/breeds/breeds-list
:param query_dict: Query String Parameters. Type - Dict
:return: Response. Type - JSON Formatted String
"""
breeds_url = f"{self.base_url}/breeds"
if isinstance(query_dict, dict):
response = self.call_api(request_type=RequestType.GET.value,
endpoint=breeds_url, payload=query_dict)
else:
raise ValueError("ERROR - Parameter 'query_dict' should be of Type Dict")
return response
def search_breeds(self, query_str: str):
"""
Function to Search Dog Breeds
- https://docs.thedogapi.com/api-reference/breeds/breeds-search
:param query_str: Query String. Type - String
:return: Response. Type - JSON Formatted String
"""
search_breeds_url = f"{self.base_url}/breeds/search"
if isinstance(query_str, str):
response = self.call_api(request_type=RequestType.GET.value,
endpoint=search_breeds_url,
payload={"q": query_str})
else:
raise ValueError("ERROR - Parameter 'query_str' should be of Type Str")
return response
def create_vote(self, payload: dict) -> str:
"""
Function to Vote on Dog Image
- https://docs.thedogapi.com/api-reference/votes/votes-post
:param payload: API Request Parameters. Type - Dict
:return: Response. Type - JSON Formatted String
"""
create_vote_url = f"{self.base_url}/votes"
if isinstance(payload, dict) and "image_id" and "value" in payload:
response = self.call_api(request_type=RequestType.POST.value,
endpoint=create_vote_url,
payload=payload)
else:
raise ValueError("ERROR - Parameter 'payload' should be of Type Dict")
return response
The sample code above consists of a DogAPI Class containing.
- List Breeds (GET Method)
- Search Breeds (GET Method)
- Create Vote (POST METHOD)
In a subsequent update of this article, we’ll cover more API Methods.
The code also contains a wrapper method around the Requests
library call_api()
.
We use the RequestType
enum class to handle each REST API Method Type e.g. GET, POST, PUT, PATCH, DELETE.
How To Test External Rest APIs?
Now that we’ve defined the Source Code let’s dive into the core idea. Python REST API unit testing for External APIs.
What should you test?
Functionality Test
Does your code do what it’s supposed to do? Does it produce the correct data and or result?
Here’s an example of some functionality unit tests that you can find in the repo under /tests
.
Environment variables for the Unit Test are specified in a pytest.ini
file
Contents of pytest.ini
. Store this file in the same directory as conftest.py
.pytest.ini
1
2
3[pytest]
env =
D:API_KEY=<YOUR_API_KEY>
- Test List Breeds (GET)
The below Code tests the list_breeds()
method by passing a custom payload and asserting against an expected responsetest_dog_api_core_basic.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20def test_list_breeds(dog_api) -> None:
"""
Unit Test to List Dog Breeds
:param dog_api: Class Object Parameter from conftest. Type - DogAPI
:return: None
"""
expected_response = [
{'weight': {'imperial': '50 - 60', 'metric': '23 - 27'},
'height': {'imperial': '25 - 27', 'metric': '64 - 69'},
'id': 2, 'name': 'Afghan Hound', 'country_code': 'AG',
'bred_for': 'Coursing and hunting', 'breed_group': 'Hound',
'life_span': '10 - 13 years',
'temperament': 'Aloof, Clownish, Dignified, Independent, Happy',
'origin': 'Afghanistan, Iran, Pakistan',
'reference_image_id': 'hMyT4CDXR',
'image': {'id': 'hMyT4CDXR', 'width': 606, 'height': 380,
'url': 'https://cdn2.thedogapi.com/images/hMyT4CDXR.jpg'}}]
actual_response = dog_api.list_breeds(query_dict={"attach_breed": 1,
"page": 1, "limit": 1})
assert actual_response == expected_response
2. Test Search Breeds and Create Vote (GET, POST)
We can repeat this type of Unit Test to cover the other 2 methods — search_breeds()
and create_vote()
.test_dog_api_core_basic.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17def test_search_breeds(dog_api) -> None:
"""
Unit Test to Search Dog Breeds
:param dog_api: Class Object Parameter from conftest.
Type - DogAPI
:return: None
"""
expected_response = [
{'weight': {'imperial': '17 - 23', 'metric': '8 - 10'}, 'height':
{'imperial': '13.5 - 16.5', 'metric': '34 - 42'},
'id': 222, 'name': 'Shiba Inu',
'bred_for': 'Hunting in the mountains of Japan, Alert Watchdog',
'breed_group': 'Non-Sporting', 'life_span': '12 - 16 years',
'temperament': 'Charming, Fearless, Keen, Alert, Confident, Faithful',
'reference_image_id': 'Zn3IjPX3f'}]
actual_response = dog_api.search_breeds(query_str="shiba")
assert actual_response == expected_response
and lastlytest_dog_api_core_basic.py
1
2
3
4
5
6
7
8
9
10
11
12
13def test_create_vote(dog_api) -> None:
"""
Unit Test to Vote on Dog Breed Images
:param dog_api: Class Object Parameter from conftest. Type - DogAPI
:return: None
"""
expected_response = {'message': 'SUCCESS', 'id': 129143,
'image_id': 'asf2', 'value': 1,
'country_code': 'AR'}
actual_response = dog_api.create_vote(
payload={"image_id": "asf2", "value": 1})
assert actual_response["message"] \
== expected_response["message"]
These types of Unit Tests verify the functionality of each method (it does what it says on the tin).
Note the use of conftest.py to define the class instance object as a pytest fixture.
Authentication Test
What happens if our Environment Variable (& API Key) is lost?
Our API will not work. So we need to be notified and look through the logs.
Here’s an example of a quick authentication check.
test_dog_api_core_basic.py
1
2
3
4
5
6
7
8
9
10
11def test_auth_api_key_error() -> None:
"""
Unit Test to Test Auth on Post Request
:return: None
"""
dog_api_temp = DogAPI()
dog_api_temp.headers = {"Content-Type": "application/json", "x-api-key": "FAKE_KEY"}
expected_response = {"ERROR": "Authorization Error. Please check API Key"}
actual_response = dog_api_temp.create_vote(payload={"image_id": "asf2", "value": 1})
assert json.loads(actual_response) == expected_response
In this case, we override the initialised class and headers with a temporary one, containing a FAKE Key string. Obviously, this is not going to work and as expected, the API returns a 401 status code.
Error Handling Test
Can your code handle errors from the server?
For example — payload errors, API limit errors or maybe the pagination times out?
Testing for all these use cases is important.
Maybe you want to add throttling to your API (reduce the # of requests/second) if the 3rd party API cannot handle it.
test_dog_api_core_basic.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16def test_list_breeds_wrong_arg_type_value_error(dog_api) -> None:
with pytest.raises(ValueError):
actual_response = dog_api.list_breeds(
query_dict="Invalid")
def test_create_vote_wrong_arg_type_value_error(dog_api) -> None:
with pytest.raises(ValueError):
actual_response = dog_api.create_vote(
payload="Invalid")
def test_create_vote_wrong_payload_value_error(dog_api) -> None:
with pytest.raises(ValueError):
actual_response = dog_api.create_vote(
payload={"image_id": "xyz"})
Schema Test
One of the biggest cases of API breakage is schema changes.
Perhaps the API releases a new version and updates the response schema and you were unaware of the release.
You can’t always trust 3rd party APIs to honour backwards compatibility.
Testing for schema changes, backwards compatibility and notifying you ASAP is extremely important.
Speed and RunTime Tests
What’s the average response time or latency of the 3rd party API?
Has it suddenly gone from 100ms to 3000ms due to a new release or issues with server capacity?
This could have a massive impact on your service and affect your users.
It’s vital you know and do something about it before your users experience it.
Should You Call The Real API?
While including a suite of tests is critical for your application to function correctly, what if you need to run these tests all the time?
Perhaps you have a monitoring environment that runs tests every hour to make sure the API is working as expected and the service is marked as healthy
.
This would mean you’d hit your API limit pretty easily, costing you more $$$ and wasting precious computing resources.
When running Unit Tests you want to be in full control of the state and reduce dependence on external services.
As you can see, calling the External API ALL the time is a bad idea.
So what’s the solution you ask?
A hybrid one.
We can gracefully “mock” the response of the External API thanks to libraries like pytest-mock
and requests-mock
.
Running Unit Tests as part of a CI/CD Pipeline, especially when releasing to Production is a good idea.
You expect your tests to pass and then promote the deployment to the next environment.
But for local or high-volume repeated environment testing, mocking is a good strategy.
Let’s look at how to do this now.
Mock The API Response Instead
How can we simulate the API response?
The same way as pilots train in a simulator or medical staff in training first practice on dummy patients?
We use Mocks and Patches.
Mocking or Patching is the practice of telling the Unit Testing framework to return a specific value for a variable, function or class object instantiation.
Let’s look at how this is achieved with some code.
test_dog_api_core_mock.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
def test_mock_list_breeds(mocker, dog_api) -> None:
mocked_call_api_value = [
{'weight': {'imperial': '44 - 66', 'metric': '20 - 30'},
'height': {'imperial': '30', 'metric': '76'}, 'id': 3,
'name': 'African Hunting Dog',
'bred_for': 'A wild pack animal', 'life_span': '11 years',
'temperament': 'Wild, Hardworking, Dutiful', 'origin': '',
'reference_image_id': 'rkiByec47',
'image': {'id': 'rkiByec47', 'width': 500, 'height': 335,
'url': 'https://cdn2.thedogapi.com/images/rkiByec47.jpg'}}]
mocker.patch('dog_api.core.DogAPI.call_api',
return_value=mocked_call_api_value)
actual_mock_response = dog_api.list_breeds(
query_dict={"attach_breed": 1, "page": 2,
"limit": 1})
assert mocked_call_api_value == actual_mock_response
def test_mock_search_breeds(mocker, dog_api) -> None:
mocked_call_api_value = [
{'weight': {'imperial': '13 - 17', 'metric': '6 - 8'},
'height': {'imperial': '13 - 14', 'metric': '33 - 36'},
'id': 139,
'name': 'Jack Russell Terrier', 'bred_for': 'Fox hunting',
'breed_group': 'Terrier',
'life_span': '12 - 14 years'}]
mocker.patch('dog_api.core.DogAPI.call_api',
return_value=mocked_call_api_value)
actual_mock_response = dog_api.search_breeds(
query_str="Jack Russell Terrier")
assert mocked_call_api_value == actual_mock_response
def test_mock_create_vote(mocker, dog_api) -> None:
mocked_call_api_value = {
'message': 'SUCCESS', 'id': 129143, 'image_id': 'asf2',
'value': 1, 'country_code': 'AR'}
mocker.patch('dog_api.core.DogAPI.call_api',
return_value=mocked_call_api_value)
actual_mock_response = dog_api.create_vote(
{"image_id": "asf2", "value": 1})
assert mocked_call_api_value == actual_mock_response
In this example, we’ll use the pytest-mock library, which is a wrapper around the patching API provided by the mock package.
If you follow the source code, you’ll see that the list_breeds()
, search_breeds()
and create_vote()
API methods use the call_api()
class method.
The call_api()
function has some logical conditions to execute a GET, PUT
request. But we don’t really care about the working of this method.
All we’re really interested in is the response of the list_breeds()
, search_breeds()
and create_vote()
methods.
So we “mock” the response of call_api()
i.e. produce a fake/mocked response.
This helps us test the functionality of the methods we’re interested in without worrying about the behaviour of underlying methods.
This can be taken further to mock a variable or even an object instantiation of a class.
This article from Chang Hsin Lee nicely explains how to use pytest-mock
in great detail.
So you may ask what’s the benefit of mocking?
Well, in case you haven’t deduced it yourself, here are a few.
Benefits Of Mocking/Patching
- Avoid calling the external API to test functionality repeatedly.
- Much faster — (slower methods and API calls) can be mocked to reduce test execution time.
- Predictable — The responses being predictable (essentially faked) allows us to write exact assert statements and guarantee responses.
Concerns with Mocking
Like all things in life, everything comes with pros and cons. Well, probably not dogs :)
While Mocking is great, there are some things you must be aware of when deciding to use mock functionality in your Unit Tests.
- Tightly coupled — Tests are tightly coupled to implementation details, specifically how you import files and modules and your code structure.
- Remember to @mock.patch every test — You have to remember to patch every test.
- Mix testing of business logic with testing technical implementation.
- Tests are uglier and less readable.
So what’s the best solution?
In a later update, you’ll learn more about using Adaptors and libraries like VCR.py` to solve some of these challenges.
This video from Harry Percival explains some of the above concerns with solutions.
Conclusion
Thanks for sticking with me through this article, I hope it’s been useful.
You came here with the goal to learn Python REST API unit testing and we covered a wide range of topics, including
- Some basics of REST APIs
- Example of working with a REST API
- Types of tests to cover
- Mocking and Patching (Benefits and Concerns)
This will help you write production-ready and more robust code to give you confidence in your abilities in delivering a good product.
Some other topics I’ve planned to expand this article or another include:
- Using Adaptors and
VCR.py
as an alternative to Mocking. - Handling various HTTP Response Status Codes from External APIs.
- Run Unit Tests as part of your CI/CD Pipeline.
- Coverage of more API Methods and variation of tests from the DogAPI.
- Test various types of Authentication
Please let me know what would be more helpful to your career (via the comments) and I’ll do my best to include them.
Till the next time… Cheers!
Additional Reading