How To Run Pytest (`python -m pytest` vs `pytest`)

Posted on Edited on In Word count in article: 2.7k Reading time ≈ 10 mins.

You may have come across the commands pytest and python -m pytest but find yourself puzzled about their differences.

Maybe you’ve seen your colleagues use either command interchangeably, and you’re wondering if they are the same.

Well, you’re not alone! It’s time to shed light on this mystery!

Pytest stands as a beacon of simplicity in Python Unit Testing and possesses the robustness needed for complex functional testing scenarios.

Intriguingly, it offers two distinct invocation methods: directly via the pytest command or through Python as a module with python -m pytest.

This exploration is crucial and understanding these nuances will enrich your knowledge and up your testing game.

In this article, you’ll dive deep into these methods, unraveling their differences and learn how to apply them.

You’ll be equipped with the insights to choose the most suitable approach for your testing.

So, let’s embark on this journey and explore different ways of invoking Pytest!

What You’ll Learn

By the end of this tutorial, you will:

  • Clearly understand the differences between invoking Pytest directly and as a Python module
  • Understand the importance of sys.path in Python module execution
  • Be able to choose the appropriate invocation method for different scenarios
  • Gain insights into the structure and execution of tests in Python projects

Prerequisites

To achieve the above objectives, the following is recommended (basic is fine):

  • Python (3.11+)
  • Pytest

We’ll shortly go onto the differences between running python -m pytest and pytest, but first, some background…

Running a Python Script/Module

A Python script, typically a .py file, is designed to be executed directly.

For example, running python script.py executes the script as a standalone program. This is your go-to method when you have a Python file intended to perform a specific task or series of tasks.

In contrast, a Python module is also a file containing Python code, but structured with definitions (like functions and classes) and statements that are intended for use in other Python programs.

Think of all the libraries and packages you’ve used in your Python projects, such as NumPy or Requests. These libraries are collections of modules, packaged and bundled for reuse.

When you use an import statement in your Python code, you’re importing these modules, bringing in abstracted external functionalities that you can then leverage in your program.

Modules are typically run using the -m flag.

This method of execution is particularly useful when you want to run a module as a script. For example, python -m <MODULE> would run the specified module as a main program.

One intriguing aspect of running Python scripts as modules (e.g., python -m script.py) is how it influences Python’s processing of the file.

python -m pytest adjusts sys.path, especially in scenarios where the directory structure is not installed as a Python package.

For instance, when a script is run as a module, its directory is added to the start of sys.path, impacting how Python finds and loads other modules.

This distinction between running scripts directly and as modules is subtle yet significant, especially when considering the structure and distribution of your Python projects.

Now let’s briefly understand SYS.PATH.

What is SYS.PATH?

sys.path is a list of strings that specifies the search path for modules.

It includes the directory containing the input script (or the current directory), followed by directories listed in the PYTHONPATH environment variable, and then the installation-dependent default.

The content of sys.path varies depending on how you invoke a Python Module.

You can add a directory to sys.path using the sys.path.append() method e.g. sys.path.append('/path/to/my_module_directory')

Imagine sys.path as a library’s index card system. When you ask for a book (or a Python module), the librarian (Python interpreter) checks the index cards (sys.path) to find out in which aisle (directory) the book (module) is located.

If the librarian can’t find the right index card, they can’t locate the book. Similarly, if Python doesn’t find the directory in sys.path, it can’t load the module.

This is often the cause for the infamous ModuleNotFoundError in Python.

So with this strong understanding, let’s see how to run tests with Pytest.

The pytest Command

Pytest is a powerful, straightforward testing framework in Python that allows you to write tests using simple assert statements.

In general, Pytest is invoked with the command pytest.

This will execute all tests in all files whose names follow the form test_*.py or *_test.py in the current directory and its subdirectories.

If your test files don’t adhere to the correct naming convention, Pytest won’t recognize them and you’ll get the pytest collected 0 items error.

Read through the article to understand how Pytest collects tests and familiarise yourself with the Python unit testing best practices.

While Pytest can be invoked directly using the pytest command, it can also be invoked as a Python Module using python -m pytest.

If you’re trying to run pytest and it doesn’t work i.e. you get a 'pytest' command not found error, this guide has your back.

The python -m pytest Command

Invoking Pytest with python -m pytest is akin to running your test scripts through the Python interpreter as a module.

This method has significant implications for your testing environment, particularly in how Python manages module resolution and path settings.

Here’s what happens under the hood:
When you execute python -m pytest, Python dynamically adjusts sys.path, which is the list of directories Python searches for modules.

The first entry in this modified path is the directory containing the module you’re running.

Consequently, this prioritizes your local directory, making sure Python first searches here for any modules you import.

Then, Python searches for the module in the directories specified by the PYTHONPATH environment variable.

Finally, it searches the installation-dependent default path, which is typically the Python installation directory.

This command becomes particularly vital in projects where multiple Python environments coexist.

By invoking Pytest this way, you ensure that you are operating within the intended environment.

This approach eliminates the risk of accidentally importing modules or dependencies from an unintended environment.

In summary, python -m pytest offers a more controlled and environment-specific way to run your tests.

It ensures that your testing process is finely tuned to your project’s structure, keeping your tests accurate and reliable.

Comparative Analysis (python -m pytest vs pytest)

At first glance, these commands might seem identical, both aiming to run tests. However, the devil is in the details, and that’s what we’ll explore.

Let’s start!

Syntax and usage differences

Invoking python -m pytest tells Python to run the Pytest module as a script, which subtly alters how Python sets up the testing environment.

Python module searching is done in the following way:

  1. Python first looks for the module in its list of built-in modules. This is like having a set of default tools always available without needing to look elsewhere.
  2. Search in sys.path list:
  • First, Python adds the directory of the script that is being run (or the current working directory if you’re using an interactive interpreter) to sys.path.
  • Second, Python includes the directories specified in the PYTHONPATH environment variable. This is indeed like consulting a guidebook for additional places.
  • Finally, Python adds the installation-dependent default directories, which include the standard library and site-packages directory where third-party packages are installed.

Similarities

  1. Core Functionality: Both commands run Pytest test suites.
  2. Test Discovery: They use the same method for discovering and running tests.
  3. Plugin Support: Both support Pytest’s extensive plugin system.

Differences

  1. Environment Handling:

    • python -m pytest: Adds the current directory to sys.path, ensuring environment consistency.
    • pytest: Relies on the existing Python environment, assuming correct setup.

  2. Module Resolution:

    • Python -m pytest: More reliable in complex environments or with multiple Python versions.
    • pytest: Simpler but requires proper configuration of the environment.

Sample Code

Consider a simple test file, test_sample.py, with the following content:

1
2
3
4
# test_sample.py

def test_addition():
    assert 1 + 1 == 2

Now let’s run this test file using python -m pytest:

1
python -m pytest test_sample.py

The output will look like this:

1
2
3
4
5
6
7
8
9
10
11
12
(venv) PS C:\VS\mypytest> python -m pytest 
C:\VS\mypytest\venv\Lib\site-packages\_pytest\config\__init__.py:482: PytestConfigWarning: pytest-catchlog plugin has been merged into the core, please remove it from your requirements.
  warnings.warn(
========================================== test session starts ==========================================
platform win32 -- Python 3.11.4, pytest-7.4.3, pluggy-1.2.0
rootdir: C:\VS\mypytest       
plugins: catchlog-1.2.2, mock-3.11.1
collected 1 item

test_python_m_pytest.py .                                                                     [100%]

=========================================== 1 passed in 0.25s ===========================================

And this is what the pytest command for running this test file will look like:

1
pytest test_sample.py

The output shall look like this:

1
2
3
4
5
6
7
8
9
10
11
12
(venv) PS C:\VS\mypytest>pytest 
C:\VS\mypytest\venv\Lib\site-packages\_pytest\config\__init__.py:482: PytestConfigWarning: pytest-catchlog plugin has been merged into the core, please remove it from your requirements.
  warnings.warn(
========================================== test session starts ==========================================
platform win32 -- Python 3.11.4, pytest-7.4.3, pluggy-1.2.0
rootdir: C:\VS\mypytest       
plugins: catchlog-1.2.2, mock-3.11.1
collected 1 item

test_python_pytest.py .                                                                     [100%]

=========================================== 1 passed in 0.23s ===========================================

As you can see for yourself, both the commands produce exactly the same output.

Both commands will correctly identify and run the test_addition function and the test cases will pass.

Project Structure and Test Execution

A typical Python project structure may look like this:

1
2
3
4
5
6
7
- Project Root
  - app/
    - __init__.py
    - module.py
  - tests/
    - __init__.py
    - test_module.py

When you run tests using python -m pytest from the Project Root, Python adds the directory from where this command is run (in this case, the Project Root) to the start of the sys.path.

This inclusion is significant because it ensures that your tests can find and import the app module, even if it’s not installed as a package in your Python environment.

On the other hand, when you simply run pytest, it operates under the assumption that your environment is already configured to find and import the necessary modules like app.module.

This means that for Pytest to successfully import the app module, the app directory needs to be discoverable within your Python environment, typically through correct path settings or by being installed as a package.

Therefore, while both commands aim to run your tests, python -m pytest provides a more fail-safe approach for handling imports in a typical Python project structure, particularly when you have a directory structure that isn’t installed as a Python package.

If you’d like to learn more about how to set the PYTHONPATH environment variable, check out this article on 4 Proven Ways To Define Pytest PythonPath and Avoid Module Import Errors.

Which One To Choose?

When you’re new to Python testing, using python -m pytest is indeed a wise choice.

This approach mitigates potential complications related to path settings and module imports.

Essentially, it simplifies handling some of the complexities of the Python environment for you.

For you pros out there, the direct pytest command offers a quick and streamlined method to run tests. This is my default go-to method for running tests.

This approach assumes that your environment is already set up correctly and that pytest will behave as expected.

The primary reason to prefer module invocation (python -m pytest) over direct invocation (pytest) hinges on ensuring that you’re using the correct installation of Pytest.

It’s not uncommon to have multiple Pytest versions installed across various environments.

In such cases, using python -m pytest ensures that the Pytest version aligned with your current interpreter is used, thereby avoiding version-related issues.

However, it’s crucial to balance the benefits of environment specificity with the potential risks of including the local directory in your Python path.

Adding the current directory to sys.path could potentially lead to conflicts or unexpected behavior if there are identically named files or modules in your project directory.

In conclusion, the choice between python -m pytest and pytest should be informed by your understanding of Python environments and the specific needs of your project.

Each method has its advantages and pitfalls, so weight these factors in carefully before making your decision.

Calling Pytest From Python Code

Did you know that you can invoke Pytest directly from your Python code, just like you would from the command line?

This approach offers more control and flexibility, especially when you’re dealing with complex testing scenarios or need to integrate testing into larger Python applications.

To call Pytest from Python, use pytest.main(). This function behaves similarly to running pytest from the command line, but instead of raising SystemExit, it returns the exit code. Here’s a basic example:

1
2
3
import pytest

retcode = pytest.main()

This command mimics typing pytest in your terminal. By default, it reads arguments from the command line arguments of the process (sys.argv).

But what if you want more control?

You can explicitly pass in options and arguments to pytest.main(), just like you would in the command line. For instance:

1
retcode = pytest.main(["-x", "mytestdir"])

This line tells Pytest to stop after the first failure (-x) and only test files in mytestdir.

You can read more about invoking Pytest in different ways and multiple arguments here.

FAQ: Addressing Common Queries

Q: Can I use both commands interchangeably?
Generally, yes. However, prefer python -m pytest in complex environments or when facing module resolution issues.

Q: Is there a performance difference between the two?
There’s no significant performance difference. The main variation lies in environment handling and module resolution.

Q: Will my Pytest plugins work with both commands?
Absolutely! Both commands fully support Pytest’s plugin architecture.

Q: Should I always use python -m pytest to be safe?
While it’s a safer option in terms of environment consistency, using pytest directly is perfectly fine for most standard setups.

Q: How do python -m pytest and pytest handle PythonPATH differently?
python -m pytest implicitly adds the current directory to PythonPATH, ensuring that the directory from which it’s run is seen as a source for modules. While pytest also adds the current directory to PythonPATH, it does so only if the current directory is already in PythonPATH.

In contrast, pytest relies on the existing PythonPATH configuration. This can lead to differences in how your tests locate and import modules.

Q: Can I use python -m pytest in a virtual environment, and will it behave differently from pytest?
Both commands work within a virtual environment. However, python -m pytest can be more consistent in ensuring that the Python interpreter associated with your virtual environment is used.

This is particularly relevant when you have multiple virtual environments or Python versions.

Q: Is there a difference in plugin loading or execution order between the two commands?
No, there is no difference in how plugins are loaded or executed. Both commands will load and execute plugins as defined in the test file.

Conclusion

That’s all, folks!

In this article, you learned about the differences between python -m pytest and pytest.

Whether you choose python -m pytest or pytest, the goal remains the same: integrating testing seamlessly into your development workflow. Experiment, find your preference, and boost your Python testing game!

Here we delved into the intricacies of Python’s testing ecosystem, uncovering how small differences in command invocation can significantly impact your testing environment and module resolution.

By dissecting these aspects, you’ve gained insight into making informed decisions that best fit your development needs.

The world of Python testing awaits, full of mysteries to uncover and challenges to conquer.

Happy testing, and may your code be bug-free and your tests always pass!

If you have ideas for improvement or would like me to cover anything specific, please send me a message via Twitter, GitHub or Email.

Till the next time… Cheers!

Additional Learning

How to Run Pytest?
Official Docs - Usage of Pytest
Python Unit Testing With Pytest
How To Use Pytest With Command Line Options
4 Proven Ways To Define Pytest PythonPath and Avoid Module Import Errors
Practical Overview Of The Top 5 Python Testing Frameworks
A Simple Guide to Fixing The ‘Pytest Collected 0 Items’” Error
Python Unit Testing Best Practices For Building Reliable Applications
7 Simple Ways To Fix The “Pytest Command Not Found” Error