Test Automation Made Easy with Pytest and Playwright

You’ve probably come across or used tools like Cypress or Selenium in test automation.

These tools though powerful, are dated, slow, and flaky.

Enter the new kid on the block, Playwright by Microsoft.

Have you heard about Playwright but have yet to use it? Does it integrate well with Pytest?

How do you configure it, why should you even consider using it, over Selenium or Cypress?

In this article, we answer all your Playwright questions.

You’ll learn how Playwright compares with Selenium, its differentiation and benefits, auto-code generation, and integration with Pytest.

You’ll also learn how to take screenshots and videos of your browser tests, do parallel cross-browser testing, use the trace viewer and so much more.

All of the above with practical code examples.

So read on and enjoy.

If you prefer to dive into the code straightaway, here’s a link to the repo.

What is Playwright?

If you’re new to automation testing, I highly recommend you check out this article on Pytest Selenium which goes into practical detail on automation tests and how to write them.

In brief, automation testing checks how your application works in a browser or across multiple browser projects.

It’s testing what happens when you submit a form, click a button, check a box, log in, and so on.

This includes verifying that page elements load and perform as expected.

It tests front-end interactions with backend processes and allows you to assert the elements against expected values. This could be on your own application or even external applications.

Playwright, released by Microsoft in January 2020 is a new tool that allows you to perform automation testing at scale, with ease.

From the docs

Playwright enables reliable end-to-end testing for modern web apps.

It’s very developer-friendly (the docs speak for themselves), much faster than similar tools, and has built-in features like auto-wait, tracing, screenshots, and video.

It also runs tests in different browser contexts providing necessary test isolation with no setup teardown overhead.

We’ll talk about the benefits and features shortly.

Playwright vs Selenium

Let’s compare how Playwright performs against the industry veteran Selenium.

Architecture

• Playwright: Playwright supports multiple languages natively and offers a single API to work with different browsers. It’s designed for parallel executions and handling modern web features like single-page applications efficiently.
• Selenium: Interacts with page elements in a more traditional, object-oriented way, which requires more boilerplate code. It operates through a webdriver which may introduce additional latency.

Browser Support

• Playwright: Supports Chromium, Firefox, and WebKit projects natively without additional drivers or configurations.
• Selenium: While Selenium supports a wide range of browsers, including the less common ones, setting up drivers for different browsers can sometimes be cumbersome.

Execution Speed

• Playwright: Tends to be faster in execution due to its more efficient handling of modern web apps and its ability to run parallel tests more natively.
• Selenium: Generally considered slower compared to Playwright, especially in complex test scenarios involving heavy AJAX and dynamic content.

API and Developer Experience

• Playwright: Offers a more consistent and unified API across supported languages, which can reduce the learning curve and improve the developer experience.
• Selenium: Has a mature API but varies slightly between languages and can be verbose, requiring more code to perform the same actions as Playwright and a greater learning curve.

Setup and Configuration

• Playwright: Offers a simpler setup with less configuration overhead. It also provides tools for generating code, managing browser binaries, and setting up the testing environment.
• Selenium: Might require more initial setup, especially when integrating with various browsers and managing their respective drivers.

CI/CD Integration

• Playwright: Integrates smoothly with modern CI/CD pipelines and provides features like tracing, video recording and screenshot capturing out of the box.
• Selenium: This can be integrated into CI/CD pipelines but may require additional tools for recording tests and capturing screenshots.

Playwright is a modern tool that’s picked up steam in the last 4 years.

Let’s look at some of its benefit-driven features.

Playwright Benefits

For convenience, I’ll list out and explain (in brief) the benefits from the Official Docs.

Any browser | Any platform | One API

• Works on any browser project — Chromium, Firefox, Webkit
• Works on Linux, Mac, Windows, CI/CD pipelines
• API support is available in Python, Java, Javascript, Typescript and .Net
• Native simulations of Chrome on Android and Safari.

Resilient | No Flaky Tests

• Unlike Selenium, where you have to handle waits for element loads, Playwright takes care of this automatically, which is a huge benefit.
• Web first assertions — this means that Playwright has an inbuilt expect statements that can assert elements on a page for example expect(page.get_by_role(“heading”, name=”Installation”)).to_be_visible().
• Tracing — It also has inbuilt tracing functionality which allows you to generate a trace of each run and import that into a Chrome Dev Tools type of inspector.

Full isolation | Fast execution

• Playwright runs each test within a browser context which is the equivalent of running a process in Incognito mode, rather than spin up a separate browser for each test. This has significantly lower setup and teardown overhead.
• It also saves authentication info and can be used across tests while maintaining full isolation.

Tooling

• Playwright has excellent tooling like Codegen that allows you to perform actions and generate Playwright test code to test those elements.
• The ability to record tests via tracing, video, or screen capture is also truly a game changer.

Many more benefits are listed in the official docs, and we’ve only scratched the surface.

I hope you’re convinced of the use cases and benefits, now let’s go into some practical code examples of how to use it.

Using Playwright — Basic Example Code

The example code used can be found in this repo.

I’ve switched to using Poetry to manage virtual environments, if you need guidance on how to set up and use Poetry, please check out this article.

First, clone the repo and install the dependencies.

1

$ poetry install

We’re going to use the pytest-playwright plugin which provides us with the inbuilt Page fixtures and auto handles setup teardown and test isolation which is extremely useful.

1

$ pip install pytest-playwright

Next, we install the browser kits.

1

$ playwright install

This will download the various browser kits to the local machine.

Now let’s look at simple Playwright tests.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

`tests/test_example.py`

import pytest  
import re  
from playwright.sync_api import Page, expect  
  
  
@pytest.mark.basic  
def test_has_title(page: Page):  
    page.goto("https://playwright.dev/")  
  
    # Expect a title "to contain" a substring.  
    expect(page).to_have_title(re.compile("Playwright"))  
  
  
@pytest.mark.basic  
def test_get_started_link(page: Page):  
    page.goto("https://playwright.dev/")  
  
    # Click the get started link.  
    page.get_by_role("link", name="Get started").click()  
  
    # Expects page to have a heading with the name of Installation.  
    expect(page.get_by_role("heading", name="Installation")).to_be_visible()

Let’s break down what’s going on here.

I’ve marked the tests as basic for selective test run purposes so you can ignore that for now.

• We import the Page fixture provided by Playwright that maintains test isolation and takes care of setup and teardown.
• We use page.goto to get a URL.
• We then use the inbuilt web assertion expect(page).to_have_title(re.compile("Playwright")) to check whether the page we got contains the title Playwright or a regex of it.

In the second test,

• We get the same page.
• We use the get_by_role locator to locate a link with the name Get started and click it.
• We expect that the new page will contain a heading with the name Installation to be visible.
• There are several locators and actions available and we’ll touch on the most important ones later in the article.

Let’s run these tests.

1

$ poetry run pytest -m basic

Now I want to take the opportunity to introduce the headed and headless mode.

Headed vs Headless Mode

In test automation, there are 2 modes of execution — headed and headless.

Headed mode means the test automation framework will open the browser and show you the HTML and execution steps, which is very convenient when developing tests and debugging.

While convenient for local development, this doesn’t do so well in a CI/CD Pipeline.

Headless, however, means running it in the background and you don’t see a browser window popup. But you can be sure it runs in the background and performs the necessary steps.

The drawback of headed mode is the sheer time, hence headless is the preferred mode of operation when running tests in a CI/CD pipeline or automated way.

Playwright runs tests by default in headless mode, which you can change by passing the --headed flag to Pytest to run it in headed mode.

You will then see the browser window popup, run the tests, and close.

1

$ poetry run pytest -m basic --headed

You can see this takes a bit more time than headless mode.

Example 2 (DuckDuckGo Search)

Let’s look at another example to understand the various locators.

tests/test_example.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

@pytest.mark.search  
def test_duckduckgo_search(page: Page):  
    # Go to the DuckDuckGo homepage.  
    page.goto("https://duckduckgo.com/")  
  
    # Type "Pytest with Eric" into the search box and press Enter.  
    page.get_by_placeholder("Search without being tracked").fill("Pytest with Eric")  
  
    # Press Enter to submit the search.  
    page.locator('button[aria-label="Search"]').click()  
  
    # Get the first search result using get_by_role  
    first_result = page.get_by_role("link", name=re.compile("Pytest")).first  
  
    # Expect the first result to contain the text "Pytest"  
    expect(first_result).to_have_text(re.compile("Pytest With Eric"))

Let’s do a quick rundown of what’s going on.

• We go to the DuckDuckGo home page.
• We search for the placeholder text and fill it with a query
• We then use the raw button locator to find the search button and click it.
• We get the first link and check that it contains “Pytest With Eric”

There are several ways to search and validate locators and this is just one of them.

Running this test

1

$ poetry run pytest -m search -v -s

Example 3 (Page Login)

Let’s look at another, slightly more complex example where you can log in to a Page.

tests/test_example.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

@pytest.mark.login  
def test_login(page: Page):  
    page.goto("https://practicetestautomation.com/practice-test-login/")  
  
    # Fill in the login form  
    page.get_by_label("username").fill("student")  
    page.get_by_label("password").fill("Password123")  
  
    # Submit the form  
    page.get_by_role("button", name="Submit").click()  
  
    # Look for "Congratulations Message" text  
    expect(page.get_by_text(re.compile("Congratulations"))).to_be_visible()  
  
    # Look for a Logout Button  
    expect(page.get_by_text(re.compile("Log out"))).to_be_visible()

In this example, we use the get_by_label locator to get the username and password fields and fill them with text.

We then used the get_by_role locator to get the button and click it.

Lastly, we expect the page to have “Congratulations” and “Log out” text.

1

$ poetry run pytest -m login -v -s

Codegen (Auto Generate Playwright Code)

One of Playwright’s interesting features is its ability to track your actions on a webpage and auto-generate code (Codegen).

You can read more about this functionality here.

Let’s test this out.

In the terminal, run

1

$ poetry run playwright codegen

This will start the Playwright Inspector and a Chrome Browser.

Click the Record button (if not red already) and perform actions like button clicks, filling in elements, etc.

Enjoy watching the auto-code generation on the right.

In this example, I navigated to https://practicetestautomation.com/practice-test-login/ and played around with different logins.

The result is here.

Fascinating, right? You can change the code language from the “Target” selection.

Here’s another example from navigating around the Pytest with Eric website.

Parallel Testing using xdist

If your machine has a lot of CPUs, you can leverage the parallel execution of Playwright tests via the pytest-xdist plugin, using the — numprocesses flag.

1

$ pytest --numprocesses auto

For example,

Screenshots and Video with Playwright

Capturing screenshots and videos is also straightforward with Playwright.

Screenshot

Add the below to your test,

tests/test_example.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

@pytest.mark.login  
def test_login(page: Page):  
    page.goto("https://practicetestautomation.com/practice-test-login/")  
  
    # Fill in the login form  
    page.get_by_label("username").fill("student")  
    page.get_by_label("password").fill("Password123")  
  
    # Submit the form  
    page.get_by_role("button", name="Submit").click()  
  
    # Look for "Congratulations Message" text  
    expect(page.get_by_text(re.compile("Congratulations"))).to_be_visible()  
  
    # Look for a Logout Button  
    expect(page.get_by_text(re.compile("Log out"))).to_be_visible()  
  
    # Take screenshot  
    page.screenshot(path="screenshot.png")

You can check out the various configuration options here.

Videos

Let’s tweak our DuckDuckGo search test to capture video.

tests/test_example.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

@pytest.mark.search  
def test_duckduckgo_search(page: Page, browser: Browser):  
  
    context = browser.new_context(record_video_dir="videos/")  
    page = context.new_page()  
  
    # Go to the DuckDuckGo homepage.  
    page.goto("https://duckduckgo.com/")  
  
    # Type "Pytest with Eric" into the search box and press Enter.  
    page.get_by_placeholder("Search without being tracked").fill("Pytest with Eric")  
  
    # Press Enter to submit the search.  
    page.locator('button[aria-label="Search"]').click()  
  
    # Get the first search result using get_by_role  
    first_result = page.get_by_role("link", name=re.compile("Pytest")).first  
  
    # Expect the first result to contain the text "Pytest"  
    expect(first_result).to_have_text(re.compile("Pytest With Eric"))  
  
    # Make sure to close, so that videos are saved.  
    context.close()

Let’s run through the changes

• Firstly, we created a new browser context and specified the directory to save our video. Think of a context like Incognito Mode.
• Then we created a new page within that context.
• Lastly, close the context at the end of the test.

Result,

Voila! we have a nice short video.

You can read more about how to configure Playwright to capture videos here.

Testing Across Browsers

One of the important features of any test automation framework is to test your application across many browser projects.

With many browsers available today, based on different projects — Chromium, Webkit, Firefox, etc. you need to be sure it works on all of them.

Fortunately, this is also very simple.

1
2
3

# Run Test on a specific browser

$ pytest test_login.py --browser webkit

1
2
3

# Run Test on multiple browsers

$ pytest test_login.py --browser webkit --browser firefox

1
2
3

# Run Test on mobile viewport

$ pytest test_login.py --device="iPhone 13"

1
2
3

# Test against branded browsers e.g. MS Edge

$ pytest test_login.py --browser-channel msedge

By default, Playwright will run your tests on Chromium, but you can specify command-line arguments to run against different browsers.

Tracing

Playwright gives you the ability to record traces, which are extremely helpful in debugging failing or flaky tests (especially on CI).

This feature is called the Trace Viewer and it can be accessed by generating a trace.zip file and uploading it to trace.playwright.dev.

To generate a trace, run

1

$ pytest --tracing on

This will generate a local trace file (default in the test-results/ folder), but you can change the file name and location.

Next, navigate to trace.playwright.dev and upload your trace.zip file.

You can see the trace like this.

This allows you to look through time and understand what happened under the hood. When combined with screenshots and videos, this is easily a powerful way to debug failing or flaky tests.

Conclusion

There’s so much more we could talk about Playwright —  Pytest fixtures, mocking, API testing, types of locators, and so on.

However, I decided to end this article here due to sheer limitations on time and scope.

If you want me to write more about some of Playwright’s interesting features, let me know via DM or email.

In this article, we started with a comparison between industry veteran Selenium and Playwright.

You then learned about the benefits of Playwright including web first assertion, auto-wait and locators.

Then we saw some practical code examples of how to write tests with Playwright, headed vs headless mode, capturing screenshots, videos and tracing.

I hope you found this useful and feel inspired to start using Playwright — whether at work or your own projects.

I think it’s a pretty powerful and modern automation tool with great documentation.

If you have any ideas for improvement or like me to cover any topics please comment below or send me a message via Twitter, GitHub or Email.

Till the next time… Cheers!

Additional Reading

Example Code Used in this Article
How to locate elements in Playwright
Playwright at PYCon US with Andrew Knight
How To Use Pytest With Selenium For Web Automation Testing
Playwright + Python: Framework for Automation Web Testing | Installation & Demo
Ultimate Guide To Pytest Markers And Good Test Management
How To Run Pytest With Poetry (A Step-by-Step Guide)
What is Setup and Teardown in Pytest? (Importance of a Clean Test Environment)