Skip to main content
Nick Baynham

Reference implementation

Python Playwright

A complete, open-source test automation platform in Python: Playwright-driven UI tests across every browser your machine has, a typed REST API client with schema-validating assertions, MongoDB seeding and state verification, and one full-stack scenario that chains all three. This guide takes you from a fresh clone to writing your own tests at each layer. Every code example on this page runs and passes against the project's dockerized sample stack.

View the source on GitHub

Introduction

Introduction

The platform tests applications at three layers. Knowing which layer a check belongs to is the single most useful skill this guide teaches.

The three testing layers and what each verifies
LayerTest suiteVerifiesBuilt on
UItests/e2e/What a user sees and does in the browserPlaywright, page objects
APItests/integration/REST contracts: status codes, payloads, schemashttpx client, assertion helpers
Databasetests/integration/Stored state behind the applicationpymongo target, seeding, state assertions

A fourth pattern, the full-stack scenario, chains all three: act in the UI, verify through the API, confirm the document in MongoDB. The repository ships a working one in tests/e2e/test_full_stack.py.

You will need Python 3.14+, PDM, GNU Make, Docker Desktop, and the Allure CLI — the README lists install commands for each. Then the first run is five commands:

First run
make install            # dependencies (PDM)
make install-browsers   # detect browsers, write config/browsers.json
make docker-up          # start the sample stack: React app, REST API, MongoDB
make test               # run everything: unit, integration, e2e
make report             # browse the Allure HTML report

UI Testing, Step by Step

UI Testing, Step by Step

UI tests are written against page objects, never raw selectors. The fixtures handle browsers, isolation, and failure evidence for you.

Three steps to a new UI test:

  1. Create a page object under tests/e2e/pages/ extending BasePage. Implement path (the route) and ready_locator (an element visible exactly when the page is usable — this is how the platform avoids sleeps). Expose locators and actions as attributes and methods.
  2. Write the test as a plain functiontaking the page-object fixture, and assert with Playwright's expect, which waits intelligently.
  3. Create unique data and clean it up — the track_item fixture deletes whatever your test created, even when the test fails.
A minimal page object
class SignInPage(BasePage):
    @property
    def path(self) -> str:
        return "/sign-in"

    @property
    def ready_locator(self) -> Locator:
        return self.heading  # visible exactly when the page is usable

A complete UI test

tests/e2e/test_example.py
from collections.abc import Callable
from uuid import uuid4

from playwright.sync_api import expect

from tests.e2e.pages.sample_app import SampleAppPage


def test_new_item_is_listed(
    sample_app: SampleAppPage, track_item: Callable[[str], str]
) -> None:
    name = track_item(f"guide-{uuid4().hex[:8]}")
    sample_app.open()
    sample_app.add_item(name)
    expect(sample_app.items().filter(has_text=name)).to_have_count(1)

Run it with make docker-up then make test-e2e. It executes once per available browser. If it fails, a full-page screenshot and a Playwright trace land in test-artifacts/, named after the test.

API Testing, Step by Step

API Testing, Step by Step

API tests use a typed httpx client and assertion helpers whose failure messages carry the response body — diagnosable from the test report alone.

Take the api fixture (an ApiClient bound to TP_API_BASE_URL), make calls with get/post/request, and assert with assert_status, assert_json, assert_json_contains, or assert_matches_schema (JSON Schema 2020-12, reporting every violation at once). Delete what you create.

A complete API test

tests/integration/test_example.py
from testplatform.api import ApiClient
from testplatform.assertions import assert_json_contains, assert_status


def test_item_lifecycle(api: ApiClient) -> None:
    created = api.post("/items", json_body={"name": "guide item"})
    assert_status(created, 201)
    item_id = created.json()["id"]

    fetched = api.get(f"/items/{item_id}")
    assert_status(fetched, 200)
    assert_json_contains(fetched, {"name": "guide item"})

    assert_status(api.request("DELETE", f"/items/{item_id}"), 204)
    assert_status(api.get(f"/items/{item_id}"), 404)

Database Testing, Step by Step

Database Testing, Step by Step

Database tests seed their own state, verify documents directly in MongoDB, and remove exactly what they seeded.

The mongo_target fixture connects to TP_MONGO_URL and fails fast with guidance when the database is down. The seeder fixture inserts documents and deletes them afterwards — by id, never by dropping collections, so it is safe against a database holding data your test does not own. Tag seeded documents with a unique marker and scope queries to it.

A complete database test

tests/integration/test_example_db.py
from uuid import uuid4

from testplatform.assertions import assert_collection_count, assert_field_values
from testplatform.db import MongoSeeder, MongoTarget


def test_seeded_order_state(mongo_target: MongoTarget, seeder: MongoSeeder) -> None:
    tag = f"guide-{uuid4().hex[:8]}"
    seeder.seed(
        "orders", {"tag": tag, "status": "new"}, {"tag": tag, "status": "paid"}
    )
    assert_collection_count(mongo_target, "orders", 2, {"tag": tag})
    assert_field_values(
        mongo_target, "orders", {"tag": tag, "status": "paid"}, {"status": "paid"}
    )

Designing Tests for the Framework

Designing Tests for the Framework

The rules that keep a growing suite fast, trustworthy, and pleasant to work in.

Test design principles
PrincipleWhat it means in practice
Choose the lowest layer that proves the behaviorA payload contract belongs in an API test, not behind a browser. A stored side effect belongs in a database assertion. A user journey belongs in the UI suite.
Isolation is non-negotiableEvery test creates its own data with unique names, cleans up what it creates, and never depends on another test having run. Whole-list equality against shared state is forbidden; filter to your own data.
Page objects own locators; tests own assertionsIf a test reaches for page.get_by_* directly, the locator belongs in a page object. If a page object asserts, that judgment belongs in the test.
Configuration comes from the environmentTests read targets through load_settings(), never hardcoded URLs. Remote targets require every TP_*_URL explicitly — the platform refuses to point a remote run at localhost defaults.
Names state behaviortest_added_item_appears_and_input_clears tells the reader what broke without opening the file.

Troubleshooting

Troubleshooting

The failures you will actually hit, each with its cause and fix. All of these were encountered for real while building the platform.

Common failures with causes and fixes
SymptomCauseFix
sample API not reachable / MongoDB not reachableThe dockerized stack is downmake docker-up and wait for healthy
Every e2e test skipped: no usable browser inventoryMissing or invalid config/browsers.jsonmake install-browsers
Coverage failure when running one test fileThe 90% gate is calibrated for the full suiteAdd --no-cov for spot runs
ValidationError naming TP_*_URL variablesRemote mode refuses defaulted localhost URLsSet every named variable explicitly
port is already allocatedAnother process owns 3100, 8100, or 27100Copy .env.example to .env; change the port and its matching URL together
A test failed — where is the evidence?Failure artifacts are captured automaticallyScreenshots and traces in test-artifacts/; open traces with playwright show-trace; make report for Allure
Windows: make not foundGNU Make is not installed by defaultchoco install make