Flask: pytest fixtures and tests that catch a real regression

pythonflaskindependent-practice

Saved on this device

Full lesson · 4 learning steps Open any exercise for focused practice

Step 1 of 4 · intro card

Flask: pytest fixtures and tests that catch a real regression

Concept and contract

Test the response contract rather than merely confirming that a request completed.

A useful test should pass for the intended behavior and fail for a relevant regression. Flask’s test client drives the request stack without a network server, and pytest fixtures can construct a fresh application for each test.

That combination makes failures reproducible. A status-only assertion can still be too weak when the endpoint returns the wrong value or leaks an unexpected field.

Mechanism and worked example

The helper writes a tiny owned pytest module into a temporary directory, supplying a controlled response body.

The helper writes a tiny owned pytest module into a temporary directory, supplying a controlled response body. The fixture builds an application and yields its test client.

pytest then executes the actual response assertions, and the helper reports its exit outcome. Temporary source is generated solely from local fixture values, not from untrusted downloaded code or production request content.

Complete worked example

python

import io
import contextlib
import tempfile
from pathlib import Path
import pytest

TEST_SOURCE = """
from flask import Flask
import pytest
@pytest.fixture
def client():
    app = Flask(__name__)
    app.config['TESTING'] = True
    @app.get('/count')
    def count():
        return BODY
    return app.test_client()
def test_count(client):
    response = client.get('/count')
    assert response.status_code == 200
    assert response.json['count'] == 2
"""

def test_contract(count, extra=False):
    body = {'count': count}
    if extra:
        body['debug'] = 'private'
    with tempfile.TemporaryDirectory(prefix='owned-flask-tests-') as folder:
        path = Path(folder) / ('test_' + Path(folder).name.replace('-', '_') + '.py')
        path.write_text('BODY = ' + repr(body) + '\n' + TEST_SOURCE)
        with contextlib.redirect_stdout(io.StringIO()):
            result = pytest.main([str(path), '-q', '--import-mode=importlib', '-p', 'no:cacheprovider'])
    return {'passed': result == 0}

Reasoning: The helper writes a tiny owned pytest module into a temporary directory, supplying a controlled response body. The fixture builds an application and yields its test client.

Counterexamples and common mistakes

The counterexample replaces the count assertion with a duplicate status assertion.

The counterexample replaces the count assertion with a duplicate status assertion. A body containing count three still passes, demonstrating a false confidence problem.

Avoid mirroring implementation statements mechanically; choose observable requirements that would catch meaningful changes. Error diagnostics are captured for this bounded runner, while a learner can run the generated test directly to inspect the ordinary pytest failure report.

Counterexample: a runnable framework mistake

python

import io
import contextlib
import tempfile
from pathlib import Path
import pytest

TEST_SOURCE = """
from flask import Flask
import pytest
@pytest.fixture
def client():
    app = Flask(__name__)
    app.config['TESTING'] = True
    @app.get('/count')
    def count():
        return BODY
    return app.test_client()
def test_count(client):
    response = client.get('/count')
    assert response.status_code == 200
    assert response.status_code == 200
"""

def test_contract(count, extra=False):
    body = {'count': count}
    if extra:
        body['debug'] = 'private'
    with tempfile.TemporaryDirectory(prefix='owned-flask-tests-') as folder:
        path = Path(folder) / ('test_' + Path(folder).name.replace('-', '_') + '.py')
        path.write_text('BODY = ' + repr(body) + '\n' + TEST_SOURCE)
        with contextlib.redirect_stdout(io.StringIO()):
            result = pytest.main([str(path), '-q', '--import-mode=importlib', '-p', 'no:cacheprovider'])
    return {'passed': result == 0}

Reasoning: The counterexample replaces the count assertion with a duplicate status assertion. A body containing count three still passes, demonstrating a false confidence problem.

Independent practice: one bounded framework change

Add an exact-key assertion requiring only count in the JSON body.

Add an exact-key assertion requiring only count in the JSON body. Preserve the existing status and value assertions.

The controlled extra mode adds a debug field that should now make the test fail, even when count remains correct. This is one additional contract assertion rather than an invitation to duplicate the whole response-building implementation in the test.

Step 1: Assert that the response contains exactly the count field. Step 2: Add the assertion inside TEST_SOURCE.

Practice starter: solve before reading the solution

python

import io
import contextlib
import tempfile
from pathlib import Path
import pytest

TEST_SOURCE = """
from flask import Flask
import pytest
@pytest.fixture
def client():
    app = Flask(__name__)
    app.config['TESTING'] = True
    @app.get('/count')
    def count():
        return BODY
    return app.test_client()
def test_count(client):
    response = client.get('/count')
    assert response.status_code == 200
    assert response.json['count'] == 2
"""

def test_contract(count, extra=False):
    # Remove this placeholder, then adapt the retained baseline:
    # Assert that the response contains exactly the count field.
    raise NotImplementedError
    body = {'count': count}
    if extra:
        body['debug'] = 'private'
    with tempfile.TemporaryDirectory(prefix='owned-flask-tests-') as folder:
        path = Path(folder) / ('test_' + Path(folder).name.replace('-', '_') + '.py')
        path.write_text('BODY = ' + repr(body) + '\n' + TEST_SOURCE)
        with contextlib.redirect_stdout(io.StringIO()):
            result = pytest.main([str(path), '-q', '--import-mode=importlib', '-p', 'no:cacheprovider'])
    return {'passed': result == 0}

Reasoning: Assert that the response contains exactly the count field. Add the assertion inside TEST_SOURCE.

Hint 1: Keep status and numeric-value assertions.
Hint 2: Verify that the debug-field mutant fails.

Solution, tests and trade-offs

Baseline fixtures run correct and incorrect counts and repeat the correct case.

Baseline fixtures run correct and incorrect counts and repeat the correct case. Practice tests the extra-field mutation as well.

The suite uses importlib mode and disables pytest’s cache provider so repeated temporary modules remain isolated. The lab also disables unrelated plugin autoload. These checks exercise pytest itself and the Flask client, rather than returning a manually computed expected boolean.

Practice solution and acceptance checks

python

import io
import contextlib
import tempfile
from pathlib import Path
import pytest

TEST_SOURCE = """
from flask import Flask
import pytest
@pytest.fixture
def client():
    app = Flask(__name__)
    app.config['TESTING'] = True
    @app.get('/count')
    def count():
        return BODY
    return app.test_client()
def test_count(client):
    response = client.get('/count')
    assert response.status_code == 200
    assert response.json['count'] == 2
    assert set(response.json) == {'count'}
"""

def test_contract(count, extra=False):
    body = {'count': count}
    if extra:
        body['debug'] = 'private'
    with tempfile.TemporaryDirectory(prefix='owned-flask-tests-') as folder:
        path = Path(folder) / ('test_' + Path(folder).name.replace('-', '_') + '.py')
        path.write_text('BODY = ' + repr(body) + '\n' + TEST_SOURCE)
        with contextlib.redirect_stdout(io.StringIO()):
            result = pytest.main([str(path), '-q', '--import-mode=importlib', '-p', 'no:cacheprovider'])
    return {'passed': result == 0}

Reasoning: Baseline fixtures run correct and incorrect counts and repeat the correct case. Practice tests the extra-field mutation as well.

Interview reasoning and next step

Expand a real application’s suite with missing credentials, foreign ownership, malformed input and transaction failure cases.

Expand a real application’s suite with missing credentials, foreign ownership, malformed input and transaction failure cases. Keep unit, request and deployment checks complementary; a test client does not prove reverse-proxy behavior or database engine equivalence.

Select a small regression that would have caught each real defect, then verify the test fails before fixing the implementation. That habit improves evidence more than raw assertion counts.

Optional stretch — after the core task works: Add an authenticated owner/foreign-user pytest fixture.

Computer practice: https://applydjinn.com/typorithm/python-frameworks-lab-20261105.zip. Install the pinned dependencies in an isolated environment, run verify.py --track flask, then edit the retained exercise baseline and check it with --practice fl-tests. The lab uses owned in-memory databases, local framework clients and tiny CPU tensors. It never reads the working ApplyDjinn database, contacts production feeds or downloads models.

Test the response contract rather than merely confirming that a request completed.