Tests that reject a changed response contract

pythondjangoindependent-practice

Saved on this device

Full lesson · 4 learning steps Open any exercise for focused practice

Step 1 of 4 · intro card

Tests that reject a changed response contract

Concept and contract

Exercise a real response and demonstrate that the test suite catches a controlled defect rather than checking success status alone.

A passing test suite is useful evidence only for the assertions it actually makes. An endpoint can keep returning 200 while changing a value that callers depend on.

This lesson runs a small Django SimpleTestCase suite against a real test-client response, then varies a controlled response value. The endpoint uses no database query, so choosing a database-forbidden simple test also makes its narrower dependency expectation explicit.

Mechanism and worked example

The suite checks status and the declared count value in separate tests.

The suite checks status and the declared count value in separate tests. The helper supplies controlled response state, creates a fresh suite and captures the test runner’s output instead of printing it into the application response.

Its returned test count confirms tests actually ran. An incorrect count should make the suite unsuccessful even though request handling itself still returns a valid JSON success response.

Complete worked example

python

from bootstrap_lab import django_setup
django_setup(__name__)

import io
import unittest
from django.http import JsonResponse
from django.test import SimpleTestCase
from django.urls import path

current_count, extra_field = 2, False

def summary(request):
    result = {"count": current_count}
    if extra_field:
        result["debug"] = True
    return JsonResponse(result)

urlpatterns = [path("summary/", summary)]

class ContractTests(SimpleTestCase):
    def test_status(self):
        self.assertEqual(self.client.get("/summary/").status_code, 200)
    def test_count(self):
        self.assertEqual(self.client.get("/summary/").json()["count"], 2)

def suite_result(count, extra=False):
    global current_count, extra_field
    current_count, extra_field = count, extra
    suite = unittest.TestLoader().loadTestsFromTestCase(ContractTests)
    result = unittest.TextTestRunner(stream=io.StringIO()).run(suite)
    return {"passed": result.wasSuccessful(), "tests": result.testsRun}

Reasoning: The suite checks status and the declared count value in separate tests. The helper supplies controlled response state, creates a fresh suite and captures the test runner’s output instead of printing it into the application response.

Counterexamples and common mistakes

The counterexample replaces the count assertion with another status assertion.

The counterexample replaces the count assertion with another status assertion. Both tests then pass for the controlled wrong value, demonstrating how superficially broad test coverage can miss the contract.

Avoid generating an expected value through the same questionable implementation as the endpoint. Also do not advertise the number of tests as proof of every property; assertions, input boundaries and failure cases determine what the evidence supports.

Counterexample: a runnable framework mistake

python

from bootstrap_lab import django_setup
django_setup(__name__)

import io
import unittest
from django.http import JsonResponse
from django.test import SimpleTestCase
from django.urls import path

current_count, extra_field = 2, False

def summary(request):
    result = {"count": current_count}
    if extra_field:
        result["debug"] = True
    return JsonResponse(result)

urlpatterns = [path("summary/", summary)]

class ContractTests(SimpleTestCase):
    def test_status(self):
        self.assertEqual(self.client.get("/summary/").status_code, 200)
    def test_count(self):
        self.assertEqual(self.client.get("/summary/").status_code, 200)

def suite_result(count, extra=False):
    global current_count, extra_field
    current_count, extra_field = count, extra
    suite = unittest.TestLoader().loadTestsFromTestCase(ContractTests)
    result = unittest.TextTestRunner(stream=io.StringIO()).run(suite)
    return {"passed": result.wasSuccessful(), "tests": result.testsRun}

Reasoning: The counterexample replaces the count assertion with another status assertion. Both tests then pass for the controlled wrong value, demonstrating how superficially broad test coverage can miss the contract.

Independent practice: one bounded framework change

Extend the count test to reject an unexpected debug field in the JSON object.

Extend the count test to reject an unexpected debug field in the JSON object. Keep the existing value and status checks and the same two-test structure.

Begin with the correct response, then a correct count plus the extra field and finally a wrong count. The task adds one response-shape assertion rather than creating a parallel testing framework or introducing a dependency the endpoint does not need.

Step 1: Reject an unexpected debug field in the count test. Step 2: Keep both value and status assertions and two executed tests.

Practice starter: solve before reading the solution

python

from bootstrap_lab import django_setup
django_setup(__name__)

import io
import unittest
from django.http import JsonResponse
from django.test import SimpleTestCase
from django.urls import path

current_count, extra_field = 2, False

def summary(request):
    result = {"count": current_count}
    if extra_field:
        result["debug"] = True
    return JsonResponse(result)

urlpatterns = [path("summary/", summary)]

class ContractTests(SimpleTestCase):
    def test_status(self):
        self.assertEqual(self.client.get("/summary/").status_code, 200)
    def test_count(self):
        self.assertEqual(self.client.get("/summary/").json()["count"], 2)

def suite_result(count, extra=False):
    # Remove this placeholder, then adapt the retained baseline:
    # Reject an unexpected debug field in the count test.
    raise NotImplementedError
    global current_count, extra_field
    current_count, extra_field = count, extra
    suite = unittest.TestLoader().loadTestsFromTestCase(ContractTests)
    result = unittest.TextTestRunner(stream=io.StringIO()).run(suite)
    return {"passed": result.wasSuccessful(), "tests": result.testsRun}

Reasoning: Reject an unexpected debug field in the count test. Keep both value and status assertions and two executed tests.

Hint 1: Compare the actual response key set with the declared contract.
Hint 2: Require the controlled extra-field variant to fail.

Solution, tests and trade-offs

The acceptance cases require a successful suite for the correct value and a failing suite for the controlled wrong value.

The acceptance cases require a successful suite for the correct value and a failing suite for the controlled wrong value. Practice adds the extra-field failure while retaining the same number of executed tests.

The companion is checking the suite’s ability to reject defects, not simply checking that a test runner exits normally. A failure in this demonstration is an expected observation when a deliberately changed endpoint violates its contract.

Practice solution and acceptance checks

python

from bootstrap_lab import django_setup
django_setup(__name__)

import io
import unittest
from django.http import JsonResponse
from django.test import SimpleTestCase
from django.urls import path

current_count, extra_field = 2, False

def summary(request):
    result = {"count": current_count}
    if extra_field:
        result["debug"] = True
    return JsonResponse(result)

urlpatterns = [path("summary/", summary)]

class ContractTests(SimpleTestCase):
    def test_status(self):
        self.assertEqual(self.client.get("/summary/").status_code, 200)
    def test_count(self):
        body = self.client.get("/summary/").json()
        self.assertEqual(body["count"], 2)
        self.assertEqual(set(body), {"count"})

def suite_result(count, extra=False):
    global current_count, extra_field
    current_count, extra_field = count, extra
    suite = unittest.TestLoader().loadTestsFromTestCase(ContractTests)
    result = unittest.TextTestRunner(stream=io.StringIO()).run(suite)
    return {"passed": result.wasSuccessful(), "tests": result.testsRun}

Reasoning: The acceptance cases require a successful suite for the correct value and a failing suite for the controlled wrong value. Practice adds the extra-field failure while retaining the same number of executed tests.

Interview reasoning and next step

Explain when SimpleTestCase, TestCase and TransactionTestCase fit different dependencies and transaction claims.

Explain when SimpleTestCase, TestCase and TransactionTestCase fit different dependencies and transaction claims. A test with real database behavior should choose an appropriate isolation model instead of silently bypassing database checks.

Browser behavior, concurrent writers and deployment settings require further layers of testing. The capstone next joins authentication, ownership, validation and CSRF so the complete request path is checked alongside its individual components.

Optional stretch — after the core task works: Add a database-dependent test with an explicitly chosen isolation model.

Computer practice: https://applydjinn.com/typorithm/python-frameworks-lab-20261105.zip. Install the pinned dependencies in an isolated environment, run verify.py --track django, then edit the retained exercise baseline and check it with --practice dj-tests. The lab uses owned in-memory databases, local framework clients and tiny CPU tensors. It never reads the working ApplyDjinn database, contacts production feeds or downloads models.

Exercise a real response and demonstrate that the test suite catches a controlled defect rather than checking success status alone.