Typorithm · medium

Loading relationships without an N+1 surprise

pythonsqlalchemyalembicdatabasesindependent-practice
Saved on this device
Full lesson · 4 learning steps Open any exercise for focused practice
1

Step 1 of 4 · intro card

Loading relationships without an N+1 surprise

0%

Concept and contract

Preserve the parent population while measuring lazy, select-in and joined collection loading.

A correct object graph can still be expensive to traverse. Lazy loading may issue a SELECT when an unloaded relationship is first accessed; repeating that access for many parents can turn one list operation into a sequence of additional database round trips. The N+1 label describes this access pattern rather than a particular exception.

The fixture reuses employers with small role collections, including an empty collection. A SELECT counter is attached only after seeding so setup statements do not pollute the measurement. The contract checks both returned parent/count pairs and the statements needed to materialize them.

Mechanism and worked example

The worked statement applies selectinload to Employer.roles and raiseload to other unplanned relationships.

The worked statement applies selectinload to Employer.roles and raiseload to other unplanned relationships. For the small nonempty fixture, one SELECT loads employers and a second loads their role collections. With no employers, only the first SELECT runs. Counts [2, 0, 1] remain [[1, 2], [2, 0], [3, 1]].

selectinload does not mean every possible production graph always needs exactly two queries: larger key sets may be batched and nested paths add work. Here the explicit query count belongs to the bounded fixture. It is useful evidence because it measures real execution instead of inferring efficiency from the source looking short.

Complete worked example

python

from sqlalchemy import create_engine, event, ForeignKey, select
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship, Session

class Base(DeclarativeBase):
    pass

class Employer(Base):
    __tablename__ = "employers"
    id: Mapped[int] = mapped_column(primary_key=True)
    roles: Mapped[list["Role"]] = relationship(back_populates="employer")

class Role(Base):
    __tablename__ = "roles"
    id: Mapped[int] = mapped_column(primary_key=True)
    employer_id: Mapped[int] = mapped_column(ForeignKey("employers.id", ondelete="RESTRICT"))
    employer: Mapped[Employer] = relationship(back_populates="roles")

from sqlalchemy.orm import selectinload, raiseload

def inspect_example(counts):
    engine = create_engine("sqlite+pysqlite://", connect_args={"autocommit": False})
    try:
        Base.metadata.create_all(engine)
        with Session(engine) as session, session.begin():
            for i, count in enumerate(counts, 1):
                session.add(Employer(id=i, roles=[Role() for _ in range(count)]))
        selects = [0]
        @event.listens_for(engine, "before_cursor_execute")
        def count_selects(connection, cursor, statement, parameters, context, many):
            if statement.lstrip().upper().startswith("SELECT"):
                selects[0] += 1
        with Session(engine) as session:
            statement = (select(Employer).order_by(Employer.id)
                         .options(selectinload(Employer.roles), raiseload("*")))
            employers = session.scalars(statement).all()
            result = [[employer.id, len(employer.roles)] for employer in employers]
        return {"rows": result, "selects": selects[0]}
    finally:
        engine.dispose()
Reasoning
The worked statement applies selectinload to Employer.roles and raiseload to other unplanned relationships. For the small nonempty fixture, one SELECT loads employers and a second loads their role collections. With no employers, only the first SELECT runs. Counts [2, 0, 1] remain [[1, 2], [2, 0], [3, 1]]. selectinload does not mean every possible production graph always needs exactly two queries: larger key sets may be batched and nested paths add work. Here the explicit query count belongs to the bounded fixture. It is useful evidence because it measures real execution instead of inferring efficiency from the source looking short.
Download the SQLAlchemy and Alembic computer lab ↗

Focused decision: context is in the complete example

python

statement = select(Employer).options(selectinload(Employer.roles), raiseload("*"))
employers = session.scalars(statement).all()
Database decision
The counter starts after fixture setup. selectinload bounds collection loading for this small population; joinedload needs explicit unique consumption and preserves the same parents.
Download the SQLAlchemy and Alembic computer lab ↗

Counterexamples and common mistakes

The controlled bug removes the eager loading options.

The controlled bug removes the eager loading options. Reading each employer.roles now produces lazy SELECT statements. The returned counts can stay correct while the query-budget assertion fails, demonstrating that value tests alone do not detect every performance regression.

Do not replace a collection loader with an ordinary filtering join and assume its parent population is unchanged. Joined eager loading also multiplies SQL rows for collections, so modern ORM consumption requires an explicit unique call. raiseload can expose unwanted attribute loading; it is not a universal promise that every flush or internal ORM operation emits no SQL.

Controlled bug: observe the rejected database contract

python

from sqlalchemy import create_engine, event, ForeignKey, select
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship, Session

class Base(DeclarativeBase):
    pass

class Employer(Base):
    __tablename__ = "employers"
    id: Mapped[int] = mapped_column(primary_key=True)
    roles: Mapped[list["Role"]] = relationship(back_populates="employer")

class Role(Base):
    __tablename__ = "roles"
    id: Mapped[int] = mapped_column(primary_key=True)
    employer_id: Mapped[int] = mapped_column(ForeignKey("employers.id", ondelete="RESTRICT"))
    employer: Mapped[Employer] = relationship(back_populates="roles")

from sqlalchemy.orm import selectinload, raiseload

def inspect_example(counts):
    engine = create_engine("sqlite+pysqlite://", connect_args={"autocommit": False})
    try:
        Base.metadata.create_all(engine)
        with Session(engine) as session, session.begin():
            for i, count in enumerate(counts, 1):
                session.add(Employer(id=i, roles=[Role() for _ in range(count)]))
        selects = [0]
        @event.listens_for(engine, "before_cursor_execute")
        def count_selects(connection, cursor, statement, parameters, context, many):
            if statement.lstrip().upper().startswith("SELECT"):
                selects[0] += 1
        with Session(engine) as session:
            statement = (select(Employer).order_by(Employer.id)
                         .options())
            employers = session.scalars(statement).all()
            result = [[employer.id, len(employer.roles)] for employer in employers]
        return {"rows": result, "selects": selects[0]}
    finally:
        engine.dispose()
Reasoning
The controlled bug removes the eager loading options. Reading each employer.roles now produces lazy SELECT statements. The returned counts can stay correct while the query-budget assertion fails, demonstrating that value tests alone do not detect every performance regression. Do not replace a collection loader with an ordinary filtering join and assume its parent population is unchanged. Joined eager loading also multiplies SQL rows for collections, so modern ORM consumption requires an explicit unique call. raiseload can expose unwanted attribute loading; it is not a universal promise that every flush or internal ORM operation emits no SQL.
Download the SQLAlchemy and Alembic computer lab ↗

Independent practice

Type the explicit loading option and the scalar consumption first, then predict the counter for a nonempty and empty parent list.

Type the explicit loading option and the scalar consumption first, then predict the counter for a nonempty and empty parent list. For independent practice switch to joinedload for the role collection and explicitly uniquify the ORM result. Preserve all employers, including those with zero roles.

The new fixture contract requires one SELECT while retaining the same returned counts. Make the loader change and the result-consumption change together; omitting unique should not be hidden with an arbitrary Python set after execution. Use the supplied complete mapping rather than inventing a different relationship or reducing the test population.

Step 1: Use joinedload for the collection and explicitly uniquify ORM results; keep one SELECT. Step 2: Preserve empty employers and the exact parent/count pairs.

Practice starter: solve before reading the solution

python

from sqlalchemy import create_engine, event, ForeignKey, select
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship, Session

class Base(DeclarativeBase):
    pass

class Employer(Base):
    __tablename__ = "employers"
    id: Mapped[int] = mapped_column(primary_key=True)
    roles: Mapped[list["Role"]] = relationship(back_populates="employer")

class Role(Base):
    __tablename__ = "roles"
    id: Mapped[int] = mapped_column(primary_key=True)
    employer_id: Mapped[int] = mapped_column(ForeignKey("employers.id", ondelete="RESTRICT"))
    employer: Mapped[Employer] = relationship(back_populates="roles")

from sqlalchemy.orm import selectinload, raiseload

def inspect_example(counts):
    engine = create_engine("sqlite+pysqlite://", connect_args={"autocommit": False})
    try:
        Base.metadata.create_all(engine)
        with Session(engine) as session, session.begin():
            for i, count in enumerate(counts, 1):
                session.add(Employer(id=i, roles=[Role() for _ in range(count)]))
        selects = [0]
        @event.listens_for(engine, "before_cursor_execute")
        def count_selects(connection, cursor, statement, parameters, context, many):
            if statement.lstrip().upper().startswith("SELECT"):
                selects[0] += 1
        with Session(engine) as session:
            statement = (select(Employer).order_by(Employer.id)
                         .options(selectinload(Employer.roles), raiseload("*")))
            employers = session.scalars(statement).all()
            result = [[employer.id, len(employer.roles)] for employer in employers]
        return {"rows": result, "selects": selects[0]}
    finally:
        engine.dispose()

# Task: Use joinedload for the collection and explicitly uniquify ORM results; keep one SELECT.
Reasoning
Use joinedload for the collection and explicitly uniquify ORM results; keep one SELECT.
Download the SQLAlchemy and Alembic computer lab ↗
  • Hint 1: Replace selectinload with joinedload in the explicit option.
  • Hint 2: Add unique before consuming the complete scalar result.

Solution, tests and trade-offs

The solution uses joinedload and session.scalars(statement).unique().all().

The solution uses joinedload and session.scalars(statement).unique().all(). The one-query result still contains each employer once, including an employer with an empty collection. The cases compare parent identities, counts and SELECT totals, so a change that merely suppresses a counter or drops empty parents will not satisfy the intended result.

A single joined query is not automatically the fastest production choice. Row multiplication, transferred data, latency, limits and relationship depth all matter. The exercise teaches how to make a loading choice explicit and measurable; it does not declare one loader universally superior for every data distribution.

Practice solution and acceptance checks

python

from sqlalchemy import create_engine, event, ForeignKey, select
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship, Session

class Base(DeclarativeBase):
    pass

class Employer(Base):
    __tablename__ = "employers"
    id: Mapped[int] = mapped_column(primary_key=True)
    roles: Mapped[list["Role"]] = relationship(back_populates="employer")

class Role(Base):
    __tablename__ = "roles"
    id: Mapped[int] = mapped_column(primary_key=True)
    employer_id: Mapped[int] = mapped_column(ForeignKey("employers.id", ondelete="RESTRICT"))
    employer: Mapped[Employer] = relationship(back_populates="roles")

from sqlalchemy.orm import joinedload, raiseload

def inspect_example(counts):
    engine = create_engine("sqlite+pysqlite://", connect_args={"autocommit": False})
    try:
        Base.metadata.create_all(engine)
        with Session(engine) as session, session.begin():
            for i, count in enumerate(counts, 1):
                session.add(Employer(id=i, roles=[Role() for _ in range(count)]))
        selects = [0]
        @event.listens_for(engine, "before_cursor_execute")
        def count_selects(connection, cursor, statement, parameters, context, many):
            if statement.lstrip().upper().startswith("SELECT"):
                selects[0] += 1
        with Session(engine) as session:
            statement = (select(Employer).order_by(Employer.id)
                         .options(joinedload(Employer.roles), raiseload("*")))
            employers = session.scalars(statement).unique().all()
            result = [[employer.id, len(employer.roles)] for employer in employers]
        return {"rows": result, "selects": selects[0]}
    finally:
        engine.dispose()
Reasoning
The solution uses joinedload and session.scalars(statement).unique().all(). The one-query result still contains each employer once, including an employer with an empty collection. The cases compare parent identities, counts and SELECT totals, so a change that merely suppresses a counter or drops empty parents will not satisfy the intended result. A single joined query is not automatically the fastest production choice. Row multiplication, transferred data, latency, limits and relationship depth all matter. The exercise teaches how to make a loading choice explicit and measurable; it does not declare one loader universally superior for every data distribution.
Download the SQLAlchemy and Alembic computer lab ↗

Interview reasoning and next step

Explain which operation causes lazy IO, why select-in loading adds a second query in this fixture, and why a joined collection requires unique consumption.

Explain which operation causes lazy IO, why select-in loading adds a second query in this fixture, and why a joined collection requires unique consumption. Distinguish SQL result rows from distinct ORM parent identities, using an employer with two roles as your concrete example.

When reviewing a service, measure the full access path through serialization rather than stopping at the first SELECT call. An unloaded relationship accessed while building a response can add queries later. Next, keep the parent population visible while computing counts with an outer join and a deliberately chosen aggregate expression.

Optional stretch — after the core task works: Compare transferred row counts for one parent with many children under both strategies.

Computer practice: https://applydjinn.com/typorithm/sqlalchemy-lab-20261107.zip. Install the pinned requirements in an isolated Python 3.13 environment and run verify.py. Edit the working exercise baseline, then run --practice sa-loading. The lab uses disposable SQLite databases and actual SQLAlchemy/Alembic calls; it does not connect to application accounts, production databases or paid services. The app includes prepared results and explained questions without an editor or Python runtime.

Preserve the parent population while measuring lazy, select-in and joined collection loading.