Typorithm · medium

Outer joins and honest aggregate populations

pythonsqlalchemyalembicdatabasesindependent-practice
Saved on this device
Full lesson · 4 learning steps Open any exercise for focused practice
1

Step 1 of 4 · intro card

Outer joins and honest aggregate populations

0%

Concept and contract

Count real child identities while retaining parents that have no matching child rows.

An aggregate is meaningful only when its population and denominator are clear. A left outer join retains a parent with no children by producing a result row whose child columns are NULL. That placeholder is not a real child record, even though it is still one SQL result row.

The fixture reports each employer and the number of actual roles that reference it. Employers with zero roles must remain in the worked report. This is a relational counting question, not a request to fetch each collection and count its Python objects one parent at a time.

Mechanism and worked example

The statement selects Employer.id and count(Role.id), outer-joins the relationship, groups by the selected parent identity and orders by that identity.

The statement selects Employer.id and count(Role.id), outer-joins the relationship, groups by the selected parent identity and orders by that identity. COUNT over the child primary key ignores the NULL placeholder. For counts [2, 0, 1], the correct report is [[1, 2], [2, 0], [3, 1]].

The selected nonaggregate expression is explicitly part of GROUP BY instead of relying on SQLite accepting an under-specified grouped SELECT. The returned rows are ordinary data. Their stable order is checked independently of the group computation, so an accidental row-order assumption cannot masquerade as a correct aggregate.

Complete worked example

python

from sqlalchemy import create_engine, event, ForeignKey, select
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship, Session

class Base(DeclarativeBase):
    pass

class Employer(Base):
    __tablename__ = "employers"
    id: Mapped[int] = mapped_column(primary_key=True)
    roles: Mapped[list["Role"]] = relationship(back_populates="employer")

class Role(Base):
    __tablename__ = "roles"
    id: Mapped[int] = mapped_column(primary_key=True)
    employer_id: Mapped[int] = mapped_column(ForeignKey("employers.id", ondelete="RESTRICT"))
    employer: Mapped[Employer] = relationship(back_populates="roles")

from sqlalchemy import func

def inspect_example(counts):
    engine = create_engine("sqlite+pysqlite://", connect_args={"autocommit": False})
    try:
        Base.metadata.create_all(engine)
        with Session(engine) as session, session.begin():
            for i, count in enumerate(counts, 1):
                session.add(Employer(id=i, roles=[Role() for _ in range(count)]))
        with Session(engine) as session:
            statement = (select(Employer.id, func.count(Role.id))
                         .outerjoin(Employer.roles).group_by(Employer.id).order_by(Employer.id))
            return [list(row) for row in session.execute(statement)]
    finally:
        engine.dispose()
Reasoning
The statement selects Employer.id and count(Role.id), outer-joins the relationship, groups by the selected parent identity and orders by that identity. COUNT over the child primary key ignores the NULL placeholder. For counts [2, 0, 1], the correct report is [[1, 2], [2, 0], [3, 1]]. The selected nonaggregate expression is explicitly part of GROUP BY instead of relying on SQLite accepting an under-specified grouped SELECT. The returned rows are ordinary data. Their stable order is checked independently of the group computation, so an accidental row-order assumption cannot masquerade as a correct aggregate.
Download the SQLAlchemy and Alembic computer lab ↗

Focused decision: context is in the complete example

python

statement = (select(Employer.id, func.count(Role.id))
             .outerjoin(Employer.roles).group_by(Employer.id))
Database decision
The child primary key distinguishes a real child from the NULL placeholder. HAVING changes which completed groups are reported without changing their counts.
Download the SQLAlchemy and Alembic computer lab ↗

Counterexamples and common mistakes

The controlled mistake counts every joined row rather than non-null Role.id values.

The controlled mistake counts every joined row rather than non-null Role.id values. An empty employer is then reported with count one, because its outer-join placeholder is included. The query is valid and nonempty, yet it misstates a central business metric.

A WHERE condition on child columns can also remove NULL-extended rows and accidentally undo the intended outer population. Decide whether a condition belongs in the join, before grouping or after grouping. Do not repair a wrong population by subtracting one everywhere: parents with real children would then become wrong in a different way.

Controlled bug: observe the rejected database contract

python

from sqlalchemy import create_engine, event, ForeignKey, select
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship, Session

class Base(DeclarativeBase):
    pass

class Employer(Base):
    __tablename__ = "employers"
    id: Mapped[int] = mapped_column(primary_key=True)
    roles: Mapped[list["Role"]] = relationship(back_populates="employer")

class Role(Base):
    __tablename__ = "roles"
    id: Mapped[int] = mapped_column(primary_key=True)
    employer_id: Mapped[int] = mapped_column(ForeignKey("employers.id", ondelete="RESTRICT"))
    employer: Mapped[Employer] = relationship(back_populates="roles")

from sqlalchemy import func

def inspect_example(counts):
    engine = create_engine("sqlite+pysqlite://", connect_args={"autocommit": False})
    try:
        Base.metadata.create_all(engine)
        with Session(engine) as session, session.begin():
            for i, count in enumerate(counts, 1):
                session.add(Employer(id=i, roles=[Role() for _ in range(count)]))
        with Session(engine) as session:
            statement = (select(Employer.id, func.count())
                         .outerjoin(Employer.roles).group_by(Employer.id).order_by(Employer.id))
            return [list(row) for row in session.execute(statement)]
    finally:
        engine.dispose()
Reasoning
The controlled mistake counts every joined row rather than non-null Role.id values. An empty employer is then reported with count one, because its outer-join placeholder is included. The query is valid and nonempty, yet it misstates a central business metric. A WHERE condition on child columns can also remove NULL-extended rows and accidentally undo the intended outer population. Decide whether a condition belongs in the join, before grouping or after grouping. Do not repair a wrong population by subtracting one everywhere: parents with real children would then become wrong in a different way.
Download the SQLAlchemy and Alembic computer lab ↗

Independent practice

First type the outer join and the child-ID count.

First type the outer join and the child-ID count. Predict the zero-role case before reading the prepared result. For independent practice, keep only employers whose actual role count is greater than zero, using HAVING after the grouping stage. Preserve the exact counts and parent ordering of those that remain.

Use a population containing both empty and nonempty employers. A dataset with no empty employers would not establish whether the new filter is doing its intended job. Keep the all-empty and no-employer cases too, because an empty result is a legitimate report rather than a reason to invent a fallback row.

Step 1: Use HAVING to keep only employers whose actual role count is greater than zero. Step 2: Keep the child-ID count, outer join and parent ordering.

Practice starter: solve before reading the solution

python

from sqlalchemy import create_engine, event, ForeignKey, select
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship, Session

class Base(DeclarativeBase):
    pass

class Employer(Base):
    __tablename__ = "employers"
    id: Mapped[int] = mapped_column(primary_key=True)
    roles: Mapped[list["Role"]] = relationship(back_populates="employer")

class Role(Base):
    __tablename__ = "roles"
    id: Mapped[int] = mapped_column(primary_key=True)
    employer_id: Mapped[int] = mapped_column(ForeignKey("employers.id", ondelete="RESTRICT"))
    employer: Mapped[Employer] = relationship(back_populates="roles")

from sqlalchemy import func

def inspect_example(counts):
    engine = create_engine("sqlite+pysqlite://", connect_args={"autocommit": False})
    try:
        Base.metadata.create_all(engine)
        with Session(engine) as session, session.begin():
            for i, count in enumerate(counts, 1):
                session.add(Employer(id=i, roles=[Role() for _ in range(count)]))
        with Session(engine) as session:
            statement = (select(Employer.id, func.count(Role.id))
                         .outerjoin(Employer.roles).group_by(Employer.id).order_by(Employer.id))
            return [list(row) for row in session.execute(statement)]
    finally:
        engine.dispose()

# Task: Use HAVING to keep only employers whose actual role count is greater than zero.
Reasoning
Use HAVING to keep only employers whose actual role count is greater than zero.
Download the SQLAlchemy and Alembic computer lab ↗
  • Hint 1: Add a grouped-condition clause after group_by.
  • Hint 2: Use a mixed fixture and an all-empty fixture.

Solution, tests and trade-offs

The solution adds HAVING count(Role.id) > 0 to the grouped statement.

The solution adds HAVING count(Role.id) > 0 to the grouped statement. It returns [[1, 2], [3, 1]] for the mixed fixture and an empty list for an employer population containing only zero roles. The acceptance cases compare complete rows, so retaining a zero group or removing a real group is observable.

SQL aggregates and ORM relationship loading solve different problems. If the output needs only counts, an aggregate can avoid materializing all children. If it needs child detail, loaders may be appropriate. Choosing between them starts from the report contract, not from a preference for whichever API happens to be shorter.

Practice solution and acceptance checks

python

from sqlalchemy import create_engine, event, ForeignKey, select
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship, Session

class Base(DeclarativeBase):
    pass

class Employer(Base):
    __tablename__ = "employers"
    id: Mapped[int] = mapped_column(primary_key=True)
    roles: Mapped[list["Role"]] = relationship(back_populates="employer")

class Role(Base):
    __tablename__ = "roles"
    id: Mapped[int] = mapped_column(primary_key=True)
    employer_id: Mapped[int] = mapped_column(ForeignKey("employers.id", ondelete="RESTRICT"))
    employer: Mapped[Employer] = relationship(back_populates="roles")

from sqlalchemy import func

def inspect_example(counts):
    engine = create_engine("sqlite+pysqlite://", connect_args={"autocommit": False})
    try:
        Base.metadata.create_all(engine)
        with Session(engine) as session, session.begin():
            for i, count in enumerate(counts, 1):
                session.add(Employer(id=i, roles=[Role() for _ in range(count)]))
        with Session(engine) as session:
            statement = (select(Employer.id, func.count(Role.id))
                         .outerjoin(Employer.roles).group_by(Employer.id).having(func.count(Role.id) > 0).order_by(Employer.id))
            return [list(row) for row in session.execute(statement)]
    finally:
        engine.dispose()
Reasoning
The solution adds HAVING count(Role.id) > 0 to the grouped statement. It returns [[1, 2], [3, 1]] for the mixed fixture and an empty list for an employer population containing only zero roles. The acceptance cases compare complete rows, so retaining a zero group or removing a real group is observable. SQL aggregates and ORM relationship loading solve different problems. If the output needs only counts, an aggregate can avoid materializing all children. If it needs child detail, loaders may be appropriate. Choosing between them starts from the report contract, not from a preference for whichever API happens to be shorter.
Download the SQLAlchemy and Alembic computer lab ↗

Interview reasoning and next step

Explain why COUNT(*) and COUNT(child_primary_key) differ after a left join, and identify the placeholder row that creates the difference.

Explain why COUNT(*) and COUNT(child_primary_key) differ after a left join, and identify the placeholder row that creates the difference. Then explain what WHERE and HAVING filter at their respective stages. State the parent population before giving a numerical result.

In real reporting, also declare deduplication and join cardinality: another one-to-many join can multiply child rows and inflate a count. A distinct count may be appropriate for a specified identity, but it is not a universal cure for a poorly defined join. Next, observe how set-based changes interact with already loaded ORM state.

Optional stretch — after the core task works: Join a second child table in an owned example and predict how cardinality can multiply counts.

Computer practice: https://applydjinn.com/typorithm/sqlalchemy-lab-20261107.zip. Install the pinned requirements in an isolated Python 3.13 environment and run verify.py. Edit the working exercise baseline, then run --practice sa-aggregates. The lab uses disposable SQLite databases and actual SQLAlchemy/Alembic calls; it does not connect to application accounts, production databases or paid services. The app includes prepared results and explained questions without an editor or Python runtime.

Count real child identities while retaining parents that have no matching child rows.