APPLYDJINN
← Python: reusable libraries and packagingProgress & review ↗

Typorithm · medium

Lazy pipelines and measurable consumption

pythonlibrariespackagingindependent-practice
Saved on this device
Full lesson · 4 learning steps Open any exercise for focused practice
1

Step 1 of 4 · intro card

Lazy pipelines and measurable consumption

0%

Concept and contract

Distinguish the number of returned values from the number of source values actually consumed.

A lazy pipeline postpones work until a consumer requests values. That matters when reading a large stream, performing expensive transformations or preserving an iterator for later use. Returning only two results does not prove that only two inputs were consumed.

The example records every source value visited by the generator and separately records the selected output. Its contract requires a nonnegative exact integer count; True is rejected even though bool is an int subclass. Both the output and the consumption trace are observable parts of this owned experiment.

Mechanism and worked example

Calling transformed constructs a generator without executing its loop.

Calling transformed constructs a generator without executing its loop. Passing that stream to islice and then materializing the slice requests only the chosen number of values. With four inputs and count two, selected contains two doubled values and consumed contains the first two original inputs.

The zero case is especially useful: neither the loop body nor its tracing append should run. Empty sources remain empty even when the requested count is larger. This is a sequential consumption contract, not a promise about parallel execution, wall-clock speed or automatically bounded storage for every downstream consumer.

Complete worked example

python

from itertools import islice

def inspect_example(values, count):
    if type(count) is not int or count < 0:
        raise ValueError('count must be a nonnegative integer')
    seen = []
    def transformed():
        for value in values:
            seen.append(value)
            yield value * 2
    stream = transformed()
    selected = list(islice(stream, count))
    return {'selected': selected, 'consumed': seen}
Reasoning
Calling transformed constructs a generator without executing its loop. Passing that stream to islice and then materializing the slice requests only the chosen number of values. With four inputs and count two, selected contains two doubled values and consumed contains the first two original inputs.
Download the reusable Python and packaging computer lab ↗

Counterexamples and common mistakes

The counterexample first converts the entire generator into a list and only then slices it.

The counterexample first converts the entire generator into a list and only then slices it. Its selected values may look correct, yet consumed includes the complete input. A result-only test would incorrectly accept this eager implementation.

Laziness can also be lost accidentally in debugging, sorting or repeated list conversion. Once an iterator is consumed, another pass does not magically recreate its source. The trace makes that ownership visible here without using network requests or an infinite sequence that could hang a learner's process.

Controlled bug: the executable contract rejects this version

python

from itertools import islice

def inspect_example(values, count):
    if type(count) is not int or count < 0:
        raise ValueError('count must be a nonnegative integer')
    seen = []
    def transformed():
        for value in values:
            seen.append(value)
            yield value * 2
    stream = transformed()
    selected = list(stream)[:count]
    return {'selected': selected, 'consumed': seen}
Reasoning
The counterexample first converts the entire generator into a list and only then slices it. Its selected values may look correct, yet consumed includes the complete input. A result-only test would incorrectly accept this eager implementation.
Download the reusable Python and packaging computer lab ↗

Independent practice

Change the yielded transformation from doubling to tripling while preserving the consumption boundary.

Change the yielded transformation from doubling to tripling while preserving the consumption boundary. Do not change how many source values islice requests. Start with count two and compare both returned arrays before checking count zero and an empty input.

The first hint is to edit the yield expression rather than moving materialization earlier. The second is to inspect consumed when the requested count is zero. It must remain empty regardless of the arithmetic. Keep validation before constructing the stream so negative counts and boolean counts fail with the documented domain error.

Step 1: Keep partial consumption bounded and change the lazy transformation to tripling. Step 2: Run the accepted baseline, edit the exercise, then check its independent solution cases; preserve caller inputs and the documented exception behavior.

Practice starter: solve before reading the solution

python

from itertools import islice

def inspect_example(values, count):
    if type(count) is not int or count < 0:
        raise ValueError('count must be a nonnegative integer')
    seen = []
    def transformed():
        for value in values:
            seen.append(value)
            yield value * 2
    stream = transformed()
    selected = list(islice(stream, count))
    return {'selected': selected, 'consumed': seen}

# Task: Keep partial consumption bounded and change the lazy transformation to tripling.
# Practice contract: Keep partial consumption bounded and change the lazy transformation to tripling.
Reasoning
Keep partial consumption bounded and change the lazy transformation to tripling.
Download the reusable Python and packaging computer lab ↗
  • Hint 1: Change the yielded transformation from doubling to tripling while preserving the consumption boundary. Do not change how many source values islice requests. Start with count two and compare both returned arrays before checking count zero and an empty input.
  • Hint 2: The first hint is to edit the yield expression rather than moving materialization earlier. The second is to inspect consumed when the requested count is zero. It must remain empty regardless of the arithmetic. Keep validation before constructing the stream so negative counts and boolean counts fail with the documented domain error.

Solution, tests and trade-offs

The solution changes only the multiplication inside the generator.

The solution changes only the multiplication inside the generator. Acceptance checks the new selected values and the unchanged consumption trace, while separate invalid-input cases protect the count boundary. This verifies both the transformation and its evaluation strategy.

A small finite trace is easier to reason about than a timing benchmark. For a production pipeline, document whether iteration is repeatable and which component owns closing the source. A generator can still retain referenced objects between yields, and a downstream list can still grow without bound if it consumes the entire stream.

Practice solution and acceptance checks

python

from itertools import islice

def inspect_example(values, count):
    if type(count) is not int or count < 0:
        raise ValueError('count must be a nonnegative integer')
    seen = []
    def transformed():
        for value in values:
            seen.append(value)
            yield value * 3
    stream = transformed()
    selected = list(islice(stream, count))
    return {'selected': selected, 'consumed': seen}
Reasoning
The solution changes only the multiplication inside the generator. Acceptance checks the new selected values and the unchanged consumption trace, while separate invalid-input cases protect the count boundary. This verifies both the transformation and its evaluation strategy.
Download the reusable Python and packaging computer lab ↗

Interview reasoning and next step

Explain how selected can be correct while consumed is wrong.

Explain how selected can be correct while consumed is wrong. Then compare a reusable iterable with a one-shot iterator and identify which object in this example is exhausted after traversal. Avoid treating generator syntax as an automatic performance guarantee.

For a stretch, compose a filtering generator before the slice and record how many raw values are needed to produce two accepted results. State the predicate and expected trace first. A bounded output count may require reading more raw values, so revise the contract deliberately rather than assuming one input always corresponds to one output.

Optional stretch — after the core task works: For a stretch, compose a filtering generator before the slice and record how many raw values are needed to produce two accepted results. State the predicate and expected trace first. A bounded output count may require reading more raw values, so revise the contract deliberately rather than assuming one input always corresponds to one output.

Computer practice: https://applydjinn.com/typorithm/python-engineering-lab-20261107.zip. Use an isolated Python 3.13 environment, install requirements.txt, then run verify.py. Edit exercises/pye-lazy.py and run --practice pye-lazy. Fixture files are created in fresh temporary directories for every case. Packaging checks build real local artifacts, install the exact wheel without an index and run outside the source tree. No package is published to PyPI. The app includes static examples, prepared results and questions; it does not run Python or provide an editor.

Distinguish the number of returned values from the number of source values actually consumed.