PyTorch: datasets, loaders and complete batches

pythontorchindependent-practice

Saved on this device

Full lesson · 4 learning steps Open any exercise for focused practice

Step 1 of 4 · intro card

PyTorch: datasets, loaders and complete batches

Concept and contract

Preserve all examples and distinguish reproducible order from shuffled training.

A Dataset defines how examples are accessed, while DataLoader groups them into batches and optionally controls order and worker processes. Those roles are separate from the model.

For small owned fixtures, a tensor-backed dataset and zero worker processes keep behavior clear and portable. A batch policy should state whether an incomplete final group is retained or dropped.

Mechanism and worked example

The helper wraps a one-dimensional value tensor in TensorDataset and uses a non-shuffled DataLoader.

The helper wraps a one-dimensional value tensor in TensorDataset and uses a non-shuffled DataLoader. Each returned batch is selected from the loader’s tuple structure.

drop_last=False preserves the final partial batch, so five examples at batch size two produce groups of two, two and one. No external files, training dataset download or multiprocessing setup is required.

Complete worked example

python

from bootstrap_lab import torch_setup
torch = torch_setup()

from torch.utils.data import TensorDataset, DataLoader

def batches(values, batch_size):
    if type(batch_size) is not int or batch_size < 1:
        raise ValueError('positive batch size required')
    dataset = TensorDataset(torch.tensor(values, dtype=torch.float32))
    loader = DataLoader(dataset, batch_size=batch_size, shuffle=False,
                        num_workers=0, drop_last=False)
    return [batch[0].tolist() for batch in loader]

Reasoning: The helper wraps a one-dimensional value tensor in TensorDataset and uses a non-shuffled DataLoader. Each returned batch is selected from the loader’s tuple structure.

Counterexamples and common mistakes

The counterexample enables drop_last, silently removing the final example when the length is not divisible by batch size..

The counterexample enables drop_last, silently removing the final example when the length is not divisible by batch size.

That may be intentional for some training designs, but it violates this collection-preservation contract and is inappropriate for ordinary evaluation. Also remember that dataset length, number of batches and number of examples are different quantities.

Counterexample: a runnable framework mistake

python

from bootstrap_lab import torch_setup
torch = torch_setup()

from torch.utils.data import TensorDataset, DataLoader

def batches(values, batch_size):
    if type(batch_size) is not int or batch_size < 1:
        raise ValueError('positive batch size required')
    dataset = TensorDataset(torch.tensor(values, dtype=torch.float32))
    loader = DataLoader(dataset, batch_size=batch_size, shuffle=False,
                        num_workers=0, drop_last=True)
    return [batch[0].tolist() for batch in loader]

Reasoning: The counterexample enables drop_last, silently removing the final example when the length is not divisible by batch size.

Independent practice: one bounded framework change

Add an optional reverse flag that reverses the owned tensor values before constructing the dataset.

Add an optional reverse flag that reverses the owned tensor values before constructing the dataset. Preserve batch boundaries, final partial groups and the caller’s input list.

This simple ordering exercise prepares for samplers without requiring random shuffling. A later extension can use a seeded Generator, but reproducibility should be explained rather than inferred from one lucky observed order.

Step 1: Add a reverse mode before building the dataset without dropping examples. Step 2: Construct an owned tensor first.

Practice starter: solve before reading the solution

python

from bootstrap_lab import torch_setup
torch = torch_setup()

from torch.utils.data import TensorDataset, DataLoader

def batches(values, batch_size):
    # Remove this placeholder, then adapt the retained baseline:
    # Add a reverse mode before building the dataset without dropping examples.
    raise NotImplementedError
    if type(batch_size) is not int or batch_size < 1:
        raise ValueError('positive batch size required')
    dataset = TensorDataset(torch.tensor(values, dtype=torch.float32))
    loader = DataLoader(dataset, batch_size=batch_size, shuffle=False,
                        num_workers=0, drop_last=False)
    return [batch[0].tolist() for batch in loader]

Reasoning: Add a reverse mode before building the dataset without dropping examples. Construct an owned tensor first.

Hint 1: Flip its example axis only when requested.
Hint 2: Keep num_workers zero and the partial final batch.

Solution, tests and trade-offs

Cases check five values in two-item groups, an empty dataset and a batch larger than the collection.

Cases check five values in two-item groups, an empty dataset and a batch larger than the collection. Practice confirms reversed ordering and an unchanged normal mode.

Exact membership matters: counting batches alone would not catch a dropped or duplicated example. The helper validates the batch size before constructing the loader and never mutates the supplied Python sequence.

Practice solution and acceptance checks

python

from bootstrap_lab import torch_setup
torch = torch_setup()

from torch.utils.data import TensorDataset, DataLoader

def batches(values, batch_size, reverse=False):
    if type(batch_size) is not int or batch_size < 1:
        raise ValueError('positive batch size required')
    tensor = torch.tensor(values, dtype=torch.float32)
    if reverse:
        tensor = tensor.flip(0)
    dataset = TensorDataset(tensor)
    loader = DataLoader(dataset, batch_size=batch_size, shuffle=False,
                        num_workers=0, drop_last=False)
    return [batch[0].tolist() for batch in loader]

Reasoning: Cases check five values in two-item groups, an empty dataset and a batch larger than the collection. Practice confirms reversed ordering and an unchanged normal mode.

Interview reasoning and next step

For real workloads choose worker count, pinning and prefetch settings according to measured hardware and data costs.

For real workloads choose worker count, pinning and prefetch settings according to measured hardware and data costs. More workers are not automatically faster, especially for tiny in-memory data or constrained hosts.

Define train, validation and test sampling separately, and inspect per-example identities when debugging missing records or leakage. Keep evaluation coverage complete unless a documented policy says otherwise.

Optional stretch — after the core task works: Introduce a seeded sampler and verify complete membership across an epoch.

Computer practice: https://applydjinn.com/typorithm/python-frameworks-lab-20261105.zip. Install the pinned dependencies in an isolated environment, run verify.py --track torch, then edit the retained exercise baseline and check it with --practice pt-data. The lab uses owned in-memory databases, local framework clients and tiny CPU tensors. It never reads the working ApplyDjinn database, contacts production feeds or downloads models.

Preserve all examples and distinguish reproducible order from shuffled training.