PyTorch: train-only preprocessing and leakage prevention

pythontorchindependent-practice

Saved on this device

Full lesson · 4 learning steps Open any exercise for focused practice

Step 1 of 4 · intro card

PyTorch: train-only preprocessing and leakage prevention

Concept and contract

Fit normalization on training data and apply it unchanged to validation.

Validation is useful only when its information does not influence the training choices being evaluated.

Leakage can occur before a model is trained: fitting a normalizer on both training and validation values lets held-out data change preprocessing. Splits should respect the task’s time, group or identity boundaries, not merely produce two arbitrary lists with different names.

Mechanism and worked example

The helper computes mean and population standard deviation using only the training tensor, then transforms validation values with those fixed parameters.

The helper computes mean and population standard deviation using only the training tensor, then transforms validation values with those fixed parameters. A constant training feature uses scale one to avoid division by zero.

The returned mean and scale expose which population was fitted. A far-away validation value should not pull the fitted mean toward itself.

Complete worked example

python

from bootstrap_lab import torch_setup
torch = torch_setup()

def normalize_split(training, validation):
    train = torch.tensor(training, dtype=torch.float32)
    valid = torch.tensor(validation, dtype=torch.float32)
    if train.ndim != 1 or train.numel() == 0 or valid.ndim != 1:
        raise ValueError('nonempty one-dimensional training data required')
    mean = train.mean()
    scale = train.std(correction=0)
    if scale.item() == 0:
        scale = torch.tensor(1.0)
    return {'mean': mean.item(), 'scale': scale.item(),
            'validation': ((valid - mean) / scale).tolist()}

Reasoning: The helper computes mean and population standard deviation using only the training tensor, then transforms validation values with those fixed parameters. A constant training feature uses scale one to avoid division by zero.

Counterexamples and common mistakes

The counterexample computes the mean over concatenated training and validation tensors.

The counterexample computes the mean over concatenated training and validation tensors. The code still runs and may make normalized validation values look more centered, but it has consumed held-out information.

Duplicate identities, future timestamps and target-derived features can create other leakage paths. No framework split helper can automatically infer all those domain-specific boundaries for an application.

Counterexample: a runnable framework mistake

python

from bootstrap_lab import torch_setup
torch = torch_setup()

def normalize_split(training, validation):
    train = torch.tensor(training, dtype=torch.float32)
    valid = torch.tensor(validation, dtype=torch.float32)
    if train.ndim != 1 or train.numel() == 0 or valid.ndim != 1:
        raise ValueError('nonempty one-dimensional training data required')
    mean = torch.cat([train, valid]).mean()
    scale = train.std(correction=0)
    if scale.item() == 0:
        scale = torch.tensor(1.0)
    return {'mean': mean.item(), 'scale': scale.item(),
            'validation': ((valid - mean) / scale).tolist()}

Reasoning: The counterexample computes the mean over concatenated training and validation tensors. The code still runs and may make normalized validation values look more centered, but it has consumed held-out information.

Independent practice: one bounded framework change

Also return the normalized training values using the already fitted mean and scale.

Also return the normalized training values using the already fitted mean and scale. Preserve the validation transform and fit population.

This adds one observed field rather than changing the estimator. Training values should have approximately zero mean after centering, while validation values need not. Do not independently refit the normalizer on validation to make both outputs look similar.

Step 1: Return normalized training values without refitting on validation. Step 2: Reuse mean and scale already computed.

Practice starter: solve before reading the solution

python

from bootstrap_lab import torch_setup
torch = torch_setup()

def normalize_split(training, validation):
    # Remove this placeholder, then adapt the retained baseline:
    # Return normalized training values without refitting on validation.
    raise NotImplementedError
    train = torch.tensor(training, dtype=torch.float32)
    valid = torch.tensor(validation, dtype=torch.float32)
    if train.ndim != 1 or train.numel() == 0 or valid.ndim != 1:
        raise ValueError('nonempty one-dimensional training data required')
    mean = train.mean()
    scale = train.std(correction=0)
    if scale.item() == 0:
        scale = torch.tensor(1.0)
    return {'mean': mean.item(), 'scale': scale.item(),
            'validation': ((valid - mean) / scale).tolist()}

Reasoning: Return normalized training values without refitting on validation. Reuse mean and scale already computed.

Hint 1: Keep held-out values out of fitted statistics.
Hint 2: Compare constant-feature outputs too.

Solution, tests and trade-offs

Fixtures compare a simple two-point training set with an outlying validation point, constant training values and invalid empty training data.

Fixtures compare a simple two-point training set with an outlying validation point, constant training values and invalid empty training data. Practice inspects both transformed populations.

Exact fitted statistics plus finite output comparisons expose leakage that a shape-only test would miss. These examples illustrate the boundary, not the statistical adequacy of a real-world train-validation design.

Practice solution and acceptance checks

python

from bootstrap_lab import torch_setup
torch = torch_setup()

def normalize_split(training, validation):
    train = torch.tensor(training, dtype=torch.float32)
    valid = torch.tensor(validation, dtype=torch.float32)
    if train.ndim != 1 or train.numel() == 0 or valid.ndim != 1:
        raise ValueError('nonempty one-dimensional training data required')
    mean = train.mean()
    scale = train.std(correction=0)
    if scale.item() == 0:
        scale = torch.tensor(1.0)
    return {'training': ((train - mean) / scale).tolist(), 'mean': mean.item(), 'scale': scale.item(),
            'validation': ((valid - mean) / scale).tolist()}

Reasoning: Fixtures compare a simple two-point training set with an outlying validation point, constant training values and invalid empty training data. Practice inspects both transformed populations.

Interview reasoning and next step

For temporal job-market data, consider training only on observations available before the evaluated period and grouping related entities consistently.

For temporal job-market data, consider training only on observations available before the evaluated period and grouping related entities consistently. Keep a final test set separate from repeated tuning.

Save preprocessing parameters with model configuration so production inference uses the same transformation. Explain what is learned from data and which population is allowed to influence it before comparing model scores.

Optional stretch — after the core task works: Design a time-based split with repeated-company identities kept under an explicit policy.

Computer practice: https://applydjinn.com/typorithm/python-frameworks-lab-20261105.zip. Install the pinned dependencies in an isolated environment, run verify.py --track torch, then edit the retained exercise baseline and check it with --practice pt-splits. The lab uses owned in-memory databases, local framework clients and tiny CPU tensors. It never reads the working ApplyDjinn database, contacts production feeds or downloads models.

Fit normalization on training data and apply it unchanged to validation.