PyTorch: eval mode and gradient-disabled inference

pythontorchindependent-practice

Saved on this device

Full lesson · 4 learning steps Open any exercise for focused practice

Step 1 of 4 · intro card

PyTorch: eval mode and gradient-disabled inference

Concept and contract

Use both model behavior mode and an appropriate inference context.

Evaluation mode and gradient mode control different mechanisms. model.eval changes behavior of modules such as Dropout and BatchNorm, while no_grad or inference_mode disables gradient recording in its scope.

One does not automatically imply the other. A prediction path should choose both deliberately, and a training loop must restore training behavior before its next intended update.

Mechanism and worked example

The helper uses a Dropout layer as an observable mode example, sets it to eval and performs the forward operation inside inference_mode..

The helper uses a Dropout layer as an observable mode example, sets it to eval and performs the forward operation inside inference_mode.

Because evaluation Dropout returns its input unchanged, the explicit input requires no gradients here; the fixture then reports output values, module mode and gradient state. The numerical behavior is deterministic and uses no random training dropout.

Complete worked example

python

from bootstrap_lab import torch_setup
torch = torch_setup()

from torch import nn

def evaluate(values):
    layer = nn.Dropout(p=0.5)
    layer.eval()
    x = torch.tensor(values, dtype=torch.float32, requires_grad=False)
    with torch.inference_mode():
        output = layer(x)
    return {'values': output.tolist(), 'training': layer.training,
            'requires_grad': output.requires_grad}

Reasoning: The helper uses a Dropout layer as an observable mode example, sets it to eval and performs the forward operation inside inference_mode.

Counterexamples and common mistakes

The counterexample leaves the layer in training mode, so its reported mode is wrong and values may be randomly dropped or scaled.

The counterexample leaves the layer in training mode, so its reported mode is wrong and values may be randomly dropped or scaled. Merely wrapping a forward pass in inference_mode does not change that module behavior.

Conversely, eval alone does not prevent graph construction for differentiable operations. Avoid using one flag as a vague shorthand for all inference requirements.

Counterexample: a runnable framework mistake

python

from bootstrap_lab import torch_setup
torch = torch_setup()

from torch import nn

def evaluate(values):
    layer = nn.Dropout(p=0.5)
    layer.train()
    x = torch.tensor(values, dtype=torch.float32, requires_grad=False)
    with torch.inference_mode():
        output = layer(x)
    return {'values': output.tolist(), 'training': layer.training,
            'requires_grad': output.requires_grad}

Reasoning: The counterexample leaves the layer in training mode, so its reported mode is wrong and values may be randomly dropped or scaled. Merely wrapping a forward pass in inference_mode does not change that module behavior.

Independent practice: one bounded framework change

Return a doubled output by applying multiplication inside the existing inference_mode block.

Return a doubled output by applying multiplication inside the existing inference_mode block. Preserve eval mode and confirm the new result does not require gradients.

This adds a differentiable operation whose graph suppression is directly observable, while retaining the simple deterministic values. Do not move the multiplication outside the context or re-enable gradients to make a label look correct.

Step 1: Double the output inside the inference-mode context. Step 2: Keep eval as the behavior mode.

Practice starter: solve before reading the solution

python

from bootstrap_lab import torch_setup
torch = torch_setup()

from torch import nn

def evaluate(values):
    # Remove this placeholder, then adapt the retained baseline:
    # Double the output inside the inference-mode context.
    raise NotImplementedError
    layer = nn.Dropout(p=0.5)
    layer.eval()
    x = torch.tensor(values, dtype=torch.float32, requires_grad=False)
    with torch.inference_mode():
        output = layer(x)
    return {'values': output.tolist(), 'training': layer.training,
            'requires_grad': output.requires_grad}

Reasoning: Double the output inside the inference-mode context. Keep eval as the behavior mode.

Hint 1: Place multiplication inside the existing context.
Hint 2: Observe the resulting requires_grad value.

Solution, tests and trade-offs

Cases inspect nonzero, mixed-sign and empty vectors and require training=False.

Cases inspect nonzero, mixed-sign and empty vectors and require training=False. Practice checks the doubled vectors and requires_grad=False.

The source tensor is owned by this helper and created without a training graph. These checks explain mode separation on a tiny layer; they are not a throughput benchmark or evidence that every production inference path is memory efficient.

Practice solution and acceptance checks

python

from bootstrap_lab import torch_setup
torch = torch_setup()

from torch import nn

def evaluate(values):
    layer = nn.Dropout(p=0.5)
    layer.eval()
    x = torch.tensor(values, dtype=torch.float32, requires_grad=False)
    with torch.inference_mode():
        output = layer(x) * 2
    return {'values': output.tolist(), 'training': layer.training,
            'requires_grad': output.requires_grad}

Reasoning: Cases inspect nonzero, mixed-sign and empty vectors and require training=False. Practice checks the doubled vectors and requires_grad=False.

Interview reasoning and next step

Use no_grad when outputs need later interactions compatible with ordinary autograd behavior, and understand the stricter semantics of inference_mode before reusing its tensors in gradient-tracked computation..

Keep BatchNorm statistics and Dropout behavior consistent with the chosen mode. Record validation metrics without optimizer steps, then explicitly return the model to train mode when continuing training.

Optional stretch — after the core task works: Compare eval and train behavior using BatchNorm with owned batches.

Computer practice: https://applydjinn.com/typorithm/python-frameworks-lab-20261105.zip. Install the pinned dependencies in an isolated environment, run verify.py --track torch, then edit the retained exercise baseline and check it with --practice pt-evaluation. The lab uses owned in-memory databases, local framework clients and tiny CPU tensors. It never reads the working ApplyDjinn database, contacts production feeds or downloads models.

Use both model behavior mode and an appropriate inference context.