Typorithm · medium

Prompt Caching

prompt-cachingllmcache-controllatency

Saved on this device

Full lesson · 8 learning steps Open any exercise for focused practice

Step 1 of 8 · intro card

Learn Prompt Caching

Context & Tokens

Prompt caching reuses a static prefix — a long system prompt or documents — across calls, cutting cost and latency. You mark the cache breakpoint with cache_control. The cache has a short time-to-live, so it helps most with rapid, repeated calls.

Many requests share a large, unchanging prefix: a detailed system prompt, instructions, or reference documents. Prompt caching stores that prefix so repeated calls reuse it instead of reprocessing it every time.

You set the breakpoint with cache_control on a content block. Everything up to that marker is cached; on the next request with the same prefix, those tokens are served from cache at a fraction of the cost and latency.

The cache is short-lived (a brief time-to-live), so the win is largest when calls repeat quickly. Keep the stable content first and the varying content after the breakpoint, or any change invalidates the cached prefix.

Cache a static prefix: system prompt or documents.

cache_control marks the cache breakpoint.

Reuse is cheaper and lower-latency on repeated calls.

const system = [{
  type: "text",
  text: largeSystemPrompt,
  cache_control: { type: "ephemeral" },
}];
// a cached prefix is cheaper and faster on reuse

Read the implementation

Connect each part of the code to the concept before you type.

1. Static prefix

A long, stable system prompt is the ideal prefix to cache and reuse across calls.

const system = [{

2. The large prompt

The big, unchanging text is what makes caching worthwhile — reusing it skips reprocessing.

  text: largeSystemPrompt,

3. Cache breakpoint

cache_control marks where the cached prefix ends; ephemeral is the short-lived cache type.

  cache_control: { type: "ephemeral" },

4. Block end

Everything up to this cache_control breakpoint is stored and reused on the next matching request.

}];

5. Why it helps

Reusing a cached prefix cuts both cost and latency, since the model skips reprocessing it.

// a cached prefix is cheaper and faster on reuse

A complete learning loop

Understand

Read the concept map and follow the code breakdown.

Reconstruct

Type the implementation, repair mistakes and work through every recall exercise.

Verify

Use the final check without hints; revisit weak concepts in short practice sessions.