Pydantic AI is the Python framework between an application’s tools and its model calls. A change merged October 20 in pydantic/pydantic-ai gives it a new way to assemble those calls: keep an instruction’s original text in the opening, then send changed state later. A todo can become “done” without rewriting the “pending” text the model first received.
That matters because prompt caches recognize matching beginnings. Editing an early instruction can break reuse of the material after it. The new option moves the update; it does not freeze the task or erase the old state.
The update moves. The opening stays.
A tool has marked a task as done. Compare how Pydantic AI assembles the next request to the model. Read each strip from top to bottom.
The default rewrites the opening. Append mode keeps that opening and puts the new value after the conversation.
The default rewrites the opening. Append mode keeps that opening and puts the new value after the conversation. Before the change, both requests contain the pending value. Append mode already wraps its identified block for later replacement.Rewrite the opening
The default for your instructions.
Append the update
Opt in on an identified instruction.
Schematic of instruction placement, not a model run or a cache-hit measurement. “Pending” remains in the append-mode opening as historical text; the later replacement with the same identity tells the model to use the new value. Provider formatting is simplified here.
Keeping this prefix stable can preserve a condition for cache reuse. Other request changes, provider rules and expiry still matter. Each update also stays in history; a new baseline or compaction changes the comparison.
Inspect the source example and its limits.
A small state change can disturb a long opening.
Developers had already encountered the tradeoff. An issue opened August 6 asked for cache-friendly dynamic instructions. In another report opened August 30, an application author described a todo update invalidating a much larger prompt prefix—and a workaround that put changing state later in the conversation.
The new contribution is a framework-owned contract for that technique. A named instruction can opt into on_change='append'. Its function still runs before each model request, including after a tool changes state. The initial value stays in place; a changed value becomes a typed history entry. An unchanged value adds nothing. Withdrawing the instruction produces a withdrawal rather than silently leaving the old value in force.
The model sees the old state and its replacement.
Pydantic labels the instruction so later text can identify what it supersedes. Saved history retains the complete replacement; for a small edit to a long block, the request can carry a shorter diff. The framework also saves a baseline record so a fresh agent instance can reconstruct the earlier prefix. That bookkeeping record is not itself sent to the model.
The provider tests check both sides of the promise: unchanged earlier request content and a delivered update after a tool changes a todo from pending to done. Other cases change a warehouse destination, withdraw it and continue through serialized history. We inspected the tests and recorded provider cases; we did not run a live model trial or measure a bill.
Delivery also has a meaningful boundary. Models that support system messages within a conversation receive updates that way. Others receive system-tagged text inside a user message. The tags do not give that text identical instruction priority. Preserving the prefix and getting the model to follow the new state are separate things to verify.
Retention and request construction solve different problems.
OpenAI’s October 20 API announcement adds a 12-hour prompt-cache option. Its guide prices those writes at 2.5 times ordinary input, compared with 1.25 times for 30 minutes; the read rate is the same for either lifetime. Longer retention can help a stable prefix reused hours later. It cannot make a changed prefix identical.
Pydantic’s work began before that announcement. It does not implement OpenAI’s retention setting, and neither source establishes savings for a particular application. The useful connection is a design choice: separate frequently changing task state from material worth keeping stable, then measure the complete workload.
