Unsloth has added a useful kind of refusal to local AI. An October 19 change in unslothai/unsloth lets its Studio workbench reject a request to load a different model when doing so would unload one already there. The setting is called “Always,” under “Keep multiple models loaded.” Its promise is about protecting existing residents, not finding room for everything.
That is a practical choice for a machine doing several jobs. A chat model may serve an agent through an API while someone starts an image or video task. Supporting both models does not establish that both can stay ready on the same GPU. Unsloth now exposes more of that decision to the person running the machine.
The change fits the company’s public pitch for a local workspace that combines chat, media generation and connections to coding agents. The inspected work is merged source, however: the latest GitHub beta release checked, from October 15, does not contain these October 19 changes. This is a look at the implementation entering that workspace, not a claim that every downloaded app already has it.
Three choices for the same finite memory
Keeping several models loaded is not new to Unsloth. An October 3 change already let multiple chat models remain available. When space ran short, it could unload idle kept models in least-recently-used order, then replace the active model. But the October 12 source snapshot still handed the GPU between chat, image and video owners by evicting the previous one.
The new cross-modality residency implementation and settings interface offer three policies. Off retains the swapping behavior and remains the default for fresh installations. When they fit keeps models together where the memory check permits, freeing models when necessary. Always refuses a load that would require displacing a protected model. Existing users who had enabled multiple-model loading migrate to “When they fit.”
The memory check considers model weights, room needed for generation and live free memory on the GPU being used. It leaves an additional 1,024 MiB margin. If it cannot price the load or measure suitable memory, it treats the load as not fitting. That is why “Always” can mean receiving a refusal rather than getting another model.
This coexistence path has a hardware and engine boundary. Unified-memory systems, including Macs, and some media engines or GPU configurations cannot use its current estimates. On those paths, “When they fit” can still unload a model; “Always” can refuse instead. The setting also does not disable training’s memory reclamation or idle expiry. Decision API servers remain outside its protection, and changing the quantization, context size or engine of the same model repository is still allowed.
For an unattended service, that narrower promise can be valuable. An unexpected model request can fail visibly instead of silently replacing the model another workflow expects to find. The API returns a recognizable refusal; the interface explains why the model was not loaded. A refusal is now one supported outcome of the resource policy.
When a swap is useful, save the work already done
Sometimes making room is exactly what the user wants. A second part of the change addresses the return trip: chat, then an image or video, then chat again.
A language model keeps a cache of calculations for tokens it has already processed. Reloading the model without that cache may require processing the conversation again. This is different from losing the saved conversation text. Unsloth’s new media-swap code can save the primary GGUF chat backend’s cache to disk before media takes its place, then make that cache available to a compatible reload.
The save belongs to the model actually unloaded. If the load never evicts chat, the unused save is discarded. Preservation must be enabled, the account must match and chat must be idle. Saving has a 45-second budget; a failure or timeout lets the media load proceed without the saved cache. Extra chat models kept alongside the primary one do not receive this preservation.
The contributor’s reported GPU trial illustrates the intended gain. After a Qwen chat model gave way to an image model and returned, its next 5,052-token turn evaluated 37 tokens with cache preservation enabled, versus all 5,052 without it. That is an author-reported example on an earlier branch head, not our benchmark or a general speed claim.
We inspected the source and test definitions, including identity checks, cancellation and slow-save cleanup. The tests use fake backends; we did not execute them or run a GPU workload. They make the intended contract inspectable without establishing results across hardware.
A better question than “does it support this model?”
The consequential change is a choice about continuity. A local workbench can protect a resident model, let several coexist where memory permits, or swap them while trying to retain expensive chat processing. Those are different operating preferences, even when the model list stays the same.
The next useful receipt is a containing release completing a fixed chat–media–chat sequence on a specified GPU: which models remain loaded, which request is refused, and how much conversation processing is repeated on return. That will show how the new policy behaves in ordinary work. The source already makes the decision clearer: loading another model need not automatically take precedence over keeping the current one ready.