Ollama has added another way to turn images and audio into the numerical representations used by local search. An October 12 change in ollama/ollama carries media through its llama-server embedding path. It is included in v0.40.4-rc0, currently a prerelease. The gain is a choice of serving backend for builders who already use Ollama—not the first arrival of multimodal retrieval.
The timing gives a concrete shape to Google’s October 6 announcement of EmbeddingGemma 2. Google’s open model places text, code, images, audio and sampled video content into a common representation space. A phrase can become a query against a collection of pictures or recordings, without first requiring a separate caption or transcript for every item.
That is a different job from asking a chatbot to read a file and write an answer. An embedding is a list of numbers used to compare meanings. The application indexes those representations, encodes a query and retrieves nearby matches. A generative model may explain the retrieved material afterward, but it does not have to perform every comparison.
A model promise meets two serving paths
Google’s developer guide describes a modular design: a 270-million-parameter text component, an optional 170-million-parameter vision encoder and a 300-million-parameter audio encoder. The full model produces 768-dimensional vectors. A text-only application need not load every media component.
Ollama already had a route to that capability. Its October 6 MLX implementation, contained in stable v0.40.3, introduced the model’s text and optional media encoders. Before the new change, the normal llama-server embedding interface accepted text; media requests needed a separate interface supplied by the MLX runner.
The new code passes text and media through the shared embedding call. In llama-server’s implementation, the request gains media markers and encoded media data. It checks that projector components—the machinery connecting media input to the model—are present before sending the request. Without them, it reports that media embeddings are unsupported.
For a local application, this removes a distinction inside the serving layer: media no longer depends on that MLX-specific interface. It does not establish that every model or hardware combination works. MLX itself already has more than one platform path, so this is not evidence that media search has suddenly escaped one hardware vendor.
What a shared media index could change
Consider an application that keeps equipment manuals, inspection photographs and spoken field notes. A query about a damaged part could search across those materials in one representation space. That is a possible use of the model, not an application we tested. The developer still has to choose document chunks, preserve references to the original files and decide whether the returned matches answer the question.
The distinction matters because the representation layer can now accept more of the original evidence. A transcript preserves words but may discard a relevant sound; a caption captures selected visual details rather than the full image. A shared multimodal embedding offers another way to compare these materials. Whether it retrieves the right item depends on the task and collection.
Ollama’s input parser also draws a useful boundary around the available route. It accepts text, image and audio fields but explicitly rejects native video input. Google documents video processing through sampled frames in its own tooling. A model’s supported content types and a particular server’s accepted requests are separate facts.
There are other paths already. Google’s sentence-transformers guide documents broader multimodal use. Ollama’s new route is valuable for people choosing how to serve the model inside an existing application, rather than proof that every developer needs to change frameworks.
More kinds of evidence, not a universal upgrade
Google’s model card supplies a counterweight to an easy “better search” headline. Its reported multilingual-text score moves only from 61.15 to 61.36, while the code score rises from 68.76 to 78.68. Those are provider results, not our evaluation. For a collection of ordinary text, the strongest reason to migrate may differ from the reason to build a new media index.
The source tests support narrower statements. The MLX implementation includes synthetic tests for encoder combinations. The new llama-server patch updates the shared call and existing tests; we did not run model inference or measure retrieval relevance, memory use or speed on either backend. Running the embedding component locally also says nothing by itself about whether the surrounding application uploads files elsewhere.
The release boundary is still moving. An October 13 follow-up repairs how a small requested context size affects the default embedding batch. It is later than the inspected rc0 package. Builders evaluating the new route need the precise version as well as the model name.
The useful next trial is a small, fixed collection of text, images and recordings with known relevant matches. Run the same queries through a containing release, keep the original file references, and compare relevance, latency and memory with the existing path. Ollama has widened the implementation choice. The reader’s next gain comes when that choice finds the right evidence in the collection they actually use.