Recent releases

This page keeps the unreleased changes and the current and previous minor releases. Notes for every older release are in CHANGELOG.md.

For upgrade planning, combine this page with the Upgrade Checklist. For breaking changes, validate against the exact release tag notes.

0.18.0#

• Fix synthesized speech playback in the macOS chat example.
• Add shared LlamadartEngine<C> lifecycle management and eager text/chat and Decision tasks with events, completion and request-scoped cancellation.
• Parse LFM2 tool arguments against their schemas, preserve unescaped inner quotes in required JSON string arguments (#1609), honor explicit parallel requests, keep reasoning non-executable and expose unfinished calls as separate diagnostics.
• Apply supported native/Web lazy tool grammars and Web prompt penalties with explicit capability checks and unchanged ordinary grammar defaults.
• Adopt native LiteRT-LM v0.19.0 and C API 1.0 with legacy compatibility, tagged Android pointer support, validated settings and typed unsupported-path errors.
• Preserve LiteRT-LM workers and model caches after unsettled native cleanup; reject incomplete runtime overrides and document process-restart recovery and unsupported text LoRA.
• Keep cached models reusable, visible and removable after cache-root moves, including iOS app updates (#1741).
• Adopt llama.cpp v0.8.0-3 for speculative rollback and model-creation ownership fixes, retaining the genuine nextn query while retiring the unused embedding query; isolate native bundle caches from older apps, verify Apple framework identity and report missing Windows speculative helpers.
• Restrict unqualified Android Vulkan Qwen3.5 combinations to bounded ordinary text; refuse larger micro-batches, media, speculation and embeddings (#948).
• Adopt stable-diffusion v0.3.0-3 for original SD checkpoint containers while retaining the SDXS tiny-decoder policy and iOS full-VAE Metal restrictions.
• Reinitialize native LiteRT-LM image-chat engines after cancellation before reusing them (#1749).
• Keep future LiteRT-LM SwiftPM pin syncs aligned with macOS app-bundle runtime discovery.
• Request optional Android vendor OpenCL access in the chat example and document LiteRT-LM GPU setup and CPU fallback.
• Document Safari 27 LiteRT-LM Web GPU support, required require-corp isolation and hidden-tab termination during large model loads (#1025).
• Aligned default WebGPU bridge assets to v0.1.82 with bounded preallocation for large wasm32 model writes.
• Pair all three Apple runtime companions at 0.18.0 with matching native binaries.

Runtime baseline: native llama.cpp v0.8.0-3 and Web use upstream v0.8.0@8642541b255bfb5dc39cd351a178be09013a55d6; native LiteRT-LM uses v0.19.0 and stable-diffusion uses v0.3.0-3. Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: fda19e047aba5a511020854437cc89e2f53e2f67f5f6cf5fec336faef3a6c362.

0.17.5#

• Preserve quoted LFM2/LFM2.5 tool-end markers inside tool arguments during parsing and streaming.
• Preserve Solar Open text across repeated content channels in parsed and streamed replies.
• Release checks reject changed Apple companion package contents without a version bump, including README-only changes.
• Keep default WebGPU bridge assets at v0.1.79 (unchanged from 0.17.4).

Runtime baseline unchanged from 0.17.4: native llama.cpp v0.8.0-1 and Web use upstream v0.8.0@8642541b255bfb5dc39cd351a178be09013a55d6; native LiteRT-LM stays v0.18.1-4 and stable-diffusion stays v0.3.0-1. Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 66ffee7a1c2c54c187526ba6a27957141b77438fed9a0f2a5afcfa2736682e9c.

0.17.4#

• Restore documentation destinations linked from published package READMEs and protect retained release notes.

• Refresh Apple companion installation guidance in 0.17.2 with unchanged native runtimes.

• Keep default WebGPU bridge assets at v0.1.79 (unchanged from 0.17.3).

Runtime baseline unchanged from 0.17.3: native llama.cpp v0.8.0-1 and Web use upstream v0.8.0@8642541b255bfb5dc39cd351a178be09013a55d6; native LiteRT-LM stays v0.18.1-4 and stable-diffusion stays v0.3.0-1. Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 66ffee7a1c2c54c187526ba6a27957141b77438fed9a0f2a5afcfa2736682e9c.

0.17.3#

• Reject projector requests on fixed LiteRT-LM custom backends before resolving files, preserving the loaded model (#1556).
• Fix native LiteRT-LM backend-SPI context recreation after automatic GPU-to-CPU fallback when reusing GPU preference hints.
• Keep default WebGPU bridge assets at v0.1.79 (unchanged from 0.17.2).

Runtime baseline unchanged from 0.17.2: native llama.cpp v0.8.0-1 and Web use upstream v0.8.0@8642541b255bfb5dc39cd351a178be09013a55d6; native LiteRT-LM stays v0.18.1-4 and stable-diffusion stays v0.3.0-1. Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 66ffee7a1c2c54c187526ba6a27957141b77438fed9a0f2a5afcfa2736682e9c.

0.17.2#

• Fix Windows native runtime overrides failing model loads with no backends are loaded (#1649).
• Preserve assistant reasoning in LFM2.5 templates that read message.thinking, including turns with tool calls (#1623).
• Reject non-finite tensorSplit sums in direct llama.cpp loads instead of silently selecting a free-memory split (#1554).
• Keep default WebGPU bridge assets at v0.1.79 (unchanged from 0.17.1).

Runtime baseline unchanged from 0.17.1: native llama.cpp v0.8.0-1 and Web use upstream v0.8.0@8642541b255bfb5dc39cd351a178be09013a55d6; native LiteRT-LM stays v0.18.1-4 and stable-diffusion stays v0.3.0-1. Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 66ffee7a1c2c54c187526ba6a27957141b77438fed9a0f2a5afcfa2736682e9c.

0.17.1#

• Aligned default WebGPU bridge assets to v0.1.79: Web now runs llama.cpp v0.8.0 like native, and a media embedding batch that takes longer than two minutes on the CPU stays on the worker instead of moving to the main thread (llama-web-bridge#165). Upstream v0.8.0@8642541b255bfb5dc39cd351a178be09013a55d6 matches native v0.8.0, which the manifest records; Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 66ffee7a1c2c54c187526ba6a27957141b77438fed9a0f2a5afcfa2736682e9c.
• Update the default llama.cpp native runtime to leehack/llamadart-native@v0.8.0-1 (llama.cpp v0.8.0), restoring Metal model loading on Apple devices. Apple apps get it with llamadart_llama_cpp_flutter 0.17.1; a locked companion 0.17.0 keeps v0.7.0-3 and builds with an upgrade warning. llama.cpp v0.8.0 refuses a hand-written grammar nested or chained more than 256 levels deep with LlamaInferenceException; generated schema grammars stay below that.
• With numberOfThreads or numberOfThreadsBatch at 0, native llama.cpp now uses at most as many threads as Platform.numberOfProcessors reports, instead of always 4, which made inference tens of times slower on hosts with fewer than 4 cores; multimodal projectors and decision encoders follow the same cap. Hosts with 4 or more cores use 4, which lowers opt-in GpuBackend.blas on hosts above 4 cores from all cores to 4; set an explicit count to raise it (#1612).
• With a bundled GPU backend module that fails to load or finds no device, such as the Windows CUDA sidecar without an NVIDIA driver, llama.cpp getResolvedGpuLayers() now reports 0, a warning names the requested backend, and getAvailableBackends() no longer lists a module that failed to load. getResolvedGpuLayers() also reports 0 for a GpuBackend.auto load on a host with no GPU device (#1598).
• ModelSplitMode.row on native llama.cpp devices without split buffers, which include every bundled GPU backend whose module loads, BLAS and an explicitly selected or fallback CPU, now throws LlamaUnsupportedException naming the device before the model loads, instead of a generic load failure; use ModelSplitMode.layer. A bundled module that fails to load or finds no device loads row on the CPU instead (#1576), and a speculative draft model now loads when its row target runs on the CPU of a host with no GPU backend, instead of failing with "Failed to load draft model" (#1593).
• On macOS x86_64, including Rosetta on Apple silicon (an x86_64 process cannot tell the two apart), llama.cpp GpuBackend.auto now runs on the CPU with 0 GPU layers, because Metal returns garbled output on Intel Macs with AMD GPUs. An explicit GpuBackend.metal or ComputeDevice.gpu still uses Metal and logs a warning; arm64 macOS is unchanged (#1597).
• On LFM2 and LFM2.5, ToolChoice.required with parallelToolCalls: true now returns every call instead of only the first; the flag also widens the grammar under ToolChoice.auto when the first system message carries the Force JSON schema. marker. A longer list needs more tokens: a reply maxTokens cuts off before the list closes finishes with length and returns no calls, including those already complete, so raise maxTokens and retry. Their tool-call parsing also no longer throws on malformed model output: a number too large to represent, a value nested too deeply or a lone quote (#1606).
• On DeepSeek R1 templates that support parallel tool calls, such as DeepSeek-R1-Distill-Qwen-32B, parallelToolCalls: true now lets the model put whitespace between calls, as llama.cpp does, so it can write the newline its template uses instead of stopping after one call. Templates without parallel calls, such as the stock 1.5B and 8B distills, still allow one call (#1581).
• Streamed thinking tags now match the final parse however a backend splits them: a </think> split across chunks no longer streams as content when the parse drops it, as in a DeepSeek R1 reply that starts with </think>, and TranslateGemma, Muse Glimmer without tools, and PEG-parsed templates without tools such as Ministral 3 Reasoning, Devstral and Nemotron 3 Nano stream thinking tags their parse keeps as text (#1582).

0.17.0#

• Breaking (backend SPI): LoRA and multimodal projector members move off LlamaBackend into the optional BackendLoraSupport and BackendMultimodalSupport interfaces. On a backend or routed runtime without one, LlamaEngine LoRA calls and projector loads throw LlamaUnsupportedException, before any download, and removeLoraSource of an adapter never applied does nothing. The built-in backends keep the same support and exception type; LlamaEngine.loadMultimodalProjector refuses a LiteRT-LM model with the same LlamaUnsupportedException message as loadMultimodalProjectorSource and setModel, including on a custom backend that reports the LiteRT-LM runtime and implements BackendMultimodalSupport, and through a router it no longer says the projector is unsupported by the active runtime (#362, #1541).
• Update the default llama.cpp native runtime to leehack/llamadart-native@v0.7.0-3, a wrapper rebuild of llama.cpp v0.7.0, read through llamadart-native hook contract 2: the hook downloads a backend sidecar archive, such as Windows x64 CUDA, only when that backend is selected, so Windows x64 apps that do not select cuda download 15 MiB instead of 703 MiB, and an overridden release that needs an unsupported contract fails the build with NativeHookContractException (#812). Linux arm64 bundles ship FP16 CPU variant modules, and every backend mode loads the best variant the CPU supports, falling back to the baseline, so mixed auto and explicit loads share one CPU backend. Apple apps get the runtime with llamadart_llama_cpp_flutter 0.17.0.
• Pin stable-diffusion-native v0.3.0-1 (stable-diffusion.cpp master-1017), which loads FP8 image weights as F16 on Metal: FP8 checkpoints now generate on Metal instead of being refused. The bounded image recipes are requalified on it, and Flutter iOS/macOS apps need llamadart_stable_diffusion_flutter 0.17.0 (#1446).
• Aligned default WebGPU bridge assets to v0.1.76: removes the retained projector copy behind the repeated Web media embedding batch abort on the wasm32 core, which stays unqualified (#1189) and can still exhaust its heap on larger contexts (llama-web-bridge#200); a speculative draft that does not fit throws LlamaUnsupportedException instead of leaking until the runtime aborts, and a model whose context does not fit fails its load instead of aborting (#198); and uncached model loads report progress as they download (#195). Upstream v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab matches native v0.7.0-2, which the manifest records; Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 90822656cef6eaf43a47eafc7fe32f2ef9c2ed56a74c6b3cae29115ba61f818a.
• Check whether a model and ModelParams can load here before any download with await engine.checkLoad(model, params: params): the LlamaLoadCheckReport is unsupported exactly when setModel would throw before downloading (LiteRT-LM on macOS x86_64, Windows arm64 or the x86_64 iOS simulator, an explicit device the platform lacks, or an option the runtime ignores), and names the runtime and device the load would use. Custom backends join through BackendLoadCheck. setModel now also rejects a LiteRT-LM model with LlamaUnsupportedException before any download, keeping the loaded model, for ComputeDevice.npu or liteRtLmEmbeddingEngine on the Web and for liteRtLmEmbeddingEngine on native platforms without qualified embeddings (Linux and Windows); these used to fail at load after the previous model was unloaded (#861).
• Load a GGUF model on chosen GPUs with ModelParams.gpuDevices (listGpuDevices indices of one backend) and set each GPU's share with tensorSplit on native llama.cpp, validated on one Metal GPU and on a Vulkan Tesla T4, with Mesa lavapipe as the second device for two-device lists and splits (two-GPU hardware has no evidence yet, and CUDA is not hardware-qualified); decision heads and multimodal projectors of the model run on a listed GPU (#1517), and indices the host lacks, a split of another length, the Web and LiteRT-LM throw LlamaUnsupportedException (#916).
• Keep Mixture-of-Experts expert weights in host memory with ModelParams.cpuMoeLayers and cache recently used experts on the GPU with the experimental moeCacheBytes budget on native llama.cpp with Metal or CUDA, validated on Metal only (CUDA and multi-GPU hosts have no hardware evidence yet); a budget on Vulkan, other backends or tensor split, and either field on the Web or LiteRT-LM, throw LlamaUnsupportedException (#1123).
• LlamaEngine.stateIdentity() reports the native runtime build, session format, context size and a digest of the context settings that shape the saved state (KV cache types, flash attention, sequences, SWA and RoPE), so an app can check a llama.cpp state file another process saved before loading it. It needs native llama.cpp with llamadart-native v0.7.0-3; Web, LiteRT-LM, custom backends and older runtimes throw LlamaUnsupportedException (#1211).
• Return images from tools with LlamaToolResultContent.content or a handler's LlamaToolOutput on native llama.cpp with a vision projector and Hermes-style templates (smoke-tested on macOS CPU and Metal with Qwen3.5-0.8B weights, a qwen3vl_merger projector and the Qwen3-VL template), rendered as llama-server renders them; check LlamaEngineCapabilities.toolResultMedia. Other formats, runtimes, audio, video and URL-only images throw LlamaUnsupportedException (#1008).
• Parse a reply as the engine does with ChatTemplateEngine.parseRendered(rendered, output), which reads how its render or engine.chatTemplate result left the thought: after the stock Seed-OSS template, Plan</seed:think>Sunny. is content, as in llama-server, where ChatTemplateEngine.parse returns reasoning Plan (#1565).
• Native LiteRT-LM under ComputeDevice.auto retries a GPU engine that fails to start on the CPU with a warning, and reports LiteRT-LM cpu; explicit gpu and npu still fail (#1034).
• Refine text-to-image output in a second pass at up to twice the size with HiresFixImageGenerationRequest and ImageHiresFix (latent upscale, bounded to 1024x1024 and 4 images) on macOS CPU or Metal with SD 1.x or SD 2.x full-VAE checkpoints; check ImageGenerationCapabilities.supportsHiresFix, and other recipes throw LlamaUnsupportedException (#1228).
• Edit images from a reference with FLUX.1 Kontext [dev] through ReferenceImageEditingRequest and ImageReferenceEditingRecipe.fluxKontext (one RGB reference at the output size, at most 1024x1024) on macOS Metal with the Kontext transformer, ae, CLIP-L and T5-XXL; check ImageGenerationCapabilities.supportsReferenceImageEditing, and other models, devices and reference counts throw LlamaUnsupportedException (#1222).
• Load an SD 1.x ControlNet as ImageModelRole.controlNet and steer generation with ControlNetImageGenerationRequest on the bounded macOS CPU recipe; other ControlNet recipes are refused at load (#1224).
• Keep an image model's text encoders or VAE on the CPU, cap the GPU memory stable-diffusion.cpp uses, or turn off its automatic fit with ImageGenerationEngine.load(placement: ImageModelPlacement(...)) on Metal and Vulkan, validated with SD-Turbo on Metal and measured on one Vulkan Tesla T4 (multi-GPU hosts have no evidence yet); a placement on a CPU engine, Android, the CPU builds or the Web, and a GPU budget with the fit off or on Vulkan without a discrete GPU, throw LlamaUnsupportedException (#806).
• Chat templates now receive enableThinking as enable_thinking on every request, including responseFormat and generic tool-call routes, as in llama.cpp, so templates such as GLM-4.6, GLM-4.7-Flash, GLM-OCR, SmolLM3, Apertus and DeepSeek-R1 distills render thinking as llama-server does for the same flag. enableThinking still defaults to true, so Apertus now deliberates and DeepSeek-R1 distills open a thought unless you pass false (#1465).
• A boolean enable_thinking or thinking chat template kwarg now overrides enableThinking, and a template's own thinking setting passed in chatTemplateKwargs, such as reasoning_effort for Hy3 and Cohere2 MoE or thinking_mode for MiniMax M3, is no longer replaced by the value llamadart derives from enableThinking, so both render as in llama-server; DeepSeek V3.1 forces its thought open from enableThinking alone (#750).
• A responseFormat or tool-call grammar after a thought the prompt leaves open now lets the model finish the thought first, as llama-server does, instead of returning the JSON as reasoning; a thought the template opens stays open with enableThinking: false, as in llama-server (#1465). A required tool call after such a thought also lets Cohere2 MoE, Nemotron 3 Nano, MiniCPM5 and Hy3 models finish it first; MiniCPM5 previously wrote the call inside the thought and returned no tool call.
• A thought that Qwen3.5, QwQ and other Hermes-style templates open in the prompt, or that DeepSeek V3.1 forces open, is kept as llama-server does: an escaped \n and a repeated <think> stay in the reasoning as generated, thinking tags after the first </think> stay in the content instead of being parsed as more reasoning, a partial </think> that ends an unfinished thought is dropped, and a <tool_call> also ends a Qwen3.5 thought (#750).
• With thinking off, DeepSeek V3.1, Qwen3, QwQ and Qwen3.5 replies keep any <think> blocks the model writes in the content, as llama-server does, instead of dropping them as hidden reasoning (#750).
• With thinking off, StepFun3.5-Flash, DeepSeek R1 distills, MiniMax M2 and Nemotron Nano v2 replies no longer stream a thought the model writes before </think> as content: content waits for </think>, <think> or the end of the reply, matching the final message at any chunking (#1532).
• With thinking off, Cohere2 MoE replies no longer stream a thought the model writes before <|END_THINKING|> as content: content waits for a thinking tag, <|START_TEXT|> or <|START_ACTION|>, where llama-server ends the thought. Command R7B and Seed-OSS templates that open the thought with thinking off hold their reply until a thinking tag, and Ministral and Solar Open replies, whose parse keeps such text as content, no longer wait (#1553, #1557). With thinking off, a Cohere2 MoE thought that ends at <|START_TEXT|> or <|START_ACTION|> is reasoning instead of content, as in llama-server; thinking tags inside a Cohere2 MoE text or action stay text; and a response schema no longer leaves the thought open (#1566).
• Cohere2 MoE and Command R7B models on native llama.cpp keep the thinking, action and response markers their GGUFs store as control tokens, as llama-server does, so their thoughts, tool calls and text parse instead of all landing in reasoning or showing up as raw JSON (#1505).
• A Cohere2 MoE thought that the prompt opens ends at a <|START_TEXT|> or <|START_ACTION|> the model writes without <|END_THINKING|>, as in llama-server, so its text and tool calls no longer land in the reasoning; with enableThinking: false and a reasoning: true chat template kwarg, its prompt ends with the template's own empty thought, without a trailing newline (#1523).
• A Seed-OSS reply to a prompt that opens no thought, as the stock template renders, has one only when it starts with <seed:think>, kept as generated up to the first </seed:think>, as in llama-server; other thinking tags stay in the content (#1561). To a prompt that opens the thought, a repeated <seed:think>, and any text before it, stay in the reasoning up to the first </seed:think>, as in llama.cpp, instead of the tag being dropped, and a thinking tag inside a tool-call argument after that thought no longer drops the call.
• Replies with parallel tool calls return one call per block instead of none, and the tool-call grammar admits those blocks, or only one call when parallel tool calls are off, as llama.cpp does: <tool_call> blocks of Qwen3-Coder XML replies, as from Qwen3-Coder, Qwen3.5 and StepFun3.5 (#1530), and <seed:tool_call> blocks of Seed-OSS replies (#1540). Nemotron 3 Nano's parallel-call grammar admits them too, where it used to admit only replies that parsed to no calls.
• ToolChoice.required now returns the tool call on DeepSeek-R1 distills, LFM2 and Functionary v3.2 instead of running to the length limit or returning the call as text: their tool-call grammars match llama.cpp's call framing, without a literal \n or a free-text LFM2 id, and DeepSeek-R1 takes one call unless parallelToolCalls is set (#1577). On Firefunction v2 it no longer makes the model write a free-text call id: its tool-call grammar, like llama.cpp's, takes only the call's name and arguments (#1565).
• Requests with an image or audio part and tools honor ToolChoice.auto instead of forcing a tool call on templates such as Qwen3-VL and Qwen3.5, and use the same tool-call grammar, triggers, parser and stops as text requests (#1463).
• A tool result whose first text is empty no longer renders a leading newline before the next text part on string-only chat templates, as llama-server does (#1468).
• A failed native stateLoadFile no longer lets the next prompt-prefix reuse skip evaluating the prompt on the cache llama.cpp cleared, which returned wrong scores and text without an error (#1491).
• Processes and isolates sharing a model cache root, such as sharedCache(), userSelected(...) or appGroup(...) directories, now download each entry once instead of returning a file another process is still writing, and never cache a file whose length differs from the one the server reported (#1444). A cache hit on a read-only model cache root returns the cached entry instead of reporting that no cached model is available.
• DefaultModelDownloadManager.remove, prune and clear no longer delete a cache entry another isolate or process is downloading: remove and clear remove the rest and throw ModelCacheEntryInUseException, and prune skips it (#1467). Overlapping remove, prune and clear calls on one cache root take turns instead of reporting each other's entries in use (#1490), and no longer fail or over-remove when they overlap in one process; a stray .REMOVAL directory in a case-insensitive cache root no longer keeps an entry reported in use (#1522); clear removes a stray .removal directory instead of reporting it in use forever (#1507); and on Windows a model file that can never be deleted, such as a read-only one, throws its FileSystemException instead of ModelCacheEntryInUseException (#1490).
• For an overridden native release, the assets.json.absent record, which notes that the release publishes no assets.json, is re-checked daily, or at once when it is dated in the future, and a record whose re-check fails or returns a page that is not JSON is kept and renewed instead of failing the build or retrying on every offline build; an assets.json that is truncated, empty, non-UTF-8 or not a manifest fails the build instead of being treated as unpublished; and a CUDA build that cannot reach its assets.json uses the cached CUDA sidecar instead of leaving CUDA out (#1490, #1506, #1522).
• The build hook reuses a cached llama.cpp bundle only while its extracted/ directory still holds what the cached archive beside it extracted to, and otherwise extracts that archive again, so a library replaced or added in the cache, such as a test stub, is no longer bundled (#1558). A cache from an earlier release is extracted again once, without network access. A cached native archive that matches its digest but lacks runtime libraries fails the build instead of being downloaded again on every build (#1490), and an incomplete cached core archive that matches its cached SHA256SUMS no longer tells an offline build to pre-seed SHA256SUMS (#1506).
• The image runtime's "not bundled" and other load-failure reasons name the iOS simulators by the build hook's bundle names, ios-arm64-sim and ios-x64-sim, instead of ios-arm64 and ios-x64 (#1565).
• The installation guide and support matrix now name libCLiteRTLM_mac.dylib as the library a sandboxed Flutter macOS app without the LiteRT-LM companion fails to load, instead of libLiteRt.dylib (#1565).