Recent releases

Release notes for the unreleased changes and the current and previous llamadart minor, with a link to the full changelog for older releases.

On this page

This page keeps the unreleased changes and the current and previous minor releases. Notes for every older release are in CHANGELOG.md.

For upgrade planning, combine this page with the Upgrade Checklist. For breaking changes, validate against the exact release tag notes.

0.17.4#

Restore documentation destinations linked from published package READMEs and protect retained release notes.
Refresh Apple companion installation guidance in 0.17.2 with unchanged native runtimes.
Keep default WebGPU bridge assets at v0.1.79 (unchanged from 0.17.3).

Runtime baseline unchanged from 0.17.3: native llama.cpp v0.8.0-1 and Web use upstream v0.8.0@8642541b255bfb5dc39cd351a178be09013a55d6; native LiteRT-LM stays v0.18.1-4 and stable-diffusion stays v0.3.0-1. Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 66ffee7a1c2c54c187526ba6a27957141b77438fed9a0f2a5afcfa2736682e9c.

0.17.3#

Reject projector requests on fixed LiteRT-LM custom backends before resolving files, preserving the loaded model (#1556).
Fix native LiteRT-LM backend-SPI context recreation after automatic GPU-to-CPU fallback when reusing GPU preference hints.
Keep default WebGPU bridge assets at v0.1.79 (unchanged from 0.17.2).

Runtime baseline unchanged from 0.17.2: native llama.cpp v0.8.0-1 and Web use upstream v0.8.0@8642541b255bfb5dc39cd351a178be09013a55d6; native LiteRT-LM stays v0.18.1-4 and stable-diffusion stays v0.3.0-1. Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 66ffee7a1c2c54c187526ba6a27957141b77438fed9a0f2a5afcfa2736682e9c.

0.17.2#

Fix Windows native runtime overrides failing model loads with no backends are loaded (#1649).
Preserve assistant reasoning in LFM2.5 templates that read message.thinking, including turns with tool calls (#1623).
Reject non-finite tensorSplit sums in direct llama.cpp loads instead of silently selecting a free-memory split (#1554).
Keep default WebGPU bridge assets at v0.1.79 (unchanged from 0.17.1).

Runtime baseline unchanged from 0.17.1: native llama.cpp v0.8.0-1 and Web use upstream v0.8.0@8642541b255bfb5dc39cd351a178be09013a55d6; native LiteRT-LM stays v0.18.1-4 and stable-diffusion stays v0.3.0-1. Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 66ffee7a1c2c54c187526ba6a27957141b77438fed9a0f2a5afcfa2736682e9c.

0.17.1#

Aligned default WebGPU bridge assets to v0.1.79: Web now runs llama.cpp v0.8.0 like native, and a media embedding batch that takes longer than two minutes on the CPU stays on the worker instead of moving to the main thread (llama-web-bridge#165). Upstream v0.8.0@8642541b255bfb5dc39cd351a178be09013a55d6 matches native v0.8.0, which the manifest records; Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 66ffee7a1c2c54c187526ba6a27957141b77438fed9a0f2a5afcfa2736682e9c.
Update the default llama.cpp native runtime to leehack/llamadart-native@v0.8.0-1 (llama.cpp v0.8.0), restoring Metal model loading on Apple devices. Apple apps get it with llamadart_llama_cpp_flutter 0.17.1; a locked companion 0.17.0 keeps v0.7.0-3 and builds with an upgrade warning. llama.cpp v0.8.0 refuses a hand-written grammar nested or chained more than 256 levels deep with LlamaInferenceException; generated schema grammars stay below that.
With numberOfThreads or numberOfThreadsBatch at 0, native llama.cpp now uses at most as many threads as Platform.numberOfProcessors reports, instead of always 4, which made inference tens of times slower on hosts with fewer than 4 cores; multimodal projectors and decision encoders follow the same cap. Hosts with 4 or more cores use 4, which lowers opt-in GpuBackend.blas on hosts above 4 cores from all cores to 4; set an explicit count to raise it (#1612).
With a bundled GPU backend module that fails to load or finds no device, such as the Windows CUDA sidecar without an NVIDIA driver, llama.cpp getResolvedGpuLayers() now reports 0, a warning names the requested backend, and getAvailableBackends() no longer lists a module that failed to load. getResolvedGpuLayers() also reports 0 for a GpuBackend.auto load on a host with no GPU device (#1598).
ModelSplitMode.row on native llama.cpp devices without split buffers, which include every bundled GPU backend whose module loads, BLAS and an explicitly selected or fallback CPU, now throws LlamaUnsupportedException naming the device before the model loads, instead of a generic load failure; use ModelSplitMode.layer. A bundled module that fails to load or finds no device loads row on the CPU instead (#1576), and a speculative draft model now loads when its row target runs on the CPU of a host with no GPU backend, instead of failing with "Failed to load draft model" (#1593).
On macOS x86_64, including Rosetta on Apple silicon (an x86_64 process cannot tell the two apart), llama.cpp GpuBackend.auto now runs on the CPU with 0 GPU layers, because Metal returns garbled output on Intel Macs with AMD GPUs. An explicit GpuBackend.metal or ComputeDevice.gpu still uses Metal and logs a warning; arm64 macOS is unchanged (#1597).
On LFM2 and LFM2.5, ToolChoice.required with parallelToolCalls: true now returns every call instead of only the first; the flag also widens the grammar under ToolChoice.auto when the first system message carries the Force JSON schema. marker. A longer list needs more tokens: a reply maxTokens cuts off before the list closes finishes with length and returns no calls, including those already complete, so raise maxTokens and retry. Their tool-call parsing also no longer throws on malformed model output: a number too large to represent, a value nested too deeply or a lone quote (#1606).
On DeepSeek R1 templates that support parallel tool calls, such as DeepSeek-R1-Distill-Qwen-32B, parallelToolCalls: true now lets the model put whitespace between calls, as llama.cpp does, so it can write the newline its template uses instead of stopping after one call. Templates without parallel calls, such as the stock 1.5B and 8B distills, still allow one call (#1581).
Streamed thinking tags now match the final parse however a backend splits them: a </think> split across chunks no longer streams as content when the parse drops it, as in a DeepSeek R1 reply that starts with </think>, and TranslateGemma, Muse Glimmer without tools, and PEG-parsed templates without tools such as Ministral 3 Reasoning, Devstral and Nemotron 3 Nano stream thinking tags their parse keeps as text (#1582).

0.17.0#

Breaking (backend SPI): LoRA and multimodal projector members move off LlamaBackend into the optional BackendLoraSupport and BackendMultimodalSupport interfaces. On a backend or routed runtime without one, LlamaEngine LoRA calls and projector loads throw LlamaUnsupportedException, before any download, and removeLoraSource of an adapter never applied does nothing. The built-in backends keep the same support and exception type; LlamaEngine.loadMultimodalProjector refuses a LiteRT-LM model with the same LlamaUnsupportedException message as loadMultimodalProjectorSource and setModel, including on a custom backend that reports the LiteRT-LM runtime and implements BackendMultimodalSupport, and through a router it no longer says the projector is unsupported by the active runtime (#362, #1541).
Update the default llama.cpp native runtime to leehack/llamadart-native@v0.7.0-3, a wrapper rebuild of llama.cpp v0.7.0, read through llamadart-native hook contract 2: the hook downloads a backend sidecar archive, such as Windows x64 CUDA, only when that backend is selected, so Windows x64 apps that do not select cuda download 15 MiB instead of 703 MiB, and an overridden release that needs an unsupported contract fails the build with NativeHookContractException (#812). Linux arm64 bundles ship FP16 CPU variant modules, and every backend mode loads the best variant the CPU supports, falling back to the baseline, so mixed auto and explicit loads share one CPU backend. Apple apps get the runtime with llamadart_llama_cpp_flutter 0.17.0.
Pin stable-diffusion-native v0.3.0-1 (stable-diffusion.cpp master-1017), which loads FP8 image weights as F16 on Metal: FP8 checkpoints now generate on Metal instead of being refused. The bounded image recipes are requalified on it, and Flutter iOS/macOS apps need llamadart_stable_diffusion_flutter 0.17.0 (#1446).
Aligned default WebGPU bridge assets to v0.1.76: removes the retained projector copy behind the repeated Web media embedding batch abort on the wasm32 core, which stays unqualified (#1189) and can still exhaust its heap on larger contexts (llama-web-bridge#200); a speculative draft that does not fit throws LlamaUnsupportedException instead of leaking until the runtime aborts, and a model whose context does not fit fails its load instead of aborting (#198); and uncached model loads report progress as they download (#195). Upstream v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab matches native v0.7.0-2, which the manifest records; Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 90822656cef6eaf43a47eafc7fe32f2ef9c2ed56a74c6b3cae29115ba61f818a.
Check whether a model and ModelParams can load here before any download with await engine.checkLoad(model, params: params): the LlamaLoadCheckReport is unsupported exactly when setModel would throw before downloading (LiteRT-LM on macOS x86_64, Windows arm64 or the x86_64 iOS simulator, an explicit device the platform lacks, or an option the runtime ignores), and names the runtime and device the load would use. Custom backends join through BackendLoadCheck. setModel now also rejects a LiteRT-LM model with LlamaUnsupportedException before any download, keeping the loaded model, for ComputeDevice.npu or liteRtLmEmbeddingEngine on the Web and for liteRtLmEmbeddingEngine on native platforms without qualified embeddings (Linux and Windows); these used to fail at load after the previous model was unloaded (#861).
Load a GGUF model on chosen GPUs with ModelParams.gpuDevices (listGpuDevices indices of one backend) and set each GPU's share with tensorSplit on native llama.cpp, validated on one Metal GPU and on a Vulkan Tesla T4, with Mesa lavapipe as the second device for two-device lists and splits (two-GPU hardware has no evidence yet, and CUDA is not hardware-qualified); decision heads and multimodal projectors of the model run on a listed GPU (#1517), and indices the host lacks, a split of another length, the Web and LiteRT-LM throw LlamaUnsupportedException (#916).
Keep Mixture-of-Experts expert weights in host memory with ModelParams.cpuMoeLayers and cache recently used experts on the GPU with the experimental moeCacheBytes budget on native llama.cpp with Metal or CUDA, validated on Metal only (CUDA and multi-GPU hosts have no hardware evidence yet); a budget on Vulkan, other backends or tensor split, and either field on the Web or LiteRT-LM, throw LlamaUnsupportedException (#1123).
LlamaEngine.stateIdentity() reports the native runtime build, session format, context size and a digest of the context settings that shape the saved state (KV cache types, flash attention, sequences, SWA and RoPE), so an app can check a llama.cpp state file another process saved before loading it. It needs native llama.cpp with llamadart-native v0.7.0-3; Web, LiteRT-LM, custom backends and older runtimes throw LlamaUnsupportedException (#1211).
Return images from tools with LlamaToolResultContent.content or a handler's LlamaToolOutput on native llama.cpp with a vision projector and Hermes-style templates (smoke-tested on macOS CPU and Metal with Qwen3.5-0.8B weights, a qwen3vl_merger projector and the Qwen3-VL template), rendered as llama-server renders them; check LlamaEngineCapabilities.toolResultMedia. Other formats, runtimes, audio, video and URL-only images throw LlamaUnsupportedException (#1008).
Parse a reply as the engine does with ChatTemplateEngine.parseRendered(rendered, output), which reads how its render or engine.chatTemplate result left the thought: after the stock Seed-OSS template, Plan</seed:think>Sunny. is content, as in llama-server, where ChatTemplateEngine.parse returns reasoning Plan (#1565).
Native LiteRT-LM under ComputeDevice.auto retries a GPU engine that fails to start on the CPU with a warning, and reports LiteRT-LM cpu; explicit gpu and npu still fail (#1034).
Refine text-to-image output in a second pass at up to twice the size with HiresFixImageGenerationRequest and ImageHiresFix (latent upscale, bounded to 1024x1024 and 4 images) on macOS CPU or Metal with SD 1.x or SD 2.x full-VAE checkpoints; check ImageGenerationCapabilities.supportsHiresFix, and other recipes throw LlamaUnsupportedException (#1228).
Edit images from a reference with FLUX.1 Kontext [dev] through ReferenceImageEditingRequest and ImageReferenceEditingRecipe.fluxKontext (one RGB reference at the output size, at most 1024x1024) on macOS Metal with the Kontext transformer, ae, CLIP-L and T5-XXL; check ImageGenerationCapabilities.supportsReferenceImageEditing, and other models, devices and reference counts throw LlamaUnsupportedException (#1222).
Load an SD 1.x ControlNet as ImageModelRole.controlNet and steer generation with ControlNetImageGenerationRequest on the bounded macOS CPU recipe; other ControlNet recipes are refused at load (#1224).
Keep an image model's text encoders or VAE on the CPU, cap the GPU memory stable-diffusion.cpp uses, or turn off its automatic fit with ImageGenerationEngine.load(placement: ImageModelPlacement(...)) on Metal and Vulkan, validated with SD-Turbo on Metal and measured on one Vulkan Tesla T4 (multi-GPU hosts have no evidence yet); a placement on a CPU engine, Android, the CPU builds or the Web, and a GPU budget with the fit off or on Vulkan without a discrete GPU, throw LlamaUnsupportedException (#806).
Chat templates now receive enableThinking as enable_thinking on every request, including responseFormat and generic tool-call routes, as in llama.cpp, so templates such as GLM-4.6, GLM-4.7-Flash, GLM-OCR, SmolLM3, Apertus and DeepSeek-R1 distills render thinking as llama-server does for the same flag. enableThinking still defaults to true, so Apertus now deliberates and DeepSeek-R1 distills open a thought unless you pass false (#1465).
A boolean enable_thinking or thinking chat template kwarg now overrides enableThinking, and a template's own thinking setting passed in chatTemplateKwargs, such as reasoning_effort for Hy3 and Cohere2 MoE or thinking_mode for MiniMax M3, is no longer replaced by the value llamadart derives from enableThinking, so both render as in llama-server; DeepSeek V3.1 forces its thought open from enableThinking alone (#750).
A responseFormat or tool-call grammar after a thought the prompt leaves open now lets the model finish the thought first, as llama-server does, instead of returning the JSON as reasoning; a thought the template opens stays open with enableThinking: false, as in llama-server (#1465). A required tool call after such a thought also lets Cohere2 MoE, Nemotron 3 Nano, MiniCPM5 and Hy3 models finish it first; MiniCPM5 previously wrote the call inside the thought and returned no tool call.
A thought that Qwen3.5, QwQ and other Hermes-style templates open in the prompt, or that DeepSeek V3.1 forces open, is kept as llama-server does: an escaped \n and a repeated <think> stay in the reasoning as generated, thinking tags after the first </think> stay in the content instead of being parsed as more reasoning, a partial </think> that ends an unfinished thought is dropped, and a <tool_call> also ends a Qwen3.5 thought (#750).
With thinking off, DeepSeek V3.1, Qwen3, QwQ and Qwen3.5 replies keep any <think> blocks the model writes in the content, as llama-server does, instead of dropping them as hidden reasoning (#750).
With thinking off, StepFun3.5-Flash, DeepSeek R1 distills, MiniMax M2 and Nemotron Nano v2 replies no longer stream a thought the model writes before </think> as content: content waits for </think>, <think> or the end of the reply, matching the final message at any chunking (#1532).
With thinking off, Cohere2 MoE replies no longer stream a thought the model writes before <|END_THINKING|> as content: content waits for a thinking tag, <|START_TEXT|> or <|START_ACTION|>, where llama-server ends the thought. Command R7B and Seed-OSS templates that open the thought with thinking off hold their reply until a thinking tag, and Ministral and Solar Open replies, whose parse keeps such text as content, no longer wait (#1553, #1557). With thinking off, a Cohere2 MoE thought that ends at <|START_TEXT|> or <|START_ACTION|> is reasoning instead of content, as in llama-server; thinking tags inside a Cohere2 MoE text or action stay text; and a response schema no longer leaves the thought open (#1566).
Cohere2 MoE and Command R7B models on native llama.cpp keep the thinking, action and response markers their GGUFs store as control tokens, as llama-server does, so their thoughts, tool calls and text parse instead of all landing in reasoning or showing up as raw JSON (#1505).
A Cohere2 MoE thought that the prompt opens ends at a <|START_TEXT|> or <|START_ACTION|> the model writes without <|END_THINKING|>, as in llama-server, so its text and tool calls no longer land in the reasoning; with enableThinking: false and a reasoning: true chat template kwarg, its prompt ends with the template's own empty thought, without a trailing newline (#1523).
A Seed-OSS reply to a prompt that opens no thought, as the stock template renders, has one only when it starts with <seed:think>, kept as generated up to the first </seed:think>, as in llama-server; other thinking tags stay in the content (#1561). To a prompt that opens the thought, a repeated <seed:think>, and any text before it, stay in the reasoning up to the first </seed:think>, as in llama.cpp, instead of the tag being dropped, and a thinking tag inside a tool-call argument after that thought no longer drops the call.
Replies with parallel tool calls return one call per block instead of none, and the tool-call grammar admits those blocks, or only one call when parallel tool calls are off, as llama.cpp does: <tool_call> blocks of Qwen3-Coder XML replies, as from Qwen3-Coder, Qwen3.5 and StepFun3.5 (#1530), and <seed:tool_call> blocks of Seed-OSS replies (#1540). Nemotron 3 Nano's parallel-call grammar admits them too, where it used to admit only replies that parsed to no calls.
ToolChoice.required now returns the tool call on DeepSeek-R1 distills, LFM2 and Functionary v3.2 instead of running to the length limit or returning the call as text: their tool-call grammars match llama.cpp's call framing, without a literal \n or a free-text LFM2 id, and DeepSeek-R1 takes one call unless parallelToolCalls is set (#1577). On Firefunction v2 it no longer makes the model write a free-text call id: its tool-call grammar, like llama.cpp's, takes only the call's name and arguments (#1565).
Requests with an image or audio part and tools honor ToolChoice.auto instead of forcing a tool call on templates such as Qwen3-VL and Qwen3.5, and use the same tool-call grammar, triggers, parser and stops as text requests (#1463).
A tool result whose first text is empty no longer renders a leading newline before the next text part on string-only chat templates, as llama-server does (#1468).
A failed native stateLoadFile no longer lets the next prompt-prefix reuse skip evaluating the prompt on the cache llama.cpp cleared, which returned wrong scores and text without an error (#1491).
Processes and isolates sharing a model cache root, such as sharedCache(), userSelected(...) or appGroup(...) directories, now download each entry once instead of returning a file another process is still writing, and never cache a file whose length differs from the one the server reported (#1444). A cache hit on a read-only model cache root returns the cached entry instead of reporting that no cached model is available.
DefaultModelDownloadManager.remove, prune and clear no longer delete a cache entry another isolate or process is downloading: remove and clear remove the rest and throw ModelCacheEntryInUseException, and prune skips it (#1467). Overlapping remove, prune and clear calls on one cache root take turns instead of reporting each other's entries in use (#1490), and no longer fail or over-remove when they overlap in one process; a stray .REMOVAL directory in a case-insensitive cache root no longer keeps an entry reported in use (#1522); clear removes a stray .removal directory instead of reporting it in use forever (#1507); and on Windows a model file that can never be deleted, such as a read-only one, throws its FileSystemException instead of ModelCacheEntryInUseException (#1490).
For an overridden native release, the assets.json.absent record, which notes that the release publishes no assets.json, is re-checked daily, or at once when it is dated in the future, and a record whose re-check fails or returns a page that is not JSON is kept and renewed instead of failing the build or retrying on every offline build; an assets.json that is truncated, empty, non-UTF-8 or not a manifest fails the build instead of being treated as unpublished; and a CUDA build that cannot reach its assets.json uses the cached CUDA sidecar instead of leaving CUDA out (#1490, #1506, #1522).
The build hook reuses a cached llama.cpp bundle only while its extracted/ directory still holds what the cached archive beside it extracted to, and otherwise extracts that archive again, so a library replaced or added in the cache, such as a test stub, is no longer bundled (#1558). A cache from an earlier release is extracted again once, without network access. A cached native archive that matches its digest but lacks runtime libraries fails the build instead of being downloaded again on every build (#1490), and an incomplete cached core archive that matches its cached SHA256SUMS no longer tells an offline build to pre-seed SHA256SUMS (#1506).
The image runtime's "not bundled" and other load-failure reasons name the iOS simulators by the build hook's bundle names, ios-arm64-sim and ios-x64-sim, instead of ios-arm64 and ios-x64 (#1565).
The installation guide and support matrix now name libCLiteRTLM_mac.dylib as the library a sandboxed Flutter macOS app without the LiteRT-LM companion fails to load, instead of libLiteRt.dylib (#1565).

0.16.5#

Reject unsupported tool-role image/audio content before chat rendering or generation instead of silently losing or relocating it.
Refuse FP8 image weights on Metal before native loading can abort the process; explicit CPU loading remains available.
Enforce bounded JSON Schema text output with native LiteRT-LM on macOS arm64 CPU and canonical Qwen3 models.
Reduce startup compilation overhead for llamadart:doctor and add opt-in isolated desktop native diagnostics with privacy-safe loader and ABI observations.
Add owned RGB image-to-image inputs and binary-mask editing for the bounded macOS CPU SD 2.x full-VAE recipe.
Document and validate complete image checkpoint containers on macOS CPU, and add verified local Safetensors checkpoint sets with owned caching for the bounded SD-Turbo macOS CPU recipe.
Add CPU state-persistence validation with cold KV restore, reduced-work controls, typed rejection and recovery checks; native restores reject token capacities outside 1..context size with LlamaArgumentException.
Keep default WebGPU bridge assets at v0.1.73 (unchanged from 0.16.4).

Web runtime baseline (unchanged): upstream llama.cpp v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab, native v0.7.0-2; Web LiteRT-LM @litert-lm/core@0.18.0. Bridge manifest SHA-256: f89c41d151b7df0cbe4f5e97f95563e31e0968c13e65a630e4e0dba28adb1d44.

0.16.4#

Opt into ordered system/user/assistant text history on LiteRT-LM Web with LlamaBackend.withTextHistory(); the chat app adopts it, and unavailable controls fail explicitly (#1152).
Keep default WebGPU bridge assets at v0.1.73 (unchanged from 0.16.3).

0.16.3#

Recommend the corrected Qwen3-0.6B LiteRT-LM model bundle for native CPU to preserve split-token emoji; replace the older model artifact rather than relying on a runtime upgrade.
Inspect desktop setup with collectLlamaRuntimeSetupReport() and dart run llamadart:doctor [--json]. Reports separate pinned publication policy from unchecked native loading, GPU/backend selection and model behavior. Exit 0 means report generation, not runtime validation (#124).
Set reproducible dates in Dart-rendered chat templates with TemplateTimeChatSession and explicit clocks for strftime_now (#859).
Rerank documents with LlamaReranker on native rank-pooled GGUF models, preserving classifier scores, labels and document indices (#323).
Generate repeating textures with independent X/Y axes through SeamlessImageGenerationRequest; the initial SDXS embedded-tiny-decoder recipe is qualified at 512×512 on CPU and Metal (#1234).
Add opt-in spatial VAE tiling with TiledImageGenerationRequest and ImageVaeTiling, qualified on SD-Turbo full VAE/TAESD with macOS CPU and Metal. Decoder buffers are about one quarter of untiled size, with longer decoding; other SD 2.x checkpoints remain unmeasured, and unsupported families, backends and circular-axis combinations are refused (#1231).
Control retained image prompt conditioning with ImageConditioningCacheParams; zero disables reuse, while ordinary settings preserve the runtime default.
Add opt-in EmbeddingGemma-2 query/document/title helpers and normalized 768/512/256/128-dimensional outputs. Existing raw embedding calls are unchanged (#1126).
Fix explicit Web GGUF GPU placement with splitMode: ModelSplitMode.none and negative mainGpu; unavailable or inactive GPUs still throw (#915).
Cancel queued Web generation, speech-to-text or speech-synthesis tasks without stopping another request running on the bridge (#1371).
Keep default WebGPU bridge assets at v0.1.73 (unchanged from 0.16.2).

0.16.2#

Update the default LiteRT-LM native runtime to leehack/litert-lm-native@v0.18.1-4. Windows x64 apps no longer need the Visual C++ Redistributable for LiteRT-LM, which now ships its own Visual C++ runtime DLLs (litert-lm-native#87); llama.cpp already did, so only the opt-in stable_diffusion runtime still needs it. On Linux and Windows, LiteRT-LM CPU inference uses the configured thread count instead of one thread: Qwen3 0.6B CPU decode is 3.3× faster on an 8-vCPU Linux arm64 host and 2.2× faster on a 4-vCPU Windows x64 host (litert-lm-native#89, #1375). Flutter Apple apps get it with llamadart_litert_lm_flutter 0.16.2.
Linux and Windows apps built with dart build cli, flutter build linux or flutter build windows load LiteRT-LM from their own bundle, so a copied or installed app no longer fails with Failed to load dynamic library 'package:llamadart/litert_lm_LiteRtLm' once the project's hook cache is out of reach (#1397).
LiteRT-LM native embedding loads throw LlamaUnsupportedException on Linux, Windows and Android x86_64 instead of failing at engine creation or crashing the process; they stay available on macOS arm64, iOS arm64 and Android arm64 (verified on a Galaxy S24 without SME and on arm64 emulators) (#1367).
Native LiteRT-LM reports length instead of stop when a reply reaches maxTokens or fills the context, and automatic sendWithTools/completeWithTools loops now run on it, stopping truncated without running tools on a cut-off reply. This needs a runtime whose stream_proxy_finish_reason_abi_version export returns 1 (litert-lm-native v0.18.0-3 or later, including the default pin); on older runtimes loops still throw LlamaUnsupportedException. LiteRtLmRuntimeClient adds supportsFinishReasons and lastFinishReason. Web LiteRT-LM is unchanged (#919).
Aligned default WebGPU bridge assets to v0.1.73: Web embed() and embedBatch() throw LlamaUnsupportedException for rank-pooled rerankers instead of returning an invalid vector built from classifier scores, and LlamaEmbeddingInputTooLongException for an input longer than the one micro-batch it must fit, instead of embedding only part of it or aborting the runtime, as native does (llama-web-bridge#194). Upstream v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab matches native v0.7.0-2, which the manifest records; Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: f89c41d151b7df0cbe4f5e97f95563e31e0968c13e65a630e4e0dba28adb1d44.
Add dart run llamadart:web_setup: it downloads the Web bridge files this version pins into web/webgpu_bridge/, checks each against the pinned release manifest and adds the loader to web/index.html; reruns keep current files, and --check verifies offline. It refuses a page that already loads the bridge, so remove a hand-written loader first. With its loader, a page whose files are missing fails the first model load at once instead of after 12 seconds (#855).
Native llama.cpp usage.promptTokens counts the tokens of an image or audio prompt, as llama-server does, instead of its positions, which M-RoPE vision models such as Qwen3.5 report far fewer of (#696).
Native llama.cpp frees each context's token batch and sampler chain, and the sampler chain of a running generation, when its worker isolate is killed, instead of leaking about 0.1 MiB per kill plus 12 bytes per vocabulary token during generation (1.8 MiB for Qwen2.5) (#1404).
Correct the LlamaVideoContent rejection text: on desktop it points to a projector without a vision encoder instead of saying video support is built out, and it names Android and iOS as the platforms whose artifacts lack video (#320).

0.16.1#

Update the default LiteRT-LM native runtime to leehack/litert-lm-native@v0.18.1-2: on SME2 CPUs (verified on Apple M4 and iPhone 16 Pro), CPU chat and audio embeddings are deterministic again (#1340) and Gemma 4 CPU decode is faster again (litert-lm-native#84; 1.56× on M4, 1.2× on iPhone 16 Pro), while its CPU prefill is 15–30% slower than v0.18.1. Output and speed on non-SME2 CPUs (Linux arm64, Windows x64) are unchanged. Image and audio embeddings shift slightly from core 0.16.0, so re-embed stored vectors of image or audio content; text-only vectors are unchanged. Flutter Apple apps get it with llamadart_litert_lm_flutter 0.16.1.
Aligned default WebGPU bridge assets to v0.1.72: Web repeat and presence penalties now count the prompt, as native does, so output for the same seed changes at the default penalty of 1.1, and tool chats (ToolChoice.auto) leave the prompt out, also as native does. In worker mode, Stop right after a generation starts now stops it, and a bare cancel() streams the text generated before the worker yields. Text containing U+0000 is no longer truncated in tokenize, embed, score and prompts; speech text containing it is rejected. Upstream v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab matches native v0.7.0-2, which the manifest records; Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 85075d7d278bcacab706b79cda65c4a1bf8098a90f1e9fdbd504c198b8c8b03a.
llama.cpp draftDflash and draftDspark generation throws LlamaArgumentException instead of aborting the process when a LiliCorr draft block (dflash.block_size, 16 when absent) does not fit one draft micro-batch: SpeculativeDecodingConfig.draftMicroBatchSize, or the inherited ModelParams.microBatchSize when it is unset (#1330, #1341).
Fix hexadecimal string escapes in embedded and custom chat templates, including Flint's template (#1335).
Replace a stale embedded LiteRT-LM runtime in the macOS build phase after a core upgrade instead of keeping the old dylib until a clean build.

0.16.0#

Add LlamaCppModelParams and LiteRtLmModelParams, ModelParams that take only the fields their runtime applies; the liteRtLm* arguments of ModelParams are deprecated in favor of LiteRtLmModelParams. Loads reject ModelParams fields the model's runtime would ignore, such as liteRtLm* fields on a GGUF model or preferMemory64 off the Web, and either type used for the other runtime, with LlamaUnsupportedException naming each field; LlamaBackend() also rejects these and LiteRT-LM platform gaps before any download when the model format is known (#858, #861).
Update the default LiteRT-LM native runtime to leehack/litert-lm-native@v0.18.1, built from upstream LiteRT-LM v0.18.1 with the ASR and stream finish-reason fixes of v0.18.0-3. Flutter Apple apps get it with llamadart_litert_lm_flutter 0.16.0. Native GPU embeddings still throw LlamaUnsupportedException because EmbeddingGemma 2 still crashes in the prebuilt WebGPU accelerator.
Update the opt-in stable_diffusion runtime to leehack/stable-diffusion-native@v0.3.0, built from stable-diffusion.cpp master-982-0518b34: Windows opens model files with non-ASCII paths, and the legacy pickle reader no longer rejects checkpoints above 2 GiB (stable-diffusion-native#23). Images keep the flash attention and direct convolution settings ImageModelParams selects instead of the runtime's new automatic per-module choice. Flutter Apple apps need llamadart_stable_diffusion_flutter 0.16.0; this core rejects earlier SD companions, whose sd_ctx_params_t is smaller.
Aligned default WebGPU bridge assets to v0.1.70: a character split across tokens streams once complete instead of as a wrong one, and detokenize returns U+FFFD for invalid or incomplete bytes. Upstream v0.7.0@620d2225cc436ab0b001e3200e590e3b286ef9ab matches native v0.7.0-2, which the manifest records; Web LiteRT-LM stays @litert-lm/core@0.18.0. Manifest SHA-256: 7ac4e6e88205396f7221cd4e9e863bdf284df3498ced58cae2ffa1840996a8bf.
Native LiteRT-LM applies GenerationParams.penalty (other than the default), presencePenalty and, in chat on bundles with a thinking channel such as Gemma 4, thinkingBudget; penalties count only the tokens of the current send, capabilities reports each control, and a value it cannot apply throws LlamaUnsupportedException instead of UnsupportedError (#1155).
Add SpeculativeDecodingConfig.draftMicroBatchSize (llama.cpp --spec-draft-n-ubatch) to give native llama.cpp speculative draft contexts a smaller micro-batch and compute buffer than the target; unset keeps the target's, and WebGPU and LiteRT-LM reject it (#1124).
Templates that open a thought in the prompt: Seed-OSS, MiniMax M2, Apriel 1.5, Xiaomi MiMo and EXAONE MoE return it as reasoning until its end tag, as llama.cpp does, and stream it once instead of as content or twice; DeepSeek V3, Seed-OSS, MiniMax M2, Apriel 1.5 and EXAONE MoE also reason before a ToolChoice.required tool call and end the thought, so the call is no longer parsed as reasoning and lost (#742).
Fix the EXAONE MoE tool-call grammar emitting the tool name unquoted, so constrained tool calls are valid JSON (#1312).
Tool schemas from ToolDefinition.fromJsonSchema may list in required a property that only additionalProperties describes, as JSON Schema allows; the tool-call grammar requires that key instead of rejecting the schema (#1090).
Fix tool-loop rollbacks on ChatSession subclasses whose addMessage stores a copy of a tool result, adds a note before it or keeps the history length fixed: the rollback could leave an orphaned tool result or note behind (#1268).
Remove the WEBGPU_BRIDGE_PATCH_SAFARI_COMPAT and WEBGPU_BRIDGE_MIN_SAFARI_VERSION options of scripts/fetch_webgpu_bridge_assets.sh and the chat app's Safari patching of the bridge: the default assets already contain both fixes, and overrides to older asset tags no longer get them.

Edit this page