Platform and backend support matrix

Use this page to check whether a runtime or feature works on a platform. LlamaBackend() picks the runtime from the model format: LiteRT-LM bundles run on LiteRT-LM, and GGUF files run on llama.cpp natively or on the WebGPU bridge in a browser. Native targets read the file header; web targets use the URL's extension or ModelSource.format (How routing works). To choose backend modules or leave a runtime out of the app, see Native runtime configuration.

At a glance#

On Windows, llama.cpp needs the latest Microsoft Visual C++ v14 Redistributable for the app's architecture (x64 or arm64), at least as new as the build tools of the bundled DLLs, which the bundles do not ship. Stock Windows Server lacks it, and the load error names the DLLs that failed to load.

Speech to text, text to speech and decision models are experimental. Image generation is a Preview: its API is experimental and may change, and it needs the opt-in stable_diffusion runtime, built for iOS 16.4 and macOS 13.3; x64 Linux and Windows CPUs need AVX2, FMA, F16C and BMI2, Windows needs the latest Microsoft Visual C++ v14 Redistributable (x64), and Android arm64 needs dot-product and fp16. The desktop models (SDXL-Lightning, FLUX.1-schnell, SD 3.5 Large Turbo, Z-Image-Turbo) need about 8 to 15 GiB and are validated on macOS Metal; NVIDIA Vulkan figures come from the native CLI. See Image generation. GGUF Qwen3-ASR accepts WAV, MP3 and FLAC; real-model checks cover all three on native macOS and in headless Chromium. LiteRT-LM ASR is CPU-only streaming recognition. Decision models run on llama.cpp and WebGPU only; on LiteRT-LM DecisionEngine.load and attach throw LlamaUnsupportedException. See Speech to text, Text to speech and Decision models.

Compute device#

ModelParams.device (ComputeDevice) applies to every runtime. auto keeps each runtime's default; cpu, gpu and npu run there or throw LlamaUnsupportedException, never on another device. See Choosing the device.

Under auto, gpuLayers: 0 or a CPU or BLAS preferredBackend selects the CPU on both llama.cpp and LiteRT-LM, and a GPU preferredBackend (Vulkan, Metal, CUDA, OpenCL or HIP) selects the LiteRT-LM GPU backend. On Linux x64 and Windows x64, the LiteRT-LM GPU backend is not CUDA; use device: cpu on hosts without a hardware Vulkan driver. Windows arm64 has no LiteRT-LM runtime. The deprecated liteRtLmBackend selector works until 1.0; an unavailable choice there throws LlamaModelException.

Features by runtime#

LlamaEngine.supportsVideo returns false on every runtime, and passing LlamaVideoContent throws LlamaUnsupportedException. Web state paths point into the bridge's WASMFS virtual filesystem; to keep state across reloads, export and import it in app code. Bridge assets v0.1.54+, the default pin among them, report the WebGPU LoRA, thinking-budget, presence-penalty, Min-P and speculative decoding capabilities; older assets report none of them. After a load, LlamaEngine.runtime names the runtime and LlamaEngine.capabilities reports the rows of this table for the loaded model: media input, embeddings, next-token scores, multi-turn chat, tools, structured output and grammars, sampling controls and speculative decoding. Guides: LoRA adapters, Tool calling, Performance tuning, Multimodal, Embeddings, Observing operations.

Known limitations#

• Native LiteRT-LM v0.18.0 iOS artifacts omit the Gemma FST constraint provider. Gemma 3/4 and FunctionGemma conversations must disable constrained decoding; Dart already does so. Ordinary generation, thinking and best-effort tool formatting remain available. Automatic tool loops and strict structured output remain unsupported.

• Native LiteRT-LM on iOS: iOS 16.4 is the declared deployment floor, not a tested one. The companion Swift package requires an iOS 16.4 app target and the v0.18.0 frameworks declare MinimumOSVersion 15.0, but nothing has been run on iOS 16.4. On a device, model load and generation have run only on an iPhone 16 Pro with iOS 18.3.2 (CPU and GPU); simulator evidence is iOS 26.4, CPU only. No other iOS version is device-verified (#831).

• Flutter macOS LiteRT-LM: the app bundle does not contain the LiteRT-LM runtime, with or without llamadart_litert_lm_flutter. An unsandboxed app run on the build machine from the project loads it from the hook cache; a sandboxed (the flutter create default), copied or distributed app throws Failed to load dynamic library 'libLiteRt.dylib' on the first CPU or GPU generation. Add the LiteRT-LM build phase (#1029).

• macOS x86_64: LiteRT-LM v0.18.0 does not load (litert-lm-native#72). A Flutter macOS app that depends on llamadart_litert_lm_flutter dies at launch on an Intel Mac or under Rosetta with symbol not found in flat namespace '_LiteRtLmGemmaModelConstraintProvider_Destroy'; a Dart program or an app without the companion gets that error from dlopen on first LiteRT-LM use. Build Flutter macOS apps arm64-only; see Intel Macs.

• iOS x86_64 simulator: no LiteRT-LM runtime is published. Apps that include LiteRT-LM must exclude the x86_64 simulator architecture; llama.cpp still builds for it.

• iOS simulator: llama.cpp on Metal returns incorrect output. On an arm64 simulator (iPhone 17 Pro, iOS 26.5, Apple silicon host) Qwen3.5 0.8B (Q4_K_M) generates garbage text on Metal and answers correctly on the CPU, and Qwen2.5 0.5B fails on Metal the same way (#950); the cause is not known. On an iOS simulator auto therefore runs llama.cpp on the CPU with 0 GPU layers. GpuBackend.metal and ComputeDevice.gpu still load on Metal there, and the load logs a warning: use them only to check a model on the simulator's Metal device. The x86_64 simulator gets the same CPU default and has not been run. iOS devices and macOS are unchanged: auto uses Metal.

• Linux x64 LiteRT-LM GPU needs a hardware Vulkan driver. On Mesa llvmpipe alone it answers a few prompts, then crashes the process; use cpu on such hosts (#572).

• Android LiteRT-LM on Adreno 750 (Galaxy S24): ComputeDevice.auto, the default, runs LiteRT-LM on the GPU, which cannot run Qwen3 0.6B. The adapter's 128 MiB storage-buffer binding limit is smaller than one of the model's weight buffers, so expect the same from any model and adapter with that mismatch. v0.17.0-8 and earlier generated wrong text there. With the pinned v0.18.0 GPU engine creation fails instead: on a Galaxy S24 the first use throws LlamaUnsupportedException when ComputeDevice.gpu was requested and LlamaModelException under auto. Gemma 4 E2B runs on that GPU. Load Qwen3 0.6B with ModelParams(device: ComputeDevice.cpu), which answers correctly on the same device (#553, upstream LiteRT-LM#3866).

• Windows x64 LiteRT-LM GPU needs litert-lm-native v0.17.0-6 or later, which bundles dxil.dll and dxcompiler.dll. The pinned runtime includes them. The pinned v0.18.0 GPU accelerator and sampler load webgpu_dawn.dll, which the hook bundles. Qwen3 0.6B ran on an NVIDIA L4 (Direct3D 12) through dart run and a dart build cli bundle; a Flutter Windows app has not been run with it.

• Android NPU depends on the device SoC, the .litertlm bundle and the LiteRT dispatch libraries (ModelParams.liteRtLmDispatchLibDir). If native LiteRT-LM cannot create an NPU engine, use cpu or gpu.

• Android llama.cpp Vulkan is experimental and device-dependent. auto runs llama.cpp on the CPU there; Vulkan runs only when the app asks for ComputeDevice.gpu or GpuBackend.vulkan. Known failures on the pinned runtime: on a Pixel 9 Pro (Mali-G715), a prompt longer than about 32 tokens evaluated in one batch returns wrong text, and a request with a grammar or tool call can then abort the process, confirmed so far with one model (Qwen3.5-0.8B Q4_0) on that one device (#948); a Galaxy A53 (Mali-G68) crashes while loading the model (#782); a Galaxy S24 (Adreno 750) crashes on the first generation (llamadart-native#79). The first of these is a llama.cpp defect at a Vulkan subgroup size of 16 (llama.cpp#28637), which llama.cpp v0.6.0 still has; the same code mis-tiles subgroup sizes below 8, which no device run has confirmed. On Android, llamadart reads the subgroup size of the Vulkan GPUs when it creates a context: on such a GPU, and on any GPU whose subgroup size it cannot read, the context decodes a text prompt in micro-batches of at most 8 tokens by default; on other GPUs it decodes the prompt in full micro-batches. On a Pixel 9 Pro, prompt evaluation with 8-token micro-batches ran at about the speed uncapped decoding did (about 34 tokens/s against about 30 tokens/s for a 59-token prompt); other GPUs are not measured. An app that has validated Vulkan on its target devices can set ModelParams.microBatchSize to choose the micro-batch size itself: an explicit value is used as given, and one above 32 brings the wrong text back on affected GPUs. The cap applies to text-prompt decoding only: it does not cover prompts with image or audio input, embeddings, decision models, text-to-speech, or speculative-decoding verification batches during generation. Those batches can exceed 32 tokens, so speculative decoding can still produce wrong output on an affected GPU. llama.cpp's Vulkan backend also needs Vulkan 1.2 from the Vulkan loader and the GPU driver. When the Vulkan GPU a load would run on reports an older version, ComputeDevice.gpu throws LlamaUnsupportedException naming the GPU and both versions, and ComputeDevice.auto loads the model on the CPU and logs a warning. A device that used to run Vulkan on a 1.1 driver now takes that path. When other GPUs of the load are usable, such as a discrete GPU beside an old one, the load leaves the old one out and logs a warning; a GPU llama.cpp would not select, like an integrated GPU beside a discrete one, changes nothing. Whether this is what crashes the Galaxy A53 is not confirmed. On Android, the exact Qualcomm vendor 0x5143, Adreno (TM) 750 driver 2150604839 has shader-compiler crashes and incorrect quantized results, including common Q4_K_M models. When this device is registered, GPU loads are refused even if another GPU was selected: projector and decision-head GPU selection can independently choose the affected device. ComputeDevice.auto loads on CPU with a warning, including when preferredBackend requests Vulkan; ComputeDevice.gpu throws LlamaUnsupportedException. CPU loads do not probe Vulkan. Detection needs the device-facts API in native v0.6.0-1 or later; an overridden runtime without it cannot enforce this guard. Other driver versions are unchanged. A shader crash workaround alone does not qualify this driver; re-enabling a model requires qualification of the final native artifact on that exact driver. See native issue #79.

  Keep auto or cpu on Android unless the app has validated Vulkan on its target devices.

• Some Vulkan drivers crash in the cooperative-matrix path; see GPU crash or device loss.

• WebGPU readiness depends on the browser, device, bridge assets and model size; see WebGPU bridge.

Pinned runtimes#

The native-assets hook currently pins llamadart-native tag v0.6.0-2 and litert-lm-native release v0.18.0 (lib/src/hook/native_release_pins.dart). Apps can override the llama.cpp release with llamadart_native_tag, which takes a vMAJOR.MINOR.PATCH, vMAJOR.MINOR.PATCH-N, bNNNN, bNNNN-N or bNNNN-llamadart.N tag; nightly cores and rebuild counters reject leading zeros. Build-hook overrides must always name an explicit tag; latest is limited to maintainer synchronization and header/binding regeneration. Keys, backend modules and local bundles: Native runtime configuration.

Apps load the bridge assets themselves; see Add the bridge to your app. Linux system libraries: Linux prerequisites.