Platform and backend support matrix

Use this page to check whether a runtime or feature works on a platform. LlamaBackend() picks the runtime from the model format: LiteRT-LM bundles run on LiteRT-LM, and GGUF files run on llama.cpp natively or on the WebGPU bridge in a browser. Native targets read the file header; web targets use the URL's extension or ModelSource.format (How routing works). To choose backend modules or leave a runtime out of the app, see Native runtime configuration.

All five public engine types share LlamadartEngine<C extends EngineCapabilities> for capability inspection and final disposal. This lifecycle interface does not change the runtime support below; see engine ownership.

Eager createTask / generateTask handles wrap the same native and Web text paths, with request-scoped subscription cancellation and observable terminal outcomes. They add no model/backend capability; see task ownership.

At a glance#

Desktop source-CLI diagnostics can opt into dart run llamadart:doctor_native --runtime llama_cpp --json (or litert_lm / stable_diffusion) to check a declared artifact in an isolated process. It checks selected symbols/static ABI markers, without device or model initialization. This adds no runtime/platform/model capability: production loader behavior, loaded release identity, emitted app assets and GPU/inference remain unchecked. See scope and limits.

On Windows, the llama.cpp and LiteRT-LM bundles ship the Visual C++ runtime DLLs they need, so they need no Visual C++ Redistributable. The opt-in stable_diffusion runtime does not ship them: machines that run it need the latest Microsoft Visual C++ v14 Redistributable (x64), which stock Windows Server lacks, and its load error names the DLLs that failed to load.

With the pinned bridge assets, Web speech-to-text and text-to-speech cancellation is request-scoped, including queued STT generation and TTS startup awaiting a capability probe.

Speech to text, text to speech and decision models are experimental. Image generation is a Preview: its API is experimental and may change, and it needs the opt-in stable_diffusion runtime. Per-platform requirements, validated models, memory figures and the bounded recipes for tiling, editing, ControlNet, hires fix, reference editing, seamless textures, module placement, the conditioning cache, FP8 weights and local checkpoint sets are in Image generation. On iOS, Metal VAE decoding requires an intrinsic SDXS tiny decoder and a selected runtime with the intrinsic tiny-decoder v1 context policy, which keeps automatic model refresh from selecting a full VAE. The pinned v0.3.0-3 runtime provides this policy. Full VAE and external-decoder Metal decoding remain refused; use ImageModelPlacement(vaeOnCpu: true) for Metal diffusion or ComputeDevice.cpu for other recipes. Tiling does not bypass this restriction; iPhone full-VAE watchdog avoidance remains unqualified (#1047).

GGUF Qwen3-ASR accepts WAV, MP3 and FLAC; real-model checks cover all three on native macOS and in headless Chromium. LiteRT-LM ASR is CPU-only streaming recognition. Decision models run on llama.cpp and WebGPU only; on LiteRT-LM DecisionEngine.load and attach throw LlamaUnsupportedException. See Speech to text, Text to speech and Decision models. Decision tasks stop preflight or discard an already-dispatched answer after the backend finishes; they do not add native interruption or support on LiteRT-LM.

Native Android Vulkan Qwen3.5 permits bounded ordinary text only on unsafe/unproven small-matmul subgroup facts: microBatchSize unset or 1..8; larger explicit values, media, speculation and embeddings are typed unsupported. CPU and complete safe-subgroup facts are exempt. See the exact predicate and qualification limits.

Compute device#

ModelParams.device (ComputeDevice) applies to every runtime. auto keeps each runtime's default; cpu, gpu and npu run there or throw LlamaUnsupportedException, never on another device. See Choosing the device.

The WebGPU bridge ignores negative mainGpu values, including with splitMode: ModelSplitMode.none; they do not select the CPU as they do in native llama.cpp. Explicit gpu still requires WebGPU after load.

Android LiteRT-LM apps targeting API 31+ need an optional libOpenCL.so manifest request on Android 12+; see Android setup. An accessible OpenCL library does not guarantee GPU initialization or correct model output. WebGPU buffer limits remain device-dependent; an automatic CPU retry is CPU execution and does not qualify a GPU path.

Under auto, gpuLayers: 0 or a CPU or BLAS preferredBackend selects the CPU on both llama.cpp and LiteRT-LM, and a GPU preferredBackend (Vulkan, Metal, CUDA, OpenCL or HIP) selects the LiteRT-LM GPU backend. On Linux x64 and Windows x64, the LiteRT-LM GPU backend is not CUDA; use device: cpu on hosts without a hardware Vulkan driver. Windows arm64 has no LiteRT-LM runtime. The deprecated liteRtLmBackend selector works until 1.0; an unavailable choice there throws LlamaModelException.

LlamaBackend() rejects a LiteRT-LM model with LlamaUnsupportedException before any download on macOS x86_64, Windows arm64 and the x86_64 iOS simulator, which have no LiteRT-LM runtime, and for an explicit device the platform's LiteRT-LM lacks, such as gpu on Linux arm64. The message names the ABI. This applies when ModelSource.format or the file name says .litertlm; a model known only from its file header is rejected when it loads.

await engine.checkLoad(model, params: params) returns the same verdict as a LlamaLoadCheckReport without downloading, reading a file or starting a runtime: unsupported with the exception the load would throw, deferred when only the file header can tell the format, unknown when the backend cannot tell, as with LlamaEngine(LiteRtLmBackend()), and otherwise supported, with the format, runtime and device the load would use. Like the setup report, it is policy: native libraries, browser assets, auto device selection, LoRA limits, download options and the file are checked by the load (backend selection).

Dart-side chat-template rendering accepts Jinja \xHH string escapes on native and web. Model architecture support depends on the pinned runtime; see chat templates.

Features by runtime#

Tool-result media are LlamaToolResultContent.content parts; LlamaEngineCapabilities.toolResultMedia reports them. Other formats, audio, video and URL-only images throw LlamaUnsupportedException, as do tool-role image/audio parts and media alongside LlamaToolResultContent under any role (tool-result media).

LlamaEngine.supportsVideo returns false on every runtime, and passing LlamaVideoContent throws LlamaUnsupportedException. Web state paths point into the bridge's WASMFS virtual filesystem; to keep state across reloads, export and import it in app code. The pinned bridge assets report the WebGPU LoRA, thinking-budget, presence-penalty, Min-P and speculative decoding capabilities; older assets report none of them (Older bridge assets). After a load, LlamaEngine.runtime names the runtime and LlamaEngine.capabilities reports the rows of this table for the loaded model: media input, embeddings, next-token scores, multi-turn chat, tools, structured output and grammars, sampling controls and speculative decoding. Guides: LoRA adapters, Tool calling, Performance tuning, Multimodal, Embeddings, Observing operations.

Known limitations#

Device, OS and driver limits on iOS, macOS, Android, Linux, Windows and the web are on Known platform issues.

Pinned runtimes#

The native-assets hook currently pins llamadart-native tag v0.8.0-3 and litert-lm-native release v0.19.0 (lib/src/hook/native_release_pins.dart).

The direct native client, service probes and speech worker share ABI-aware discovery. Public selection admits legacy runtimes and C API 1.0.x; unknown or future versions are refused before engine calls. The source pin adopts published native v0.19.0 (owner f173fe9b, upstream 83ce6067). The selected loader requires macOS 14+ before loading any core or companion library, including legacy libraries and custom paths. Final default-discovery and Apple embedding/signature checks remain in #1677; the source pin does not publish core or companion packages. Web stays @litert-lm/core@0.18.0. The Flutter companion's iOS 16.4 floor remains separate. Apps can override the llama.cpp release with llamadart_native_tag; accepted tag forms, keys, backend modules and local bundles: Native runtime configuration.

Apps load the bridge assets themselves; see Add the bridge to your app. Linux system libraries: Linux prerequisites.