llamadart: on-device LLMs for Flutter and Dart

llamadart is a Dart package for on-device inference. It runs GGUF models through llama.cpp and .litertlm bundles through LiteRT-LM, on Android, iOS, macOS, Linux, Windows and the web, behind one Dart API. Web support is experimental.

Who this is for#

• Flutter and Dart developers who want AI features that run on the user's device: prompts stay private, and there is no inference server or API key.
• Apps that must work offline. Native targets need no network once the model is on the device; on the web, the page fetches the runtime and model over the network, then runs inference in the browser.
• Teams that want one codebase across mobile, desktop and web instead of a separate SDK per platform.
• Tools that need an OpenAI-compatible HTTP endpoint backed by a local model: see the OpenAI-compatible server example.

What you can build#

• Streaming chat and text generation, with structured JSON output.
• Tool calling driven by the model's chat template.
• Embeddings for search and retrieval.
• Image and audio input with multimodal models.
• Speech to text and text to speech.
• Image generation from text prompts on native platforms, with the opt-in stable-diffusion.cpp runtime (Preview).
• Decision models that answer typed questions without generating text.
• Runtime LoRA adapters.

Engines#

• LlamaEngine: text generation, chat, embeddings and tool calling. ChatSession is the stateful chat wrapper over it.
• ImageGenerationEngine: image generation (Preview).
• SpeechToTextEngine and TextToSpeechEngine: speech to text and text to speech.
• DecisionEngine: typed answers from decision models.

An engine reaches its runtime through LlamaBackend. The glossary defines these terms.

Where to go next#

• New to llamadart: Install it, then run the Quickstart.
• Building a Flutter app: follow the Flutter chat app tutorial.
• Choosing a model: Finding models and Model families.
• Answering from your own documents: the RAG tutorial.
• Shipping: the support matrix, performance tuning and troubleshooting.
• Upgrading: the upgrade checklist.