On-device LLM inference for Flutter and Dart

Run LLMs on the device, from one Dart API

llamadart runs GGUF (llama.cpp) and LiteRT-LM models inside your app on Android, iOS, macOS, Linux, Windows and the web. Prompts stay on the device, there is no inference server or API key, and native apps work offline once the model is downloaded.

$ dart pub add llamadart

Get started Try the web demo API reference

bin/main.dart

import 'dart:io';

import 'package:llamadart/llamadart.dart';

Future<void> main() async {
  final engine = await LlamaEngine.load(
    LlamaModel(
      ModelSource.parse(
        'hf://unsloth/SmolLM2-135M-Instruct-GGUF/'
        'SmolLM2-135M-Instruct-Q2_K.gguf',
      ),
    ),
  );

  final chat = ChatSession(engine, systemPrompt: 'You are concise.');
  await for (final chunk in chat.create([
    const LlamaTextContent('What is quantization?'),
  ])) {
    stdout.write(chunk.text);
  }
  await engine.dispose();
}