Installation

Prerequisites#

• Dart SDK >= 3.10.7
• Flutter SDK >= 3.38.0 (if you build Flutter apps)
• Flutter iOS builds require a minimum deployment target of 16.4 or newer
• Flutter macOS builds require a minimum deployment target of 14.0 or newer

Apple deployment targets#

If you build a Flutter Apple app, set your app project deployment target before running the app. iOS needs 16.4 or newer; macOS needs 14.0 or newer. If your iOS app still uses CocoaPods, set the Podfile platform too.

platform :ios, '16.4'

In Xcode, set IPHONEOS_DEPLOYMENT_TARGET = 16.4 or MACOSX_DEPLOYMENT_TARGET = 14.0 for the relevant Runner configurations.

Add dependency#

dependencies:
  llamadart: ^0.8.6

For Flutter iOS/macOS apps that should link Apple XCFrameworks through Swift Package Manager, also add the runtime companion packages you need:

dependencies:
  llamadart: ^0.8.6
  llamadart_llama_cpp_flutter: ^0.0.5 # GGUF / llama.cpp
  llamadart_litert_lm_flutter: ^0.0.2 # .litertlm / LiteRT-LM

The companion packages are published independently from the packages/ subdirectories in the main llamadart repository.

Then resolve packages:

dart pub get
# or
flutter pub get

What happens on first run/build#

On the first dart run / flutter run for a native target, llamadart:

1. Detects platform and architecture.
2. Resolves matching runtime artifacts from leehack/llamadart-native and leehack/litert-lm-native.
3. Wires them into your app through native assets. Flutter iOS/macOS builds use SwiftPM-linked XCFrameworks when the matching companion packages are present.

No local C++ toolchain setup is required for consumers.

Optional native source and backend selection#

You can configure the native runtime source and backend modules per target in your pubspec.yaml:

hooks:
  user_defines:
    llamadart:
      # Optional. Defaults to llamadart's tested native runtime pin.
      # Use a leehack/llamadart-native release tag when testing another build.
      llamadart_native_tag: b9776

      # Optional. GitHub repository slug or github.com URL.
      llamadart_native_repository: leehack/llamadart-native

      # Optional. Takes precedence over GitHub downloads when set.
      # Relative paths are resolved from the pubspec defining this config.
      # llamadart_native_path: ./native-bundles

      llamadart_native_backends:
        platforms:
          android-arm64:
            backends: [vulkan]
            cpu_profile: full # default: full; use compact for baseline-only CPU
          linux-x64: [vulkan, cuda]
          windows-x64: [vulkan, cuda]

Module availability is platform/arch specific and tied to the selected native bundle tag. If llamadart_native_tag points at a release without a matching bundle asset, the native-assets hook fails while downloading that asset. See Platform & Backend Matrix for the current per-target module list.

Native source overrides are for compatibility testing. They do not regenerate Dart FFI bindings or symbol lookups, so the selected binary still must be ABI- and symbol-compatible with the default leehack/llamadart-native@b9776 runtime.

Available native tags are published on the leehack/llamadart-native releases page. You can also list them with the GitHub CLI:

gh release list --repo leehack/llamadart-native --limit 20

Before overriding, confirm the release includes the asset for your target. The hook downloads files named llamadart-native-<bundle>-<tag>.tar.gz, for example llamadart-native-windows-x64-b9776.tar.gz. For local testing, llamadart_native_path may point directly at a bundle archive, at an extracted bundle directory, or at a directory containing <tag>/<bundle>/, <bundle>/, or the expected archive file.

For android-arm64, CPU variant policy is configurable:

• cpu_profile: full (default) includes all Android ARM CPU variants.
• cpu_profile: compact keeps baseline CPU variant only.
• cpu_variants: [...] (advanced) selects exact variants and overrides profile.

Canonical cpu_variants values:

• android_armv8.0_1
• android_armv8.2_1
• android_armv8.2_2
• android_armv8.6_1
• android_armv9.0_1
• android_armv9.2_1
• android_armv9.2_2

Key differences:

• android_armv8.2_1: DOTPROD
• android_armv8.2_2: DOTPROD + FP16_VECTOR_ARITHMETIC
• android_armv9.2_1: DOTPROD + FP16_VECTOR_ARITHMETIC + MATMUL_INT8 + SVE + SME
• android_armv9.2_2: android_armv9.2_1 + SVE2

Selection precedence:

1. cpu_variants (if present and valid)
2. cpu_profile
3. default cpu_profile: full

If requested modules are unavailable for a target, llamadart falls back to safe defaults and logs warnings.

Verify installation quickly#

Run a minimal script that loads a GGUF model and generates 1 token:

dart run your_app.dart

If the runtime initializes and model loads successfully, your setup is complete.