Download and cache models

On native targets, LlamaEngine.load and setModel download a remote ModelSource into a package-managed cache, verify it, and load the cached local file. Later loads reuse the cached file without a network request. Loading itself is covered in Model lifecycle.

Show download progress in an app#

ModelDownloadController wraps any ModelDownloadManager and emits UI-ready snapshots: a stage, a progress fraction, cancel and retry, and an error message with URL userinfo, query strings and fragments redacted, including in redirect failures and invalid-source diagnostics. Cancellation messages also remove known URL secrets repeated in the model filename. It does not depend on Flutter.

final controller = ModelDownloadController(
  manager: DefaultModelDownloadManager.auto(
    appPrivateCacheDirectory: appCacheModelsDirectory,
  ),
);

final subscription = controller.snapshots.listen((snapshot) {
  switch (snapshot.stage) {
    case ModelDownloadTaskStage.checkingCache:
      print('Checking cache for ${snapshot.source?.displayName}');
      break;
    case ModelDownloadTaskStage.downloading:
      final percent = snapshot.fraction == null
          ? 'unknown'
          : '${(snapshot.fraction! * 100).toStringAsFixed(1)}%';
      print('Downloading $percent');
      break;
    case ModelDownloadTaskStage.ready:
      print('Ready at ${snapshot.entry?.filePath}');
      break;
    case ModelDownloadTaskStage.failed:
      print(snapshot.errorMessage);
      break;
    case ModelDownloadTaskStage.cancelled:
      print('Cancelled; retry is available: ${snapshot.canRetry}');
      break;
    default:
      break;
  }
});

try {
  final entry = await controller.start(
    ModelSource.parse('hf://owner/repo/model-Q4_K_M.gguf'),
    options: ModelLoadOptions(maxRetries: 3),
  );
  await engine.setModel(LlamaModel(ModelSource.path(entry.filePath)));
} catch (_) {
  if (controller.snapshot.canRetry) {
    // Wire this to a Retry button.
    await controller.retry();
  }
} finally {
  await subscription.cancel();
  await controller.dispose();
}

• Stages: idle, resolving, checkingCache, downloading, verifying, ready, failed, cancelled. The cache check only drives UI state; ready follows the manager's own ensureModel(...) validation and any checksum.
• cancel() requests cooperative cancellation; the pending start(...) then throws LlamaStateException and the stage becomes cancelled. retry() after failed or cancelled reuses the last source and options.
• The controller owns cancellation: call controller.cancel() and leave ModelLoadOptions.cancelToken unset, or start(...) throws.
• Passing the original ModelSource to LlamaEngine.load or setModel instead of entry.filePath loads the same cached file only when the engine uses the same download manager: pass it as store: ModelFileStore(downloadManager: manager). An engine with its own manager looks in its own cache root and downloads the file again.
• On web, pass a custom manager for browser storage; the default manager's operations throw LlamaUnsupportedException there.

Hugging Face hf:// references#

hf://owner/repo/path/to/file names one .gguf or .litertlm file and resolves to https://huggingface.co/owner/repo/resolve/<revision>/path/to/file with download=true. The revision defaults to main. The cache key uses the stable hf:// identity, not the resolved URL.

final main = ModelSource.parse(
  'hf://unsloth/Qwen3.5-0.8B-GGUF/Qwen3.5-0.8B-Q4_K_M.gguf',
);

final litert = ModelSource.parse(
  'hf://litert-community/gemma-4-E2B-it-litert-lm/gemma-4-E2B-it.litertlm',
);

final tagged = ModelSource.parse(
  'hf://owner/repo@v1.0.0/model-Q4_K_M.gguf',
);

// Use ?revision= when the revision contains `/`, such as PR refs.
final pullRequestRef = ModelSource.parse(
  'hf://owner/repo/model-Q4_K_M.gguf?revision=refs/pr/12',
);

// The same pieces, built from app state.
final source = ModelSource.huggingFace(
  repoId: 'owner/repo',
  revision: 'main',
  filePath: 'model-Q4_K_M.gguf',
);

Native targets pick the runtime from the downloaded file's header, so a URL without a model extension, such as https://example.com/download?id=42, still loads. Web targets route by the URL's extension: pass format: ModelFormat.liteRtLm for a LiteRT-LM URL without one. See How routing works.

Private and gated repositories#

Pass credentials through ModelLoadOptions, never in the source string:

final engine = await LlamaEngine.load(
  LlamaModel(ModelSource.parse('hf://owner/private-repo/model-Q4_K_M.gguf')),
  download: ModelLoadOptions(bearerToken: hfToken),
);

Bearer tokens and custom headers go only on download requests, and never to more than one origin: a model and projector on different hosts throw LlamaArgumentException when credentials are set. They are not part of ModelSource.canonicalKey, cache metadata or toString().

Signed HTTP(S) URLs differ: canonicalKey keeps the full URL, and cacheKey hashes it so distinct signed URLs stay distinct. Cache metadata and toString() redact the query string, fragment and userinfo, but do not log or persist canonicalKey for a signed URL.

Download options#

final cancelToken = ModelDownloadCancelToken();
final engine = await LlamaEngine.load(
  LlamaModel(ModelSource.url(Uri.parse('https://example.com/model.gguf'))),
  store: ModelFileStore(
    downloadManager: DefaultModelDownloadManager.auto(
      appPrivateCacheDirectory: appCacheModelsDirectory,
    ),
  ),
  download: ModelLoadOptions(
    cachePolicy: ModelCachePolicy.preferCached,
    sha256: '0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef',
    bearerToken: hfToken,
    cancelToken: cancelToken,
    resume: true,
    maxRetries: 3,
  ),
  onProgress: (progress) {
    final fraction = progress.fraction;
    if (fraction != null) {
      print('download progress: ${(fraction * 100).toStringAsFixed(1)}%');
    }
  },
);

ModelLoadOptions.defaults is preferCached, resume: true and maxRetries: 3.

cancelToken.cancel() is cooperative: the download stops at its next checkpoint and the load throws LlamaStateException, which is never retried. Other download failures are typically LlamaModelException; check cancelToken.isCancelled in the catch to tell a cancel from a failure.

Disposing the engine also cancels package-managed downloads from setModel and the deprecated loadModelSource, including ModelParams.loras adapters. The load promptly throws LlamaStateException, even if a custom resolver or download manager ignores cancellation, and no late result loads a model. The manager receives a linked token; disposal leaves the caller's model and adapter tokens unchanged. The underlying transfer stops at its next cancellation checkpoint and can retain a partial file for resume, but never promotes a cancelled download to a complete cached file.

download applies to every remote file of the load. A local ModelSource.path(...) takes only the cancel token and, when the model has no projector, sha256. ModelDownloadManager.ensureModel(...) called directly with a local source and a non-default cache policy, cacheDirectory, auth headers, resume: false or a non-default maxRetries throws LlamaUnsupportedException instead of ignoring the option.

Cache policies#

• preferCached (default): reuse a completed cache entry; otherwise download.
• refresh: download again and replace the cached file atomically.
• cacheOnly: throw without a network request when the entry is missing.
• noCache: download to a temporary entry that later loads do not reuse. Call remove(entry.cacheKey) or clear() when done, or use prune(...).

Choose the cache location#

Without configuration, LlamaEngine, ModelDownloadController and DefaultModelDownloadManager() cache models in a per-user shared cache on desktop and server, and in the app's cache directory on Android and iOS: the directory Flutter's getApplicationCacheDirectory() returns. It survives app updates and restarts, is not backed up, and the OS clears it only under storage pressure. llamadart finds it without Flutter, so no path_provider setup is needed.

To put every default download somewhere else, set a global directory once at startup, before the first model load. It is a static, so set it in each isolate that creates engines or managers:

// For example, Flutter's application support directory, which the OS never
// clears. Exclude it from backups yourself: models are large.
DefaultModelDownloadManager.globalCacheDirectory = p.join(
  (await getApplicationSupportDirectory()).path,
  'models',
);

DefaultModelDownloadManager.auto(...) keeps one call site for every platform: desktop and server use a per-user shared cache; Android and iOS use the app-private directory you pass, else the app cache directory.

// Desktop/server shared cache; app-private directory on Android/iOS.
final crossPlatformManager = DefaultModelDownloadManager.auto(
  appPrivateCacheDirectory: appCacheModelsDirectory,
);

// Desktop/server: per-user cache shared by llamadart apps.
final desktopManager = DefaultModelDownloadManager.sharedCache();

// Mobile: an app-private directory resolved by the app.
final mobileManager = DefaultModelDownloadManager.appPrivate(
  cacheDirectory: appCacheModelsDirectory,
);

// Android sharing across developers: only a directory the user granted.
final androidUserLibrary = DefaultModelDownloadManager.userSelected(
  cacheDirectory: userGrantedModelLibraryDirectory,
);

// iOS/macOS sharing between apps in the same App Group.
final appGroupLibrary = DefaultModelDownloadManager.appGroup(
  cacheDirectory: appGroupModelsDirectory,
);

To choose a different mobile directory in Flutter, resolve it with path_provider (getApplicationSupportDirectory() only when the app manages its backup policy) and pass it as appPrivateCacheDirectory, or pass androidAppPrivateCacheDirectory and iosAppPrivateCacheDirectory to resolve both up front. Only if the app cache directory cannot be found does the default fall back to Directory.systemTemp/llamadart/models, which the OS may clear, and log a one-time LlamaLogger warning.

DefaultModelDownloadManager.globalCacheDirectory, when set, replaces the default root on every platform; an explicit directory still wins over it.

• Pass namespace: 'your.namespace' to auto(...) or sharedCache(...) to replace the llamadart segment, or cacheDirectory to force a root.
• sharedCache() never invents a shared folder on Android or iOS: it throws without cacheDirectory. Android shares a model library only through a directory the user granted; iOS only within an App Group. Apps from unrelated iOS developers can load user-picked files but have no writable shared cache.
• DefaultModelDownloadManager(), which LlamaEngine uses when you pass no manager, uses the same defaults as auto(), but falls back to the system temp directory when no home or cache directory exists; auto() and sharedCache() report an error instead.
• On web, DefaultModelDownloadManager is a placeholder whose operations (ensureModel, list and the rest) throw LlamaUnsupportedException. Browser model caches are origin-scoped.

Inspect and clean the cache#

final manager = DefaultModelDownloadManager.auto(
  appPrivateCacheDirectory: appCacheModelsDirectory,
);

final cached = await manager.list();
final entry = await manager.get(
  ModelSource.parse('hf://owner/repo/model.gguf').cacheKey,
);
if (entry != null) {
  await manager.remove(entry.cacheKey);
}
await manager.prune(
  maxAge: const Duration(days: 30),
  maxBytes: 20 * 1024 * 1024 * 1024,
);
await manager.clear();

Large downloads on mobile#

• Show progress and a cancel control, download one large GGUF at a time, and ask users to keep the app open.
• Do not cancel on every lifecycle pause: Android and iOS may let a short screen lock or app switch continue, and an eager cancel guarantees a restart.
• Downloads are foreground Dart HTTP requests. If the OS suspends or kills the app, the request can fail; a later session resumes from the .part file when resume is possible (see Reference).
• For downloads that must continue in the background, implement ModelDownloadManager in the app or a platform package: an Android foreground service or system DownloadManager with a notification, or iOS background URLSession tasks.
• For device-level sharing, use userSelected(...) on Android only after the user grants every participating app the same directory (do not request All Files Access by default), and appGroup(...) on iOS only for apps in the same App Group. Apps from unrelated developers should accept user-picked files through the document picker and copy them into the app cache: the loaders take file paths, not content URIs or file descriptors.

Limits#

• hf:// names one file. A multimodal model's mmproj GGUF is a separate source; see Multimodal.
• Sharded GGUF files are not expanded. Pick a single-file GGUF.
• llamadart does not list repository files or pick a quantization. Copy the exact path from the repository's Files and versions tab.
• URL-loading web backends accept only unauthenticated preferCached loads. Auth headers, checksums, cancel tokens, other cache policies, cacheDirectory, resume: false and custom retries throw LlamaUnsupportedException there; use a native target for those. A ModelSource.path there is a URL relative to the document, or a blob: URL.

Reference: resume, locking and cache metadata#

Atomic writes. Downloads go to a .part file, which becomes the cached model only after the HTTP stream and any SHA-256 check succeed.

Resume. A retry or resume sends an HTTP Range request only when the partial file has a validator (ETag or Last-Modified); otherwise it restarts from byte zero. A server that answers a Range request with 200 OK also restarts it from byte zero.

Locking. Stable-cache downloads are serialized per cache entry within the process, including across DefaultModelDownloadManager instances that share a cache root. A same-entry caller waits for the active operation; cache-reusing policies then re-check the cache. Different entries download in parallel. Concurrent refresh calls each refresh in turn. noCache downloads are not coalesced. Cancelling a waiting caller takes effect after the active operation finishes and does not cancel it; cancelling the active download releases the lock so a later caller can retry or resume from a safe .part file.

Metadata. Each entry has a versioned metadata.json sidecar next to the model file. An entry is reused only when the sidecar matches the cache key, file name and file path, and the file matches the recorded length and any supplied or stored SHA-256. If the file is intact but the sidecar is missing, malformed or from an unsupported schema version, the manager rebuilds the sidecar without a network request, so cacheOnly survives metadata damage. A missing file or a failed length or checksum check counts as a cache miss, and cache-reusing policies download again.