Download and cache models

On native targets, LlamaEngine.load and setModel download a remote ModelSource into a package-managed cache, verify it, and load the cached local file. Later loads reuse the cached file without a network request. Loading itself is covered in Model lifecycle.

Show download progress in an app#

ModelDownloadController wraps any ModelDownloadManager and emits UI-ready snapshots: a stage, a progress fraction, cancel and retry, and an error message with URL userinfo, query strings and fragments redacted, including in redirect failures and invalid-source diagnostics. Cancellation messages also remove known URL secrets repeated in the model filename. It does not depend on Flutter.

final controller = ModelDownloadController(
  manager: DefaultModelDownloadManager.auto(
    appPrivateCacheDirectory: appCacheModelsDirectory,
  ),
);

final subscription = controller.snapshots.listen((snapshot) {
  switch (snapshot.stage) {
    case ModelDownloadTaskStage.checkingCache:
      print('Checking cache for ${snapshot.source?.displayName}');
      break;
    case ModelDownloadTaskStage.downloading:
      final percent = snapshot.fraction == null
          ? 'unknown'
          : '${(snapshot.fraction! * 100).toStringAsFixed(1)}%';
      print('Downloading $percent');
      break;
    case ModelDownloadTaskStage.ready:
      print('Ready at ${snapshot.entry?.filePath}');
      break;
    case ModelDownloadTaskStage.failed:
      print(snapshot.errorMessage);
      break;
    case ModelDownloadTaskStage.cancelled:
      print('Cancelled; retry is available: ${snapshot.canRetry}');
      break;
    default:
      break;
  }
});

try {
  final entry = await controller.start(
    ModelSource.parse('hf://owner/repo/model-Q4_K_M.gguf'),
    options: ModelLoadOptions(maxRetries: 3),
  );
  await engine.setModel(LlamaModel(ModelSource.path(entry.filePath)));
} catch (_) {
  if (controller.snapshot.canRetry) {
    // Wire this to a Retry button.
    await controller.retry();
  }
} finally {
  await subscription.cancel();
  await controller.dispose();
}

• Stages: idle, resolving, checkingCache, downloading, verifying, ready, failed, cancelled. The cache check only drives UI state; ready follows the manager's own ensureModel(...) validation and any checksum.
• cancel() requests cooperative cancellation; the pending start(...) then throws LlamaStateException and the stage becomes cancelled. retry() after failed or cancelled reuses the last source and options.
• The controller owns cancellation: call controller.cancel() and leave ModelLoadOptions.cancelToken unset, or start(...) throws.
• Passing the original ModelSource to LlamaEngine.load or setModel instead of entry.filePath loads the same cached file only when the engine uses the same download manager: pass it as store: ModelFileStore(downloadManager: manager). An engine with its own manager looks in its own cache root and downloads the file again.
• On web, pass a custom manager for browser storage; the default manager's operations throw LlamaUnsupportedException there.

Hugging Face hf:// references#

hf://owner/repo/path/to/file names one .gguf or .litertlm file and resolves to https://huggingface.co/owner/repo/resolve/<revision>/path/to/file with download=true. The revision defaults to main. The cache key uses the stable hf:// identity, not the resolved URL.

final main = ModelSource.parse(
  'hf://unsloth/Qwen3.5-0.8B-GGUF/Qwen3.5-0.8B-Q4_K_M.gguf',
);

final litert = ModelSource.parse(
  'hf://litert-community/gemma-4-E2B-it-litert-lm/gemma-4-E2B-it.litertlm',
);

final tagged = ModelSource.parse(
  'hf://owner/repo@v1.0.0/model-Q4_K_M.gguf',
);

// Use ?revision= when the revision contains `/`, such as PR refs.
final pullRequestRef = ModelSource.parse(
  'hf://owner/repo/model-Q4_K_M.gguf?revision=refs/pr/12',
);

// The same pieces, built from app state.
final source = ModelSource.huggingFace(
  repoId: 'owner/repo',
  revision: 'main',
  filePath: 'model-Q4_K_M.gguf',
);

Native targets pick the runtime from the downloaded file's header, so a URL without a model extension, such as https://example.com/download?id=42, still loads. Web targets route by the URL's extension: pass format: ModelFormat.liteRtLm for a LiteRT-LM URL without one. See How routing works.

Private and gated repositories#

Pass credentials through ModelLoadOptions, never in the source string:

final engine = await LlamaEngine.load(
  LlamaModel(ModelSource.parse('hf://owner/private-repo/model-Q4_K_M.gguf')),
  download: ModelLoadOptions(bearerToken: hfToken),
);

Bearer tokens and custom headers go only on download requests, and never to more than one origin: a model and projector on different hosts throw LlamaArgumentException when credentials are set. They are not part of ModelSource.canonicalKey, cache metadata or toString().

Signed HTTP(S) URLs differ: canonicalKey keeps the full URL, and cacheKey hashes it so distinct signed URLs stay distinct. Cache metadata and toString() redact the query string, fragment and userinfo, but do not log or persist canonicalKey for a signed URL.

Download options#

final cancelToken = ModelDownloadCancelToken();
final engine = await LlamaEngine.load(
  LlamaModel(ModelSource.url(Uri.parse('https://example.com/model.gguf'))),
  store: ModelFileStore(
    downloadManager: DefaultModelDownloadManager.auto(
      appPrivateCacheDirectory: appCacheModelsDirectory,
    ),
  ),
  download: ModelLoadOptions(
    cachePolicy: ModelCachePolicy.preferCached,
    sha256: '0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef',
    bearerToken: hfToken,
    cancelToken: cancelToken,
    resume: true,
    maxRetries: 3,
  ),
  onProgress: (progress) {
    final fraction = progress.fraction;
    if (fraction != null) {
      print('download progress: ${(fraction * 100).toStringAsFixed(1)}%');
    }
  },
);

ModelLoadOptions.defaults is preferCached, resume: true and maxRetries: 3.

cancelToken.cancel() is cooperative: the download stops at its next checkpoint and the load throws LlamaStateException, which is never retried. Other download failures are typically LlamaModelException; check cancelToken.isCancelled in the catch to tell a cancel from a failure. A cache directory that cannot be created throws LlamaModelException naming the directory. A TLS certificate or handshake failure throws LlamaModelException saying so, and is not retried.

Disposing the engine also cancels package-managed downloads from setModel, loadMultimodalProjectorSource and the deprecated loadModelSource, including ModelParams.loras adapters. The load promptly throws LlamaStateException, even if a custom resolver or download manager ignores cancellation, and no late result loads a model. The manager receives a linked token; disposal leaves the caller's model and adapter tokens unchanged. The underlying transfer stops at its next cancellation checkpoint and can retain a partial file for resume, but never promotes a cancelled download to a complete cached file.

download applies to every remote file of the load. A local ModelSource.path(...) takes only the cancel token and, when the model has no projector, sha256, and so does a remote source that a custom ModelResolver maps to a local file, such as an offline mirror. ModelDownloadManager.ensureModel(...) called directly with a local source and a non-default cache policy, cacheDirectory, auth headers, resume: false or a non-default maxRetries throws LlamaUnsupportedException instead of ignoring the option.

Cache policies#

• preferCached (default): reuse a completed cache entry; otherwise download.
• refresh: download again and replace the cached file atomically.
• cacheOnly: throw without a network request when the entry is missing.
• noCache: download to a temporary entry that later loads do not reuse. Call remove(entry.cacheKey) or clear() when done, or use prune(...).

Choose the cache location#

Without configuration, LlamaEngine, ModelDownloadController and DefaultModelDownloadManager() cache models in a per-user shared cache on desktop and server, and in the app's cache directory on Android and iOS: the directory Flutter's getApplicationCacheDirectory() returns. It survives app updates and restarts, is not backed up, and the OS clears it only under storage pressure. llamadart finds it without Flutter, so no path_provider setup is needed.

To put every default download somewhere else, set a global directory once at startup, before the first model load. It is a static, so set it in each isolate that creates engines or managers:

// For example, Flutter's application support directory, which the OS never
// clears. Exclude it from backups yourself: models are large.
DefaultModelDownloadManager.globalCacheDirectory = p.join(
  (await getApplicationSupportDirectory()).path,
  'models',
);

DefaultModelDownloadManager.auto(...) keeps one call site for every platform: desktop and server use a per-user shared cache; Android and iOS use the app-private directory you pass, else the app cache directory.

// Desktop/server shared cache; app-private directory on Android/iOS.
final crossPlatformManager = DefaultModelDownloadManager.auto(
  appPrivateCacheDirectory: appCacheModelsDirectory,
);

// Desktop/server: per-user cache shared by llamadart apps.
final desktopManager = DefaultModelDownloadManager.sharedCache();

// Mobile: an app-private directory resolved by the app.
final mobileManager = DefaultModelDownloadManager.appPrivate(
  cacheDirectory: appCacheModelsDirectory,
);

// Android sharing across developers: only a directory the user granted.
final androidUserLibrary = DefaultModelDownloadManager.userSelected(
  cacheDirectory: userGrantedModelLibraryDirectory,
);

// iOS/macOS sharing between apps in the same App Group.
final appGroupLibrary = DefaultModelDownloadManager.appGroup(
  cacheDirectory: appGroupModelsDirectory,
);

To choose a different mobile directory in Flutter, resolve it with path_provider (getApplicationSupportDirectory() only when the app manages its backup policy) and pass it as appPrivateCacheDirectory, or pass androidAppPrivateCacheDirectory and iosAppPrivateCacheDirectory to resolve both up front. Only if the app cache directory cannot be found does the default fall back to Directory.systemTemp/llamadart/models, which the OS may clear, and log a one-time LlamaLogger warning.

DefaultModelDownloadManager.globalCacheDirectory, when set, replaces the default root on every platform; an explicit directory still wins over it.

• Pass namespace: 'your.namespace' to auto(...) or sharedCache(...) to replace the llamadart segment, or cacheDirectory to force a root.
• sharedCache() never invents a shared folder on Android or iOS: it throws without cacheDirectory. Android shares a model library only through a directory the user granted; iOS only within an App Group. Apps from unrelated iOS developers can load user-picked files but have no writable shared cache.
• DefaultModelDownloadManager(), which LlamaEngine uses when you pass no manager, uses the same defaults as auto(), but falls back to the system temp directory when no home or cache directory exists; auto() and sharedCache() report an error instead.
• On web, DefaultModelDownloadManager is a placeholder whose operations (ensureModel, list and the rest) throw LlamaUnsupportedException. Browser model caches are origin-scoped.

Inspect and clean the cache#

final manager = DefaultModelDownloadManager.auto(
  appPrivateCacheDirectory: appCacheModelsDirectory,
);

final cached = await manager.list();
final entry = await manager.get(
  ModelSource.parse('hf://owner/repo/model.gguf').cacheKey,
);
if (entry != null) {
  await manager.remove(entry.cacheKey);
}
await manager.prune(
  maxAge: const Duration(days: 30),
  maxBytes: 20 * 1024 * 1024 * 1024,
);
try {
  await manager.clear();
} on ModelCacheEntryInUseException catch (error) {
  // Every other entry is gone; these are in use elsewhere.
  print('Kept ${error.entryDirectories.length} entries in use.');
}

remove and clear never wait for an entry in use: they remove the rest and then throw ModelCacheEntryInUseException naming the entries they kept. prune skips entries in use, and entries used after it listed them, and leaves them out of its result. See Locking for what counts as in use.

Large downloads on mobile#

• Show progress and a cancel control, download one large GGUF at a time, and ask users to keep the app open.
• Do not cancel on every lifecycle pause: Android and iOS may let a short screen lock or app switch continue, and an eager cancel guarantees a restart.
• Downloads are foreground Dart HTTP requests. If the OS suspends or kills the app, the request can fail; a later session resumes from the .part file when resume is possible (see Reference).
• For downloads that must continue in the background, implement ModelDownloadManager in the app or a platform package: an Android foreground service or system DownloadManager with a notification, or iOS background URLSession tasks.
• To share one copy of a model between apps that can open the same directory, see the separate shared model store package.
• For device-level sharing, use userSelected(...) on Android only after the user grants every participating app the same directory (do not request All Files Access by default), and appGroup(...) on iOS only for apps in the same App Group. Apps from unrelated developers should accept user-picked files through the document picker and copy them into the app cache: the loaders take file paths, not content URIs or file descriptors.
• A download into an App Group container holds a file lock there (see Locking). iOS terminates an app that is suspended while holding a lock in a shared container (0xdead10cc), so finish or cancel the download, or keep it running with a background task, before the app is suspended.

Limits#

• hf:// names one file. A multimodal model's mmproj GGUF is a separate source; see Multimodal.
• Sharded GGUF files are not expanded. Pick a single-file GGUF.
• llamadart does not list repository files or pick a quantization. Copy the exact path from the repository's Files and versions tab.
• URL-loading web backends accept only unauthenticated preferCached loads. Auth headers, checksums, cancel tokens, other cache policies, cacheDirectory, resume: false and custom retries throw LlamaUnsupportedException there; use a native target for those. A ModelSource.path there is a URL relative to the document, or a blob: URL.

Reference: resume, locking and cache metadata#

Atomic writes. Each download attempt writes its own <file>.<random id>.part file and renames it onto the model path only after the HTTP stream succeeds, the file's length matches the total the server reported (when it reported one for the decoded body; not for gzip-encoded responses) and any SHA-256 check passes. A shorter file is kept and resumed; a longer one is discarded and downloaded again. A failed or cancelled attempt leaves its bytes in <file>.part for the next resume.

Resume. A retry or resume sends an HTTP Range request only when the partial file has a validator (ETag or Last-Modified); otherwise it restarts from byte zero. A server that answers a Range request with 200 OK also restarts it from byte zero.

Locking. Stable-cache operations are serialized per cache entry across DefaultModelDownloadManager instances, isolates and processes that share a cache root, such as two apps using sharedCache(), apps granted the same userSelected(...) directory, or an iOS app and its extension in one appGroup(...) container. Each entry has an exclusive OS file lock in <cache root>/.locks/. A same-entry caller waits for the active operation; cache-reusing policies then re-check the cache, including the file's length and checksum, so a waiter reuses the finished download or downloads again. Different entries download in parallel.

remove, prune and clear take the same lock without waiting for it, so they never delete an entry another isolate or process is downloading or verifying, including from that download's onProgress callback. They do wait for each other: removals from one cache root take turns through <cache root>/.locks/.removal.lock, so two overlapping clear calls both succeed instead of reporting each other's entries in use. On Windows an entry whose model file a process has open or mapped, such as a loaded model, is also in use; on other platforms deleting it leaves the open file readable. A model file Windows denies deleting for good, such as a read-only file or one an ACL protects, throws that FileSystemException instead. Windows reports a mapped file and a denied delete with the same error, so llamadart tells them apart by opening the file for writing: a file that can be written but not deleted still counts as in use. noCache downloads take no lock, so clear removes one in progress and that download fails. clear keeps <cache root>/.locks/: deleting a lock file while another process waits on it would let two processes hold the same entry's lock. Its empty files are safe to delete only when no process uses the cache root. Concurrent refresh calls each refresh in turn. noCache downloads are not coalesced. In the same isolate, cancelling a waiting caller takes effect after the active operation finishes; a caller waiting on another isolate or process stops waiting at once. Neither cancels the active download. Cancelling the active download releases the lock so a later caller can retry or resume from <file>.part.

The OS releases the lock when the holding process exits, including after a crash or kill, and the next holder resumes from the largest part file left behind and deletes the others.

Only callers that take the lock are excluded from each other. Processes running llamadart 0.16 or older do not take it, and a manager whose cache root is not writable or whose file system refuses locks logs a warning once and falls back to serializing within its own isolate. Such a writer can download the same entry at the same time as a locked one, and a locked caller can resume from a part file that writer is still extending. A resume writes each byte at the offset its Range request named and only when the server's validator still matches, so both writers put the same bytes at the same offsets; any file longer than the server's total is discarded. The cost is a repeated download, not a corrupt entry. Without the lock, crashed attempts' part files also stay in place. Network file systems (NFS, SMB) are not supported for concurrent use from several machines; their locking depends on the server and client configuration.

Metadata. Each entry has a versioned metadata.json sidecar next to the model file. An entry is reused only when the sidecar matches the cache key, file name and file path, and the file matches the recorded length and any supplied or stored SHA-256. If the file is intact but the sidecar is missing, malformed or from an unsupported schema version, the manager rebuilds the sidecar without a network request, so cacheOnly survives metadata damage. A missing file or a failed length or checksum check counts as a cache miss, and cache-reusing policies download again.