WebGPU Bridge

On this page

Web mode uses an external JavaScript bridge runtime consumed by llamadart.

Ownership#

Bridge source and build: leehack/llama-web-bridge
Published bridge assets: leehack/llama-web-bridge-assets
This repository consumes those artifacts

Runtime load order#

example/chat_app/web/index.html uses local-first loading on localhost for development validation, and CDN-first loading for normal hosted deployments:

On localhost: local asset first, then CDN fallback
On hosted deployments: CDN asset first, then local fallback

Fetch pinned local assets with:

WEBGPU_BRIDGE_ASSETS_TAG=v0.1.10 ./scripts/fetch_webgpu_bridge_assets.sh

Compatibility and safeguards#

Web backend remains experimental.
CPU fallback is available through bridge runtime routing.
Safari compatibility guard and fallback behavior are integrated in this repo.
Legacy bridge assets may be forced to CPU in Safari when GPU layers are requested.

Large single-file web model loading requires a cross-origin isolated page:

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp (or credentialless)
Runtime check: window.crossOriginIsolated === true

Runtime overrides#

You can override bridge asset source/version before loader startup:

<script>
  window.__llamadartBridgeAssetsRepo = 'leehack/llama-web-bridge-assets';
  window.__llamadartBridgeAssetsTag = 'v0.1.10';
  // Optional knobs:
  // window.__llamadartBridgeEnableMem64 = false;
  // window.__llamadartBridgeAllowAutoRemoteFetchBackend = false;
  // window.__llamadartBridgeRemoteFetchChunkBytes = 4 * 1024 * 1024;
  // window.__llamadartBridgeThreadPoolSize = 2;
  // window.__llamadartBridgeBootstrapVerbose = true;
</script>

Contract reference#

Bridge contract details (global shape, required methods, compatibility targets):

doc/webgpu_bridge.md

Edit this page