Step-by-step setup guide for running AI models directly on your hardware GPU inside the browser.
WebLLM is a cutting-edge engine that executes Large Language Models entirely inside the web browser utilizing your graphics card (GPU) through the standardized WebGPU API.
WebGPU performance depends on available GPU video memory (VRAM):
| Model Parameter Size | Recommended VRAM | Sample Models in Catalog |
|---|---|---|
| 0.5B – 1.5B (Lightweight) | 1.5 GB – 2.5 GB | Qwen2.5-0.5B, SmolLM2-1.7B, Qwen2.5-1.5B |
| 3B – 4B (Balanced) | 3 GB – 4.5 GB | Llama-3.2-3B, Phi-3.5-mini |
| 7B – 8B (High Quality) | 6 GB – 8 GB | Llama-3.1-8B-Instruct, Mistral-7B-Instruct |
Chromium-based browsers provide top-tier performance and reliability with WebGPU.
Go to Settings > System and ensure "Use hardware acceleration when available" is enabled.
chrome://settings/system
Paste into your address bar, set the flag to Enabled, and relaunch your browser:
chrome://flags/#enable-unsafe-webgpu
On Linux (especially when running Wayland or dedicated NVIDIA/AMD GPUs), enable this setting to connect WebGPU compute directly to the browser window without overloading the CPU:
chrome://flags/#force-enable-webgpu-interop
Open the internal graphics status page and verify that WebGPU is listed as "Hardware accelerated":
chrome://gpu
WebGPU is in active rollout in Firefox and requires enabling specific preferences:
Type into the address bar and click "Accept the Risk and Continue":
about:config
Search for this preference and toggle its value to true:
dom.webgpu.enabled
Allows Firefox to use WebGPU even if your driver is not yet on the official whitelist:
gfx.webgpu.ignore-status
Review the Graphics > WebGPU section to confirm hardware adapter detection:
about:support
When configuring a profile with the WebLLM provider, next to the model selector you will find a gear icon button that reveals the advanced performance panel. These dropdown selectors allow you to fine-tune VRAM usage and inference speed according to your GPU capabilities:
context_window_size): Dropdown selector for the maximum token capacity in the KV Cache (e.g., 2048, 4096, 8192, 16384, 32768). By default, browser builds often limit this to 2048 or 4096 to prevent out-of-memory errors on integrated or mobile GPUs. If your GPU has ample VRAM, increasing this retains longer conversations and large documents without truncation.
prefill_chunk_size): Dropdown selector for the number of tokens processed concurrently by the GPU during prompt evaluation (512, 1024, 2048, 4096). Higher values take full advantage of dedicated GPUs, reducing time to first token (TTFT).
The first choice in each dropdown is "Model default" (default), instructing ZeroChat to preserve the model's standard default configuration. These settings are stored as part of the connection profile.
If you wish to undo the technical changes made to your browser or ZeroChat and return to factory defaults, follow the instructions below:
Navigate back to the flag URL and select Default from the dropdown menu:
chrome://flags/#enable-unsafe-webgpu
Click the Relaunch button that appears at the bottom of the screen to restart the browser with default settings.
If you adjusted multiple parameters and want to return everything to out-of-the-box defaults, open the main flags page and click the "Reset all" button in the upper-right corner:
chrome://flags
In chrome://settings/system, you can toggle "Use hardware acceleration when available" according to your standard preference.
Type into your address bar and accept the warning:
about:config
Search for each modified preference (they will be displayed in bold, indicating a custom non-default value):
dom.webgpu.enabled (factory default: false)gfx.webgpu.ignore-status (factory default: false or non-existent)Click the Reset button (the curved back-arrow icon at the right edge of the row) or double-click the row so it returns to false and is no longer bold.
Close and restart Firefox so that graphics and GPU subsystems reinitialize under default parameters.
Local WebLLM models are stored in the browser's persistent storage (Cache Storage / IndexedDB) and can take up several gigabytes. If you want to delete them to reclaim disk space:
webllm/model entries, and click Delete.ZeroChat executes WebLLM through an in-memory self-contained classic Web Worker, ensuring full compatibility in both Google Chrome and Mozilla Firefox in a purely static web setup (without requiring local build steps or special browser flags). The JavaScript runtime is persistently cached in Cache Storage and automatically refreshed every 7 days when an active internet connection is available (with seamless offline fallback when disconnected).
Settings → Clear all data deletes storage for the current origin, including RAG, attachments, downloaded models, IndexedDB, Cache Storage and browser-private files. Close other tabs before confirming. Deletion is irreversible and affects applications sharing the origin. Repeat it for each HTTP or HTTPS origin you used. It does not delete files downloaded to your computer, HttpOnly cookies or the browser's internal HTTP cache.
If preparation fails, the model row displays the error detail received from WebLLM. Copy the complete message to diagnose hardware resource access and GPU WebGPU acceleration.