Running a Local LLM in the Browser with Wllama

I have been curious about what it would feel like to move an LLM completely into the browser.

No backend.

No API bill.

No server-side inference hiding behind the UI.

Just the user’s machine, a browser tab, WebAssembly, and a local model file.

This is my attempt to understand that workflow using @wllama/wllama inside a Next.js app. I spend a lot of time building interfaces with Next.js, Tailwind, and Framer Motion, so the idea of adding a private, local AI layer directly into the frontend felt too interesting not to try.

The dream is simple: download a GGUF model, run it locally in the browser, stream tokens into React state, and keep everything client-side.

The reality was a little more painful — but also pretty magical once it worked.

Why Wllama?

Wllama lets you run GGUF models in the browser using WebAssembly. In my case, I was looking at small-ish quantized models such as:

• Gemma-style GGUF models
• Llama 3.2 1B Instruct GGUF

The browser is not a free infinite GPU, of course. Model size matters a lot. Even quantized models can still be huge. A model around 1B–2B parameters may still mean downloading gigabytes or close to it depending on the quantization.

But the upside is very compelling:

• Inference can happen locally.
• User prompts do not need to leave the browser.
• There is no backend inference server to maintain.
• After caching, reloads can be much faster.

For privacy-first experiments, offline-capable tools, and small local assistants, this feels like a very promising direction.

The First Problem: Next.js and Browser-Only APIs

The first thing I ran into was the usual Next.js reality: server-side rendering does not like browser-only libraries.

Wllama depends on browser APIs such as:

• window
• Worker
• WebAssembly-related browser runtime behavior

If I imported it normally at the top of a component, the build could fail because Next.js may try to evaluate it in a server environment.

The fix was to dynamically import Wllama only on the client:

const { Wllama } = await import('@wllama/wllama');

This was one of those small but important reminders: if a library assumes the browser exists, do not let Next.js touch it during SSR.

The Second Problem: WASM Path Configuration

One of the first errors I hit was:

"default" is missing from pathConfig

That error was confusing at first, but the meaning was simple enough: Wllama needs to know where its compiled WASM files are.

I had to explicitly point it to the browser runtime files, for example from unpkg:

const pathConfig = {
  default: 'https://unpkg.com/@wllama/wllama/esm/wasm/wllama.wasm',
  'multi-thread/wllama.worker.mjs':
    'https://unpkg.com/@wllama/wllama/esm/multi-thread/wllama.worker.mjs',
};

Without this, the engine does not know where to fetch the WASM binary or worker module from.

The Third Problem: Model Size and Caching

Running models locally in the browser sounds clean until you remember that the model still has to get to the browser somehow.

A quantized GGUF file can still be very large. That means the first load can be expensive, especially if the model is hundreds of megabytes or more.

The browser Cache API becomes important here. The idea is that after the model chunks are downloaded once, later loads can come from local disk cache rather than the network.

That does not make the first experience cheap, but it changes the second experience dramatically.

My main takeaway here:

Browser-local AI is not just about inference. It is also about model delivery, storage, and cache strategy.

The Error That Looked Scary

Another error I ran into was:

[json.exception.out_of_range.403] key 'prompt' not found

I also saw variations like:

RangeError: Invalid typed array length

These felt like deep C++/WASM crashes bubbling up into JavaScript. In my case, the issue came from using the wrong shape of API call.

Wllama V3 moved toward an OpenAI-style completion API. I was initially trying to pass older-style arguments such as plain strings or n_predict, but the newer API expected a configuration object with fields like:

• prompt
• max_tokens
• stream
• onData

The working mental model became: treat completion calls more like an OpenAI-compatible request object.

The React Strict Mode “Double Typing” Bug

This one was subtle and kind of funny after the fact.

At one point, streamed output looked like this:

lliikkee  tthhiiss

The cause was how I was updating React state while streaming tokens.

I was appending text chunks directly by mutating the last message object:

last.content += chunk;

React Strict Mode can run updates in a way that exposes unsafe mutations. Because I was mutating the existing object reference, streamed chunks could appear duplicated.

The fix was to clone both the array and the message object every time:

setMessages((prev) => {
  const updated = [...prev];
  const lastIndex = updated.length - 1;

  updated[lastIndex] = {
    ...updated[lastIndex],
    content: updated[lastIndex].content + text,
  };

  return updated;
});

This felt like a good reminder that streaming UI is where sloppy state updates become very visible.

The Pattern That Worked for Me

Here is the cleaned-up core pattern I ended up with. This is not a full app, but it captures the important pieces:

import { useRef, useState } from 'react';

const pathConfig = {
  default: 'https://unpkg.com/@wllama/wllama/esm/wasm/wllama.wasm',
  'multi-thread/wllama.worker.mjs':
    'https://unpkg.com/@wllama/wllama/esm/multi-thread/wllama.worker.mjs',
};

const modelUrl =
  'https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF/resolve/main/Llama-3.2-1B-Instruct-Q4_K_M.gguf';

export default function LocalChat() {
  const [messages, setMessages] = useState([]);
  const wllamaRef = useRef(null);

  async function initEngine() {
    // Dynamic import prevents Next.js SSR issues.
    const { Wllama } = await import('@wllama/wllama');

    if (!wllamaRef.current) {
      wllamaRef.current = new Wllama(pathConfig);
    }

    await wllamaRef.current.loadModelFromUrl(modelUrl, {
      parallelDownloads: 1,
      progressCallback: (progress) => {
        console.log(`Loading: ${progress}%`);
      },
    });
  }

  async function generateResponse(userText) {
    if (!wllamaRef.current) {
      throw new Error('Wllama engine is not initialized yet.');
    }

    setMessages((prev) => [
      ...prev,
      { role: 'user', content: userText },
      { role: 'assistant', content: '' },
    ]);

    const prompt = `<|start_header_id|>user<|end_header_id|>\n\n${userText}<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n`;

    await wllamaRef.current.createCompletion({
      prompt,
      max_tokens: 400,
      stream: true,
      onData: (chunk) => {
        const text =
          chunk.choices?.[0]?.text ||
          chunk.choices?.[0]?.delta?.content ||
          '';

        if (!text) return;

        setMessages((prev) => {
          const updated = [...prev];
          const lastIndex = updated.length - 1;

          updated[lastIndex] = {
            ...updated[lastIndex],
            content: updated[lastIndex].content + text,
          };

          return updated;
        });
      },
    });
  }

  return null;
}

The important pieces are:

• Dynamically import @wllama/wllama.
• Explicitly provide the WASM path configuration.
• Load the GGUF model from a URL.
• Use the newer completion object shape with prompt, max_tokens, and stream.
• Treat streaming React state immutably.

What I Learned

This experiment made browser-local AI feel both closer and harder than I expected.

The exciting part is obvious: seeing tokens stream into a React UI while knowing there is no server doing the generation is genuinely fun. It feels like the browser has become a small local runtime for intelligence.

The hard parts are also obvious:

• WASM setup can be fragile.
• Next.js SSR needs to be handled carefully.
• Model files are large.
• Caching matters a lot.
• Streaming state updates need to be very deliberate.
• API version changes can produce strange low-level errors.

Still, I like the direction. For certain use cases, local browser inference feels like a meaningful shift: private by default, offline-friendly, and surprisingly capable when paired with small models.

It is not frictionless yet. But when it works, it really does feel like developer magic 🙂

References

• Uploaded browser-local AI image
• Wllama WASM binary on unpkg
• Wllama multi-thread worker on unpkg
• Llama 3.2 1B Instruct GGUF model file on Hugging Face