Why the Fastest LLM Load Test Runs in Your Browser

The friction is the failure

The dirty truth of load testing AI endpoints is that the tooling friction is so high the test frequently never happens. The intent is there, someone knows they should load-test the new streaming feature before launch. Then they hit the wall: which auth does staging use? Where do I paste the token? Why does my quick browser-console script return a CORS error? Do I really have to deploy a proxy just to send traffic at our own API? Each obstacle is small; together they’re enough that “I’ll load-test it” quietly becomes “it looked fine in the demo, ” and the feature ships on vibes. The most expensive load test is the one whose setup cost meant it was skipped.

This isn’t a discipline problem; it’s an architecture problem. The dominant tools are script runners, k6, Locust, JMeter, that live outside the browser. That choice imposes a fixed setup cost on every test: a runtime to install, a script to author and maintain, secrets to wire in, and the cross-origin problem to solve. Powerful for a dedicated performance team running a planned campaign. Wildly disproportionate for an engineer who wants to answer “does this endpoint hold up at 50 concurrent users?” before lunch.

Why a web page literally cannot load-test your API

Start with the constraint everyone hits and few understand: a script running in an ordinary web page is forbidden from calling most cross-origin APIs. This is the same-origin policy, the foundational browser security mechanism. A page at app.example.com cannot freely make requests to api.example.com or to api.openai.com unless the target server opts in by sending the right CORS headers (Access-Control-Allow-Origin).

For anything with side effects, a POST with a JSON body and an Authorization header, i.e. exactly an LLM call, the browser first fires a preflight OPTIONS request, and only proceeds if the server explicitly approves the origin, method, and headers. The crucial part: the target server controls whether the call is allowed at all. Your production LLM endpoint, sensibly, does not send permissive CORS headers to arbitrary origins. So a page script trying to load-test it gets a wall of CORS errors before a single byte of real load is sent.

That’s why script-based tools route around the browser entirely (a server-side runner has no same-origin policy) or make you stand up a CORS proxy, an extra service that forwards requests and strips the policy. The proxy is itself a piece of infrastructure to deploy, secure, and keep alive, and it sits in the request path skewing the very latency you’re trying to measure. You’re now load-testing your proxy as much as your endpoint.

Why a browser extension is a different animal

Here is the architectural unlock. A browser extension is not a web page. When it declares host permissions in its manifest, its background context becomes a privileged origin for those hosts. Per Chrome’s extension networking documentation, an extension with host permissions can make cross-origin fetches to those hosts without being subject to the CORS restrictions that bind page scripts. No preflight wall, no Access-Control-Allow-Origin requirement, no proxy.

Two consequences follow, and both are exactly what a load test needs:

• No proxy, no CORS dance. The extension calls your real endpoint directly, so the latency you measure is the endpoint’s, not a proxy’s. The numbers are honest.
• It reuses your real session. Because it runs in your browser with your cookies and tokens, it authenticates exactly as you do. There’s nothing to copy out of DevTools, the request that the extension sends is the request your app would send, against the gated endpoint your users actually hit.

Streaming is native here, not bolted on

AI load testing has a second requirement most generic tools handle poorly: you must read the response as a stream to measure time-to-first-token and inter-token latency. The browser’s fetch() exposes the response body as a ReadableStream, which is the cleanest streaming primitive in any runtime, and it’s right there, no SSE library, no parser to wire up. The same loop that drives the request captures TTFT, ITL, and tokens/sec in one pass:

// Runs in an extension context, no CORS, real session, native streaming
async function loadTest(url, body, headers, concurrency, total) {
  const samples = [];
  async function one() {
    const t0 = performance.now();
    const res = await fetch(url, {            // extension: not blocked by CORS
      method: 'POST', headers, body: JSON.stringify(body),
      credentials: 'include',                 // reuse the live session
    });
    const reader = res.body.getReader();      // ReadableStream, streaming is native
    let ttft = null, tokens = 0, last = t0, gaps = [];
    while (true) {
      const { done, value } = await reader.read();
      if (done) break;
      const now = performance.now();
      if (ttft === null) ttft = now - t0;     // time to first token
      else gaps.push(now - last);             // inter-token latency
      last = now; tokens += countSSE(value);
    }
    const total_ms = performance.now() - t0;
    samples.push({ ttft, tps: tokens / (total_ms / 1000),
                   itl: avg(gaps), total_ms });
  }
  // fire `total` requests `concurrency` at a time
  await pool(concurrency, Array.from({ length: total }, () => one));
  return summarize(samples);                  // p50/p95/p99 TTFT, TPS, goodput
}

Try the cross-origin version of that loop from a normal page and the very first fetch dies on preflight. The extension context is what makes the honest version possible at all.

Where script runners still win, and where they don’t

This isn’t “extensions beat k6 at everything.” For a planned, massive, distributed campaign generating millions of requests from many machines, a server-side runner is the right tool and always will be. But that describes a small fraction of the load tests that should happen. The vast majority are the quick, frequent checks an engineer should run before merging or launching: “does my streaming endpoint hold TTFT under 50 concurrent users with my real auth?” For that, the common case, the friction of the script runner is precisely what guarantees the check gets skipped.

Compare the workflows honestly. The script-runner path: install runtime, write script, extract and inject token, solve CORS (proxy or config), run, parse output, keep the script alive as the API changes. The browser-native path: open the tab where you’re already logged in, set concurrency, click run, read p95 TTFT and goodput. One of these gets done on a busy Tuesday. The other gets a Jira ticket that ages out.

The metrics fall out for free

Because the extension reads the raw stream, the AI-specific metrics that generic HTTP tools struggle with come naturally: TTFT (covered in time to first token is your real SLA), goodput under SLO (goodput, not throughput), inter-token latency, tokens/sec, and per-request token cost. And because the workflow is so light, the same measurement drops straight into a CI load gate without a separate, heavyweight harness to maintain.

This is the entire design thesis of Hit: meet the engineer where they already are, logged into the app, in the browser, and remove every reason the load test wouldn’t get run. No proxy, because the extension isn’t bound by CORS. No token juggling, because it reuses the live session. No streaming parser to build, because ReadableStream is native. The fastest LLM load test runs in your browser not as a gimmick, but because the browser-extension runtime is the only one that erases the two problems, cross-origin and auth, that make every other approach slow to start.

The bottom line

Load tests don’t fail because the tool can’t generate enough traffic; they fail because the setup cost meant nobody ran them. A web page can’t load-test a cross-origin API at all, the same-origin policy and CORS preflight see to that, which forces script-based tools into proxies and out-of-browser runners that carry a fixed, test-killing setup tax. A browser extension with host permissions is a privileged context that bypasses CORS and inherits your real session, turning a half-day of plumbing into a 30-second run with native streaming metrics. The fastest test is the one that actually runs. Optimize for that, and you stop shipping AI features that were never load-tested at all.