A realtime chat UI streaming in Next.js has three jobs: relay the model's tokens as they arrive, let the user press Stop, and fail visibly when the stream breaks halfway. The first is easy. The second and third are where demos turn into bugs.
I built the whole path (route handler, SSE parser, client component) and tested it against a stub server that speaks the Anthropic streaming format. I did not have an API key, so nothing here was run against the live API. What I did run is labelled below. A bug I hit along the way, a stream that hangs forever, is the most useful thing in this post.
What does the architecture look like?
Browser (Chat.tsx) --POST /api/chat--> Route handler --POST stream:true--> Messages API
^ | |
|<------ plain text chunks ------------|<-------- SSE events --------------|
The route handler does not forward the raw event stream. It parses the vendor's server-sent events, keeps only the text, and sends the browser plain text. That keeps the vendor's wire format out of your client code, and means you can later swap providers by changing one file. If you route through a gateway, see streaming with a TypeScript gateway client.
What does the Messages API stream actually contain?
From Anthropic's streaming docs (read on 2026-10-31): set "stream": true and the response is server-sent events. A text answer is a message_start, then content blocks made of content_block_start, many content_block_delta events, and content_block_stop, then message_delta and a final message_stop. ping events can appear anywhere. An error event can appear mid-stream, and the docs' example is an overloaded_error. Text arrives as:
event: content_block_delta
data: {"type": "content_block_delta","index": 0,"delta": {"type": "text_delta", "text": "Hello"}}
The error case is the one that bites: the HTTP status was already 200 when it arrives, so a check on res.ok is not enough. Your code has to watch for the event.
The docs also show input_json_delta (tool calls) and thinking_delta (extended thinking). A basic chat UI ignores both; this code only forwards text_delta.
Step 1: a tiny SSE parser
Network chunks do not respect event boundaries. One chunk can end mid-line, or hold three events. So buffer until you see a blank line:
// lib/sse.ts
export type SseEvent = { event: string; data: string };
export class SseParser {
private buffer = "";
push(chunk: string): SseEvent[] {
this.buffer += chunk.replace(/\r\n/g, "\n");
const events: SseEvent[] = [];
let idx: number;
while ((idx = this.buffer.indexOf("\n\n")) !== -1) {
const raw = this.buffer.slice(0, idx);
this.buffer = this.buffer.slice(idx + 2);
let event = "message";
const data: string[] = [];
for (const line of raw.split("\n")) {
if (line.startsWith(":")) continue; // comment
if (line.startsWith("event:")) event = line.slice(6).trim();
else if (line.startsWith("data:")) data.push(line.slice(5).trimStart());
}
if (data.length > 0) events.push({ event, data: data.join("\n") });
}
return events;
}
}
Step 2: turn the vendor stream into a text stream
// lib/stream.ts
import { SseParser } from "./sse";
type StreamData = {
type: string;
delta?: { type: string; text?: string };
error?: { type: string; message: string };
};
export function textStreamFromAnthropic(
upstream: ReadableStream<Uint8Array>,
): ReadableStream<Uint8Array> {
const decoder = new TextDecoder();
const encoder = new TextEncoder();
const parser = new SseParser();
const reader = upstream.getReader();
return new ReadableStream<Uint8Array>({
async pull(controller) {
// pull() is not called again until something is enqueued, so keep
// reading until this call produces output (or the stream ends).
let enqueued = false;
while (!enqueued) {
const { done, value } = await reader.read();
if (done) {
controller.close();
return;
}
for (const ev of parser.push(decoder.decode(value, { stream: true }))) {
let data: StreamData;
try {
data = JSON.parse(ev.data) as StreamData;
} catch {
continue; // ignore malformed event
}
if (data.type === "error") {
controller.error(new Error(data.error?.type ?? "upstream_error"));
await reader.cancel();
return;
}
if (data.type === "content_block_delta" && data.delta?.type === "text_delta" && data.delta.text) {
controller.enqueue(encoder.encode(data.delta.text));
enqueued = true;
}
}
}
},
cancel(reason) {
return reader.cancel(reason); // browser pressed Stop or closed the tab
},
});
}
The bug that hung my first test
My first version of pull() read one upstream chunk and returned. When that chunk held only a ping or half an event, nothing was enqueued, and the stream never called pull() again. The test sat there until I killed it. A ReadableStream calls pull() again only after you enqueue something, so a pull() that produces no output must keep looping. That is the while (!enqueued) loop above, and it is easy to miss because a happy-path demo with tidy chunks works fine without it.
Step 3: the route handler
// app/api/chat/route.ts
import { textStreamFromAnthropic } from "../../../lib/stream";
export const runtime = "nodejs";
type ChatMessage = { role: "user" | "assistant"; content: string };
const API_URL = process.env.ANTHROPIC_API_URL ?? "https://api.anthropic.com/v1/messages";
function isMessages(x: unknown): x is ChatMessage[] {
return (
Array.isArray(x) && x.length > 0 && x.length <= 50 &&
x.every((m) =>
typeof m === "object" && m !== null &&
((m as ChatMessage).role === "user" || (m as ChatMessage).role === "assistant") &&
typeof (m as ChatMessage).content === "string" &&
(m as ChatMessage).content.length < 20_000)
);
}
export async function POST(req: Request): Promise<Response> {
const body: unknown = await req.json().catch(() => null);
const messages = (body as { messages?: unknown } | null)?.messages;
if (!isMessages(messages)) {
return Response.json({ error: "messages must be a non-empty array of {role, content}" }, { status: 400 });
}
const upstream = await fetch(API_URL, {
method: "POST",
signal: req.signal, // client disconnect or Stop aborts the upstream call
headers: {
"content-type": "application/json",
"x-api-key": process.env.ANTHROPIC_API_KEY ?? "",
"anthropic-version": process.env.ANTHROPIC_VERSION ?? "2023-06-01",
},
body: JSON.stringify({
model: process.env.ANTHROPIC_MODEL, // a model ID from Anthropic's models page
max_tokens: 1024,
stream: true,
messages,
}),
});
if (!upstream.ok || !upstream.body) {
// Do not forward the vendor body: it can echo request details.
return Response.json({ error: `upstream returned ${upstream.status}` }, { status: 502 });
}
return new Response(textStreamFromAnthropic(upstream.body), {
headers: {
"content-type": "text/plain; charset=utf-8",
"cache-control": "no-store, no-transform",
"x-accel-buffering": "no", // stop nginx from buffering the stream
},
});
}
Notes on choices:
- Model ID is an env var. Model names change. Read the current ID from Anthropic's models page and set
ANTHROPIC_MODEL; do not hardcode a name from a blog post, including this one. anthropic-versionis required. Anthropic's docs (checked 2026-10-31) say every request must send it, list2026-10-21as the latest version and2023-06-01as an earlier one, and point to a migration guide. I defaulted to the older value and made it configurable; I could not test either against the live API, so read the migration notes before choosing. The official SDKs set this header for you.- Validate the body. Cap the message count and length. An open streaming endpoint with no limits is a way to burn your API credit.
- Add authentication and rate limiting before this goes on the internet. Neither is in the sample.
Step 4: the client component
// components/Chat.tsx (trimmed; the full file also renders the form)
"use client";
import { useEffect, useRef, useState } from "react";
type Msg = { role: "user" | "assistant"; content: string };
type Status = "idle" | "streaming" | "error";
export default function Chat() {
const [messages, setMessages] = useState<Msg[]>([]);
const [input, setInput] = useState("");
const [status, setStatus] = useState<Status>("idle");
const abortRef = useRef<AbortController | null>(null);
const scrollRef = useRef<HTMLDivElement>(null);
const stickToBottom = useRef(true);
useEffect(() => {
const el = scrollRef.current;
if (el && stickToBottom.current) el.scrollTop = el.scrollHeight;
}, [messages]);
async function send() {
const text = input.trim();
if (!text || status === "streaming") return;
const history: Msg[] = [...messages, { role: "user", content: text }];
setMessages([...history, { role: "assistant", content: "" }]);
setInput("");
setStatus("streaming");
stickToBottom.current = true;
const controller = new AbortController();
abortRef.current = controller;
try {
const res = await fetch("/api/chat", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ messages: history }),
signal: controller.signal,
});
if (!res.ok || !res.body) throw new Error(`HTTP ${res.status}`);
const reader = res.body.pipeThrough(new TextDecoderStream()).getReader();
for (;;) {
const { done, value } = await reader.read();
if (done) break;
setMessages((prev) => {
const next = prev.slice();
const last = next[next.length - 1];
next[next.length - 1] = { ...last, content: last.content + value };
return next;
});
}
setStatus("idle");
} catch (err) {
if (controller.signal.aborted) setStatus("idle"); // Stop: keep partial text
else { console.error(err); setStatus("error"); }
} finally {
abortRef.current = null;
}
}
// ...render: a role="log" aria-live="polite" div, and a Stop button that calls
// abortRef.current?.abort() while status === "streaming"
}
Details worth keeping:
TextDecoderStreamhandles multi-byte characters split across chunks, which matters for non-English text and emoji. Decoding each chunk withnew TextDecoder().decode(chunk)withoutstream: truecan corrupt them.- Stop is not an error. When the user aborts,
fetchthrows anAbortError; checkcontroller.signal.abortedand keep the partial answer. - Only auto-scroll when the reader is at the bottom. My component tracks
stickToBottomfrom the scroll position (within 48 px of the end). Without it, a long answer yanks the page away from someone reading the top. - Accessibility.
role="log"witharia-live="polite"andaria-busywhile streaming lets screen readers announce new text without interrupting. I did not test with a screen reader. - Re-render cost. Updating state per token is fine for chat-length text. If you render markdown, memoise the finished messages so only the last one re-renders.
What did I run, and what happened?
- Typecheck.
tsc --stricton all four files (route,lib/stream.ts,lib/sse.ts,Chat.tsx) using the TypeScript and React types from this site's ownnode_modules: no errors. I did not runnext buildor open a browser, so the component is typechecked, not exercised. - Route against a stub. I compiled the route and library to CommonJS and ran the real
POSThandler against a local HTTP stub that emits Anthropic-style SSE (message_start, ping, text_delta events) and splits the byte stream at awkward points, including mid-event.
1 happy: status 200 | chunks ["Hel","lo, ","wor","ld!"] | joined "Hello, world!"
upstream got stream flag: true | model: stub-model
2 bad body: 400 {"error":"messages must be a non-empty array of {role, content}"}
3 error: text before failure "Hel" | reader rejected with: overloaded_error
4 abort: upstream connection closed early: true
Case 3 is the mid-stream error event: the client gets the text that arrived, then a rejected read, which Chat.tsx turns into the "interrupted" message. Case 4 shows cancelling the downstream reader closes the upstream connection. The stub is my own approximation of the documented format, not Anthropic's server.
Production checklist
- Test the deployed URL with
curl -Nand confirm chunks arrive progressively. Proxies and some platforms buffer or cap streaming responses. - Set a
max_tokensyou can afford, and limit input size. - Add auth, per-user rate limits and a spend cap. This route is an open door to your API key's budget.
- Treat model output as untrusted when you render it. If you render markdown or HTML, sanitise it. See prompt injection explained if the chat can read external content.
- Log upstream status codes and error event types, not message contents.
- For a first pass at Claude from Node, the TypeScript Claude API guide covers the SDK. For agents that stream tool calls and state, see streaming AI agents with LangGraph.
What I did not verify
- Any call to the real Anthropic API: no key, so request shape, headers, the
2023-06-01default and billing on abort are unrun. next build,next devand browser behaviour. Route handler streaming in your Next.js version and host should be tested there.- Hosting-specific limits (timeouts, response size, buffering).
- Tool use and extended thinking streams; the code ignores them.
