OpenAI-Compatible Gateway in TypeScript and Next.js: OpenAI SDK, Vercel AI SDK and Streaming

Most gateway tutorials are Python. If your product is a Next.js app, you need the TypeScript version: one client, a streaming route that does not buffer, and a plan for the SDK quirks. This post gives you that code, tested against a local mock gateway, with one compatibility trap I hit while testing.

Disclosure: AICredits links on this site are affiliate links. I had no AICredits key, so I did not call AICredits from this code. The homepage says the gateway works with the OpenAI Python and TypeScript SDKs, LangChain and the Vercel AI SDK. I verified the code against my own mock gateway, not against theirs. For the key-less pieces I could check on the real service, see the conformance test.

Versions I ran on 2026-10-16: openai 7.34.0, ai 7.0.142, @ai-sdk/openai-compatible 3.0.70, Node 22. SDK APIs move, so check your installed versions.

How do you create one client for any gateway?

import OpenAI from "openai";

export function makeClient(baseURL: string, apiKey: string) {
  return new OpenAI({ baseURL, apiKey, maxRetries: 2, timeout: 20_000 });
}

export async function ask(client: OpenAI, model: string, prompt: string) {
  const r = await client.chat.completions.create({
    model,
    messages: [{ role: "user", content: prompt }],
  });
  return { text: r.choices[0].message.content ?? "", tokens: r.usage?.total_tokens ?? null };
}

Read the base URL and key from environment variables on the server (process.env.LLM_BASE_URL, process.env.LLM_API_KEY), never from client code. For AICredits the base URL on its site is https://api.aicredits.in/v1, and model IDs come from its catalog. The model ID lookup script shows how to find them.

The SDK's maxRetries handles simple transient errors. For fallback across models, wrap it with the logic from retries, timeouts and fallback.

How do you stream to the browser?

export async function* streamText(client: OpenAI, model: string, prompt: string) {
  const stream = await client.chat.completions.create({
    model,
    stream: true,
    messages: [{ role: "user", content: prompt }],
  });
  for await (const chunk of stream) {
    const piece = chunk.choices[0]?.delta?.content;
    if (piece) yield piece;
  }
}

export function streamResponse(client: OpenAI, model: string, prompt: string): Response {
  const enc = new TextEncoder();
  const body = new ReadableStream({
    async start(controller) {
      try {
        for await (const piece of streamText(client, model, prompt)) controller.enqueue(enc.encode(piece));
      } catch (e) {
        controller.enqueue(enc.encode("\n[error: upstream failed]"));
      } finally {
        controller.close();
      }
    },
  });
  return new Response(body, { headers: { "Content-Type": "text/plain; charset=utf-8" } });
}

In a Next.js App Router route handler you return streamResponse(...) from POST. I did not run it inside a Next.js server; I ran the functions under Node. Note the chunk.choices[0]?. with an optional chain: some gateways send a final usage chunk with an empty choices array, and indexing it directly would throw.

What did the test show?

I ran these against a local mock (not AICredits) with the real openai package, on 2026-10-16:

ask          -> {"text":"pong","tokens":17}
streamText   -> "pong"
Response     -> 200 text/plain; charset=utf-8 "pong"
bad key      -> 401 AuthenticationError
unknown model-> 404 NotFoundError

The SDK turns HTTP errors into typed exceptions (AuthenticationError for 401, NotFoundError for 404), so your catch blocks can branch on the class or on e.status. I also type-checked the file with tsc --strict, which passed.

How does it look with the Vercel AI SDK?

The AI SDK has an OpenAI-compatible provider for custom endpoints:

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { generateText, streamText } from "ai";

const gateway = createOpenAICompatible({ name: "gateway", baseURL: process.env.LLM_BASE_URL!, apiKey: process.env.LLM_API_KEY! });

const { text, usage } = await generateText({ model: gateway("openai/gpt-6-luna"), prompt: "ping" });

Against the mock, generateText returned the text and a usage object with input, output and total tokens, and streamText yielded the text.

What is the finish_reason trap?

My first mock streamed text pieces and a usage chunk but never sent a chunk with a finish_reason. The OpenAI SDK accepted it. The Vercel AI SDK logged this error on the stream:

InvalidResponseDataError [AI_InvalidResponseDataError]: Response stream ended without a finish reason.

The text still came through in my run, but the error surfaced. After I added a final chunk with finish_reason: "stop" to the mock, the error went away. The lesson is not that either SDK is wrong. Strictness differs, so a gateway that works in one client can complain in another. If you see this against a real gateway, test with the conformance script, which now includes a finish-reason check, and report it to the vendor with the raw stream.

What should you not do?

• Do not expose the key to the browser. Route through the server.
• Do not trust usage blindly while streaming. Some gateways only send it when you ask. The OpenAI API has a stream_options setting for this; check whether your gateway honours it before you rely on streamed usage for billing.
• Do not hardcode model IDs in many places. Keep one map from feature to model so you can change it, and so per-feature cost reports stay meaningful.
• Do not retry blindly after the stream starts. A retry restarts the answer.

For a Claude-specific TypeScript guide, see the Claude API in Node and Next.js. To compare gateways, read AICredits vs OpenRouter. The AICredits signup is an affiliate link.

What I could not verify

• Any of this against the real AICredits service (no key).
• Running inside an actual Next.js server, including edge runtime constraints.
• Tool calling and structured output through these SDKs against a real gateway.