Custom Base URL in LangChain, LlamaIndex and LiteLLM: Working Code and Three Gotchas

The three Python frameworks that most Indian AI teams reach for all accept a custom base URL, and each one has a different way to trip you. This post shows the working call for LangChain, LlamaIndex and LiteLLM, then the three errors I hit on purpose and how to fix them. Everything was run on 2026-10-16 against a local mock gateway I wrote, which speaks the chat completions format. It is not AICredits.

Disclosure: AICredits links on this site are affiliate links. I had no AICredits key, so I did not test these frameworks against it. The AICredits homepage says it works with LangChain and OpenAI-compatible SDKs. That is the vendor's claim. For the model IDs to use, see the model ID lookup script, and for existing end-to-end tutorials see building an AI app with AICredits and LangChain.

Versions: Python openai 2.54.0, langchain-openai 1.7.2, llama-index-llms-openai-like 0.8.1, litellm 1.105.1.

What is the working code for each?

OpenAI SDK, as the baseline:

from openai import OpenAI
c = OpenAI(base_url=BASE, api_key=KEY)
r = c.chat.completions.create(model=MODEL, messages=[{"role": "user", "content": "ping"}])

LangChain:

from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model=MODEL, base_url=BASE, api_key=KEY)
llm.invoke("ping").content

LlamaIndex:

from llama_index.llms.openai_like import OpenAILike
from llama_index.core.llms import ChatMessage
llm = OpenAILike(model=MODEL, api_base=BASE, api_key=KEY, is_chat_model=True)
llm.chat([ChatMessage(role="user", content="ping")]).message.content

LiteLLM:

import litellm
r = litellm.completion(model="openai/" + MODEL, api_base=BASE, api_key=KEY,
                       messages=[{"role": "user", "content": "ping"}])

All four returned the mock's reply:

openai SDK              OK   content='pong' streamed='pong' tokens=17
langchain-openai        OK   'pong'
llama-index OpenAILike  OK   'pong'
litellm                 OK   'pong'

The OpenAI SDK also streamed and reported usage. LangChain's stream() and bind_tools() worked too, with the mock returning a tool call: [{'name': 'get_weather', 'args': {'city': 'Pune'}, ...}].

Gotcha 1: LlamaIndex needs is_chat_model=True

Left at its defaults, OpenAILike failed with a 404 on my mock:

LlamaIndex OpenAILike (default flags)       ERR  NotFoundError: Error code: 404 - {'error': {'message': 'unknown route', ...
LlamaIndex OpenAILike (is_chat_model=True)  OK   'pong'

By default it appears to treat the model as a completion model and call a different route. Gateways differ in what they serve there, and many only serve chat completions. Set is_chat_model=True for chat models. If you need function calling, check the class options in your installed version, as those flags have changed across releases.

Gotcha 2: LiteLLM needs a provider prefix

Without a prefix, LiteLLM refused the call:

LiteLLM model='mock/ok' (no provider prefix)  ERR  BadRequestError: litellm.BadRequestError: LLM Provider NOT provided. ...
LiteLLM model='openai/mock/ok'                OK   pong

LiteLLM reads the first part of the model string as the provider. openai/ means "use the OpenAI-compatible route", and the mock received the name without that prefix (otherwise it would have 404ed on an unknown model). This matters for gateways whose model IDs already contain a slash. A catalog ID like openai/gpt-6-luna should therefore be written openai/openai/gpt-6-luna in LiteLLM: the first openai/ selects the route, the rest is the gateway's model ID. That last step is my inference from the behaviour above, not something I ran against AICredits, so test it. If you run LiteLLM as a proxy with budgets and virtual keys, read LiteLLM proxy for India.

Gotcha 3: your framework hides what was sent

When something fails, the framework error tells you little. Two habits save time:

1. Test the raw call first. Use the standard-library conformance script with the same base URL, key and model. If it passes there, the problem is framework configuration, not the gateway.
2. Log the model string at the call site. The three frameworks transform the model name differently (LiteLLM strips a prefix; LangChain passes it through). Print what you pass, and check the gateway's catalog for exactly that ID.

Which features should you test before relying on them?

The frameworks add features on top of chat: streaming, tool calls, structured outputs, embeddings. Each depends on the gateway and the underlying model. In my test, LangChain streaming and tool binding worked because my mock implemented them. A real gateway may not for every model. Run these before shipping:

• Streaming, including a final usage chunk.
• A tool call with arguments that parse as JSON.
• JSON mode or structured output, with validation on your side. See Instructor for structured outputs.
• Embeddings through the framework's embedding class, which uses a different endpoint. In AICredits' catalog, embedding models list embeddings in supported_apis and image models list images_generations, so check each model's field.

What about n8n, Dify and CrewAI?

I did not test them. n8n, Dify and similar tools expose "OpenAI-compatible" credentials with a base URL field, but I did not read their current documentation for this post, so I will not give field names. The same two rules apply: use the gateway's /v1 URL and the catalog's exact model ID. Existing posts cover the surrounding tools: Dify vs LangChain vs LlamaIndex and the n8n and Claude API tutorial.

If you want an INR wallet and a single key to try this, the AICredits signup is an affiliate link. Start with a cheap model and the conformance script.

What I could not verify

• Any of this against AICredits' authenticated API.
• The double-prefix form for LiteLLM with real catalog IDs (inferred).
• Behaviour of other framework versions; APIs move often.