Hinglish is the way a lot of people in India actually type to a bot: Hindi grammar, Roman letters, English nouns, no fixed spelling. The practical answer for prompting it is to keep your instructions in English, say explicitly which script the reply must use, and then check that rule with code instead of trusting the model. This post gives you a short Python script that does the checking, with the output I got when I ran it on 2026-10-09.
Status: the vendor statements come from Anthropic's multilingual support page, which I fetched and read as full text on 2026-10-09. The Hinglish sentences are my own, written for this post, and no native speaker has reviewed them. Prompts are untested templates. The script output is real. Example replies in the checker demo are hand-written, not model output.
What do the vendor docs say about Hinglish?
Nothing directly, and that is the useful finding. Here is what the Anthropic page (platform.claude.com, multilingual support) says that applies:
- It recommends stating the desired input and output languages explicitly, and says the system prompt is the most reliable place to put the instruction.
- It says to "use native scripts" and submit text in native script rather than transliteration for optimal results. Roman Hindi is transliteration, so this is a warning that Hinglish input may do worse than Devanagari input. The page gives no measurement for that.
- Its performance table lists Hindi and Bengali as a percentage of English performance on a translated MMLU set: Hindi 96.7% for Claude Sonnet 4.5 (marked deprecated on the page) and 92.4% for Claude Haiku 4.5. Those are older models and standard Hindi, not code-mixed text.
I also fetched OpenAI's text guide and Google's Gemini text generation page and searched the text for Hindi, multilingual and transliteration. Nothing came up. That is not proof the vendors have no guidance elsewhere. It means I found no vendor claim about Hinglish quality, so none appears in this post.
For the wider picture on Hindi and other Indian languages, start with prompting in Hindi and Indian languages. It covers Unicode normalization, digits and evaluation rubrics. This post is narrower: the Roman-script, code-mixed case.
How do you decide which script the reply should use?
Script is the one property you can detect exactly, so route on it first.
| User message | Example (my own wording) | Default reply |
|---|---|---|
| Latin letters, Hindi words | Mera order abhi tak nahi aaya | Roman Hinglish |
| Devanagari only | मेरा ऑर्डर अभी तक नहीं आया | Devanagari, product names in Latin |
| Both scripts in one message | Meeting कल 5 baje hai | Pick one by rule, say which, test it |
| Latin letters, English words | Where is my order? | English |
The mixed row is where bots wobble. Write the rule down, for example "if more than half of the letters are Devanagari, reply in Devanagari, else Roman", and implement it in code, so the model is told the answer instead of asked to guess.
What does a Hinglish system prompt look like?
This is an untested template. Fill the routing line from your script, not from the model.
You are a support assistant for [COMPANY]. Instructions are in English.
The customer wrote in: [ROMAN_HINGLISH | DEVANAGARI | ENGLISH].
Reply in exactly that style:
- ROMAN_HINGLISH: Hindi grammar in Roman letters. Keep English words the customer used (order, refund). Do not use any Devanagari characters.
- DEVANAGARI: Devanagari for Hindi words; product names and technical terms stay in Latin letters.
- ENGLISH: plain English.
Address the customer as "aap". Never use tu, tum, tera or tumhara.
Use only ASCII digits 0-9. Write dates as DD-MM-YYYY.
If you do not know the order status, say so and offer a human handoff. Do not guess.
Keep the reply under 80 words.
Each line maps to something the checker below can verify: the script line, the pronoun line, the digit line, the date line and the length line. The last "do not guess" line cannot be checked by a regex, which is why it needs test cases in your eval set (see building evaluation datasets).
Can you check a Hinglish reply with code?
Mostly yes, for the mechanical rules. The module below has four parts: a script profile, a rough romanized-Hindi score, a spelling normalizer and a reply checker. It uses only the standard library plus tiktoken for counting. The word list is small and hand-picked. I left out words that are also common English words (such as "he", "to", "do" and "par") on purpose, because they would make English look like Hindi.
import re
import unicodedata
DEV = re.compile(r"[ऀ-ॿ]")
LAT = re.compile(r"[A-Za-z]")
DEV_DIGITS = re.compile(r"[०-९]")
HINDI_WORDS = {
"hai", "hain", "kya", "kyaa", "kia", "nahi", "nahin", "nhi", "kal", "baje",
"bhej", "mujhe", "mera", "meri", "aap", "aapka", "ka", "ki", "ke", "ko",
"se", "mein", "hum", "kar", "karo", "kab", "kb", "tak", "abhi", "aaya",
"milega", "ho", "gaye", "din", "bahut", "kaise", "kyon", "ya", "aur", "toh",
}
CANON = {"kyaa": "kya", "kia": "kya", "nahin": "nahi", "nhi": "nahi",
"kb": "kab", "mujhko": "mujhe"}
def script_profile(text):
dev, lat = len(DEV.findall(text)), len(LAT.findall(text))
if dev and lat:
return "mixed"
return "devanagari" if dev else ("latin" if lat else "none")
def romanized_hindi_score(text):
w = re.findall(r"[A-Za-z]+", text.lower())
return sum(1 for x in w if x in HINDI_WORDS) / len(w) if w else 0.0
def normalize(text):
text = unicodedata.normalize("NFC", text)
text = re.sub(r"([?!.])\1+", r"\1", text)
return "".join(CANON.get(t.lower(), t) if t.isalpha() else t
for t in re.findall(r"[A-Za-z]+|[^A-Za-z]+", text))
def check_reply(reply, user_script, max_words=80):
problems = []
rp = script_profile(reply)
if user_script == "latin" and rp in ("devanagari", "mixed"):
problems.append("reply has Devanagari but user wrote Latin")
if user_script == "devanagari" and rp == "latin":
problems.append("reply is all Latin but user wrote Devanagari")
if DEV_DIGITS.search(reply):
problems.append("Devanagari digits in reply")
if re.search(r"\b(tu|tum|tera|tumhara|tujhe|tumhe)\b", reply.lower()):
problems.append("informal pronoun (tu/tum family)")
if re.search(r"\b\d{4}-\d{2}-\d{2}\b", reply):
problems.append("ISO date; expected DD-MM-YYYY")
if len(reply.split()) > max_words:
problems.append(f"{len(reply.split())} words (limit {max_words})")
return problems
I ran the full version (the same functions plus a test driver) with Python 3.13 and tiktoken 0.14.0. First, script profile, romanized-Hindi score and the normalized text for six inputs:
latin 0.89 'Mera order abhi tak nahi aaya, kab tak milega?' -> 'Mera order abhi tak nahi aaya, kab tak milega?'
latin 0.89 'Mera order abhi tak nhi aaya, kb tak milega??' -> 'Mera order abhi tak nahi aaya, kab tak milega?'
devanagari 0.00 'मेरा ऑर्डर अभी तक नहीं आया, कब तक मिलेगा?' -> 'मेरा ऑर्डर अभी तक नहीं आया, कब तक मिलेगा?'
mixed 0.50 'Meeting कल 5 baje hai, reminder bhej do' -> 'Meeting कल 5 baje hai, reminder bhej do'
latin 0.00 'Where is my order?' -> 'Where is my order?'
latin 0.00 'Please check my refund status for order 4471' -> 'Please check my refund status for order 4471'
Read this honestly. The normalizer folded "nhi" to "nahi", "kb" to "kab" and "??" to "?", so the second input now matches the first. That is the point of it: keyword rules, caches and test sets see one spelling instead of several. The romanized score separates the Hinglish sentences (0.89) from the English ones (0.00). The Devanagari input scores 0.00 because the score only looks at Latin words, which is correct, since you route Devanagari by script. The mixed message scored 0.50, because "Meeting" and "reminder" are English and the Devanagari word is not counted. On messages of one or two words this score will be noisy, so it is a hint and not a classifier.
Next, the reply checker on six hand-written replies. These are not model outputs:
latin | Aapka order 12-10-2026 tak aa jayega. Hum track ka | []
latin | आपका ऑर्डर कल तक आ जाएगा। | ['reply has Devanagari but user wrote Latin']
latin | Tumhara order 2026-10-12 tak aayega, tu chinta mat | ['informal pronoun (tu/tum family)', 'ISO date; expected DD-MM-YYYY']
devanagari | Your order will arrive soon. | ['reply is all Latin but user wrote Devanagari']
latin | Order ₹ २५० ka hai. | ['reply has Devanagari but user wrote Latin', 'Devanagari digits in reply']
latin | word word word word word word word word word word | ['90 words (limit 80)']
What this does not catch: a reply that passes every rule and is still wrong, rude, or says the wrong order date. It also cannot tell good Hinglish from stiff Hinglish. Those need a native reader on a sample.
Does Hinglish really cost more tokens?
I counted the six messages with tiktoken:
| Message | o200k_base | cl100k_base |
|---|---|---|
| Hinglish, Roman: Mera order abhi tak nahi aaya, kab tak milega? | 15 | 16 |
| Same sentence, Devanagari | 14 | 42 |
| Mixed script: Meeting कल 5 baje hai, reminder bhej do | 12 | 14 |
| English: Where is my order? | 5 | 5 |
| English: Please check my refund status for order 4471 | 10 | 10 |
Two readings. Per message, the Hinglish question cost three times the English question on o200k_base, though the English one is a shorter sentence, so this is a rough picture and not a fair ratio. The Devanagari version cost 42 tokens on the older cl100k_base against 16 for Roman letters, and nearly the same on o200k_base. That is consistent with the larger 20-sentence measurement in the tokenizer cost post, which also explains why "type in Roman to save tokens" is advice tied to one tokenizer. I did not count Claude or Gemini tokenizers because I had no API key here. Use the provider's token-count endpoint on your own messages.
What should your Hinglish test set contain?
Build 30 real messages before you tune anything. Write them as a spreadsheet with an expected reply style, and include these on purpose:
- The same request in three spellings (kya, kyaa, kia; nahi, nahin, nhi).
- A message with typos and repeated punctuation (kb tak milega??).
- A mixed-script message.
- A message in English only, to confirm the bot does not force Hindi.
- An abusive or angry message, to check that the bot stays on "aap".
- A message that asks for something the bot must not do, so the refusal itself is checked for script and tone.
- A short message of one or two words ("refund?") that breaks your language guess.
Run each through the model, then the checker, then give the failures and a random sample of passes to a native reader. A rubric with meaning, register and naturalness columns is in the Indian-languages post. If the output is a translation of a fixed document rather than a conversation, use the separate translation quality checklist.
Should you convert Roman Hinglish to Devanagari before the model sees it?
Unknown, and I will not guess. The vendor guidance favours native script, and the token table shows Devanagari is cheaper on newer tokenizers but far costlier on older ones. Transliteration also adds a step that can introduce its own errors, since Roman spelling is ambiguous (one Roman string can map to several Devanagari words). If you want to try it, set it up as an experiment:
- Take your 30 test messages.
- Arm A: send them as typed. Arm B: transliterate to Devanagari first, with a separate prompt or tool.
- Keep every other part of the prompt fixed, and have a native reader rate the replies blind on meaning and register.
- Compare cost per message using your model's token counter.
Pick the arm that wins on meaning first and cost second. If the difference is small, take the simpler pipeline, which is arm A.
Where does this fit in a real bot?
The routing and checking above belong in the code around the model call, not in the prompt. A common shape is: normalize the incoming text, compute the script profile, fill the style line of the template, call the model, run check_reply, and retry once or hand off to a person if a rule fails. If you are building on WhatsApp, the Twilio and Claude tutorial shows the plumbing, and prompts for customer support in Indian BPOs has more support-specific templates. Do not log customer messages containing personal data into your test set without redaction.
Before you ship, read the replies for the three things regexes cannot see: wrong facts, a tone that does not suit your brand, and Hinglish that a native speaker would not write.



