Writing Tool Descriptions for LLM Agents: A Schema Checklist and Linter

Your agent keeps calling search when it should call lookup_order, passes "yes" where the schema wants a boolean, and now and then asks the user for an API key that your code already holds. The model isn't being careless. It only knows what your tool definitions told it, and most definitions say very little.

A good tool description for an LLM agent answers four questions before the model has to guess: what does this tool do, when should I use it, what exactly goes in each field, and what will it not give me. This post turns the vendors' written advice into a 12-rule schema checklist, then into a linter script I ran on a made-up set of support tools, before and after a rewrite.

Two limits up front. The linter checks structure, not whether the model behaves better, and I did not run any of these tools against a live model. Everything vendor-related below comes from the pages I read on 2026-10-09: Anthropic's define tools page and OpenAI's function calling guide, both fetched with curl and converted to text, so check exact wording there. For the broader design thinking see how to design tools for AI agents and the tool design lesson. This post is the mechanical pass you run after that thinking is done.

What should a tool description for an LLM agent contain?

Anthropic's page calls detailed descriptions "by far the most important factor in tool performance" and lists what to cover:

• What the tool does.
• When it should be used, and when it shouldn't.
• What each parameter means and how it changes behaviour.
• Caveats, including what the tool does not return if the name alone doesn't make that clear.

It suggests at least 3 to 4 sentences per tool. Its own example contrasts "Gets the stock price for a ticker." with a version that names the exchanges supported, the currency returned, when to use it and what it will not provide.

OpenAI's guide says the same thing from the other side with its "intern test": can a human use the function correctly given nothing but what you gave the model? If not, the questions that human would ask belong in the description.

The 12-rule checklist

Each rule below maps to a check in the script further down. Where a rule comes from a vendor page, the source is named. Where it is my heuristic, it says so.

Two set-level checks sit on top: more than 20 tools at once (OpenAI's soft suggestion, S01) and two tools whose descriptions overlap heavily (my Jaccard heuristic, S02), because overlapping tools are where "which one do I call" mistakes come from.

A few of these deserve a sentence each.

L07 is a refusal risk, not just style. Anthropic's page says a parameter that asks for the model's thinking or step-by-step reasoning may lead to a reasoning_extraction refusal, and recommends asking for a short explanation or supporting evidence instead. Rename reasoning to explanation and describe it as "one or two sentences naming the policy clause or order fact".

L08 and L12 pull in opposite directions, and that's fine. OpenAI's strict mode needs every property in required, with optional fields expressed as a nullable type. So the fix for an optional parameter is often to remove it: decide in code, not in the model.

L10 is the one that fails loudly. Anthropic's page says each input_examples entry must be valid for the input_schema or the request returns a 400. A typo in an example can take the whole request down, so check it before deploy rather than in production.

The linter

Install jsonschema (I used 4.26.0 on Python 3.13) and save this as lint_tools.py. It reads a JSON file holding a list of tool definitions in Anthropic's name / description / input_schema shape. It uses no model and no network.

import json
import re
import sys
from itertools import combinations

from jsonschema import Draft202012Validator

NAME_RE = re.compile(r"^[a-zA-Z0-9_-]{1,128}$")
VAGUE_NAMES = {"search", "get", "run", "do", "handle", "process", "tool", "data", "action", "query"}
REASONING_PARAMS = {"reasoning", "thoughts", "thinking", "chain_of_thought", "rationale"}
KNOWN_BY_CODE = {"api_key", "session_id", "user_id", "tenant_id", "auth_token"}
WHEN_HINTS = ("use this", "use when", "should be used", "call this", "when the user", "do not use", "will not")
ENUM_HINT = re.compile(r"\b(either|one of|must be|can be)\b.*['\"]", re.I)


def sentences(text):
    return [s for s in re.split(r"(?<=[.!?])\s+", text.strip()) if s]


def words(text):
    return set(re.findall(r"[a-z]{4,}", text.lower()))


def lint_tool(tool, strict=False):
    out = []
    name = tool.get("name", "")
    desc = tool.get("description", "")
    schema = tool.get("input_schema", {})
    props = schema.get("properties", {})

    if not NAME_RE.match(name):
        out.append(("FAIL", "L01", f"name {name!r} breaks ^[a-zA-Z0-9_-]{{1,128}}$"))
    if name in VAGUE_NAMES:
        out.append(("WARN", "L02", f"name {name!r} is generic; prefix a service or object"))
    if len(sentences(desc)) < 3:
        out.append(("WARN", "L03", f"description has {len(sentences(desc))} sentence(s); docs suggest 3-4 or more"))
    if not any(h in desc.lower() for h in WHEN_HINTS):
        out.append(("WARN", "L04", "description never says when to use the tool or what it will not do"))
    for p, spec in props.items():
        if not spec.get("description"):
            out.append(("WARN", "L05", f"param {p!r} has no description"))
        if spec.get("type") == "string" and "enum" not in spec and ENUM_HINT.search(spec.get("description", "")):
            out.append(("WARN", "L06", f"param {p!r} lists allowed values in prose; use enum"))
        if p.lower() in REASONING_PARAMS:
            out.append(("WARN", "L07", f"param {p!r} asks for reasoning; ask for a short explanation instead"))
        if p.lower() in KNOWN_BY_CODE:
            out.append(("FAIL", "L08", f"param {p!r} is something your code already knows; inject it, do not ask the model"))
    bools = [p for p, s in props.items() if s.get("type") == "boolean"]
    if len(bools) >= 2:
        out.append(("WARN", "L09", f"boolean params {bools} allow contradictory calls; use one enum"))
    for i, ex in enumerate(tool.get("input_examples", [])):
        errs = sorted(Draft202012Validator(schema).iter_errors(ex), key=lambda e: list(e.path))
        for e in errs:
            out.append(("FAIL", "L10", f"input_examples[{i}] invalid: {e.message}"))
    if strict:
        if schema.get("additionalProperties") is not False:
            out.append(("FAIL", "L11", "strict mode needs additionalProperties: false"))
        missing = [p for p in props if p not in schema.get("required", [])]
        if missing:
            out.append(("FAIL", "L12", f"strict mode needs every property in required; missing {missing}"))
    return out


def lint_set(tools, strict=False, max_tools=20, overlap=0.5):
    report = {t.get("name", "?"): lint_tool(t, strict) for t in tools}
    set_level = []
    if len(tools) > max_tools:
        set_level.append(("WARN", "S01", f"{len(tools)} tools; OpenAI suggests fewer than 20 at once"))
    for a, b in combinations(tools, 2):
        wa, wb = words(a.get("description", "")), words(b.get("description", ""))
        if wa and wb:
            j = len(wa & wb) / len(wa | wb)
            if j >= overlap:
                set_level.append(("WARN", "S02", f"{a['name']} and {b['name']} descriptions overlap (Jaccard {j:.2f})"))
    return report, set_level


def main(path, strict):
    tools = json.load(open(path))
    report, set_level = lint_set(tools, strict)
    fails = warns = 0
    for name, items in report.items():
        for sev, code, msg in items:
            print(f"{sev} {code} {name}: {msg}")
            fails += sev == "FAIL"
            warns += sev == "WARN"
    for sev, code, msg in set_level:
        print(f"{sev} {code} (set): {msg}")
        warns += 1
    print(f"-- {path}: {len(tools)} tools, {fails} FAIL, {warns} WARN")
    return 1 if fails else 0


if __name__ == "__main__":
    sys.exit(main(sys.argv[1], "--strict" in sys.argv))

Run it as python3 lint_tools.py tools.json for the general rules, or add --strict to include L11 and L12. The exit code is 1 when anything fails, so it drops straight into CI.

Before: four invented support tools

These tools are invented for this post and belong to no real product. They pack in common tool-list mistakes: a vague name, a space in a name, prose where an enum belongs, an API key as a parameter, two booleans, a reasoning field and an invalid example. Save as tools_before.json.

[
  {"name": "search",
   "description": "Searches stuff.",
   "input_schema": {"type": "object",
     "properties": {"q": {"type": "string"},
                    "kind": {"type": "string", "description": "Either 'orders' or 'tickets'."}},
     "required": ["q"]}},
  {"name": "lookup customer",
   "description": "Finds a customer record by email address. Returns the profile and the plan name.",
   "input_schema": {"type": "object",
     "properties": {"email": {"type": "string", "description": "Customer email"},
                    "api_key": {"type": "string", "description": "Key for the CRM"}},
     "required": ["email", "api_key"]}},
  {"name": "find_customer",
   "description": "Finds a customer record by phone number. Returns the profile and the plan name.",
   "input_schema": {"type": "object",
     "properties": {"phone": {"type": "string", "description": "Customer phone"}},
     "required": ["phone"]}},
  {"name": "refund",
   "description": "Refunds an order.",
   "input_schema": {"type": "object",
     "properties": {"order_id": {"type": "string", "description": "Order id"},
                    "approve": {"type": "boolean", "description": "Approve the refund"},
                    "deny": {"type": "boolean", "description": "Deny the refund"},
                    "reasoning": {"type": "string", "description": "Think step by step about why"}},
     "required": ["order_id"]},
   "input_examples": [{"order_id": "ORD-1042", "approve": true},
                      {"order_id": 1043, "approve": "yes"}]}
]

Output of python3 lint_tools.py tools_before.json --strict, run on 2026-10-09:

WARN L02 search: name 'search' is generic; prefix a service or object
WARN L03 search: description has 1 sentence(s); docs suggest 3-4 or more
WARN L04 search: description never says when to use the tool or what it will not do
WARN L05 search: param 'q' has no description
WARN L06 search: param 'kind' lists allowed values in prose; use enum
FAIL L11 search: strict mode needs additionalProperties: false
FAIL L12 search: strict mode needs every property in required; missing ['kind']
FAIL L01 lookup customer: name 'lookup customer' breaks ^[a-zA-Z0-9_-]{1,128}$
WARN L03 lookup customer: description has 2 sentence(s); docs suggest 3-4 or more
WARN L04 lookup customer: description never says when to use the tool or what it will not do
FAIL L08 lookup customer: param 'api_key' is something your code already knows; inject it, do not ask the model
FAIL L11 lookup customer: strict mode needs additionalProperties: false
WARN L03 find_customer: description has 2 sentence(s); docs suggest 3-4 or more
WARN L04 find_customer: description never says when to use the tool or what it will not do
FAIL L11 find_customer: strict mode needs additionalProperties: false
WARN L03 refund: description has 1 sentence(s); docs suggest 3-4 or more
WARN L04 refund: description never says when to use the tool or what it will not do
WARN L07 refund: param 'reasoning' asks for reasoning; ask for a short explanation instead
WARN L09 refund: boolean params ['approve', 'deny'] allow contradictory calls; use one enum
FAIL L10 refund: input_examples[1] invalid: 'yes' is not of type 'boolean'
FAIL L10 refund: input_examples[1] invalid: 1043 is not of type 'string'
FAIL L11 refund: strict mode needs additionalProperties: false
FAIL L12 refund: strict mode needs every property in required; missing ['approve', 'deny', 'reasoning']
WARN S02 (set): lookup customer and find_customer descriptions overlap (Jaccard 0.64)
-- tools_before.json: 4 tools, 10 FAIL, 14 WARN

Without --strict the same file gives 4 FAIL and 14 WARN, so the strict rules add six failures on top.

After: rewrite by the checklist

Here is what the checklist does to the worst tool. The refund tool becomes orders_decide_refund: one verb-object name with a service-style prefix, a description that states when to use it and what it can't do, one decision enum in place of two booleans, and an explanation field in place of reasoning. Examples are valid, every property is required and extras are refused.

{
  "name": "orders_decide_refund",
  "description": "Records a refund decision for one order. Use this only after you have checked the order status and the refund policy text you were given. Approving starts the refund in the payment system and cannot be undone from this tool. Denying only records the decision. It will not contact the customer.",
  "input_schema": {
    "type": "object",
    "properties": {
      "order_id": {"type": "string", "description": "Order id exactly as returned by support_search_records, e.g. ORD-1042."},
      "decision": {"type": "string", "enum": ["approve", "deny"], "description": "The refund decision."},
      "explanation": {"type": "string", "description": "One or two sentences naming the policy clause or order fact that supports the decision."}
    },
    "required": ["order_id", "decision", "explanation"],
    "additionalProperties": false
  },
  "input_examples": [
    {"order_id": "ORD-1042", "decision": "approve", "explanation": "Delivered damaged; policy allows a refund within 14 days."}
  ]
}

The other three changes were structural. search became support_search_records with a record_type enum. The two customer lookups merged into crm_find_customer with a lookup_field enum (email or phone) and a value, which is the consolidation Anthropic recommends and removes the overlap warning. The api_key parameter disappeared; your code attaches credentials, not the model. I haven't printed those two rewrites, but they follow the same pattern as the refund tool above: four- or five-sentence descriptions that say when to use the tool and what it won't return, enums for closed sets, valid examples, and additionalProperties set to false. My rewritten file held those three tools. Run on 2026-10-09 with --strict:

-- tools_after.json: 3 tools, 0 FAIL, 0 WARN

That's the headline number, and it deserves suspicion, which is the next section.

What the linter can't tell you

I wrote a deliberately useless tool to see whether the checks could be gamed:

{"name": "crm_find_customer",
 "description": "Finds things. Use this when the user wants things. It will not find other things.",
 "input_schema": {"type": "object",
   "properties": {"value": {"type": "string", "description": "A value."}},
   "required": ["value"], "additionalProperties": false}}

-- tools_gamed.json: 1 tools, 0 FAIL, 0 WARN

It passes with a clean sheet, because three sentences, a "use this when" phrase and a described parameter satisfy every rule. So treat a clean lint as "no cheap mistakes left", not as proof of a good description. Only two checks here have teeth against nonsense: L10 (a real schema validation) and the strict-mode checks (L11, L12). The rest are proxies.

What to do after the linter:

1. Write ten to twenty realistic user requests, including ones that should trigger no tool and ones that sit between two tools.
2. Run them against the model you ship with the old and new definitions.
3. Count wrong-tool calls, invalid arguments and unnecessary calls for each. The agent evals from scratch post shows a harness for this.

I haven't done step 2 for the tools above, so I can't tell you the rewrite is better in practice. It's untested.

Decisions the vendors leave to you

Examples in the schema or not. Anthropic's optional input_examples helps with nested or format-sensitive inputs, and each example is checked against the schema. OpenAI's guide warns that adding examples may hurt reasoning models. If you run a reasoning model, test with and without. If you add examples, keep them few and valid, because L10 exists for a reason.

One tool with an action parameter, or several. Anthropic recommends grouping related operations (create_pr, review_pr, merge_pr) into one tool with an action parameter. That reduces selection ambiguity but widens one schema. A split is still right when the actions have different risk: orders_decide_refund above moves money, and you may want an approval gate on it alone. The MCP security checklist has more on gating by tool.

How many tools. OpenAI suggests fewer than 20 at the start of a turn as a soft target and points to tool search for the rest. Anthropic documents a defer_loading option and says tools found through tool search are appended inline, leaving the cached prefix untouched. Measure at your own count.

Strict mode. OpenAI recommends always enabling it. Anthropic documents strict: true on tool definitions and, for some current models, recommends auto tool choice with strict tool use instead of forcing a tool, since its page says any and tool return a 400 on those models. Read the model list on the page for the model you use, since it changes.

Changing descriptions later has a cost. Both vendors cache prompt prefixes, and on Anthropic, editing tool definitions invalidates the entire cache (tools, system and messages). Tune descriptions in development, then freeze them, and serialise the schema keys in a fixed order so you don't cause misses by accident. The mechanics are in prompt caching for long-running agent loops.

Pre-merge checklist

Copy this into your pull-request template for any change to a tool definition.

• Name is service_verb_object style, matches the regex, and isn't a bare verb.
• Description has what it does, when to use it, when not to, and what it won't return.
• Every parameter has a description with a format or an example value.
• Closed sets are enum; no prose lists of allowed values.
• No parameter for secrets, user ids or anything your code already has.
• No boolean pairs; no "reasoning" field.
• input_examples pass the linter.
• Linter exit code is 0, with --strict if you use strict mode.
• Ten or more test requests re-run, and the wrong-tool count didn't go up.
• Tool list order and key order are fixed, so caches survive.

If your tools are served over MCP rather than defined in your request, the same descriptions still reach the model, so run the linter on the definitions your server returns. The MCP versus function calling post explains how that path differs, and structured outputs and tool calling across providers covers the schema differences between vendors.

What I did not verify

• No tool in this post was run against a live model. The before-and-after claim is about structure only.
• The vendor pages were read through curl and a text converter on 2026-10-09. Wording, model lists and limits may change; check the live pages.
• L02, L03 and L04 are heuristics. The word list, the sentence counter and the phrase list will miss some cases and flag others wrongly. Tune them to your style.
• The checklist covers Anthropic-style and OpenAI-style function definitions. Google's Gemini function declarations were not read for this post.