Why a Model's Tool List Should Not Be Global

A model does not need your full catalog of API operations in every request. It needs a small set of actions that are actually allowed in the current conversation state. When a team sends the same list of 200 functions to both a large reasoning model and a cheaper compact model, it is testing not the models' capabilities, but the system's tolerance for poor design.

The tool list affects three things at once: the chance of choosing correctly, the space available in the context, and execution safety. A large model can sometimes handle an overloaded catalog. A small model is more likely to choose a function because of a chance match between words in the description, omit a required parameter, or call an action that was never needed. This is not a reason to abandon small models. It is a reason to stop treating the function set as a permanent part of the system prompt.

One global catalog breaks action selection

A global API catalog is convenient only for the developer who wants to generate a JSON Schema once and never revisit it. For the model, it mixes actions from different domains, roles, and stages of a process.

Imagine a support assistant for an online store. The shared registry contains get_order, cancel_order, refund_payment, create_return, change_delivery_address, apply_discount, block_account, reset_password, and dozens of other functions. The user writes: "The courier never arrived. Refund the money for order 8241." If the model sees refund, cancellation, delivery complaint, and manual discount operations at the same time, it has to do more than extract the order number. It has to reconstruct the right business process.

The cancel_order function may look appropriate because of the word "refund." But the order has already been handed to the courier, so it is too late to cancel it. refund_payment may be unavailable until the status and reason have been checked. The correct sequence usually starts by reading the order, then checking whether delivery actually failed, and then creating a complaint or offering a refund according to the rules. If all these transitions are presented to the model as equal buttons, an error becomes the expected outcome.

The problem gets worse when tools have similar names:

• search_customer searches for an account by contact details;
• get_customer reads a record by internal ID;
• get_customer_orders returns orders;
• get_order reads one order;
• update_customer changes a profile.

A person sees the difference after a quick look at the documentation. The model sees several semantically similar descriptions competing for a single call. The weaker the model and the shorter the user's request, the higher the cost of that competition.

You cannot fix this with a prompt instruction such as "choose the correct function carefully." The model is already trying to do that. The application must reduce the number of wrong options before the request reaches the model.

Context size is not the same as the ability to distinguish functions

A large context window lets you put more schemas, documentation, and history into a request. It does not give the model a separate search mechanism for the catalog, nor does it make similar operations less similar.

The official Gemini documentation recommends keeping the active set to roughly 10 to 20 relevant functions and selecting them dynamically when the overall catalog is large. This is not a universal limit or a magic number. It is a useful acknowledgment that selection quality deteriorates when a model receives too many competing options. The same documentation describes the allowed_function_names parameter, which can be used to limit the available calls.

There is another practical reason not to fill the context with schemas. A tool description consists of more than its name. It can include purpose text, a JSON Schema, properties, enumerations, required fields, examples, and sometimes error-handling rules. A hundred carefully documented functions can easily push the user's history, previous call results, and the documents the model needs to make a decision out of the request.

This is especially noticeable in long agent chains. After each call, the history gains the operation name, arguments, result, call ID, and provider metadata. Gemini's documentation on combining tools notes that returned call parts become part of the history and are counted in requests. You can increase the context window. You cannot make history free and error-proof.

Separate two tasks:

1. The context window determines how much information you can pass and retain between steps.
2. Tool calling quality determines whether the model chooses the right operation, creates valid arguments, and understands the result.

A model with a huge window can still have mediocre function selection. A small model with a short window can be highly accurate if you give it five clearly separated operations. Comparing models only by the number of context tokens is meaningless for an agent task.

The active set should follow the task state

Pass functions according to the stages of the process, not according to the organizational structure of your API. At the first step, the model often needs a search or read operation. At the second, after facts have been checked, it may need to create a request. An operation that actually changes money or permissions should appear only after the application code has checked the required conditions.

For the same order refund, the route might look like this:

1. The user reports the problem and provides the order number.
2. The application gives the model only get_order and find_orders_by_contact if the number is missing.
3. After reading the order, the application determines the available transitions based on status, country, time limit, and user role.
4. The model receives create_delivery_claim or prepare_refund if those operations are allowed.
5. The executor validates the parameters and either performs the action or requests confirmation.

This is not a rigid flow that destroys the usefulness of an LLM. The model still extracts data from free text, chooses between permitted branches, asks clarifying questions, and explains the result. But it no longer decides whether your system grants permission for a risky transition.

This approach is useful for internal systems as well. In a financial assistant, the set at the analysis stage might include reading transactions, finding a contract, and calculating a limit. At the payment-preparation stage, it might receive a function for creating a draft. It is better not to add the payment-submission function until the server has checked the amount, authorization, and required fields. If the user says "pay it now," the model should not bypass the rules simply because it can see the corresponding function name.

AI Router lets you keep the familiar OpenAI-compatible request format when changing models, so the active-set router should live in your application rather than being tied to one provider. You can then test the same task on several models without rewriting client code.

Similar functions should be separated before the model sees them

The most common mistake in tool schemas is not a missing required field. Teams often try to present internal CRUD methods as an interface for a system that has to reason.

For example, instead of three operations, create_ticket, update_ticket, and assign_ticket, teams sometimes publish ticket_mutation with an action parameter. This may reduce the number of functions, but it often makes the schema worse: the model has to guess the action string, field compatibility, and permitted transition. Combine operations only when they have the same goal, the same permissions, and nearly the same lifecycle.

The opposite extreme is harmful too. There is no need to publish set_shipping_city, set_shipping_street, set_shipping_house, and set_shipping_apartment if the system can safely update the address as one validated object. The user thinks of an address as one object. The model should see one update_delivery_address operation with a clear schema as well.

A good function answers one practical question. Its name describes the action, and its description explains where it applies. Compare:

{
  "name": "refund_payment",
  "description": "Return money for a paid order only after the order record confirms that a refund is allowed. Do not use to cancel an unpaid order or create a delivery claim.",
  "parameters": {
    "type": "object",
    "properties": {
      "order_id": {
        "type": "string",
        "description": "Internal order identifier returned by get_order"
      },
      "reason": {
        "type": "string",
        "enum": ["delivery_failure", "duplicate_charge", "approved_return"]
      }
    },
    "required": ["order_id", "reason"],
    "additionalProperties": false
  }
}

The English wording is not the important part. What matters is that the description rules out two similar but incorrect uses, while the reason field does not force the model to invent an internal reason code. Use enum for closed sets of values. The Gemini documentation recommends the same approach for parameters with a limited list of allowed values.

Do not hide business rules only in the tool description. Text helps the model choose an action, but it does not replace server-side validation. additionalProperties: false prevents an extra argument from being passed in implementations that support it, but it does not check whether the user is entitled to refund money for this particular order.

Small models need a narrower contract

A small model can be a good dispatcher for simple, frequent, and well-bounded tasks. It is often cheaper and faster than a larger model, but that advantage disappears if you load it with a huge function library and make it reconstruct your business logic.

Four constraints are useful for a small model. They do not replace testing, but they sharply reduce random decisions.

• Give it only the functions for the current process stage.
• Use short, distinct names without internal abbreviations.
• Make required parameters explicit and use enumerations.
• Require clarification when a call lacks an identifier, date, amount, or user consent.

Do not make a compact model plan a chain of eight calls in one response. Break execution into short turns: the model chooses an action, the application executes it and returns the result, and then the active set is formed again. This cycle adds network requests, but it localizes errors. If the model misclassifies the request, it will not have time to create a customer, change a contract, and send a notification in a single response.

The tool-choice mode matters too. The OpenAI API has modes where the application can disable tools, leave the choice to the model, require a call, or force a specific tool. A forced call is useful not as a universal workaround, but after your code has made a deterministic decision. If the server has already determined that only get_order is needed, do not make the model choose again among ten operations.

A strict schema helps with the shape of arguments, but not with intent. A model can produce perfect JSON for the wrong function. Therefore, always distinguish selection errors from parameter errors in your evaluation. Otherwise, the team will see a high share of valid JSON and mistakenly conclude that the agent is ready to take action.

A catalog of hundreds of operations should be searched by code, not by the model's attention

When you have hundreds of functions, do not pass them all even to a strong model. Add a selection layer. It does not have to be a complex vector database, and it should not give the LLM the right to expand its own access.

A practical router builds candidates from several signals: the current product module, the user's role, the object's state, the request language, and an explicitly recognized intent. It then adds a small set of functions allowed in that state to the request. Semantic search over descriptions can help find candidates, but it should not be the only filter for dangerous actions.

An example contract between the router and the model call:

{
  "conversation_state": "order_delivered_claim",
  "user_role": "support_agent",
  "allowed_tools": [
    "get_order",
    "create_delivery_claim",
    "prepare_refund"
  ],
  "blocked_tools": [
    "refund_payment",
    "cancel_order",
    "change_delivery_address"
  ]
}

The blocked_tools field does not need to be sent to the model. It is useful in the decision log and in router tests. In a month, you will be able to answer an uncomfortable question: was the refund function absent because the model did not remember it, or because the application correctly blocked it?

There is a popular but poor piece of advice: "Give the model all the tools. It will figure out which one to use." This approach looks good in demos, where the set is small, the operations are harmless, and the user's request is perfectly phrased. In production, it creates two types of failure. First, the model chooses a plausible but incorrect function. Then the team adds even longer descriptions, making the catalog heavier, more expensive, and more confusing.

If your platform uses MCP, the rule does not change. MCP solves compatibility between a client and a tool server, not the question of authorization. A server can announce hundreds of operations, but the client must choose which ones are available in the current conversation. Gemini's documentation for Remote MCP explicitly provides allowed_tools to limit server tools.

Evaluate the route, not just the final answer

The test "the model returned the right text" says almost nothing about whether an agent is suitable. A user may receive a polite message even if the model chose the wrong operation, the application rejected it, and the model then apologized successfully.

Build a task set where each case has known allowed functions, forbidden functions, required arguments, and an acceptable sequence. Add deliberately difficult examples: an incomplete order number, two orders in one message, a request to break a rule, conflicting dates, an outdated status in the history, and products with similar names.

For each run, save a record like this:

{
  "case_id": "refund_after_failed_delivery_017",
  "active_tools": ["get_order", "create_delivery_claim", "prepare_refund"],
  "expected_first_tool": "get_order",
  "model_tool": "prepare_refund",
  "arguments_valid": true,
  "policy_allowed": false,
  "executor_result": "blocked_missing_status_check"
}

This record shows the difference between three events that are often combined into one metric: the model created valid JSON, the model chose a reasonable function, and the system allowed the action. In the example, the JSON is valid, but the call is premature. If you count only schema validity, you will miss a dangerous defect.

At a minimum, track four results:

• first-function selection accuracy;
• argument accuracy after server-side normalization;
• the share of unnecessary calls;
• the share of forbidden or premature actions.

Also track justified clarification questions separately. A model that asks for the order number when the identifier is missing is better than one that confidently invents an identifier and calls a function. In high-risk domains, including banking, medicine, and public services, this distinction matters more than the average response length.

Parallelism and autonomy require a clear boundary

Parallel calls reduce latency when operations are independent. You can retrieve a customer's profile and order list at the same time if both functions only read data and do not depend on each other. The OpenAI documentation describes the parallel_tool_calls setting, while the Gemini documentation supports parallel and sequential calls. API support does not mean that every group of operations can safely run in parallel.

Do not parallelize reading and changing the same object when the read determines whether the change is allowed. Do not start two calls that spend from the same limit. Do not allow the model to create a refund and apply a discount at the same time "as compensation" until the code has checked whether the actions are compatible.

It is useful to divide tools into three classes:

• reads that do not change state;
• preparation, which creates a draft or calculation;
• execution, which changes money, permissions, data, or an external obligation.

The model can work relatively freely with read operations. Preparation requires context checks and an idempotency key. Execution requires server-side authorization, logging, and, in some processes, human confirmation. This is not bureaucracy surrounding the model. It is a normal engineering boundary between suggesting an action and taking the action itself.

Do not choose a model based on one impressive tool calling example. Use your own operations, reduce the active set to the functions that are genuinely allowed, add tests for similar actions, and inspect call traces. You will then see where you need a stronger model and where a small one is enough. In most cases, the first fix will not be changing the model, but removing 90 percent of the catalog from its view.