Chat Reasoning Model (Thinking)

A chat search that takes more than ten seconds to answer erodes the user's trust, even when the final answer is excellent. The Modelo de Raciocínio (Thinking) field, in Sistema›IA, tackles exactly that: it lets you use a fast model for the mechanical part of the AI's work and reserve the most capable model for the part the user actually reads. Per the measurements, chat response time drops to less than half, with no loss in text quality.

Required permission: CONFIG_EDIT. Available since: 2026-07-20.

The problem this solves

A search in the main page chat took about 11 seconds to answer. The production measurement — question "a fusopar tem processos abertos?", conversation mrssp5lqssxmd — showed where the time went:

Almost 90% of the time went into a single decision: which of the 28 tools do I call? gemini-2.5-pro is a reasoning model and, without a cap, goes into dynamic thinking — it decides on its own how much to think, and it thinks a lot, even for a mechanical choice.

A control measurement, with the same trivial prompt and no tools, confirms the difference between models:

gemini-2.5-pro     2,309 ms / 2,317 ms
gemini-2.5-flash     730 ms /   717 ms

None of this was a lack of resources: the nodes were at 1% CPU, the platform cache was using 30 MiB of a 6 GiB limit, and the graph database answered the query in 79 ms. It was the expensive model doing work the fast model solves just as well.

What the setting does

The chat resolves a question in two distinct steps, with different requirements:

1. Selecting the tool — a mechanical decision ("do I need a graph query or a semantic search?"). It needs neither an expensive model nor long reasoning.
2. Writing the answer — where text quality matters. It stays on the chat model, with no reasoning cap.

The Modelo de Raciocínio (Thinking) field chooses the model for step 1 only. Step 2 always uses the Modelo de Chat (LLM) configured right above it, on the same screen.

Leaving the field at "Mesmo do modelo de chat (sem separação)" (same as the chat model, no separation) keeps the previous behavior: a single model for the whole cycle.

Recommended configuration

Expected effect based on the measurements: the selection step drops from ~9.8 s to the 2–3 s range, taking the search from ~12 s to ~5 s, without changing who writes the answer the user reads.

Step by step

1. Go to Sistema›IA.
2. Confirm the Modelo de Chat (LLM) you want (e.g. gemini-2.5-pro).
3. In the Modelo de Raciocínio (Thinking) field, select a fast model (e.g. gemini-2.5-flash). The choice applies to the whole platform — every context.
4. Ask a test question in Chat and compare the response time. To roll back, select "Mesmo do modelo de chat (sem separação)".

The reasoning cap

Regardless of the chosen model, the selection step now sends a reasoning token cap (thinkingConfig.thinkingBudget) to Gemini. That alone already cuts latency, even without switching models.

The value comes from the installation configuration, set by the operator at deploy time — not from this screen:

chat:
  thinkingBudget: 512     # → CHAT_THINKING_BUDGET variable

The cap is never applied when the final answer is written: there the field is dropped from the request and the provider returns to its default behavior.

How to verify it is active

In the records of the platform's inference component, the function-calling line now reports the budget:

Gemini function-calling: model=gemini-2.5-flash tools=28 contents=1 thinkingBudget=512

And in the chat records, the start of the cycle reports the separation:

Function-calling loop iniciado: 28 tool(s) disponíveis, modelo=gemini-2.5-pro (seleção de tool em gemini-2.5-flash)

If thinkingBudget=default shows up, the cap did not reach the inference engine — check the CHAT_THINKING_BUDGET variable in the chat's installation configuration.

Compatibility with any Google model

The thinking cap (thinkingConfig) is not accepted by every model, and support is not predictable from the name: production measurements showed gemini-3.5-flash accepting a zero budget, gemini-3.6-flash rejecting it — and, in one case, returning 200 with an empty response to a valid budget.

That is why the platform measures at call time instead of predicting: if the model rejects the thinkingConfig (400 error) or returns a response with no text and no tool call while the field is present, the inference component retries exactly once without the field (and with the output cap doubled). This applies to function-calling and to streaming. The practical effect: any Google model selectable on the screen works in the chat — models that do not support the cap simply pay the latency of dynamic thinking.

If the model still returns no content, the user receives a clear message in Portuguese naming the model and suggesting switching it in Configure → AI — never an empty answer. In the logs, look for retentando sem thinkingConfig and terminou sem conteudo.

Thought signatures. Gemini 3.x models attach a signature (thoughtSignature) to each tool call and require it to come back verbatim in the next turn's history — otherwise the API rejects the request with a 400 (missing a thought_signature). The platform captures the signature from the response, carries it through the function-calling loop and returns it in the history; calls constructed by the platform itself (e.g. textual-narration conversion) use Google's documented bypass sentinel. No configuration is required.

Precedence

1. An explicit model passed in the call (the model field, in integrations via the DATTA API) — turns the separation off: whoever asked for a specific model wants it for the whole cycle.
2. Modelo de Raciocínio from this screen (llmThinkingModel).
3. Neither of the two: the whole cycle uses the Modelo de Chat (llmModel).

Where it is stored

llmThinkingModel is a domain-settings field, written to all contexts: like llmModel, it is a platform choice, not a context choice. The platform exposes a dedicated endpoint to write it — see the API reference. The value is persisted in the runtime_config.json file, on the platform's permanent configuration storage.

References

• Search and chat guide
• User's custom prompt
• OpenSearch resources — the other finding from the same investigation
• The architecture of the function-calling cycle is detailed in the chat architecture documentation.