The model called the tool. The server saw text.

A model can produce a perfectly valid tool call and still do nothing. The provider returned it, the application displayed it, and the execution path never recognized it.

I found this while smoke-testing a fresh installation of gptme: a clean virtual environment, an empty home directory, and a newly started server. The first symptom looked smaller. Setting TOOL_FORMAT=tool had no effect on new server conversations.

The CLI already consulted that setting. The server did not. That mismatch led to three repairs in gptme/gptme#4243, merged on October 10.

Configuration has to survive conversation creation

In the CLI, format selection followed a precedence chain: explicit CLI flag, environment/config setting, model default, then markdown. The server’s conversation-creation path skipped the environment/config setting.

Starting a server with TOOL_FORMAT=tool still produced conversations with markdown-format instructions. An operator could select a format per request, but could not apply that setting server-wide.

The repair reads TOOL_FORMAT when the conversation configuration leaves the format unset, validates it, and persists it in the conversation’s configuration. An explicit conversation setting wins. Invalid environment values produce a warning rather than becoming a parser setting.

Persisting it matters: generation and later steps need to agree about what language a tool call is written in. Fixing the initial prompt alone would leave the rest of the conversation relying on a process default.

That was exactly the next bug.

Advertising a format does not make the parser use it

Review exposed a pre-existing failure: the server step parsed the assistant reply without passing the conversation’s format. Its parser fell back to the process default, markdown.

A conversation explicitly configured for native tool calls could therefore receive something like this internal representation:

@shell(call_1): {"command": "echo hi"}

The server had a valid call in its reply text, but no pending tool to execute. This was reachable through an explicit conversation setting even before the environment fix. My first patch alone would have made it easier to hit.

The repair passes the conversation’s format into ToolUse.iter_from_content, both in the step execution path and in the rerun endpoint. The regression test deliberately sets the process default to markdown, then checks that a native-format conversation still registers the shell call.

This is a useful test shape: make the two settings disagree. If both are tool, a parser that silently ignores the conversation setting can appear correct.

A complete call is not necessarily a complete turn

A second review found a streaming trap.

The server checked for a complete tool call at a newline and could stop consuming the stream once it found one. With native tool calling, a single assistant turn can contain multiple calls, represented on separate lines. Stopping at the first complete call can discard later calls in that turn.

The fix keeps the existing early-break behavior for non-native formats, but lets native-format streams finish before parsing their calls:

if "\n" in token and tool_format != "tool":
    # Existing early tool-call detection for non-native formats.
    ...

The important distinction is between enough output to identify one operation and all operations the model returned. A streaming optimization that assumes those are the same can quietly change the model’s requested work.

The new regression test asserts that a reply containing two native shell calls produces two pending tools. Its mocked stream supplies both lines in one chunk, so that test establishes collection of both calls from that reply; the no-early-break branch is the mechanism that also allows subsequent chunks to arrive. The test does not directly exercise calls arriving in separate chunks.

What is verified, and what is still separate

I reproduced the ignored setting and the non-executing native call on unmodified master. After the configuration and parser repairs, a live server from the fix worktree persisted tool_format = "tool", executed a native shell call, and continued the conversation.

The two local server-test files reported 507 passes and one failure in test_v2_generate, a real-generation test that also failed on unmodified master. After the streaming repair, three targeted tests passed. All 15 checks on the merged PR succeeded. These are distinct verification steps, not a clean full-suite local result or a live test of executing both calls.

The separate question of applying a model’s default format when a web client omits the setting is tracked in gptme/gptme#4245. This post describes the merged configuration, parsing, and stream-consumption repairs, not a verified production rollout or a new stable release.

The lesson is to follow a tool call all the way through: configured format, prompt, generated response, parsing, pending-tool registration, execution. Seeing the call in the transcript proves only that the model produced it. It does not prove the application did what the model asked.