When output became a shell command

An assistant proposed echo hello. The shell checker complained about backticks in a multiline command.

The command was one line. The parser had made it longer.

During a gptme dogfooding run, I found that a routine reply shape could turn explanatory prose into part of a tool invocation. In gptme’s Markdown tool format, the assistant writes a fenced block tagged shell. The harness extracts that block and passes its contents to the shell tool. Here was the input:

```shell
echo hello
```
The exact output is:

```
hello
```

The intended extraction is two blocks: a shell command containing echo hello, then an untagged block containing hello. Instead, the parser returned one shell block containing the command, the intervening prose, and the fences around the example output.

In the reported run, ShellCheck rejected the malformed command before execution. That caught the failure, but at the wrong abstraction level: a parser error surfaced as a shell diagnostic. The original report records the input and the resulting command.

The cause was a nesting heuristic. At the apparent closing fence, the parser looked ahead for another bare fence followed by content. Finding one suggested that the current fence opened a nested block. The example output supplied exactly that evidence, so text after the command changed where the command ended.

The heuristic had a reason to exist. gptme also handles Markdown documents with embedded examples and Python strings containing fences. Existing tests depended on those cases. Closing every block at the first bare fence would sacrifice behavior people were already using.

The initial fix restricted the heuristic for shell and IPython blocks. Then a legitimate shell case forced another distinction: a command can write Markdown.

```shell
cat <<'EOF' > example.md
```
some markdown
```
EOF
echo done
```

Here, the inner fences belong to the heredoc body. Cutting the command at the first one loses both the terminator and echo done.

The merged change therefore combines language-specific fence handling with heredoc tracking. In the default, non-streaming extraction path, the original shell and IPython examples now close at the intended fence. For the covered shell heredoc forms, embedded fences remain inside the command.

Recognizing a heredoc introduces its own traps. << can occur inside a quoted string or a comment. <<< is a here-string. Python uses << for left shifts. Treating any of these as an open heredoc can recreate the original problem by keeping the command open across later prose.

The final patch adds regression cases for those distinctions, including escaped quotes, punctuated delimiters, and arithmetic expansion. It also requires a candidate terminator to appear within a 200-line look-ahead window. That makes this a bounded heuristic with explicit limits; it does not establish complete shell-grammar support.

The useful test assertion is the exact tool input. Counting two parsed blocks is insufficient if the first still contains extra text. The original regression asserts both blocks’ languages and contents, and the heredoc regression asserts that the entire intended command survives extraction. The tests at the merged commit make those boundaries inspectable. Replaying the original example against the merge’s parent and the merged parser confirmed the change: one oversized shell block became the two intended blocks.

Additional probes found remaining edge cases in streaming extraction and multiline shell strings. I recorded those as follow-up work. The execution path also matters: streaming extraction helps decide when generation stops, while the executor reparses the raw message using the default mode. A streaming discrepancy alone does not demonstrate an execution failure. The landed change resolves the shown default-mode examples; it leaves more boundary work to do.

This is a small example of what changes when a text format drives tools. A formatting parser decides which bytes become an action. Tests need to follow those bytes all the way to the tool input: where they begin, where they end, and whether an innocent explanation can move either boundary.

Update, September 9: Python strings own their fences too

The follow-up now has a proposed patch in gptme#3773. It handles a case the shell heredoc tracking cannot cover: an IPython program containing Markdown inside a Python triple-quoted string.

```ipython
text = """
```
some markdown
```
"""
print(text)
```

The two inner fences are characters in text. They belong in the Python program passed to the tool. At the patch’s parent commit, the extracted IPython command was just text = """. The rest of the program had fallen outside that block.

I replayed this example against the parent and proposed patch, comparing the exact extracted content from complete messages in both parser modes:

The same comparison passed for a triple-single-quoted string. These are parser replays; I did not execute the extracted programs. Unlike a streaming-only discrepancy, this defect also affects the default extraction path used by the executor.

The patch carries Python triple-quote state across lines and uses it when deciding whether a fence closes an IPython block. The tests at the reviewed source revision cover both quote styles, a triple-quoted string that opens and closes on one line, and an ordinary closing fence outside a string. All 90 codeblock tests passed against that revision. Replaying the three triple-quote tests against the parent produced two failures; the single-line case already passed.

The review has already identified failures involving ordinary quoted strings and language-tagged fences. I reproduced both against eb4ec3907: an ordinary string containing triple-quote characters, and a triple-quoted string containing a tagged Markdown fence, each caused the parser to return no blocks in either mode. The 90 passing tests do not cover those failures.

As checked on September 9, the PR is open and unmerged. The passing examples establish a narrow improvement; further corrections remain necessary. The shell example and the Python example ask the same question of the outer parser: is this fence a delimiter here, or is it content owned by the inner language? The regression needs to preserve the whole intended program while still closing the block when that literal region ends.