“Tests passed”, no file on disk: debugging a local agent from the plumbing up

TL;DR

My local agent reported a successful build and green tests, and the project stayed empty. I blamed the model first. In reality, the project wasn't even mounted in the agent's container, and until that was sorted out, I couldn't say anything about the models. Once the mount was fixed, Devstral did write its files. Qwen2.5-Coder 7B, tested later in OpenCode, still announces a tool call without executing it, at least with my Ollama setup. What I take away from it: you check the disk, not the agent's own report.

---

Table of contents

• The setup
• The symptom
• The project wasn't in the container
• The fix, and three details not to miss
• Devstral really writes
• Qwen, on the other hand, announces without executing
• What I wouldn't do again
• Where to look, and in what order

---

The setup

A VPS with no GPU, running Ollama and Hermes (Nous Research's agent) in two Docker containers on a private agentic network. I covered the installation in an earlier article. I didn't know it when I wrote it, but the problem in this one was already sitting in there.

The symptom

I started with qwen2.5-coder:7b: roughly 7.6 billion parameters, quantized to Q4_K_M, and tagged “tools” by Ollama, so it's supposed to know how to call tools. In Hermes, everything looked fine. write_file … calls scrolled by, then a terminal("dotnet build"), then “Build succeeded” and “Tests passed”. Yet in the project, not a single file.

Devstral 24B, for its part, hadn't done much better on a more ambitious benchmark: a small ASP.NET Core API with EF Core, PostgreSQL, a service, xUnit tests, Gherkin scenarios and a GitHub Actions pipeline. About sixteen and a half minutes of work, for a text answer instead of actual execution, a wrong route, a service left out of dependency injection, an inconsistent project structure, GitHub Actions targeting .NET 6, Gherkin that couldn't run, and no real build demonstrated. My provisional conclusion at the time: being good at generating code doesn't make a good agent.

I won't pretend otherwise: at that point, I was mostly annoyed. These benchmarks run on CPU, every attempt is counted in minutes, and I could watch the hands of my watch go round for nothing. Luckily, during those waits, I was helping my daughter with her homework. They weren't a total loss for everyone.

So I had two models that seemed to be making up their tool calls. As it turned out, I wasn't asking the right question.

The project wasn't in the container

Hermes's only mount was ~/.hermes to /opt/data, its state directory. The project I was asking it to work on simply didn't exist inside the container. That's exactly the command from the installation article, and it's also the only one shown in the examples of the Hermes Docker documentation (checked in October 2026). In those docs, /opt/data holds the agent's state, not its workspace. Nothing there is wrong, it just covers half of what you need.

Whatever the agent wrote, it couldn't end up in the project. Maybe the writes went somewhere else, into the container's writable layer or into the only mounted directory, the state one. Maybe they never happened. Without a transcript I can't tell, and that's the whole problem: as long as the environment hides the result, you can't conclude anything about the model. In other words, all those benchmark minutes hadn't measured much, apart from the time I'd lost, which did nothing for my frustration.

The fix, and three details not to miss

It all comes down to -v "$PWD:/workspace", which makes the current directory the agent's workspace. I put it in a zsh function:

unalias hermes 2>/dev/null
hermes() {
  sudo docker run -it --rm \
    --network agentic \
    -v "$HOME/.hermes:/opt/data" \
    -v "$PWD:/workspace" \
    -w /workspace \
    -e HERMES_WRITE_SAFE_ROOT=/workspace \
    hermes-local "$@"
}

First, zsh. I already had a hermes alias, and when I defined a function with the same name, I got:

defining function based on alias `hermes'
parse error near `()'

With the name () syntax, zsh refuses to define a function whose name comes from an alias, unless the ALIAS_FUNC_DEF option is set. Hence the unalias on the first line.

Next, HERMES_WRITE_SAFE_ROOT, which I set to /workspace when re-validating. The current documentation presents it as a restriction: it blocks write_file and patch outside the listed roots. I didn't write down whether my version required it, or why. I keep it as an explicit guardrail for those two tools, knowing it doesn't cover shell commands: there, only permissions stand in the way.

Which brings me to permissions. Hermes runs as UID/GID 10000, while the project belongs to my ubuntu user (1000). I sorted that out with ACLs:

sudo apt install -y acl
sudo setfacl -R -m u:1000:rwX,u:10000:rwX <project>
sudo setfacl -R -d -m u:1000:rwx,u:10000:rwx <project>

The first setfacl handles what already exists (the capital X only adds execute permission to directories and to files that are already executable). The second sets a default ACL, which files created afterwards will inherit, whether the agent or I create them.

Devstral really writes

To re-validate, I asked a cloud model (gpt-5.6-sol), then Devstral, for a real write_file. This time, two files showed up on the VPS: sol-tool-test.txt and devstral-tool-test.txt.

Devstral, which Mistral presents as designed to run inside code agents, wasn't “unable to act” after all. The “no build demonstrated” from my first benchmark was mostly measuring my environment. The defects in the code itself, on the other hand, still need to be reassessed.

Qwen, on the other hand, announces without executing

Concluding that it was all down to the mount would still be wrong.

Later, in OpenCode (a different harness, running in a rootless Dev Container), I asked both models to create a file with specific content. Devstral 24B made a real write call: file created, content correct, in just under 14 minutes, at about 0.4 tokens/s, with a transport error and a retry along the way. Qwen2.5-Coder 7B, in its 64K variant, understood the instruction and laid out a plausible-looking write(…), in 8 minutes 35. No file created.

To be clear, I didn't re-test Qwen in Hermes after the fix.

That said, I'm not going to conclude that Qwen2.5-Coder can't call tools. A Hermes issue (#5867), still open as of October 2026, describes the same symptom with qwen2.5-coder:14b through Ollama: tool calls come back as JSON in the response text instead of the structured tool_calls field, and Hermes treats them as a regular reply without ever executing them. The reported workaround is switching providers on the Hermes side. The Qwen2.5 function calling documentation, which doesn't mention the Coder variant, also warns that the models don't always follow the protocol, even with the right prompt and template.

What I wouldn't do again

Judge a model by what it says. “Build succeeded” is generated text. It's nothing like an exit code.

Count on -v to warn me about a mistake. When the source path doesn't exist, -v quietly creates it as an empty directory, whereas --mount type=bind,… fails loudly. A single typo in the path is enough to reproduce this exact symptom, without a single message.

Forget the flip side of $PWD. If you run the function from $HOME, the agent gets everything its permissions can reach, including ~/.hermes (configuration, OAuth tokens, API keys) mounted a second time under /workspace, and the safe root then covers all of $HOME. So run it from the project root, and apply the ACLs to the project only, never to $HOME or to a tree that holds .env files or other secrets.

If I were starting over, I'd look at HERMES_UID and HERMES_GID first, which the current Hermes documentation and docker-compose.yml offer to align the container user with the owner of the mount. I don't know whether that was already available in my version, or with my locally built hermes-local image, so check it on yours.

Where to look, and in what order

Looking back, when the disk doesn't back up the reported success, this is the order I'd look in:

1. The environment. Is the project visible in the container, at the right path, and can the agent's user write to it? A dumb test (“create this file with this content”) and an ls on the host are enough to find out.
2. The harness and its parser. Does the tool call arrive in structured form, or as text that looks like one? If a cloud model passes the same test, the environment and the harness are in the clear. On the Ollama side, you still have to check whether the call comes back in tool_calls or in the text.
3. The model. Only at that point, and judging by the concrete result: the file is there, the build re-run by hand passes, the exit code has been read.

And this isn't a quirk reserved for small local models. Recent work (Guo et al., ICML 2026, Advani, 2026), mostly on frontier models, describes agents claiming they're done while the environment says otherwise.

---

The Hermes sandbox can finally see the project, and a local model really writes in it. What's left is re-running the tests the environment had skewed. In the meantime, I've at least learned not to take the agent at its word, and to go look at the disk.