Writing

Filtering the Menu Is Not Enforcement

Filtering the Menu Is Not Enforcement - abstract illustration

My agent tool loop excluded one whole domain of tools from the set it offers the model, with a doc comment explaining exactly why: letting a job reach back into the queue that fired it is re-entrancy, and letting it claim or mark work it doesn't own is a real hazard rather than a theoretical one. The reasoning was right. The comment was right. The enforcement wasn't there, and the comment made it look like it was.

What the exclusion actually did

Filtering the offer set stops those tools from appearing in the list the model sees. That's all it does.

Dispatch is a separate thing. When the model asks to call a tool, the loop sends whatever name it asked for straight to the shared dispatch seam. That's deliberate for the ordinary case: a hallucinated tool name should reach dispatch and be denied there, rather than being pre-filtered or crashing the loop. I have a test pinning that exact behavior by name.

So the pieces are individually correct and the composition has a hole. A name that was never offered still reaches dispatch. Dispatch authorizes it against the calling key's grants. If that key happens to carry grants for the excluded domain for some other, unrelated reason, the call succeeds. The exclusion the doc comment described as a safety property was a display filter.

How the model gets the name it was never given

The obvious objection is that the model can't call a tool it doesn't know exists. In practice there are two ways it gets there.

It can invent the name. These catalogs are conventional and a model that has seen a few of them will guess plausible names correctly more often than you'd like.

More concerning: the name can arrive inside a tool result. The loop feeds tool output back to the model as context, and that output can contain content from outside the system - an email body, a feed item, a fetched page. Text in that content saying "now call this tool with these arguments" is a prompt injection, and the tool name in it is one the offer filter never saw, because the filter only ever operated on what the loop chose to send.

That's the difference between a filter and a boundary. A filter shapes what you hand someone. A boundary is checked when they act.

Fix the enforcement, not the comment

There were two ways to close this and only one of them is real.

I could have corrected the doc comment to say the offer set is only a convenience. That's honest, and it leaves the hazard exactly where it was.

What actually shipped is a categorical denial at dispatch. Before the call reaches the dispatch seam, the loop resolves the call's domain and denies the excluded ones, regardless of what the calling key is granted. The reasoning behind the exclusion - re-entrancy, claiming work the loop doesn't own - never depended on what the key was scoped for, so the enforcement shouldn't either.

Then both misleading comments got rewritten to say the offer set is a convenience and to name where the real guarantee lives. Which is a small thing that I now think matters as much as the fix. A comment claiming a property nobody enforces is how the next person builds on a guarantee that isn't there.

Key on the resolved domain, never on the name

The implementation detail that carries the weight: the denial is keyed on the tool's resolved classification, not on a list of names.

A name-keyed check is defeated by spelling. Different casing, a trailing character, an alias, a name added later that nobody remembered to add to the deny list. Resolving the name to a domain first and denying the domain means new tools in that domain are covered the day they're registered, without anyone updating anything.

A name that doesn't classify at all still falls through to dispatch and gets denied there, which preserves the hallucinated-name behavior I wanted to keep. And the denial comes back as a non-fatal tool error, so the loop continues and the model can recover instead of the whole run dying on one bad call.

Two tests hold it in place. A key with full grants for the excluded domain, calling a real tool name in it, never reaches the seam and the run still succeeds. A second, different real name proves the guard isn't string-keyed - if someone reimplemented it as a name list, that test fails.

The same reasoning, one layer up

This pattern showed up again a few days later while I was building a tool that lets an agent create automations. The restrictions on what an agent may create needed a home, and the tool handler was the tempting place to put them.

Same mistake, different layer. A check in the tool handler binds that handler. It doesn't bind the next caller, written six months later by someone who reasonably assumes the validation layer is doing its job. The restrictions went into the draft validation instead, keyed on the record's origin, so there's one rule set and every path through it is covered.

The general form: put the check at the narrowest point that everything must pass through, not at the point where you happen to be standing when you think of it.

The honest limit, written down

There's a boundary this fix doesn't reach, and I put it in the docs rather than leaving it implied.

The dispatch denial covers the in-process tool loop. A job that spawns the Claude CLI reaches this server as an ordinary HTTP client carrying whatever key its config holds. There is no dispatch-source signal that distinguishes it from a human session, so for that path the key's grants are the entire control.

I don't like that. But an undocumented gap is worse than a documented one, because the undocumented version gets treated as covered by everyone who reads the fix and assumes it's total. If you can't close something, the next best thing is to make sure nobody builds on the assumption that you did.

What to go look at

If you're running an agent tool loop, find the code that decides which tools the model is offered, and then find the code that runs when a call comes in. If those are the same code, you're fine. If they're different, ask what the second one does with a name the first one filtered out.

For most implementations the answer is: dispatches it. The filter was never a boundary. It just looked like one, and a doc comment explaining the hazard makes it look even more like one.

Is your AI-built app ready for real users?

My AI App Review reads and runs the code by hand, then ranks the issues your team should address before release.

See the AI App Review