draft, pending human review

Benign-corpus deletion bound

How often Intercis would have removed a tool call from our own benign coding-agent traffic. Tenant Dogfood, agent claude-code-mac, a snapshot of 2,844 governed tool calls captured 2026-08-29 at 19:29 UTC. The window opened on 2026-08-25 and was still filling when the snapshot was taken.

This page is a draft. It is excluded from search and from the sitemap, it is not linked from the navigation, and the figure on it is a worst-case upper bound rather than a confirmed result. Every flagged call is counted as a deletion of benign work until a human has ruled on it, and no human has ruled on any of them yet.

at most 16.84% 479 flagged of 2,844 governed tool calls. Wilson 95% score interval 15.51% to 18.26%, computed over all 2,844 calls rather than over the flagged subset. 283 of the 479 are calls enforce mode would have removed; the other 196 are injection-scan rows that never block. The breakdown is two screens down; read it before you quote this figure.

Read this as the worst case

  • All 479 flagged calls carry confirmed_benign = false. That is the state before review, not a finding.
  • Every one of them is counted here as a deletion of legitimate work, which is the least favourable assumption available.
  • A confirmed figure would replace this bound, and it can only come out lower.
  • The denominator is every governed call, not the allowed subset. Allowed calls are by construction exactly the calls the patterns did not match, so measuring against them alone would be circular.
  • Nothing was actually removed from the wire. The tenant was in observe mode for this window, so each of these is a counterfactual: what enforce mode would have done.

Flagged calls by policy

All 479, grouped by the policy that matched.

Policy Flagged calls
injection-instruction-override191
shell-rm68
db-drop53
cred-grep-secrets52
llm-classifier39
cred-env-secrets15
exfil-curl-post14
infra-terraform-destroy12
db-truncate9
shell-del6
container-nsenter5
injection-role-hijack4
infra-kubectl-delete3
exfil-rsync2
container-mount1
cred-passwd1
file-wipe1
injection-prompt-extraction1
privesc-chown1
supply-chain-pip1

Three different layers wrote those rows and only two of them can remove anything. Read the next section before quoting the 16.84%.

What enforce mode would actually have removed

The bound above counts every flagged row. The rows come from three layers, and the injection scanner is not one that blocks.

Layer Rows Would enforce mode remove the call
Deny-list patterns244Yes
LLM classifier39Yes, when the classifier answers
Injection scanner196No, it never blocks
  • The three injection- policies in the table above are written by the prompt-injection scanner, which is observe-only. grep -rn "injection" apps/proxy/deny_list.py returns nothing: the deny list has no injection rule. Those rows are written with the action injection_scan and the verdict fixed to observe, so they are a scan result recorded next to the request rather than a tool call being removed.
  • So the number to hold us to for enforce mode is 283 of 2,844, which is 9.95%, and 244 of those come from the deterministic patterns. It is still a worst case: every one of the 283 is counted as legitimate work removed until a human rules otherwise.
  • 283 is itself an upper bound on the classifier's share. When the classifier is unavailable the proxy allows the call, so a window with an outage in it produces fewer classifier rows, not more.

Arithmetic on the table above: 244 deny-list rows plus 39 classifier rows plus 196 injection rows is 479, and 283 divided by 2,844 is 9.95%. The injection- policy string is built in apps/proxy/main.py from the pattern name in apps/proxy/injection.py, with the verdict written as a literal observe. That is the whole reason those rows cannot be counted as removals.

How it is measured

Every governed tool call from the Dogfood tenant's claude-code-mac agent in this snapshot carries a verdict: allow when nothing matched, observe when a deny-list pattern or the classifier would have blocked the call, and the same observe on an injection-scan row, which is why the count has to be split by layer before it means anything. Request bodies are credential-masked before any hash is taken, and the published fixture stores SHA-256 hashes only: never a body, never a secret.

# build the fixture from the snapshot
python scripts/build_benign_corpus.py

# re-derive the bound from it
python -m pytest apps/proxy/tests/test_benign_corpus_prod.py