Revised September 2026. The benchmark figures in the original version of this post came from an earlier build of AEGIS and were not reproducible. The harness has since been rebuilt; this version argues from the current numbers, which are. Lesson 4 changed the most — and got better for it.
I shipped AEGIS as an MIT-licensed deterministic evaluation harness for tool-using AI agents earlier this year. The project page describes what it does. This post is about the six things building it taught me — most of them about deciding what to leave out.
1. The boring decisions are the load-bearing ones.
Three early choices ended up doing all the work. Deterministic scoring instead of LLM-judged evaluation. N-gram cosine similarity instead of sentence embeddings. Mocked tool execution instead of real shells. Each of them sounded like a downgrade from a more sophisticated alternative. None of them were.
The deterministic-versus-LLM-judged call is the cleanest example. The temptation was to spin up a second model to evaluate the first — it sounds research-grade, it scales, and the literature is full of people doing it. The cost is that you've replaced experimental variance with someone else's experimental variance, plus a model dependency that drifts every six months when the judge model gets updated. AEGIS scoring is rule-based. The same trace produces the same number on someone else's machine, six months from now, with no shared infrastructure. That property turned out to be more valuable than any single quality of judgment.
The n-gram-versus-embeddings call was similar. Sentence embeddings catch a wider class of semantic similarity. They also ship as a model dependency, demand GPU pressure during evaluation, and produce decisions you can't explain to a reviewer who isn't already a specialist. Character n-grams plus a small attack corpus catch most of what's relevant in a security context where attacker payloads are constrained to particular shapes. When that approximation eventually breaks, the layer is small enough that swapping it costs hours, not weeks.
The mocked-tools call I won't relitigate; it should be obvious from a security-engineering standpoint that letting fuzz tests run real shell commands is how you turn a research harness into a vulnerability.
What I underestimated, going in, is how much velocity you get when each of your three core decisions is something you can defend in one sentence. The complicated alternatives all needed paragraphs.
2. Trace before metrics, always.
AEGIS writes a typed JSONL event log first and derives every aggregate metric from it in a separate, second pass. That structure cost about an extra week up front compared with writing summaries inline. It paid for itself by the third experiment.
The point isn't that traces are nice to have. The point is that compressing before you understand is the most expensive mistake in evaluation work. You don't yet know what slice you're going to want. You don't know which axis will turn out to matter. The first time someone asks you "how does this break out by attack class versus by encoding style?" — and they will — you want to be able to re-aggregate from the raw events, not re-run the whole experiment because you only kept the totals.
The general lesson, which I'd already learned in a different form working on observability: the lowest-cardinality thing you can ship that's still individually informative wins. For AEGIS that was per-decision events. For incident response it's structured log lines. For training data it's per-session records. Anything coarser is a trap.
3. Don't trust the dashboard.
I built a Streamlit app on top of the trace data for visual exploration — policy outcome comparison, benchmark summaries, trace inspection. It got used. It also nearly became a problem.
The temptation, once a dashboard exists, is to use it as the source of truth. Re-runs feel slower than reloading the dashboard. New questions get answered in the dashboard's UI shortcuts rather than in the trace data. After a while, the dashboard's view becomes the result, and the underlying trace is whatever the dashboard happens to be querying right now.
I caught this early because of one rule I committed to from the start: if a dashboard number ever disagrees with a manual aggregate of trace.jsonl, the dashboard is wrong by construction. That phrasing is deliberate. The dashboard isn't an unreliable narrator; it is, by construction, a viewer over the trace. The trace cannot be wrong relative to itself. Once you frame it that way, "trust the dashboard" becomes obviously the wrong instinct.
I'd have been better off if the dashboard had a watermark on every chart that read "derived view, not source of truth." I might still add it.
4. The number that refuses to improve is the most useful number.
The current benchmark, with a permissive policy so the guards are the only thing acting: no guards, 20 of 20 attacks succeed. Keyword guard, 14 of 20. Semantic guard, 14 of 20. Both layered — still 14 of 20, now with two of ten benign controls falsely blocked.
An earlier build of this harness produced a much more flattering curve, the kind where each added layer visibly buys you something. I could not reproduce it, and the honest response to a number you can't reproduce is to stop publishing it and find out what the harness actually measures. The rebuilt harness produces a worse-looking number that holds — and the trace behind it says something more interesting than the old curve ever did.
The 14 surviving attacks are not a marginally tougher version of the 6 that get caught. Every guard stops 100% of the attacks that contain literal tokens like exfil, and 0% of the attacks that avoid that vocabulary. The 70% attack success is not partial defense. It is total defense against one evasion tier and no defense at all against the other two. And the layered stack detects exactly what the semantic guard alone detects, while inheriting the keyword guard's false positives — layering wasn't neutral, it was strictly worse.
The lesson survived the rewrite intact, and sharpened: the residual after a defense isn't a smaller version of the same threat — it's a different threat, and a defense that plateaus is telling you the next unit of work belongs in a different layer entirely, not in tightening the ones you have. Optimising the existing guards further only catches more of what they already catch, which, past the first tier, is nothing.
5. Hold the LLM integration.
Plugging a real LLM into AEGIS was the obvious next step from week one. I deliberately didn't.
The reason was epistemic hygiene. As soon as a real LLM is in the loop, every disagreement between two runs has two possible explanations: the guard layer changed, or the model did. With both moving simultaneously you can't isolate either. The deterministic mock runner sounds like a placeholder. It's actually a discipline — it forced me to validate that the engine logic, the corpus, the scoring pipeline, and the trace format were all stable on their own terms before adding the source of variance everyone in the field already accepts as unavoidable. The reproducibility failure that forced the rebuild made the same point from the other direction: an apparatus you can't hold still can't measure anything.
It's also why the caveat on the project page isn't small print. The figures characterize the guard stack, not any real model, and presenting them as an attack success rate for GPT‑4 or Claude would be exactly the kind of claim this project exists to check.
General form of the lesson: add a source of variance only after you've stabilised everything that didn't need to vary.
6. Property-based fuzzing surfaces what unit tests can't.
Each guard layer in AEGIS is exercised by Hypothesis with generated payloads. Hypothesis surfaced edge cases the unit tests would never have hit: Unicode normalisation differences between input and corpus, regex catastrophic backtracking on adversarial inputs, threshold instabilities at the boundary where a similarity score sits two epsilons away from the cutoff and small perturbations flip the decision.
None of those were in any test plan I wrote. They couldn't be — I didn't know they existed. The fuzzer found them by being more imaginative about input space than I was.
What I underestimated about property-based testing isn't that it finds bugs. The literature is full of that claim. What I underestimated is that each fuzzer finding becomes a permanent regression test. Hypothesis writes the failing input out as a deterministic seed, you commit it, and it runs forever. The fuzzer is a long-term net under the project, not a one-time scan.
If I were starting AEGIS today, the property tests would go in on day one, not at the point where I had something to test.
What I'd do differently
One thing, mainly: I'd document the design decisions in the repo as the project went, instead of writing them up at the end. By the time I was assembling the write-up a lot of the reasoning had already faded into "of course it's like that". The most useful artefact for a future maintainer — including a future me — is the chain of small choices that weren't obvious at the time. I lost most of that reasoning by deferring the writing.
Everything else — including publishing the correction you read at the top of this post — I'd do the same way.