Founder, Light Emerge · Agent reliability research and engineering
I find where AI agents fail — runaway loops, duplicate execution, truncated tool calls — measure it reproducibly, and fix it upstream.
lightemerge.com · Google Scholar · ORCID · LinkedIn
- Six paper acceptances across NeurIPS 2026 workshops, on how LLM agents fail and recover
- 3 plugins added to the official Hermes Agent plugin catalog (4 merged PRs)
- Bugs found and fixes filed in crewAI, Hugging Face smolagents, LangChain.js, OpenAI Codex and SchemaStore
- Credited on 3 security advisories in nexus-memory — 1 high, 2 moderate (details below)
Three issues reported privately in nexus-memory, fixed and published with regression tests, credited as finder in the advisory text.
| Advisory | Issue | Severity | Fixed in |
|---|---|---|---|
| GHSA-2h94-xx3x-chrr | Guardrail path rules never matched Windows paths — protected subpaths and parent deletes were allowed | High | 0.22.35 |
| GHSA-m446-mvgq-w52p | Cubic backtracking in the destructive-command patterns of the pre-execution check | Moderate | 0.22.34 |
| GHSA-455p-688x-56r8 | Quadratic regex in wikilink extraction (CVSS 3.1: 6.5) | Moderate | 0.22.32 |
Write-up: Three security advisories in an agent memory layer — how each was found, the measured curve, the fix, and where every number can be checked.
Sorted by weight: named organization, a quantified finding, a fix.
| Project | What I found or fixed | Status |
|---|---|---|
| crewAI | Three retry layers stack, so one 429 becomes 27 identical requests | Reported #8332 · fix #8333 open |
| Hugging Face smolagents | Client retries and ApiModel retries multiply — one rate limit becomes 9 requests; Retrying ignores Retry-After and the first retry already waits 2–3× wait_seconds |
Reported #3034, #3037 · fix #3036 open |
| NousResearch/hermes-agent | 3 plugins added to the official plugin catalog — degeneracy-guard, provenance-gate, skill-lineage; core fixes filed for repetition loops, for a stream hook that cannot ask a response to stop, and for a missing platform in the tool-path hooks |
4 merged (UTC), all by @teknium1 (#141341 2026-10-20, #141935 2026-10-20, #142381 2026-10-24, #144339 2026-10-25) · 4 core PRs open (#141479, #142974, #144657, #141340) |
| LangChain.js | Truncated streamed tool-call arguments are repaired into a valid-looking tool call instead of being dropped | #11994 open |
| OpenAI Codex | A streamed reply that turns into a repetition loop runs to the end — the CLI sends no output cap | #55279 open |
Ecosystem fixes, merged or reported upstream:
| Project | What I found or fixed | Status |
|---|---|---|
| SchemaStore | package.json accepted an empty version string; two cargo.json constraints let through detailed dependencies cargo rejects |
#6421 merged · #6394, #6435 reported, fixed upstream |
| scamgate | A long command line kept extract() busy for about 40 seconds; it now reads call-like text in one pass |
#21, #22 merged |
| nexus-memory | Reported 3 vulnerabilities and wrote the fixes; credited in all three advisories — guardrail path rules that never matched Windows paths (high), cubic backtracking in the destructive-command patterns, and ReDoS in wikilink extraction; Windows import failure and a test-portability fix (#12, open) | #10, #11, #12 open · advisories published 2026-10-28 |
| Vercel | vercel.json declares draft-04 but uses draft-06+ keywords, so strict validators reject it |
#17639 open |
| Compose spec | No schema constraint requires image, build or provider, yet the loader rejects a service with none of them |
#933 open |
Projects listed in 4 curated awesome lists through 6 merged pull requests.
- agent-double-exec-bench — seven agent frameworks against a server that fails on purpose: one 429 becomes 27 identical requests, and some frameworks decide whether to retry by reading the error text.
- agent-runaway-bench — what clients do when the model loops mid-reply: five clients across four frameworks return the entire loop as the final answer — 300,048 characters — and none of them stops.
- agent-cutoff-bench — a tool call cut off mid-argument still runs: LangGraph writes 16 of 40 lines to disk and reports success.
- retrieval-abstention-bench — zero-dependency abstention test: the retriever declines 86.0% of questions that are unanswerable by construction against 3.3% of the answerable set, a +82.7 point gap (95% bootstrap CI +77.7 to +87.3).
- The runs behind those numbers are archived: 10.5281/zenodo.23808696 — Agent failure-mode benchmarks: raw measurement records, 721 files, CC-BY-4.0.
- hermes-degeneracy-guard — refuses degenerate, repetition-looped tool calls before they execute; 39 legitimate payloads in the test corpus, and none of them prompts.
- hermes-provenance-gate — flags memory and skill writes that claim a user decision the user's own words do not back; precision 1.000 (34/34), recall 0.694 (34/49) on a hand-labelled set.
- hermes-skill-lineage — records where each skill came from, hash-chained, so a self-rewriting agent's skill set can be audited; 131 passed, 1 skipped at v0.1.8.
- greencheck — mutation testing for quality gates and metrics: the bundled demo gate catches most mutations and still lets one bad configuration through — the difference between "the check ran" and "the check worked".
- precheck — freezes a check before an agent runs, hash-locked, then asks whether that check could have failed at all. Zero dependencies, no network.
- agent-pushgate — leak, commit-message, push-range and ledger gates whose self-tests must go red when sabotaged; 68 self-test cases across five gates.
- authority-chain — hash-chained SQLite event log with base-revision tickets and countersigning for protected documents; all 57 error codes are exercised by a test.
- self-auditing-agent — a public audit log where every claim carries a rerunnable command;
gatecheckships inside it.
Constraint recovery in two-agent language-model loops — measured, not asserted:
- Volume or Coupling? A Scale-Dependent Dissociation in Constraint Recovery of Language-Model Loops — Zenodo, 10.5281/zenodo.21200851
- Accepted versions at four NeurIPS 2026 workshops: New in ML, EvoRobust, Resource-Aware Agentic AI (RAAAI), and CL4FMAgents.
- Recovery or Repetition? Feedback Policy and Verbatim Relay in a Two-Agent Language-Model Loop — Research Square, 10.21203/rs.3.rs-11287076/v1
- Evidence before assertion. Every claim in these repos ships with the command that reproduces it. When the evidence does not support the claim, the claim goes.
- Gates that can say no. A check that has never failed is a check that never ran, so I sabotage my own gates to prove they go red.
- Publish failures. The MISS, the regression and the broken version stay in the record. A repository that only shows wins is not evidence.