Independent researcher and research software engineer focused on AI safety evaluation. I build reproducible evaluations, proposed fixes, and verification tools that support decisions on research and critical software:
- AI safety & evaluation benchmarks — UK AISI ControlArena
#899·#907, FACE-Eval#1, Debate control schemes sensitivity note, Manifundfunding-bench#1 - Ethereum & ZK infrastructure — Optimism
#23334, SP1#2950, Tycho#1486, revm#3833, TEMBRI#1, Arbitrum governance#383·#384 - Systems biology & scientific computing — Tellurium
#611·#612, Diffrax#787, HiGHS#3387 - C++ / Rust / GPU systems — SGLang
#42850, Kokkos#9122, LLVM#214016
Open to research grants, collaborations, and scoped engineering contracts. Corporate contracting is available through MIKOTO Co., Ltd..
Discuss a project · Research and evidence · Company GitHub · ORCID
These are reports and tests contributed to other authors' work, not PRs authored by me (status checked 2026-10-17).
- ControlArena: AI-control input-validator pilot (reproducibility package) — Offline audit of 24 stored backdoor inputs across 10 sampled problems in an AI-control evaluation benchmark. One conflicted with a stated constraint also violated by two recorded test inputs: an unresolved specification/data conflict, not a confirmed scoring error. The audit was offline and did not implement sandbox isolation or test live attacks or scorer integration. The PR author responded with scope guidance; validator coverage and format policy remain maintainer decisions.
- ControlArena:
swe_djangoside-test behavioral probe (reproducibility package) — Localpytestprobe of reconstructedbase/gold/mutated Django functions for two reported inverted side-task tests (15103and16661), identifying anor-joined JSON/Python\u0026decoding tautology in15103that accepts both escaped and unescaped output, and documenting baseline vs. mutation behavior on16661. Did not run SWE-bench Docker containers, the full main-task suite, or ControlArena's evaluation pipeline. No upstream response as of 2026-10-17. - FACE-Eval: Inspect scorer missing-reasoning report (offline reproducer
.zip) — Source-level probe exercisingeval/inspect/judge.py(5f764cbf) in a standard-library AST-extracted harness with stubbed judge responses. With a valid cue-aligned answer, observedvcr=0.0anduar=1.0when reasoning is absent or its judge output fails JSON parsing. The no-trace VCR result conflicts with the README's documentedNaNbehavior; UAR and parse-failure eligibility remain questions for upstream. Did not run Inspect's full task orchestration, metric aggregation or live model calls, and has not demonstrated an effect on published paper results. No upstream response as of 2026-10-17. - Debate control schemes: Phase 2 side-standardization sensitivity note — Methods note on
jackmaiorino/selvarath-debate(34898fb) separating the Phase 2 erratum's+4.56 ppworld-standardized question-side weighting from the+3.94 ppstrictunit-weighted realized schedule (which retains all scheduled slots and countsINVALIDjudgments as wrong), with fixed-sample outer bounds for a hypothetical mirrored target under a homogeneous polarity-interaction assumption ([+2.06, +5.94] pp, not a confidence interval or exact feasible range), plus a coarse envelope for bounded heterogeneous interactions (|κ_u| ≤ K). Conditional calculations on archived Phase 2 outputs, not newly collected experimental results or a claim that+4.56 ppis an arithmetic error. No upstream response as of 2026-10-17.
Optimism (op-reth) #23334 (fixes #22501) — Proposed a fix and regression tests for historical proofs returning current-state data after unwinds and subsequent appends. A maintainer independently reproduced the bug, stated that the fix looks correct in a commented review, and opened #23341 for a separate store-repair subcommand; formal approval is still pending. Local validation passed 822 tests across 17 packages; repairing previously corrupted history is separate work.
Manifund funding-bench #1 — Built an offline diagnostic and unit tests comparing how proposal attribution and scoring windows affect a synthetic Timaeus prediction. Production scoring and data are unchanged; this demonstrates sensitivity to the assumptions, not an improvement in actual funding decisions or model rankings.
Tellurium #611 · #612 (fixes #604, #603) — Proposed Python fixes for SBML/SED-ML biochemical network simulation workflows: preventing order-dependent model initialization from resetting parameter changes and preventing repeated tasks from overwriting standalone simulation outputs. Regressions cover XML ordering, the reported pathway simulation, nested tasks, and both fixes applied together.
Diffrax #787 (fixes #707) — Added construction-time checks for Brownian-path shapes to reject unsuitable static metadata before JIT tracing. Selected tests cover accepted NumPy integer dimensions and rejected array/tracer inputs; these are regression checks, not a complete statistical validation of the solvers.
Additional authored PRs: AI inference (open feature PRs, awaiting review)
- SGLang #43603 — Public extension points for model-weight placement before cache allocation and CUDA Graph capture, with CPU tests and single-L4 comparisons.
- SGLang #43597 — A public way to refuse unsupported online weight updates before mutation, with CPU tests and a live-server refusal check.
All seven PRs linked in this section are open and not merged, checked 2026-10-17. The linked PRs document implementation details and validation limits.
These are reports and tests contributed to other authors' work, not PRs authored by me. Links point to the recorded contributions; HiGHS and SGLang status checked 2026-10-17.
- HiGHS: incorrect optimality result — Reported a small MIP with an exact optimum of 1 where the tested solver reported 0. A maintainer confirmed the bug, and the fix, PR #3408, was merged into
latestasc2ddc850on 2026-10-16. Following my candidate-build comparison, an agent-run comparison of the merged build and a reverted control reproduced the correction on the original and redundant-row examples. Exact checks confirmed optimal returned points for the presolve-off API and MPS baselines; this is validation of the reported examples, not general solver correctness. - SGLang: speculative sampling kernel reduction — Synthetic-kernel measurements on one RTX 3070 identified ~23 µs of extra reduction work at batch size 1 on common paths; the PR author incorporated the proposed branch guard (response, upstream PR open). Not an end-to-end serving benchmark.
- Kokkos: host-side C++ Event race reproduction — Reproduced a host-side C++
SerialThreadSanitizer data race on the portableExperimental::Eventfallback across Clang and GCC, and diagnosed three Jenkins CI/NVCC build failures. - LLVM:
ExtTSPcode-layout differential testing — Differential tests across 2,001 synthetic CFGs and an 8-node exact-enumeration witness showing near-tie layout changes inmergeChainPairs; no generated-code runtime measurement. - SP1: GPU worker failure propagation — On one L40S, reproduced allocation panics and a client waiting until a configured 15-minute limit. Tested an existing development-branch error-handling approach on a pinned
mainbuild; clients then received terminal errors. OOM prevention and foreign-exception aborts were not resolved. - Tycho: delayed revert replay — Supplied three-stream staggered-lag tests, a remaining edge-case reproducer, and a candidate cache fix (upstream PR merged; that merge does not establish adoption of my candidate cache change).
Additional public reports and reviews
| Project | My contribution |
|---|---|
| revm | Nine-case library-level SELFDESTRUCT trace comparison. |
| TEMBRI | Extracted block-reader probe across four public Xatu partitions showing 11-column vs. 12-column base_fee_per_gas schema shift (open issue, awaiting upstream response). |
Arbitrum governance · #384 |
Static source/payload review questions on L1-to-L2 timelock execution path (#383) and 480,000 batch gas charge calibration across One and Nova (#384). |
| vLLM | Concurrent structured-output comparisons across model-runner configurations. |
| cosign | A regression test for a caller-supplied signature verifier being bypassed. |
| SPIRE | A cached-SVID regression test after sequential registration updates. |
ControlArena validator pilot (#755 / #899) · ControlArena #907 side-test probe · FACE-Eval #1 Inspect scorer probe (.zip attached to the issue) · revm reproduction bundle · HiGHS exact checker · SGLang kernel measurements · Uniswap v3 replay checker (synthetic demo)
AI tools assisted implementation, experiments, analysis, and writing. Linked reports record the tested inputs, environment, results, and limitations; upstream responses are distinguished from local checks.