Skip to content
View akatsukiwork0320's full-sized avatar

Highlights

  • Pro

Block or report akatsukiwork0320

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
akatsukiwork0320/README.md

Satoshi Kawasaki

Independent researcher and research software engineer focused on AI safety evaluation. I build reproducible evaluations, proposed fixes, and verification tools that support decisions on research and critical software:

Open to research grants, collaborations, and scoped engineering contracts. Corporate contracting is available through MIKOTO Co., Ltd..

Discuss a project · Research and evidence · Company GitHub · ORCID

AI safety: selected reports and independent validation

These are reports and tests contributed to other authors' work, not PRs authored by me (status checked 2026-10-17).

  • ControlArena: AI-control input-validator pilot (reproducibility package) — Offline audit of 24 stored backdoor inputs across 10 sampled problems in an AI-control evaluation benchmark. One conflicted with a stated constraint also violated by two recorded test inputs: an unresolved specification/data conflict, not a confirmed scoring error. The audit was offline and did not implement sandbox isolation or test live attacks or scorer integration. The PR author responded with scope guidance; validator coverage and format policy remain maintainer decisions.
  • ControlArena: swe_django side-test behavioral probe (reproducibility package) — Local pytest probe of reconstructed base/gold/mutated Django functions for two reported inverted side-task tests (15103 and 16661), identifying an or-joined JSON/Python \u0026 decoding tautology in 15103 that accepts both escaped and unescaped output, and documenting baseline vs. mutation behavior on 16661. Did not run SWE-bench Docker containers, the full main-task suite, or ControlArena's evaluation pipeline. No upstream response as of 2026-10-17.
  • FACE-Eval: Inspect scorer missing-reasoning report (offline reproducer .zip) — Source-level probe exercising eval/inspect/judge.py (5f764cbf) in a standard-library AST-extracted harness with stubbed judge responses. With a valid cue-aligned answer, observed vcr=0.0 and uar=1.0 when reasoning is absent or its judge output fails JSON parsing. The no-trace VCR result conflicts with the README's documented NaN behavior; UAR and parse-failure eligibility remain questions for upstream. Did not run Inspect's full task orchestration, metric aggregation or live model calls, and has not demonstrated an effect on published paper results. No upstream response as of 2026-10-17.
  • Debate control schemes: Phase 2 side-standardization sensitivity note — Methods note on jackmaiorino/selvarath-debate (34898fb) separating the Phase 2 erratum's +4.56 pp world-standardized question-side weighting from the +3.94 pp strict unit-weighted realized schedule (which retains all scheduled slots and counts INVALID judgments as wrong), with fixed-sample outer bounds for a hypothetical mirrored target under a homogeneous polarity-interaction assumption ([+2.06, +5.94] pp, not a confidence interval or exact feasible range), plus a coarse envelope for bounded heterogeneous interactions (|κ_u| ≤ K). Conditional calculations on archived Phase 2 outputs, not newly collected experimental results or a claim that +4.56 pp is an arithmetic error. No upstream response as of 2026-10-17.

Selected authored pull requests

Ethereum infrastructure

Optimism (op-reth) #23334 (fixes #22501) — Proposed a fix and regression tests for historical proofs returning current-state data after unwinds and subsequent appends. A maintainer independently reproduced the bug, stated that the fix looks correct in a commented review, and opened #23341 for a separate store-repair subcommand; formal approval is still pending. Local validation passed 822 tests across 17 packages; repairing previously corrupted history is separate work.

AI evaluation and funding benchmarks

Manifund funding-bench #1 — Built an offline diagnostic and unit tests comparing how proposal attribution and scoring windows affect a synthetic Timaeus prediction. Production scoring and data are unchanged; this demonstrates sensitivity to the assumptions, not an improvement in actual funding decisions or model rankings.

Systems biology and scientific computing

Tellurium #611 · #612 (fixes #604, #603) — Proposed Python fixes for SBML/SED-ML biochemical network simulation workflows: preventing order-dependent model initialization from resetting parameter changes and preventing repeated tasks from overwriting standalone simulation outputs. Regressions cover XML ordering, the reported pathway simulation, nested tasks, and both fixes applied together.

Diffrax #787 (fixes #707) — Added construction-time checks for Brownian-path shapes to reject unsuitable static metadata before JIT tracing. Selected tests cover accepted NumPy integer dimensions and rejected array/tracer inputs; these are regression checks, not a complete statistical validation of the solvers.

Additional authored PRs: AI inference (open feature PRs, awaiting review)
  • SGLang #43603 — Public extension points for model-weight placement before cache allocation and CUDA Graph capture, with CPU tests and single-L4 comparisons.
  • SGLang #43597 — A public way to refuse unsupported online weight updates before mutation, with CPU tests and a live-server refusal check.

All seven PRs linked in this section are open and not merged, checked 2026-10-17. The linked PRs document implementation details and validation limits.

Other selected reports and independent validation

These are reports and tests contributed to other authors' work, not PRs authored by me. Links point to the recorded contributions; HiGHS and SGLang status checked 2026-10-17.

  • HiGHS: incorrect optimality result — Reported a small MIP with an exact optimum of 1 where the tested solver reported 0. A maintainer confirmed the bug, and the fix, PR #3408, was merged into latest as c2ddc850 on 2026-10-16. Following my candidate-build comparison, an agent-run comparison of the merged build and a reverted control reproduced the correction on the original and redundant-row examples. Exact checks confirmed optimal returned points for the presolve-off API and MPS baselines; this is validation of the reported examples, not general solver correctness.
  • SGLang: speculative sampling kernel reduction — Synthetic-kernel measurements on one RTX 3070 identified ~23 µs of extra reduction work at batch size 1 on common paths; the PR author incorporated the proposed branch guard (response, upstream PR open). Not an end-to-end serving benchmark.
  • Kokkos: host-side C++ Event race reproduction — Reproduced a host-side C++ Serial ThreadSanitizer data race on the portable Experimental::Event fallback across Clang and GCC, and diagnosed three Jenkins CI/NVCC build failures.
  • LLVM: ExtTSP code-layout differential testing — Differential tests across 2,001 synthetic CFGs and an 8-node exact-enumeration witness showing near-tie layout changes in mergeChainPairs; no generated-code runtime measurement.
  • SP1: GPU worker failure propagation — On one L40S, reproduced allocation panics and a client waiting until a configured 15-minute limit. Tested an existing development-branch error-handling approach on a pinned main build; clients then received terminal errors. OOM prevention and foreign-exception aborts were not resolved.
  • Tycho: delayed revert replay — Supplied three-stream staggered-lag tests, a remaining edge-case reproducer, and a candidate cache fix (upstream PR merged; that merge does not establish adoption of my candidate cache change).
Additional public reports and reviews
Project My contribution
revm Nine-case library-level SELFDESTRUCT trace comparison.
TEMBRI Extracted block-reader probe across four public Xatu partitions showing 11-column vs. 12-column base_fee_per_gas schema shift (open issue, awaiting upstream response).
Arbitrum governance · #384 Static source/payload review questions on L1-to-L2 timelock execution path (#383) and 480,000 batch gas charge calibration across One and Nova (#384).
vLLM Concurrent structured-output comparisons across model-runner configurations.
cosign A regression test for a caller-supplied signature verifier being bypassed.
SPIRE A cached-SVID regression test after sequential registration updates.

Reusable research artifacts

ControlArena validator pilot (#755 / #899) · ControlArena #907 side-test probe · FACE-Eval #1 Inspect scorer probe (.zip attached to the issue) · revm reproduction bundle · HiGHS exact checker · SGLang kernel measurements · Uniswap v3 replay checker (synthetic demo)

AI tools assisted implementation, experiments, analysis, and writing. Linked reports record the tested inputs, environment, results, and limitations; upstream responses are distinguished from local checks.

Pinned Loading

  1. control-arena-validator-pilot-repro control-arena-validator-pilot-repro Public

    Offline 10-problem, 24-record secret-input validator pilot and specification audit for ControlArena APPS dataset (#755 / #899).

    Python

  2. revm-selfdestruct-trace-repro revm-selfdestruct-trace-repro Public

    Pinned library-level reproduction of revm #3833: SELFDESTRUCT trace payloads, filter matches, reference outputs and an independent checker.

    Python

  3. highs-analytic-center-repro highs-analytic-center-repro Public

    Exact verification and an 11-variable reproducer for a HiGHS 1.15.1 Optimal 0 result on a MIP with exact optimum 1.

    Python

  4. sglang-speculative-sampling-repro sglang-speculative-sampling-repro Public

    Reproduction bundle for SGLang speculative sampling fallback: pinned sources, synthetic GPU measurements, raw results and a conditional-reduction patch.