Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 44 additions & 7 deletions agentic-ai/Claude/README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Claude Code Config

Version-controlled source of truth for `~/.claude/` settings, hooks, rules, and
on-demand docs. Running `install.sh` copies the settings template and links the
rest of the configuration.
on-demand docs, plus the hook scripts shared with Codex. Running `install.sh`
copies the Claude settings template and links the shared configuration.

## Activation

Expand All @@ -24,18 +24,27 @@ This will:
that harness's config dir exists)
- Symlink each `Claude/output-styles/*.md` into `~/.claude/output-styles/`
- Symlink this `railguard.yaml` → `~/.railguard.yaml`
- Symlink each `hooks/*.sh` script into `~/.claude/hooks/`
- Symlink each `hooks/*.sh` script into both `~/.claude/hooks/` and
`~/.codex/hooks/`
- Merge the custom hook registrations into `~/.codex/hooks.json` while
preserving unrelated hooks
- Install or update the Railguard fork with
`cargo install --git https://github.com/ulises-c/railguard` (requires
Rust/cargo; an existing binary is kept with a warning when cargo is unavailable)
- Run `railguard install` to register it as a global PreToolUse hook

Restart Claude Code after running.
Restart both Claude Code and Codex after running so each reloads its hook
configuration.

> **Note:** `settings.json` sets `bypassPermissions` at the user level, so it applies to **all projects**, not just this repo.

## What this configures

The hook scripts below are deployed for both Claude Code and Codex. Claude hook
registration lives in `settings.json`; the installer idempotently merges the
custom registrations into `~/.codex/hooks.json` after Railguard registers its
own hooks, preserving unrelated entries.

### `bypassPermissions`
Claude auto-approves all tool calls without prompting. The hooks below act as the safety gate.

Expand Down Expand Up @@ -66,17 +75,17 @@ Blocks dangerous or escalation-prone shell commands:
- `sudo` (escalation must be explicit — run yourself)
- `git add -A`, `git add --all`, `git add .` (bulk staging can silently include secrets)

### PreToolUse: `validate-write.sh` (Write / Edit / MultiEdit)
### PreToolUse: `validate-write.sh` (Write / Edit / MultiEdit / apply_patch)
Blocks writes to sensitive file paths:
- `~/.ssh/`, `~/.aws/`, `~/.gnupg/`, `~/.config/gh`
- `/etc/`, `/usr/`, `/boot/`, `/sys/`, `/proc/`

### PostToolUse: `post-edit-shellcheck.sh` (Write / Edit / MultiEdit)
### PostToolUse: `post-edit-shellcheck.sh` (Write / Edit / MultiEdit / apply_patch)
After any shell script edit, runs `shellcheck --severity=error`. Exits 2 if errors are found, forcing Claude to fix them before continuing.

Skips gracefully if `shellcheck` is not installed.

### PostToolUse: `post-test-runner.sh` (Write / Edit / MultiEdit)
### PostToolUse: `post-test-runner.sh` (Write / Edit / MultiEdit / apply_patch)

After any source file edit, auto-detects and runs the project test suite. Detection order: `.claude/test-cmd` override → `Cargo.toml` → `go.mod` → `pyproject.toml`/`pytest.ini` → `package.json` → `Makefile`. Skips non-source extensions (md, json, yaml, etc.) and projects with no recognized test suite.

Expand Down Expand Up @@ -169,6 +178,34 @@ submodule) and their origin plus refresh steps are recorded in

## Testing the hooks

Run the repeatable Codex benchmark first. It exercises fixed Railguard and
custom-hook protocol cases in a disposable Git repository, emits TAP with
per-case timings, and exits nonzero on any regression. It does not execute the
dangerous commands in its fixtures or modify live Codex configuration.

```bash
# Benchmark the installed binary
bash agentic-ai/Claude/benchmark-codex-hooks.sh

# Run the identical cases against a development build
RAILGUARD_BIN=/path/to/railguard/target/debug/railguard \
bash agentic-ai/Claude/benchmark-codex-hooks.sh

# Validate deployed hook links and registrations separately
bash agentic-ai/Claude/validate.sh
```

Registration alone does not make Codex run the hooks: Codex also requires
per-hook trust, which it records in `config.toml` the first time an interactive
session encounters each hook. Until then `codex exec` silently skips them (the
`hooks` feature is on by default in current Codex; only an explicit
`hooks = false` under `[features]` disables the engine outright). For
non-interactive verification, `codex exec --dangerously-bypass-hook-trust`
runs registered hooks without persisted trust — use it only for hooks you
authored.

The commands below remain useful for quick, individual hook probes:

```bash
# Should exit 2 (blocked)
echo '{"tool_input":{"command":"rm -rf /"}}' | bash agentic-ai/Claude/hooks/validate-bash.sh
Expand Down
263 changes: 263 additions & 0 deletions agentic-ai/Claude/benchmark-codex-hooks.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,263 @@
#!/usr/bin/env bash
# Deterministic Codex hook benchmark. Runs fixed protocol cases in disposable state.
set -uo pipefail

SCRIPT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
HOOKS_DIR="$SCRIPT_DIR/hooks"
RAILGUARD_BIN=${RAILGUARD_BIN:-railguard}
CASE_INDEX=0
FAILURES=0
LAST_STATUS=0
LAST_OUTPUT=""
LAST_ERROR=""
LAST_DETAIL=""

resolve_railguard() {
if [[ "$RAILGUARD_BIN" == */* ]]; then
[[ -x "$RAILGUARD_BIN" ]] || return 1
else
RAILGUARD_BIN=$(command -v "$RAILGUARD_BIN") || return 1
fi
}

require_command() {
command -v "$1" >/dev/null 2>&1 || {
printf 'Bail out! required command is unavailable: %s\n' "$1"
exit 2
}
}

resolve_railguard || {
printf 'Bail out! Railguard binary is unavailable: %s\n' "$RAILGUARD_BIN"
exit 2
}
require_command bash
require_command git
require_command jq
require_command shellcheck

BENCH_ROOT=$(mktemp -d "${TMPDIR:-/tmp}/codex-hook-benchmark.XXXXXX")
PROJECT_DIR="$BENCH_ROOT/project"
OUTSIDE_DIR="$BENCH_ROOT/outside"
RAILGUARD_STATE="$BENCH_ROOT/railguard-state"
# Railguard's self-integrity check reads $HOME/.codex/hooks.json and denies
# everything when hooks exist there without a railguard entry. Point HOME at a
# bench-local copy registered the way `railguard install` leaves it, so results
# never depend on the live machine's Codex configuration.
BENCH_HOME="$BENCH_ROOT/home"

cleanup() {
rm -rf -- "$BENCH_ROOT"
}
trap cleanup EXIT

mkdir -p "$PROJECT_DIR/.claude" "$PROJECT_DIR/src" "$OUTSIDE_DIR" "$RAILGUARD_STATE" "$BENCH_HOME/.codex"
printf '{"hooks":{"PreToolUse":[{"matcher":"","hooks":[{"type":"command","command":"railguard hook --event PreToolUse"}]}]}}\n' \
> "$BENCH_HOME/.codex/hooks.json"
git init --quiet "$PROJECT_DIR"
printf 'true\n' > "$PROJECT_DIR/.claude/test-cmd"
printf '#!/usr/bin/env bash\nprintf "benchmark\\n"\n' > "$PROJECT_DIR/valid.sh"
printf '#!/usr/bin/env bash\nlocal value=1\nprintf "%%s\\n" "$value"\n' > "$PROJECT_DIR/invalid.sh"
printf 'pub fn benchmark_probe() {}\n' > "$PROJECT_DIR/src/probe.rs"
printf 'version: 1\nblocklist: []\nfence:\n enabled: true\n allowed_paths:\n - "%s"\n' \
"$PROJECT_DIR" > "$PROJECT_DIR/railguard.yaml"

now_ms() {
local ms
ms=$(date +%s%3N 2>/dev/null)
if [[ -z "$ms" || "$ms" == *N* ]]; then
if command -v python3 >/dev/null 2>&1; then
ms=$(python3 -c 'import time; print(int(time.time() * 1000))')
else
ms=$(( $(date +%s) * 1000 ))
fi
fi
printf '%s\n' "$ms"
}

fail_case() {
LAST_DETAIL=$1
return 1
}

capture_hook() {
local hook=$1
local input=$2
local stdout_file="$BENCH_ROOT/hook.stdout"
local stderr_file="$BENCH_ROOT/hook.stderr"

printf '%s\n' "$input" | bash "$hook" > "$stdout_file" 2> "$stderr_file"
LAST_STATUS=$?
LAST_OUTPUT=$(< "$stdout_file")
LAST_ERROR=$(< "$stderr_file")
}

capture_railguard() {
local tool_name=$1
local tool_input=$2
local stdout_file="$BENCH_ROOT/railguard.stdout"
local stderr_file="$BENCH_ROOT/railguard.stderr"
local input

input=$(jq -nc \
--arg session_id "codex-hook-benchmark-$$-$CASE_INDEX" \
--arg cwd "$PROJECT_DIR" \
--arg tool_name "$tool_name" \
--argjson tool_input "$tool_input" \
'{session_id: $session_id, cwd: $cwd, hook_event_name: "PreToolUse", tool_name: $tool_name, tool_input: $tool_input, tool_use_id: "benchmark"}')

printf '%s\n' "$input" | env \
HOME="$BENCH_HOME" \
RAILGUARD_HOME="$RAILGUARD_STATE" \
RAILGUARD_NO_KILL=1 \
"$RAILGUARD_BIN" hook --client codex --event PreToolUse \
> "$stdout_file" 2> "$stderr_file"
LAST_STATUS=$?
LAST_OUTPUT=$(< "$stdout_file")
LAST_ERROR=$(< "$stderr_file")
}

expect_status() {
local expected=$1
local context=$2
[[ $LAST_STATUS -eq $expected ]] || fail_case "$context: expected exit $expected, got $LAST_STATUS"
}

expect_output() {
local filter=$1
local context=$2
jq -e "$filter" <<< "$LAST_OUTPUT" >/dev/null 2>&1 \
|| fail_case "$context: unexpected output: $LAST_OUTPUT"
}

case_railguard_safe_noop() {
capture_railguard Bash "$(jq -nc '{command: "git status --short"}')"
expect_status 0 "Railguard safe response" || return 1
expect_output 'type == "object" and length == 0' "Codex safe response must omit permissionDecision"
}

case_railguard_hard_deny() {
capture_railguard Bash "$(jq -nc '{command: "rm -rf /"}')"
expect_status 0 "Railguard hard deny" || return 1
expect_output '.hookSpecificOutput.permissionDecision == "deny"' "destructive command must be denied"
}

case_railguard_approval_becomes_deny() {
capture_railguard Bash "$(jq -nc '{command: "npm publish"}')"
expect_status 0 "Railguard approval response" || return 1
expect_output \
'.hookSpecificOutput.permissionDecision == "deny" and (.hookSpecificOutput.permissionDecisionReason | ascii_downcase | contains("requires human approval"))' \
"Codex approval-required response must be an actionable deny"
}

case_railguard_outside_fence() {
capture_railguard Write "$(jq -nc --arg path "$OUTSIDE_DIR/probe.txt" '{file_path: $path, content: "probe"}')"
expect_status 0 "Railguard path fence response" || return 1
expect_output '.hookSpecificOutput.permissionDecision == "deny"' "write outside project fence must be denied"
}

case_validate_bash_safe() {
capture_hook "$HOOKS_DIR/validate-bash.sh" \
"$(jq -nc '{tool_name: "Bash", tool_input: {command: "git status --short"}}')"
expect_status 0 "safe custom Bash hook"
}

case_validate_bash_bulk_stage() {
capture_hook "$HOOKS_DIR/validate-bash.sh" \
"$(jq -nc '{tool_name: "Bash", tool_input: {command: "git add --all"}}')"
expect_status 2 "bulk staging custom Bash hook"
}

case_validate_bash_patch_text() {
capture_hook "$HOOKS_DIR/validate-bash.sh" \
"$(jq -nc --arg cwd "$PROJECT_DIR" --arg command $'*** Begin Patch\n*** Update File: README.md\n@@\n+sudo --version\n*** End Patch' '{cwd: $cwd, tool_name: "apply_patch", tool_input: {command: $command}}')"
expect_status 0 "apply_patch text must not be parsed as a shell command"
}

case_validate_write_sensitive_patch() {
capture_hook "$HOOKS_DIR/validate-write.sh" \
"$(jq -nc --arg cwd "$PROJECT_DIR" --arg command $'*** Begin Patch\n*** Update File: /etc/benchmark-probe\n@@\n+probe\n*** End Patch' '{cwd: $cwd, tool_name: "apply_patch", tool_input: {command: $command}}')"
expect_status 2 "sensitive Codex patch path"
}

case_shellcheck_valid_patch() {
capture_hook "$HOOKS_DIR/post-edit-shellcheck.sh" \
"$(jq -nc --arg cwd "$PROJECT_DIR" --arg command $'*** Begin Patch\n*** Update File: valid.sh\n*** End Patch' '{cwd: $cwd, tool_name: "apply_patch", tool_input: {command: $command}}')"
expect_status 0 "valid shell patch"
}

case_shellcheck_invalid_patch() {
capture_hook "$HOOKS_DIR/post-edit-shellcheck.sh" \
"$(jq -nc --arg cwd "$PROJECT_DIR" --arg command $'*** Begin Patch\n*** Update File: invalid.sh\n*** End Patch' '{cwd: $cwd, tool_name: "apply_patch", tool_input: {command: $command}}')"
expect_status 2 "invalid shell patch"
}

case_test_runner_patch() {
capture_hook "$HOOKS_DIR/post-test-runner.sh" \
"$(jq -nc --arg cwd "$PROJECT_DIR" --arg command $'*** Begin Patch\n*** Update File: src/probe.rs\n*** End Patch' '{cwd: $cwd, tool_name: "apply_patch", tool_input: {command: $command}}')"
expect_status 0 "Codex source patch test runner" || return 1
[[ "$LAST_ERROR" == *'post-test-runner: true passed'* ]] \
|| fail_case "test runner did not report the isolated test command: $LAST_ERROR"
}

run_case() {
local name=$1
local function_name=$2
local start_ms end_ms elapsed_ms

CASE_INDEX=$(( CASE_INDEX + 1 ))
LAST_DETAIL=""
start_ms=$(now_ms)
if "$function_name"; then
end_ms=$(now_ms)
elapsed_ms=$(( end_ms - start_ms ))
printf 'ok %d - %s # time=%dms\n' "$CASE_INDEX" "$name" "$elapsed_ms"
else
end_ms=$(now_ms)
elapsed_ms=$(( end_ms - start_ms ))
FAILURES=$(( FAILURES + 1 ))
printf 'not ok %d - %s # time=%dms\n' "$CASE_INDEX" "$name" "$elapsed_ms"
printf '# %s\n' "${LAST_DETAIL:-case failed without diagnostics}"
fi
}

CASE_NAMES=(
'Railguard safe Codex response is a no-op'
'Railguard hard block is a deny'
'Railguard approval requirement becomes a Codex deny'
'Railguard rejects a write outside the project fence'
'custom Bash hook allows a safe command'
'custom Bash hook blocks bulk staging'
'custom Bash hook ignores apply_patch body text'
'custom write hook blocks a sensitive patch path'
'post-edit ShellCheck accepts a valid patch'
'post-edit ShellCheck catches an invalid patch'
'post-edit test runner executes for a Codex source patch'
)
CASE_FUNCTIONS=(
case_railguard_safe_noop
case_railguard_hard_deny
case_railguard_approval_becomes_deny
case_railguard_outside_fence
case_validate_bash_safe
case_validate_bash_bulk_stage
case_validate_bash_patch_text
case_validate_write_sensitive_patch
case_shellcheck_valid_patch
case_shellcheck_invalid_patch
case_test_runner_patch
)
TOTAL_CASES=${#CASE_NAMES[@]}
RAILGUARD_VERSION=$("$RAILGUARD_BIN" --version 2>/dev/null || printf 'unknown')
printf 'TAP version 13\n'
printf '1..%d\n' "$TOTAL_CASES"
printf '# railguard=%s\n' "$RAILGUARD_BIN"
printf '# version=%s\n' "$RAILGUARD_VERSION"
printf '# workspace=%s\n' "$BENCH_ROOT"

for (( case_offset = 0; case_offset < TOTAL_CASES; case_offset++ )); do
run_case "${CASE_NAMES[$case_offset]}" "${CASE_FUNCTIONS[$case_offset]}"
done

printf '# result=%d passed, %d failed\n' "$(( TOTAL_CASES - FAILURES ))" "$FAILURES"
[[ $FAILURES -eq 0 ]]
Loading
Loading