Repository navigation
fix(ci): fix LLM review crash and deliver override alerts to Slack - #169
Conversation
… override alerts The review of #156 crashed with "string indices must be integers": Bedrock returned `findings` as a JSON string, format_review indexed it, and the apply was blocked with risk FAILED. review_plan now normalises the tool answer (parses stringified arrays/objects, drops unusable entries, rejects unknown risk). The plan-review Lambda bundles the same script, so app reviews get it. Override alerts were reported as "sent" without checking Slack's reply. Payloads were hand-built JSON strings with no size limits, so long HIGH findings (>3000 chars per section) came back invalid_blocks and nobody saw them. Both notify scripts now go through slack_post.py: JSON-escaped, size-limited blocks, Slack's reply checked, plain-text fallback on rejection, CI error annotation if still undelivered. A FAILED review is labelled as such instead of "HIGH". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Terraform Plan🚧 Changes detected — Plan: 12 to add, 8 to change, 0 to destroy. Plan output
LLM ReviewRisk: 🔴 HIGH Plan creates 12 resources (app-peers-sync Lambda with EventBridge rules, IAM role, service discovery namespace) and updates 8 in place, mainly IAM guardrail and developer boundary policies, with 0 destroys; the boundary changes remove the ability to manage IGWs/NAT gateways and Route53 hosted zone deletion and add a DenyAppDNS statement, and the Lambda role gets security group editing rights.
|
approve-override.yml's "Risk Override Approved" Slack step has always logged "No webhook URL available": javabin-ci-override-approver could only invoke the apply gate. Grant ssm:GetParameter on that one parameter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Terraform Plan🚧 Changes detected — Plan: 0 to add, 3 to change, 0 to destroy. Plan outputLLM ReviewRisk: 🟡 MEDIUM Plan updates 3 existing resources in place (0 add, 0 destroy): adds an SSM read permission for a Slack webhook parameter to the CI override approver role, restricts Google identity provider login to the java.no hosted domain, and redeploys the plan-review Lambda code.
|
Why
LLM review failed: string indices must be integers, not 'str'. Bedrock returnedfindingsas a JSON string, andformat_reviewindexed it.notify-high-risk.shandnotify-slack.shbuilt JSON by string interpolation, posted withcurl -s > /dev/null, and always printed "sent".ok.invalid_blocks.What
javabin-ci-override-approvercan nowssm:GetParameterthe override-alerts webhook. The "Risk Override Approved" step had always loggedNo webhook URL available(seen in every approve-override run), so approvals never reached Slack.review-plan.py:normalize_review()parses stringified arrays and objects, keeps plain-string findings, drops unusable entries, and rejects risks outside LOW/MEDIUM/HIGH. The plan-review Lambda bundles this file, so app reviews are fixed too.scripts/slack_post.py, used by both notify scripts:::error::annotation if delivery still fails. The job doesn't fail.CLAUDE.mdscript table anddocs/ci-workflow.md.How verified
python3 -m unittest discover -s scripts/tests: 126 tests, OK. New tests cover stringified findings, long findings, special characters, the fallback and the double failure.python3 -m unittest discover -s terraform/lambda-src/tests: 194 tests, OK.ok, and the old FAILED-case payload is accepted.Follow-ups
🤖 Generated with Claude Code