Re-render rpk docs when the overrides file changes - #1863
Conversation
The overrides file is the curated-content store for generated rpk pages, but nothing regenerated the pages when it changed: an overrides edit merged and the rendered docs kept the old content until the next release-driven regeneration. docs#1862 made this visible by fixing dozens of overrides whose pages will not update until the next release regen runs. New workflow triggers on pushes to main or beta that touch docs-data/rpk-overrides.json or its schema, re-renders the full rpk tree from the newest committed snapshot (pure from-json render: no rpk binary, no diff, no What's new), and opens a PR with the changed pages using the actions bot token so checks run. Idempotent no-op runs exit green without a PR, and the PR only touches generated pages so it cannot re-trigger the workflow.
✅ Deploy Preview for redpanda-docs-preview ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
📝 WalkthroughWalkthroughThe workflow runs when RPK override files change on Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant GitHubActions
participant SnapshotResolver
participant RPKDocumentationGenerator
participant GitHub
GitHubActions->>SnapshotResolver: select newest non-diff RPK snapshot
SnapshotResolver-->>GitHubActions: return selected snapshot
GitHubActions->>RPKDocumentationGenerator: render documentation
RPKDocumentationGenerator-->>GitHubActions: generated files
GitHubActions->>GitHub: detect changes
alt Generated files changed
GitHubActions->>GitHub: authenticate and create pull request
else No generated changes
GitHubActions-->>GitHubActions: exit without pull request
end
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
.github/workflows/rerender-rpk-docs-on-overrides-change.yml (1)
68-68: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick winPin the generator to an exact version.
^5.3.0can resolve to a later 5.x release, so the same overridden config can generate different RPK docs after a dependency update. Use the exact generator version, such as5.3.0.Proposed fix
- npx --yes -p `@redpanda-data/docs-extensions-and-macros`@^5.3.0 doc-tools generate rpk-docs \ + npx --yes -p `@redpanda-data/docs-extensions-and-macros`@5.3.0 doc-tools generate rpk-docs \🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/rerender-rpk-docs-on-overrides-change.yml at line 68, Update the doc-tools generator invocation to pin `@redpanda-data/docs-extensions-and-macros` to the exact 5.3.0 version instead of using the ^5.3.0 range, while leaving the rpk-docs generation command unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/rerender-rpk-docs-on-overrides-change.yml:
- Around line 69-70: Update
.github/workflows/rerender-rpk-docs-on-overrides-change.yml lines 69-70 to pass
steps.snapshot.outputs.snapshot through the step environment and use the quoted
SNAPSHOT variable for --from-json. Also update lines 100-106 to use the quoted
GITHUB_REF_NAME and GITHUB_SHA environment variables with printf instead of
embedding GitHub expressions in the shell script.
---
Nitpick comments:
In @.github/workflows/rerender-rpk-docs-on-overrides-change.yml:
- Line 68: Update the doc-tools generator invocation to pin
`@redpanda-data/docs-extensions-and-macros` to the exact 5.3.0 version instead of
using the ^5.3.0 range, while leaving the rpk-docs generation command unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: e2b4ec98-b831-41ab-83e5-fdc000fc087a
📒 Files selected for processing (1)
.github/workflows/rerender-rpk-docs-on-overrides-change.yml
| --from-json "${{ steps.snapshot.outputs.snapshot }}" \ | ||
| --summary-file /tmp/pr-summary.md |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Pass expression values through environment variables, not shell templates. GitHub expands expressions before Bash parses the script. This lets repository-derived or dispatch-derived values become shell syntax.
.github/workflows/rerender-rpk-docs-on-overrides-change.yml#L69-L70: passsteps.snapshot.outputs.snapshotthroughenvand use"$SNAPSHOT"..github/workflows/rerender-rpk-docs-on-overrides-change.yml#L100-L106: use"$GITHUB_REF_NAME"and"$GITHUB_SHA"withprintf.
Proposed fix
- name: Re-render rpk docs
+ env:
+ SNAPSHOT: ${{ steps.snapshot.outputs.snapshot }}
run: |
npx --yes -p `@redpanda-data/docs-extensions-and-macros`@^5.3.0 doc-tools generate rpk-docs \
- --from-json "${{ steps.snapshot.outputs.snapshot }}" \
+ --from-json "$SNAPSHOT" \
--summary-file /tmp/pr-summary.md {
- echo "Automated re-render of the generated rpk reference pages after an overrides change on \`${{ github.ref_name }}\` (${{ github.sha }})."
+ printf 'Automated re-render of the generated rpk reference pages after an overrides change on `%s` (%s).\n' \
+ "$GITHUB_REF_NAME" "$GITHUB_SHA"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| --from-json "${{ steps.snapshot.outputs.snapshot }}" \ | |
| --summary-file /tmp/pr-summary.md | |
| - name: Re-render rpk docs | |
| env: | |
| SNAPSHOT: ${{ steps.snapshot.outputs.snapshot }} | |
| run: | | |
| npx --yes -p `@redpanda-data/docs-extensions-and-macros`@^5.3.0 doc-tools generate rpk-docs \ | |
| --from-json "$SNAPSHOT" \ | |
| --summary-file /tmp/pr-summary.md |
| --from-json "${{ steps.snapshot.outputs.snapshot }}" \ | |
| --summary-file /tmp/pr-summary.md | |
| { | |
| printf 'Automated re-render of the generated rpk reference pages after an overrides change on `%s` (%s).\n' \ | |
| "$GITHUB_REF_NAME" "$GITHUB_SHA" | |
| echo | |
| echo "Review focus: the changed pages should reflect exactly the merged overrides edit, nothing else. The generator version is pinned to the same range the release regeneration uses, so unrelated churn here means the branch missed a regeneration and this PR is catching it up." | |
| echo | |
| cat /tmp/pr-summary.md 2>/dev/null || true |
🧰 Tools
🪛 zizmor (1.28.0)
[info] 69-69: code injection via template expansion (template-injection): may expand into attacker-controllable code
(template-injection)
📍 Affects 1 file
.github/workflows/rerender-rpk-docs-on-overrides-change.yml#L69-L70(this comment).github/workflows/rerender-rpk-docs-on-overrides-change.yml#L100-L106
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/rerender-rpk-docs-on-overrides-change.yml around lines 69
- 70, Update .github/workflows/rerender-rpk-docs-on-overrides-change.yml lines
69-70 to pass steps.snapshot.outputs.snapshot through the step environment and
use the quoted SNAPSHOT variable for --from-json. Also update lines 100-106 to
use the quoted GITHUB_REF_NAME and GITHUB_SHA environment variables with printf
instead of embedding GitHub expressions in the shell script.
Source: Linters/SAST tools
micheleRP
left a comment
micheleRP
left a comment
There was a problem hiding this comment.
Approving the mechanism, with one ordering requirement that I would treat as a merge gate rather than a nicety.
This workflow must merge after #1865, not before. It runs --from-json, so it never installs a plugin and never extracts flags. Combined with #1865 that is a live regression on the published rpk connect run page.
The chain, all verified with the published 5.3.1 against main plus the three merged overrides PRs:
- The committed snapshot has
rpk connect runwith 0 flags. Connect flag data only exists via live extraction. - Until now that was masked: the override carried a hand-written 7-row Flags table that rendered unconditionally. #1865 removes it, correctly, because #225 made the extracted table authoritative.
- So a
--from-jsonregeneration today renders that page with no Flags section at all: Usage straight to Global flags. Not a stale table, no table.
Which means: merge #1865 while this is live, the overrides change triggers a re-render, and the page silently loses its flag documentation.
The mitigation works, and it is just sequencing:
--plugin connect --rpk-bin -> Extracted flags for 16 command(s)
-> Saved versioned JSON to docs-data/rpk-v26.2.1-rc2.json
-> snapshot connect run flags: 13
then --from-json -> Flags sections: 1, rows: 13
So once the re-dispatch has persisted extracted connect flags into the committed snapshot, from-json regeneration is safe and this workflow is fine. Concretely, before merging #1865 or this, confirm the committed docs-data/rpk-v26.2.1-rc2.json has 13 flags on rpk connect run.
Suggested follow-up in the generator: nothing catches this class today. A command that rendered flags previously and now renders none is almost always a data problem, not a real change. A warning on a nonzero-to-zero flag-count transition would have surfaced this before it reached a page, and it generalizes to any plugin family whose flags come only from extraction.
Ordering, tested
Four open PRs edit rpk-overrides.json. I test-merged all pairs: #1838, #1860, and #1865 are mutually clean in any order, and only #1862 conflicts, with both #1838 and #1865, in either direction. Running #1838 then #1860 then #1865 as a real sequence is clean, leaves valid JSON, and produces all four intended effects at once.
So: re-dispatch, confirm the snapshot has connect flags, then #1860 and #1865, then this, then rebase #1862.
|
Follow-up: the ordering requirement in my review has now been independently re-verified in a fresh pass, link by link from primary sources rather than from my earlier notes, and it holds — with two refinements that sharpen the merge gate. Re-verified on a confirmed-pristine checkout of current main plus the merged overrides PRs: the committed snapshot's connect subtree has zero flags everywhere ( The two refinements:
The nonzero-to-zero flag-count warning suggested earlier now has direct evidence behind it: the log for the failing case contains zero errors and zero warnings. |
2 participants
What
Closes an automation gap surfaced by #1862:
docs-data/rpk-overrides.jsonis the curated-content store for the generated rpk reference pages, but no workflow watched it. An overrides edit merged and the rendered pages kept the old content until the next release-driven regeneration, so the repo carried overrides the published docs did not reflect.New workflow: on any push to
mainorbetathat touches the overrides file or its schema, re-render the full rpk tree from the newest committed snapshot and open a PR with the changed pages.Design
doc-tools generate rpk-docs --from-json <newest snapshot>with no diff and no What's new update. No rpk binary is needed because the snapshot already carries the tree and flags, so the run takes about two minutes.sort -V(GA ranks above rc), same as the plugin receiver workflow in ci: regenerate a single rpk plugin's docs when the plugin releases #1834.secrets.GITHUB_TOKENPRs do not trigger Actions).betapicks the workflow up through the regular main to beta sync.Sequencing
Merge after redpanda-data/docs-extensions-and-macros#225 publishes doc-tools 5.3.0 (the workflow resolves
@^5.3.0at runtime), the same constraint as #1834. Once this is in, merging #1860, #1861, and #1862 automatically produces the re-render PR that brings the pages in line with the fixed overrides.Validation
Workflow parses clean (act listing and YAML check). The generation command, snapshot resolution, idempotence, and no-changes behavior are the paths already exercised end to end in the #225 validation (18+ full regeneration rounds on a pristine clone, including repeated no-op re-renders).
Related PRs (rpk docs automation train)
See redpanda-data/docs-extensions-and-macros#225 for the train overview.
Jira
Prevention layer for DOC-2408, and part of the fix chain for DOC-2407: once merged, overrides changes like #1838 regenerate the affected pages automatically.