Fix github-discussion-query payload scaling by eliminating jq argument growth - #55560
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
Thanks for tackling the discussion-query pagination limit! This fix — switching from Since this is still a draft, here's what would help get it across the finish line:
The core refactor looks solid — swapping to temp files + Once the tests are in place and the checklist is complete, this should be ready for merge!
|
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
github-discussion-query payload scaling by eliminating jq argument growth
There was a problem hiding this comment.
Pull request overview
Reworks github-discussion-query to use temporary files, avoiding shell argument-length limits for large result sets.
Changes:
- Uses file-backed page merging and jq filtering.
- Adds automatic temporary-file cleanup.
- Regenerates all importing workflow lock files.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/shared/github-queries-mcp-script.md |
Implements scalable file-backed discussion processing. |
.github/workflows/daily-regulatory.lock.yml |
Regenerates the imported query script. |
.github/workflows/daily-performance-summary.lock.yml |
Regenerates the imported query script. |
.github/workflows/smoke-copilot.lock.yml |
Regenerates the imported query script. |
.github/workflows/smoke-copilot-arm.lock.yml |
Regenerates the imported query script. |
.github/workflows/smoke-copilot-aoai-entra.lock.yml |
Regenerates the imported query script. |
.github/workflows/smoke-copilot-aoai-apikey.lock.yml |
Regenerates the imported query script. |
Review details
💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.
- Files reviewed: 7/8 changed files
- Comments generated: 0
- Review effort level: Balanced
|
🎉 This pull request is included in a new release. Release: |
github-discussion-querysilently degraded coverage for higher--limitvalues because discussion payloads were passed through jq via shell arguments/here-strings, eventually hitting OS argument-length limits. This capped effective scan depth and caused daily report cross-check gaps.Pagination/data path hardening
--argjson all "$OUTPUT") with temp-file-backed merges (jq -s) so payload size no longer depends on shell argument limits.sincepagination semantics and early-stop behavior while removing large argument transport from the hot path.Filter/output execution path
jq "$JQ_FILTER" <<< "$OUTPUT"jq "$JQ_FILTER" "$OUTPUT_FILE"Temp-file lifecycle safety
trapcleanup for all intermediates used during page extraction, merge, and final projection.