Skip to content

ci: finish the Pullfrog timeout quest on DeepSeek V4.1 Flash - #7714

Merged
kixelated merged 5 commits into
mainfrom
quest/m0/ci/pullfrog-timeout
Nov 4, 2026
Merged

kixelated merged 5 commits into
mainfrom
quest/m0/ci/pullfrog-timeout

Conversation

@kixelated

@kixelated kixelated commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Pullfrog reviews were hitting the action's one-hour run limit on large PRs (41178266611, 41362378749, 41404718134, all at 60m+).

Approach

The quest finishes with no workflow change:

  • This draft's earlier attempt added effort: medium and timeout: 1h to .github/workflows/pullfrog.yml. That change is reverted. Pullfrog's OSS proxy forces high effort (main.ts#L496), so the input did nothing. 1h is already the default.
  • The chosen model, DeepSeek V4.1 Flash, is already what runs. On the OSS route, Pullfrog's server picks the model (utils/runContext.ts proxyModel). The workflow's model: input is not "explicit" (utils/payload.ts#L385-L388), so on this route it is ignored (utils/modelAccess.ts, then main.ts clears resolvedModel when proxyModel is set). DeepSeek Flash (deepseek/deepseek-flash) is the OSS-funded default (models.ts AUTO_TIER_TARGET). Every inspected run logs proxy: oss → openrouter/deepseek/deepseek-v4.1-flash, including the 10-31 and 11-02 timeouts. Runs since 2026-11-03 ~18:30Z log the dated pin deepseek-v4.1-flash-1017. A model: line in the workflow would be dead config, and it would fail if the repo ever left the OSS route without a DeepSeek key, so none is added.
  • The trial below meets the Goal, so quest rm deletes the quest and its link in quest/m0/ci/README.md.

Trial

moq-dev/moq#7729 (135 files, +1055/−545):

  • Marked ready for review at 2026-11-03T23:18:12Z; the pullfrog[bot] review posted at 23:29:11Z. That is 10m59s, under one hour.
  • Run 41738063791: Run agent took 11m41s (23:18:27Z to 23:30:08Z).
  • The run logged model openrouter/deepseek/deepseek-v4.1-flash-1017 (proxy), effort high, timeout 1h (default).

Wider sample from the Actions API (600 runs, 2026-11-01T17:35Z to 2026-11-03T23:30Z): the last run that took ≥55 minutes was 41404718134, at 2026-11-02 07:27Z. Of roughly 450 completed runs since then, none came close; the longest of the latest 93 took 35 minutes (a "Fix CI" run).

Decisions

  • ✅ Effort (maintainer, 2026-11-03): Pullfrog forces high effort on the OSS route, so effort can't be lowered. This supersedes the earlier effort-first brief, so the effort: medium input is dropped.

  • Review model (maintainer, 2026-11-03, after asking for public rankings):

    • ✅ DeepSeek V4.1 Flash
    • Muse Spark Contributor
    • MiniMax M2

    It was already Pullfrog's OSS default for this repo, chosen server-side. The repo needs no change, and nothing needs clicking in the Pullfrog console.

  • ✅ Completion (maintainer, 2026-11-03): trial on one large PR and record a review under one hour. Met above, so the quest is deleted.

Impact

  • Public API: none.
  • Wire: none.
  • CI: pullfrog.yml is byte-identical to main.

Follow-ups

  • Optional maintainer action: if you want to stop Pullfrog from changing the OSS default under this repo, select DeepSeek Flash explicitly in the Pullfrog console's model setting for moq-dev/moq rather than the automatic default. This is not verified from here; the console isn't readable from the repo.
  • The missing @moq/libav@0.1.0 / absent Nix setup error in Pullfrog's dependency install (41614007828) belongs to Claim the new npm names.

Validation

  • actionlint and zizmor --offline --quiet .github: passed.
  • quest check --base origin/main: passed (563 documents).
  • git diff --check: passed.
  • just ci not run (the brief says not to build); the diff against main only deletes a quest and its link.

Review

Local quest review --new: Codex, gpt-6.1-sol, medium effort, round 1, reviewed head 72665d431419ee243f42617ab1509c224cb855f7: no findings.

(Written by Claude Opus 5.5)

kixelated and others added 3 commits November 3, 2026 01:51
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
Co-Authored-By: GPT-6.1 Sol <noreply@openai.com>
@kixelated kixelated changed the title Pullfrog reviews finish inside their time limit ci: request medium-effort Pullfrog reviews Nov 3, 2026
kixelated and others added 2 commits November 3, 2026 15:33
Pullfrog's OSS proxy already runs DeepSeek V4.1 Flash (the funded default,
chosen server-side) and forces high effort, so the workflow's effort input
was a no-op. Drop it, and delete the quest: a 135-file review now posts in
11 minutes, inside the one-hour run limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title ci: request medium-effort Pullfrog reviews ci: finish the Pullfrog timeout quest on DeepSeek V4.1 Flash Nov 3, 2026
@kixelated

Copy link
Copy Markdown
Collaborator Author

Design change: the effort: medium / timeout: 1h workflow inputs are reverted, and the quest is deleted.

The maintainer decided on 2026-11-03 that effort can't be lowered (the OSS proxy forces high) and picked DeepSeek V4.1 Flash as the review model. Pullfrog's server already picks that model for this repo on the OSS route, and the workflow's model: input is ignored there. Every inspected run, including the timeouts, logged deepseek-v4.1-flash. The one-large-PR trial (moq-dev/moq#7729, 135 files) posted its review in 10m59s, so the quest's Goal is met with no config change. Details are in the PR body.

(Written by Claude Opus 5.5)

@kixelated
kixelated marked this pull request as ready for review November 3, 2026 23:35
@coderabbitai

coderabbitai Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 4 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: fdd03690-4859-4c66-a61a-3f26ea49598e
Commits

Reviewing files that changed from the base of the PR and between 9294812 and 72665d4.

Files selected for processing: 2
  • quest/m0/ci/README.md
  • quest/m0/ci/pullfrog-timeout.md
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed head 72665d4.

No actionable findings. The diff removes the finished quest and index entry without changing the workflow. The linked trial job completed successfully, and its log confirms the proxy model, high effort, and one-hour default described in the PR. Recording the server-selected default rather than adding an ignored workflow input is a reasonable direction.

Verification: GitHub-only static review of the quest diff and linked trial job/log. I did not independently reproduce the wider run sample or run CI.

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

  • Delete the Pullfrog timeout quest — removes quest/m0/ci/pullfrog-timeout.md and its bullet in quest/m0/ci/README.md; the quest's stated goal (large reviews finish under the run limit) is recorded as met by the #7729 trial, with the effort/timeout inputs established as dead on the OSS route so no workflow edit is needed.
  • No workflow change — pullfrog.yml is unchanged from main, matching the stated revert of the earlier effort: medium / timeout: 1h attempt.
  • No dangling references — no remaining links to pullfrog-timeout or the removed title anywhere in the tree, and the README list stays well-formed.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash-1017 (free via Pullfrog for OSS) | 𝕏

@kixelated
kixelated added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit bb3a5fb Nov 4, 2026
12 checks passed
@kixelated
kixelated deleted the quest/m0/ci/pullfrog-timeout branch November 4, 2026 00:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant