[jerelvelarde]

Review status

Marked ready for maintainer review at the user’s request. That is a request for review; it is not a claim that this is ready to merge or to enable automatic posting.

Fifteen broad review rounds preceded this branch. The thirteen commits after the previously reviewed head c602d6c answer a later, separate round of review updates and are not the output of those earlier rounds. Six of them carry the earlier concerns from that round:

• Expected tool failures — directories, symlinks and submodules, invalid paths, and GitHub transport and API errors — return bounded tool results to the investigator instead of ending the run, so it can correct the lookup or fall back to other evidence. Cancellation and the investigation budget still terminate the run.
• GitHub evidence reads authenticate with the existing App installation at read-only contents permission, bounded by the investigation deadline and a ten-second installation request deadline. One investigation can stop waiting without canceling another caller’s shared token request, and incomplete configuration cannot silently fall back to anonymous access.
• An interrupted delivery keeps the owed handoff reason, waits for a possibly in-flight post, and warns the human when delivery is unknown. A successful post records confirmation even when the row stays pending for escalation.
• The whole web response serializes with its footer last, keeping technical details and sources ahead of it in saved suggestions, shadow records and streaming output, while the native separate details pane is preserved.

The current amendment round adds five further concerns, each with its own red/green record:

• The Discord footer is counted inside the message budget, with UTF-16 boundary and indentation handling corrected.
• An exhausted GitHub rate limit is classified from its response headers rather than inferred.
• Evidence-auth failure is reported as a safe operator diagnostic — an allowlisted category name, never a credential value — leaving the recoverable auth_unavailable result the model sees unchanged.
• Validated code stays literal across both web serializations and in both validated fields; only the unvalidated disclaimer is sanitized, and the legacy formatter path is preserved.
• Deterministic handoff reasons the pipeline already knows are preserved and carried across into a durable consumer, so a known reason reaches the human-facing ticket note, the system message and the recorded-delivery retry instead of stopping at pipeline metadata.

Two documentation-only follow-ups clarify that disclaimer-only sanitization applies to the web branch and that deployment configuration needs review before rollout.

These have accepted source-and-test review, not completed review confirmation: a scoped twenty-cluster confirmation at runtime head b09050e is still running after six interrupted jobs were restarted, and the broader original-PR confirmation and the original 130-case / 90-disposition promotion audit are unfinished. The two original-PR reason-propagation defects found in this review are resolved by the producer and durable-consumer changes in these commits; the broader audit remains open. No convergence claim is made.

Current PR head: cb9f8dc. CI run 37772441929 and security run 37772442032 are green at this exact head. CI run 37701387192 and security run 37701387244 passed at runtime head b09050e. Local checks are recorded below. Review confirmation and the broader audit remain unfinished.

Summary

Support replies could repeat the report, guess API compatibility, or bury the useful answer in a long response. This change introduces a bounded support investigator using GPT-5.6 Luna and the OpenAI Agents SDK. It validates structured evidence, runs a separate verifier, and renders a concise human paragraph with expandable technical details and sources on GitHub and in web QA.

Luna also handles confidence verification, ticket classification, and sentiment. The default path requires only OPENAI_API_KEY. The existing queue, one-primary-reply-per-ticket behavior, delivery adapters, feedback calibration, and durable escalation workflow remain in place. Delivery fixes preserve expanded details in saved suggestions and shadow records, and report unresolved handoff state instead of silently accepting it.

Runtime requirement: Node 24+ for local and container execution, aligned with CI and the OpenAI dependency stack.

Old vs. new responses

The table compares historical failure examples with actual saved output from the new pipeline, evaluated September 21, 2026, on build d8c932f. It shows the public body; AI disclosure and attribution footers are omitted here for readability.

Evidence scope: D is a verified excerpt from an original Outpost reply. A, B, C, and E describe reconstructed regression fixtures, not original transcripts. All five new outputs are real live runs. No replies were posted.

Case	Previous response	New pipeline's actual public body	Assessment
A · Deep Agents subagents	Reconstructed fixture: treated support as uncertain, suggested a speculative workaround, and sent the developer to a GitHub discussion.	Yes. Deep Agents supports subagents: the top-level agent can spawn specialized agents for delegated tasks, and CopilotKit’s integration can stream those subagent interactions to the frontend.	Direct, 26-word answer. Version scope and evidence are collapsed below it.
B · Rendering delegation	Reconstructed fixture: mixed API generations and hedged an unverified render-hook name.	This needs a maintainer review to give you a reliable next step.	The draft failed the exact evidence-quote check. The handoff avoids publishing it, but gives no useful clarification.
C · Maintainer follow-up	Reconstructed fixture: praised a maintainer and claimed it could not read other replies.	Yes—the reporter confirmed that aligning the package versions restored the chat, so the rendering issue appears resolved. The exact framework, API generation, and package versions were not provided, so the underlying compatibility mismatch cannot be identified from the available thread.	Reads the supplied resolution correctly, but adds little and discusses missing context. A no-reply policy is still needed.
D · Dependency audit, #6423	Verified original excerpt: “I haven't read the source files or run resolution myself, so I won't confirm causation or prescribe the specific fix”	This needs a maintainer review to give you a reliable next step.	Less noise, but no technical resolution. The investigator could not verify historical manifests and lockfile within its tool budget.
E · MCP SSE headers, #6927	Reconstructed fixture: praised the reproduction, explained its own limitations, and deferred to engineering.	The report matches the current v2 source: the SSE config exposes a raw headers map, while the transport requires those headers under requestInit. The current main branch already contains the proposed mapping, so this appears fixed in source, but the published version containing it cannot be determined without the exact installed @copilotkit/runtime version.	A concrete finding in 53 words, with installed-version uncertainty visible. Code and sources are collapsed.

The earlier September 17–18 pipeline run remains archived separately; it is not relabeled as this build.

The original D response had 281 whitespace-delimited body words/tokens, including Markdown heading markers and excluding its opening disclosure and footers; it had no dropdown. The new handoff body has 12 words. That is a readability improvement, not a technical resolution.

See the complete new response format: Deep Agents subagents

The following is the saved formatted output. The inner disclosure is the actual technical-details dropdown.

Yes. Deep Agents supports subagents: the top-level agent can spawn specialized agents for delegated tasks, and CopilotKit’s integration can stream those subagent interactions to the frontend.

Technical details and sources

The Deep Agents documentation explicitly lists “spawn subagents for specialized tasks” as a capability. CopilotKit’s integration documentation also states that users can interact with subagents as they spawn. This support belongs to the Deep Agents backend; CopilotKit provides the frontend surface and AG-UI connection rather than defining the subagent execution model.

Applies to: Deep Agents integration; no specific package version or CopilotKit API generation was provided.

API version: unknown

Sources

• Source 1

---

_{🤖 Generated by CopilotKit AI Support · Documentation}

See the complete new response format: MCP SSE authentication headers

The following is the saved formatted output. It cites the source retrieved during evaluation; it does not establish a minimum fixed release or an executed reproduction.

The report matches the current v2 source: the SSE config exposes a raw headers map, while the transport requires those headers under requestInit. The current main branch already contains the proposed mapping, so this appears fixed in source, but the published version containing it cannot be determined without the exact installed @copilotkit/runtime version.

Technical details and sources

The verified evidence supports the reported defect mechanism and correction:

• MCPClientConfigSSE documents headers?: Record<string, string>.
• The current v2 runtime constructs SSE transport options as { requestInit: { headers: serverConfig.headers } }, rather than passing the raw map as the second argument.
• The same source catches MCP connection failures, logs that the server is being skipped, and continues, which explains why the agent can appear without those tools.

For a checkout containing the current source, the relevant implementation is:

transport = new SSEClientTransport(
  new URL(serverConfig.url),
  serverConfig.headers
    ? { requestInit: { headers: serverConfig.headers } }
    : undefined,
);

This finding is scoped to CopilotKit v2. The retrieved implementation is from main; that proves the fix exists in source but does not prove which published package release includes it. Provide the exact @copilotkit/runtime version, including any prerelease tag, to verify whether upgrading resolves the issue for the affected installation.

Applies to: CopilotKit v2 BuiltInAgent MCP SSE configuration; verified in current main source. Published-version applicability remains unresolved until the exact @copilotkit/runtime version is known.

API version: v2

Sources

• Source 1
• Source 2

---

_{🤖 Generated by CopilotKit AI Support · Documentation}

Architecture and behavior

Area	Old baseline	New implementation
Generation	Fixed docs/code retrieval followed by one Claude Sonnet 4.6 call	Luna investigation through the OpenAI Agents SDK and Responses API
Investigation	No tools on the generator	Read-only, adaptive CopilotKit/AG-UI docs/code search, supplied-thread reading, source-at-ref lookup, and tagged-release lookup
Bounds	Single generation call	Six tool calls, eight turns, 60-second investigation deadline; tools are disabled after the sixth call to leave room for synthesis
Response contract	Free-form Markdown, mainly prompt-controlled	answer / partial / route, one summary paragraph up to 80 words, details, API generation, applicability, evidence quotes, and internal handoff reason
Rendering	Collapsed long code blocks on GitHub	One GitHub technical-details dropdown; native disclosure in web QA; Discord splits to ordinary messages, and Slack and Teams still fall through to the legacy format
Verification	Haiku judgment using truncated draft/source context	Separate Luna judgment over the complete bounded draft/evidence and supplied conversation, followed by grounding checks
Publication controls	Grounding suppression; low confidence did not generally withhold the draft	Invalid output, unusable verification, intentional routing, or a verifier score below 0.40 withholds the technical draft and produces a concise handoff
Auxiliary models	Haiku defaults for confidence, classification and sentiment	Luna defaults throughout; strict structured results and preserved degraded states

Ticket classification preserves recognized critical incidents when the model fails or underestimates urgency. Regression tests distinguish incident reports from negation, prevention, and diagnostic questions.

The investigator and verifier receive available author/timestamp metadata. The thread tool reads supplied history; it does not fetch missing remote comments or claim the remote thread is complete. Version-aware search and evidence checks reduce definite v1/v2 mismatches. Source reads can pin commits, but search-result citations are not universally pinned. Main-branch code is not release evidence.

The style linter is connected in report mode by default; AI_DRAFT_LINT_MODE=enforce enables its blocking rules. Exact quote matching checks provenance, not whether every conclusion follows. The verifier is a separate call using the same model family, so it is not independent ground truth.

Validation and live evaluation

Runtime validation recorded at b09050e (tree 7baae56), eleven commits after the previously reviewed head c602d6c. Current PR head cb9f8dc adds two documentation-only commits: the formatter comment now explicitly scopes the web behavior, and rollout instructions accurately describe configuration review. The formatter’s executable output is byte-identical to b09050e, all other runtime source is unchanged, and both changed files pass a fresh formatting check. GitHub CI run 37772441929 and security run 37772442032 passed at cb9f8dc, including build, schema checks, formatting, lint, typechecks and tests.

• 5,901 tests passed at runtime head b09050e: 4,599 core tests across 78 files, and 1,302 downstream tests across 94 files — 856 web, 84 worker, 73 Teams, 66 Discord, 64 root scripts, 60 GitHub App, 52 Slack, 47 Linear sync. The downstream suites were run fresh against the current build output. The core suite was measured twice at this same head and tree and reused rather than repeated; the web suite aliases core to src, so it exercises source rather than the compiled artifact.
• Eleven cold typechecks passed: four core projects with their build info deleted first, plus each of the seven workspaces that depend on the core package. discord-mcp and docs were deliberately skipped — neither depends on it.
• Lint exits clean with no errors and the same 40 pre-existing warnings across 27 files. The five warnings inside a changed test file sit at identical lines on both sides of the diff; nothing was repaired or suppressed.
• Formatting passed --check on all 16 changed Prettier-supported files. The seventeenth, schema.prisma, is outside the Prettier glob and was checked with prisma validate instead, which passed against a placeholder connection string — static schema parsing, no database contacted. Its only change is a comment block, and the original migration is byte-unchanged, so the existing column is untouched.
• The full production build passed: 10 tasks, 10 successful. All 8 packages and apps within the change's dependency reach were cache misses and executed fresh; the 2 cached tasks (docs, discord-mcp) are outside it.
• Compiled modules were loaded and inspected after that build: 54 AI runtime exports and 21 queue exports. The public surface moves by exactly one addition versus c602d6c, with none removed. The new GitHub evidence-auth module is module-private — reached from the support agent, absent from the package index — and its only module specifiers are two Octokit packages already declared at c602d6c, so no dependency entered the runtime graph.

The reasoning-continuity proof — the Agents SDK 0.18.0 converter reading and the two-response interaction showing encrypted reasoning carried across a stateless request — is a dated historical record from an earlier session, not a new live call on this head. The new App-installation authentication has been exercised only against synthetic credentials and offline SDK requests; live installation authentication is still unverified.

This amendment changes no Dockerfile and no runtime dependency, so no container build was run and no container runtime claim is made for it; the earlier c602d6c image proof is archived and does not carry forward. No provider, GitHub, or platform call was made during these checks, no service was deployed, and no support reply was posted. The local credentials and /qa check remains tied to preview a4bf059, which is still the old build and does not contain any of these commits.

The five-case output comparison remains the September 21 live run at d8c932f. It is not a replay of this head, and tests and verifier scores do not establish real-world support accuracy.

Regression coverage includes the SDK tool loop, malformed/segmented structured output, evidence validation, investigation budgets, provider failures, rendering, stream boundaries, and delivery/escalation invariants.

The five-case evaluation was rerun September 21, 2026, against a freshly emitted AI build of d8c932f. The raw artifact records the commit and compiled-module hashes:

• Five live Luna pipeline replays produced one answer, two partial answers, and two handoffs. Classification and sentiment returned valid, non-degraded results in all five; this establishes call/schema health, not semantic accuracy.
• In an earlier, separate September 17–18 live test, the Luna verifier withheld 5/5 reconstructed historical bad replies, scoring them 0.05–0.18 against the 0.40 threshold.
• Local replay times were 21.4–35.6 seconds. Total input ranged from 11,844 to 74,892 tokens, including cached tokens and discarded drafts. There is no matched old-system latency or cost measurement.

This is not a controlled Sonnet-versus-Luna comparison. Historical failures predate prompt improvements already present in the old code baseline. A/B use reconstructed questions, C uses a synthetic resolved conversation, and D/E use real issue openers without later comments. Retrieval uses current evidence, not snapshots from the original reply dates. Model, prompts, tools, validation, and rendering changed together; these results do not isolate model quality or estimate production accuracy.

Rollout and remaining work

Keep posting disabled during further evaluation. This change does not deploy a service or enable external replies. Before enabling automatic posting:

• Give Discord a compact summary with an on-demand details interaction; details currently remain ordinary visible messages.
• Carry validated-code rendering into Slack and Teams, which still fall through to the legacy format. The collapsed GitHub section and the web disclosure are the only two surfaces this change actually gives a distinct presentation.
• Preserve useful clarification questions instead of replacing invalid or routed drafts with generic handoffs (B).
• Add a no-reply decision for resolved conversations and maintainer-owned follow-ups (C).
• Improve historical repository-layout and published-package evidence lookup (D).
• Deduplicate retrieved context, improve citation pinning, and measure the token/coverage tradeoff.
• Run matched old/new cases with frozen evidence and blind human review, grading added value, factual/version correctness, unnecessary handoffs, and appropriate silence.

Luna is the default here because that was an explicit user direction to route all AI calls through it. That direction is still being reconciled against the proposal to stay on Anthropic until a matched evaluation exists, so the default is a rollout decision rather than a settled benchmark result. For explicit rollback, set AI_RESPONSE_PROVIDER=anthropic, configure the Anthropic key, and clear or replace OpenAI-specific model overrides. Provider failures never silently switch providers.

The paragraph-plus-dropdown presentation follows the Orca response on #3434; that comment is a layout reference, not a correctness benchmark.

[jerelvelarde]

All three are in, pushed as 59965af. Thanks for keeping the branch current with main while this sat — I'd done the work locally and never pushed it, which is the second time I've left you waiting on something that was already finished. Sorry.

The App permission

The docs now carry it in two places: step 3 of the cutover, and the App setup list (Issues: Read & write, Discussions: Read & write, Contents: Read-only, with a note that Metadata is automatic). There's also an explicit procedure for an App that already exists — an org owner or App manager sets Contents under Permissions & events, and the installation then has to accept the updated permissions, because GitHub issues no narrowed token until it does.

I could not answer your actual question. gh api /orgs/CopilotKit/installations needs admin:org and my token doesn't have it, so I can't see what the live App actually holds. That one still needs someone who can open the App's settings page. If it already has Contents, nothing to do beyond the docs; if it doesn't, the installation needs to accept the update before the OpenAI default deploys, or every evidence read fails from that moment.

Worth recording what the failure looks like so it's recognisable: the model's tool result reports auth_unavailable, and the worker logs [SupportAgent] GitHub evidence authentication unavailable: with category token_exchange_failed — the same category any failed exchange gets, so the log alone won't tell you it's a missing permission rather than a bad key.

The 2000-character cut

Fixed, and wider than you described. The guard isn't just "drop a trailing high surrogate" — the store refuses three things, so all three are handled: a high surrogate with no low one after it, a low surrogate with no high one before it (a reversed pair is two separate faults, not one), and U+0000. Each becomes U+FFFD before the string is bounded.

Two things your note led to that I wouldn't have found otherwise:

• It isn't only the low-confidence reason. The withheld-response reason and the delivery-failure reason take the same path into the same jsonb payload, so all three are bounded and cleaned through one helper now.
• Blankness has to be decided after cleaning, not before. A reason that is entirely ill-formed cleans to nothing, and the old order would have stored an empty string and summoned a human with no reason at all. There's now a named fallback for that case rather than silence.

The comment names Prisma 6.19.3's first write as the exact place it fails, so the next person doesn't have to rediscover which layer refuses it.

The per-ticket token

Done — sharedInstallationAuth resolves one resolver per process, so the token one investigation mints is the one the next reads back, and the in-flight-sharing comments are now true instead of aspirational.

Two things fell out of it that are worth knowing:

• The resolver key can't contain |. @octokit/auth-app@7.2.2 caches tokens in a |-joined string, so a value containing one comes back truncated at the separator — a cached ghs_a|b returns as the bearer ghs_a. Excluded from the key rather than left to surface as a mystery 401.
• A shared resolver has to honour refresh, or a token GitHub has already rejected gets pinned for the rest of the process. It now retires a refused token the way auth-app itself does instead of rebuilding the strategy.

The deployment preconditions

In the docs as a cutover step: OPENAI_API_KEY on worker and web, old claude-* overrides cleared, with the reason spelled out — validateConfig runs when a pipeline is built, not at boot, so an Anthropic-only deployment doesn't fail loudly. Every AI_RESPONSE job fails before a pending row exists, retries, dead-letters, and the reporter gets nothing with no handoff, while sentiment quietly degrades to neutral.

State

Branch merged up to your 8e031ee and pushed. Full repo suite fresh (not cached): 6462 tests across 178 files, 11/11 turbo tasks, exit 0. Repo-wide typecheck clean, 11/11.

One scheduling note, since you said "before this goes anywhere near production": outpost is in Jordan's Node advisory thread as an open question — seven of our eight images are node:20-alpine, which is EOL and stays vulnerable, and release-bot's node:22.20 is below the fixed 22.23.4. I've answered the exposure question separately (short version: no Express, no body-parser, nothing decompresses Brotli anywhere in outpost, so the pre-auth decoder path doesn't exist here — but the images still need bumping). Not this PR's problem, but it probably decides the deploy window.

[NathanTarbert]

Thanks Jerel, this all looks great.

The shared resolver does what the comments now say. Every ticket in a process reads from the same token cache, and a new key or installation still gets a fresh one. The refresh handling is careful too. A late refusal on an old token can't throw away a newer one, and investigations that refresh at the same time share one exchange instead of piling up. Good catch on the | in auth-app's cache. That would have been a really confusing 401 to chase down.

The reason cleaning is more thorough than what I asked for, and I'm glad you followed it into the withheld and delivery-failure reasons. Deciding blankness after cleaning is the right call. A reason that cleans down to nothing still reaches a person with a named fallback, instead of an empty note. The deployment steps read clearly as well, especially the part about why an Anthropic-only deployment fails quietly rather than at boot.

On the App permission, I'll check the installation myself and make sure Contents is granted and accepted before the OpenAI default goes out. Thanks for writing down what the failure looks like in the logs. That will save whoever hits it some time.

One thing that's next door to this, and fine as a follow-up rather than here. The reply row created at ai-response.ts:988 stores the model's text as it came back, and so do suggestedResponse and the shadow SYSTEM row. That text never goes through the cleaner. It's parsed from JSON, so if the model copied a \u0000 or a lone surrogate out of a source it quoted, that message.create would fail. The reply and the handoff would both be lost, which is the same thing storableReason now prevents for the reasons. It was already like this before this PR, and I'd expect it to be rare. If you agree it's worth closing, running the response text through the same cleaner without the 2000 bound would cover it.

Approving. Thanks again for all the care on this one!