Skip to content

copilot.exe fatal-aborts under load (BEX64 / 0xc0000409) — reproducible across 1.0.57 and 1.0.60 on Windows ARM64 #3687

Open

Description

@JW-Sthlm

Summary

The interactive CLI process (copilot.exe) hard-aborts with a Windows fatal-app-exit instead of shutting down gracefully. It fires most reliably when several sessions start at once (Windows Terminal tab restore) under memory pressure, but it has also fired mid-session. The terminal has to be closed and reopened each time.

Over the last 5 days Windows Error Reporting logged 45 crashes of copilot.exe on this machine, across two CLI versions. This is reproducible and ongoing, not a one-off.

Likely the same root cause as #3250 (BYOK + parallel subagents) and #2335 (multiple hard crashes on Windows 26200). Same exception path and OS build. Filing separately because the trigger here is general load / simultaneous startup with the stock provider, and the multi-version data may help.

Environment

  • Copilot CLI version: 1.0.60-0 (also reproduced on 1.0.57.0)
  • Bundled Node: v24.16.0
  • OS: Windows 11 Pro, build 10.0.26200
  • Arch: ARM64
  • Plan: Enterprise
  • Host: Windows Terminal, pwsh
  • RAM: 32 GB. Crashes correlate with free RAM dropping to roughly 2 GB.

Crash signature (Windows Application event log, Event ID 1000 / WER 1001)

Faulting application name: copilot.exe, version: 1.0.60.0
Faulting module name:      copilot.exe, version: 1.0.60.0
Exception code:            0xc0000409   (STATUS_STACK_BUFFER_OVERRUN / __fastfail)
Exception Data (P9):       0000000000000007  (FAST_FAIL_FATAL_APP_EXIT)
Event Name:                BEX64
IsFatal:                   1
Fault offset:              0x0000000001cdd3dc

0xc0000409 with subcode 7 is __fastfail(FAST_FAIL_FATAL_APP_EXIT), the path the bundled Node/V8 runtime takes when it calls abort() on a fatal error (uncaught fatal, V8 fatal, or JavaScript heap out of memory). The faulting module is copilot.exe itself because the CLI ships as a Node single-executable, so V8 runs inside the binary. This is a deliberate runtime self-termination, not native memory corruption.

Two distinct WER fault buckets, one per version

Version WER bucket Crashes (5 days)
1.0.57.0 2212194956551539350 28
1.0.60.0 1899762302600568482 17

The 1.0.57 to 1.0.60 update changed the bucket but did not fix the abort path.

Crash clustering (auto-restart loop)

Crashes arrive in tight bursts, consistent with crash, auto-restart, crash again under the same memory pressure. Examples from one morning:

09:30:29, 09:30:36, 09:30:49, 09:30:53, 09:30:58, 09:30:58   (6 crashes in ~30s)
09:08:10 x4   (four sessions aborting in the same second)

Steps to reproduce (best-effort, load-dependent)

  1. On Windows ARM64, have several saved interactive sessions, each with a few MCP servers configured (mix of stdio node/npx and remote HTTP).
  2. Launch many at once, for example restore 6+ Windows Terminal tabs simultaneously, each running copilot --resume.
  3. While the MCP servers all cold-start and free RAM falls toward ~2 GB, one or more copilot.exe processes fatal-abort with the signature above.

Not a guaranteed single-step repro, but the correlation with simultaneous startup and low free memory is consistent across dozens of occurrences.

What I ruled out

  • Not a clean shutdown. The structured session logs that end with [shutdown] Shutdown complete are headless prompt-mode runs, not the interactive TUI. The interactive process leaves no graceful log line, only the WER record.
  • Not MCP-config-specific. Reproduces after trimming to 7 global MCP servers and with an empty project-level config.
  • Not a Windows Terminal hang. This is copilot.exe faulting, captured by WER with a full dump reference.

Expected behavior

The runtime should not hard-abort under memory pressure or concurrent startup. At minimum the fatal reason should be recoverable from a log, not only from Windows Error Reporting.

Asks

  1. Surface the actual fatal reason before aborting. Today the only artifact is the WER record. If this is heap OOM, writing the V8 fatal / heap-limit context to the session log (not just terminal stderr, which dies with the process) would confirm it.
  2. If it is heap OOM, consider a higher default old-space limit for the bundled runtime, or backpressure when many MCP servers initialize concurrently.
  3. Guard against the crash-restart loop: if a session aborts on startup, avoid immediately relaunching into the same memory-pressure condition.

Attachments available on request

WER dump (WER.*.dmp) and the full per-timestamp crash timeline.

Activity

  1. JW-Sthlm commented on Jun 5, 2026

    @JW-Sthlm
    Author

    Update: this is NOT memory exhaustion. Stronger evidence points to a deterministic fault in the remote-export / request path.

    After more data I need to correct my own framing above. The OOM/memory-pressure angle is largely a red herring. Three findings:

    1. Identical fault offset across every crash

    All 15 crashes logged on this machine share the exact same fault offset 0x0000000001cdd3dc (CLI 1.0.60.0). Heap-out-of-memory aborts crash wherever allocation happens to fail, so their offsets vary. A single constant offset across many crashes indicates one specific code path hitting __fastfail, not random heap exhaustion.

    2. Crashes with plenty of free RAM

    The most recent cluster fired with 13.5 GB free of 32 GB and only 3 interactive tabs open. Memory pressure was not present.

    3. Synchronized crash timestamps across independent processes

    Crashes arrive at near-identical wall-clock seconds across separate copilot.exe processes:

    10:00:02  x3   (three sessions, same second)
    09:08:10  x4   (four sessions, same second)
    

    Independent processes aborting at the same instant points to a shared timed trigger, not per-process memory growth.

    Correlation: repoless remote export

    The crashed sessions all run with --remote and the feature flag REPOLESS_REMOTE_EXPORT: true. The last log line of the backend before death is:

    [INFO] --- Start of group: Sending request to the AI model ---
    

    The synchronized timing is consistent with a periodic remote-export flush (or an aligned interval timer in the remote path) firing across all sessions at once. All affected sessions are remote-steerable.

    Repro refinement

    • Run multiple interactive sessions with --remote enabled (repoless remote export on).
    • Crashes occur on a roughly aligned cadence across sessions, independent of memory headroom.

    Next step I am testing

    Disabling --remote (no repoless remote export) to confirm whether the crashes stop. Will report back. If maintainers can confirm whether REPOLESS_REMOTE_EXPORT has an interval-driven flush, that would help localize the fixed offset 0x1cdd3dc.

    Environment unchanged from the original report: Windows 11 build 10.0.26200, ARM64, CLI 1.0.60-0, bundled Node v24.16.0, Enterprise plan.

  2. added
    area:platform-windowsWindows-specific: PowerShell, cmd, Git Bash, WSL, Windows Terminal
    area:sessionsSession management, resume, history, session picker, and session state
    on Jun 5, 2026
  3. added theissue type on Jun 5, 2026
  4. JW-Sthlm commented on Jun 5, 2026

    @JW-Sthlm
    Author

    Decisive update: NOT remote export, NOT memory. Deterministic fault in the active request path.

    Disabling --remote did not stop the crashes. With NO_REMOTE=1 set at user level and the dead sessions confirmed running with REPOLESS_REMOTE_EXPORT=false, copilot.exe still aborts at the same fixed offset 0x0000000001cdd3dc. So both earlier theories are out:

    • Remote export: ruled out. 6 crashes with remote export disabled, same offset.
    • Memory exhaustion: ruled out. One crash cluster fired with 7.8 GB free of 32 GB.

    What the crashes have in common

    The faulting sessions are resumed (--resume) interactive sessions that were actively processing a model request when they died. Their events.jsonl last-write timestamp matches the crash second exactly. Sample (local time):

    session 9a1c6220  events write 10:42:46  ->  crash 10:42:47
    session 39bfb408  events write 10:34:35  ->  crash 10:34:37
    session a18ccc6b  events write 10:42:46  ->  crash 10:42:47
    

    Transcripts are modest (2 to 6 MB), so this is not a large-context issue. The structured session log ends mid-loop on lines like:

    [INFO] CompactionProcessor: Utilization 9.1% (83928/922000 tokens) below threshold 80%
    [INFO] --- Start of group: Sending request to the AI model ---
    [INFO] --- End of group ---
    

    The constant fault offset across every crash (now 20+ on this machine, all 0x1cdd3dc on 1.0.60.0) indicates a single deterministic code path hitting __fastfail, not a resource/timing artifact. Multiple concurrent sessions raise the hit rate, which is why bursts of 4 crash in the same second, but a single active session also crashes.

    Dump incoming

    I have enabled WER LocalDumps (full dump) for copilot.exe. I will attach a minidump from the next crash so the offset 0x1cdd3dc can be symbolized against your private symbols. If you can map that offset now, it likely points straight at the faulting function.

    Environment unchanged: Windows 11 build 10.0.26200, ARM64, CLI 1.0.60-0, bundled Node v24.16.0, Enterprise plan.

  5. minny1205xx-cmyk commented on Jun 5, 2026

    @minny1205xx-cmyk
  6. JW-Sthlm commented on Jun 16, 2026

    @JW-Sthlm
    Author

    Update from the same machine: root-caused this. It is not heap OOM, and there are actually two distinct failure modes that the original report blurred together. I hand-parsed nine of these minidumps (ARM64, so x64 dump tooling will not load them) plus the surrounding session logs.

    It is not memory pressure

    The crashing process's own telemetry at crash time: v8_used_heap about 52 MB against a v8_heap_size_limit of about 4.3 GB (1.2 percent used), RSS about 150 MB, on a 32 GB machine. The 0xc0000409 with exception data 0x7 is FAST_FAIL_FATAL_APP_EXIT, a deliberate abort() from the bundled Node/V8 runtime. It is real, but it is not triggered by heap exhaustion. The original "free RAM drops to ~2 GB" correlation was a coincidence of load, not the cause.

    Failure mode A: the detached remote-export flush child aborts on shutdown

    The processes producing these WER dumps are short-lived 4 to 5 thread children, not the interactive TUI. Each crashing PID matches a log line from its parent session:

    [DEBUG] RemoteSessionExporter: spawned detached flush process (pid <PID>)
    

    These children are spawned during the shutdown dispose sequence (only when remote session export is active) to flush session events, and they abort() instead of exiting. The full crashing call stack sits inside copilot.exe (the Node single-executable); no MCP server or native addon is on the stack.

    The "clustering under load" in the original report is really one-flush-per-shutdown. A wrapper that repeatedly starts and stops copilot --remote sessions produces one flush child per teardown, and on win32-arm64 each one aborts. Parallel teardowns produce the bursts. It correlated with "load" only because load meant more session churn.

    Confirmed workaround for mode A: run without --remote. No RemoteSessionExporter, so the detached flush child is never spawned, and the aborts stop. This isolates the fault to the remote-session-export path on win32-arm64.

    Failure mode B: the foreground TUI dies without restoring the terminal

    This is the one that actually disrupts work, and the original report under-described it. When the foreground interactive process dies uncleanly (hard abort, or a non-zero engine exit), it does not run its terminal-restore path. The TUI enables mouse tracking (DECSET 1003 plus SGR encoding 1006) and focus reporting (1004) at startup and is supposed to disable them on exit. After an unclean death those modes stay on, so the shell is left emitting raw control sequences on every scroll, mouse move, or focus change, for example:

    <65;102;34M      (SGR mouse motion)
    [I  [O           (focus in / focus out)
    

    The terminal becomes unusable and the only recovery is closing and reopening every tab. Notably this mode does not always leave a WER dump, because a plain non-zero exit is not a fastfail.

    Asks

    1. Surface the fatal reason before aborting. Today the only artifact is the WER record. Writing the V8 or libuv fatal context to the session log (not just terminal stderr, which dies with the process) would confirm the trigger.
    2. Make the detached exporter exit non-fatally on flush failure rather than calling abort().
    3. Install a terminal-restore handler that runs on SIGTERM, SIGINT, and the fatal/abort path, so an unclean exit still sends the DECRST resets (mouse 1000/1002/1003/1006, focus 1004, bracketed paste 2004, alternate screen 1049). This would stop mode B from wrecking the terminal regardless of why the process died.

    Environment

    Payload 1.0.63, bundled Node v24.16.0, Windows 11 build 26200, ARM64, Enterprise. The same flows on x64 here do not fastfail. Happy to share a parsed crashing stack and the exception record on request.

  7. dimanardiyansah123-stack commented on Jun 16, 2026

    @dimanardiyansah123-stack

    pgawascmlm.vip

  8. JW-Sthlm commented on Jun 21, 2026

    @JW-Sthlm
    Author

    Update from the same machine: root-caused this. It is not heap OOM, and there are actually two distinct failure modes that the original report blurred together. I hand-parsed nine of these minidumps (ARM64, so x64 dump tooling will not load them) plus the surrounding session logs.

    It is not memory pressure

    The crashing process's own telemetry at crash time: v8_used_heap about 52 MB against a v8_heap_size_limit of about 4.3 GB (1.2 percent used), RSS about 150 MB, on a 32 GB machine. The 0xc0000409 with exception data 0x7 is FAST_FAIL_FATAL_APP_EXIT, a deliberate abort() from the bundled Node/V8 runtime. It is real, but it is not triggered by heap exhaustion. The original "free RAM drops to ~2 GB" correlation was a coincidence of load, not the cause.

    Failure mode A: the detached remote-export flush child aborts on shutdown

    The processes producing these WER dumps are short-lived 4 to 5 thread children, not the interactive TUI. Each crashing PID matches a log line from its parent session:

    [DEBUG] RemoteSessionExporter: spawned detached flush process (pid <PID>)
    

    These children are spawned during the shutdown dispose sequence (only when remote session export is active) to flush session events, and they abort() instead of exiting. The full crashing call stack sits inside copilot.exe (the Node single-executable); no MCP server or native addon is on the stack.

    The "clustering under load" in the original report is really one-flush-per-shutdown. A wrapper that repeatedly starts and stops copilot --remote sessions produces one flush child per teardown, and on win32-arm64 each one aborts. Parallel teardowns produce the bursts. It correlated with "load" only because load meant more session churn.

    Confirmed workaround for mode A: run without --remote. No RemoteSessionExporter, so the detached flush child is never spawned, and the aborts stop. This isolates the fault to the remote-session-export path on win32-arm64.

    Failure mode B: the foreground TUI dies without restoring the terminal

    This is the one that actually disrupts work, and the original report under-described it. When the foreground interactive process dies uncleanly (hard abort, or a non-zero engine exit), it does not run its terminal-restore path. The TUI enables mouse tracking (DECSET 1003 plus SGR encoding 1006) and focus reporting (1004) at startup and is supposed to disable them on exit. After an unclean death those modes stay on, so the shell is left emitting raw control sequences on every scroll, mouse move, or focus change, for example:

    <65;102;34M      (SGR mouse motion)
    [I  [O           (focus in / focus out)
    

    The terminal becomes unusable and the only recovery is closing and reopening every tab. Notably this mode does not always leave a WER dump, because a plain non-zero exit is not a fastfail.

    This also affects VS Code's integrated terminal. In the latest repro the standalone Copilot terminal and the VS Code terminal both showed the same polluted state, while the editor/webview stayed alive. The VS Code terminal was also holding repeated MCP connect delay messages, which suggests the terminal host itself is getting contaminated once the Copilot session dies uncleanly.

    Screenshot refs:

    • copilot-image-1b1fba.png: blocked model list until I signed in with the demo account.
    • copilot-image-4996b7.png: VS Code editor still alive, but both terminal surfaces polluted after the Copilot session died.

    Asks

    1. Surface the fatal reason before aborting. Today the only artifact is the WER record. Writing the V8 or libuv fatal context to the session log (not just terminal stderr, which dies with the process) would confirm the trigger.
    2. Make the detached exporter exit non-fatally on flush failure rather than calling abort().
    3. Install a terminal-restore handler that runs on SIGTERM, SIGINT, and the fatal/abort path, so an unclean exit still sends the DECRST resets (mouse 1000/1002/1003/1006, focus 1004, bracketed paste 2004, alternate screen 1049). This would stop mode B from wrecking the terminal regardless of why the process died.

    Environment

    Payload 1.0.63, bundled Node v24.16.0, Windows 11 build 26200, ARM64, Enterprise. The same flows on x64 here do not fastfail. Happy to share a parsed crashing stack and the exception record on request.

  9. patniko commented on Oct 26, 2026

    @patniko
    Collaborator

    Thanks for the report. We've shipped fixes in this area since this was filed, and we believe this may now be resolved. Could you retest on the latest release (Copilot CLI v1.0.103) and let us know whether you still see it? If we don't hear back in a couple of weeks, we may close this issue — you can always reopen it.

    Triaged as part of a Copilot SDK backlog review, with help from GitHub Copilot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:platform-windowsWindows-specific: PowerShell, cmd, Git Bash, WSL, Windows Terminalarea:sessionsSession management, resume, history, session picker, and session statemore-info-needed

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions