[niemyjski]

Moves Svelte real-time updates to authenticated SSE while retaining server WebSocket compatibility, existing message payloads, and the EnableWebSockets configuration alias. Replica-local ownership protects token revocation; expiring Redis leases recover connection capacity after replica loss. Scoped tokens cannot receive credential-bearing notifications. Latest main is integrated, including current frontend imports, environment configuration, layout behavior, and SSL settings.

Validation on e8a917f82: backend build passed; 73 focused backend tests including real Redis expiry, 24 frontend tests, and four isolated Chromium SSE tests passed without retries. Full frontend validation and production build passed. Hosted CI passed on this revision: all four API shards, all six browser shards, both aggregate checks, client tests, website, and container builds.

Remaining gates: the PR remains draft with historical CHANGES_REQUESTED and needs reviewer re-approval. Full local application journeys were not rerun. Production ingress streaming still needs validation; the chart keeps push disabled.

Verification and implementation details

• Two-replica shared-bus coverage verifies fanout and owner-only token revocation. An ephemeral Redis fixture verifies atomic connection limits and stale-lease recovery.
• Bounded queues prioritize critical events and force reconnect/resynchronization when invalidations are lost. HTTP/2 development proxy cancellation releases upstream streams.
• The main merge required migrating SSE imports and test mocks to package subpaths and the shared environment module. Main's new layout behavior and SSL option are preserved. Push exception logs follow current repository conventions.
• All 37 review threads were inspected and are already resolved; no review decision was dismissed.
• Final-head Build CI passed. Publication and deployment jobs were skipped.

[Copilot]

Pull request overview

This PR migrates real-time push from WebSockets to SSE while keeping a temporary WebSocket compatibility path for rolling deploys, updates both frontends to consume SSE, and adds rollout/shutdown configuration for long-lived push connections.

Changes:

• Adds SSE connection, middleware, broker fanout, diagnostics, and EnablePush configuration.
• Replaces Svelte/Angular WebSocket clients with fetch/stream-based SSE clients and updates tests.
• Adds Kubernetes drain settings and documents push being disabled until ingress supports SSE.

Reviewed changes

Copilot reviewed 30 out of 30 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/Exceptionless.Web/Hubs/SseConnection.cs	Adds bounded SSE write queue, deduplication, keep-alives, and disposal.
src/Exceptionless.Web/Hubs/SseConnectionManager.cs	Manages SSE connections, keep-alives, cleanup, and metrics.
src/Exceptionless.Web/Hubs/SseMiddleware.cs	Adds authenticated /api/v2/push SSE endpoint.
src/Exceptionless.Web/Hubs/WebSocketPushMiddleware.cs	Adds temporary WebSocket compatibility middleware.
src/Exceptionless.Web/Hubs/WebSocketConnectionManager.cs	Reworks WebSocket manager for compatibility fanout and metrics.
src/Exceptionless.Web/Hubs/MessageBusBroker.cs	Fans out push messages to SSE and WebSocket managers.
src/Exceptionless.Web/Hubs/MessageBusBrokerMiddleware.cs	Removes old WebSocket-only middleware.
src/Exceptionless.Web/Startup.cs	Wires SSE and WebSocket push middleware behind EnablePush.
src/Exceptionless.Web/Program.cs	Adds host shutdown timeout for drain alignment.
src/Exceptionless.Web/Bootstrapper.cs	Registers the SSE connection manager.
src/Exceptionless.Web/Utility/Handlers/ThrottlingMiddleware.cs	Renames push throttle exemption to SSE-oriented naming.
src/Exceptionless.Core/Configuration/AppOptions.cs	Replaces EnableWebSockets with EnablePush and legacy fallback.
src/Exceptionless.Core/Bootstrapper.cs	Updates push-disabled startup warning.
src/Exceptionless.Core/Utility/AppDiagnostics.cs	Adds SSE/WebSocket push connection counters.
src/Exceptionless.Web/ClientApp/src/lib/features/websockets/sse-client.svelte.ts	Adds Svelte SSE client with reconnect and stream parsing.
src/Exceptionless.Web/ClientApp/src/lib/features/websockets/sse-client.test.ts	Adds Svelte SSE client unit tests.
src/Exceptionless.Web/ClientApp/src/lib/features/websockets/web-socket-client.svelte.ts	Removes Svelte WebSocket client.
src/Exceptionless.Web/ClientApp/src/lib/features/websockets/web-socket-client.test.ts	Removes WebSocket client tests.
src/Exceptionless.Web/ClientApp/src/routes/(app)/+layout.svelte	Switches app layout push integration to SSE.
src/Exceptionless.Web/ClientApp.angular/components/websocket/websocket-service.js	Rewrites Angular legacy push service to SSE.
tests/Exceptionless.Tests/Hubs/SseTests.cs	Adds SSE manager, broker, and deduplication tests.
tests/Exceptionless.Tests/Hubs/SseIntegrationTests.cs	Adds HTTP pipeline tests for SSE endpoint behavior.
tests/Exceptionless.Tests/Hubs/FakeHttpResponse.cs	Adds fake response helper for SSE tests.
tests/Exceptionless.Tests/Hubs/WebSocketCompatibilityTests.cs	Adds WebSocket compatibility and dual-fanout tests.
tests/Exceptionless.Tests/Hubs/WebSocketTests.cs	Removes old WebSocket broker tests.
tests/Exceptionless.Tests/Hubs/WebSocketConnectionManagerTests.cs	Removes old WebSocket manager tests.
tests/Exceptionless.Tests/Hubs/TestWebSocket.cs	Updates test WebSocket encoding and close count layout.
tests/Exceptionless.Tests/appsettings.yml	Enables push for test host configuration.
tests/http/push.http	Adds manual SSE smoke requests for localhost.
k8s/exceptionless/templates/api.yaml	Adds drain lifecycle settings and disables push pending ingress migration.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 30 out of 30 changed files in this pull request and generated 3 comments.

[niemyjski]

Addressed the outstanding review feedback in the latest pushes: restored the Angular legacy client to the existing WebSocket path with only a deprecation note, tightened SSE/WebSocket lifecycle cleanup, fixed the SSE queue and keepalive backpressure edge case, and stopped the Svelte client from retrying when push is disabled.\n\nLocal validation is green (, , , , and Using launch settings from /Users/blake/Projects/Exceptionless/copilot-worktrees/Exceptionless/niemyjski-stunning-robot/tests/Exceptionless.Tests/Properties/launchSettings.json...
Running tests from /Users/blake/Projects/Exceptionless/copilot-worktrees/Exceptionless/niemyjski-stunning-robot/tests/Exceptionless.Tests/bin/Debug/net10.0/Exceptionless.Tests.dll (net10.0|arm64)
[+94/x0/?0] Exceptionless.Tests.dll (net10.0|arm64)(2s)

[+94/x0/?0] Exceptionless.Tests.dll (net10.0|arm64)(5s)

[+239/x0/?0] Exceptionless.Tests.dll (net10.0|arm64)(8s)

[+338/x0/?0] Exceptionless.Tests.dll (net10.0|arm64)(11s)

[+483/x0/?0] Exceptionless.Tests.dll (net10.0|arm64)(14s)

[+506/x0/?0] Exceptionless.Tests.dll (net10.0|arm64)(17s)

[+569/x0/?0] Exceptionless.Tests.dll (net10.0|arm64)(20s)

[+684/x0/?0] Exceptionless.Tests.dll (net10.0|arm64)(23s)

[+743/x0/?0] Exceptionless.Tests.dll (net10.0|arm64)(26s)

[+835/x0/?0] Exceptionless.Tests.dll (net10.0|arm64)(29s)

skipped Exceptionless.Tests.Pipeline.EventPipelineTests.GeneratePerformanceDataAsync (0ms)
Used to create performance data from the queue directory
from /Users/blake/Projects/Exceptionless/copilot-worktrees/Exceptionless/niemyjski-stunning-robot/tests/Exceptionless.Tests/bin/Debug/net10.0/Exceptionless.Tests.dll (net10.0|arm64)
[+900/x0/?1] Exceptionless.Tests.dll (net10.0|arm64)(32s)

[+1042/x0/?1] Exceptionless.Tests.dll (net10.0|arm64)(35s)

[+1151/x0/?1] Exceptionless.Tests.dll (net10.0|arm64)(38s)

skipped Exceptionless.Tests.Repositories.EventRepositoryTests.GetAsyncPerformanceAsync (0ms)
Performance Testing
from /Users/blake/Projects/Exceptionless/copilot-worktrees/Exceptionless/niemyjski-stunning-robot/tests/Exceptionless.Tests/bin/Debug/net10.0/Exceptionless.Tests.dll (net10.0|arm64)
skipped Exceptionless.Tests.Repositories.EventRepositoryTests.GetAsync (0ms)
elastic/elasticsearch-net#2463
from /Users/blake/Projects/Exceptionless/copilot-worktrees/Exceptionless/niemyjski-stunning-robot/tests/Exceptionless.Tests/bin/Debug/net10.0/Exceptionless.Tests.dll (net10.0|arm64)
[+1236/x0/?3] Exceptionless.Tests.dll (net10.0|arm64)(41s)

[+1277/x0/?3] Exceptionless.Tests.dll (net10.0|arm64)(44s)

[+1424/x0/?3] Exceptionless.Tests.dll (net10.0|arm64)(47s)

[+1567/x0/?3] Exceptionless.Tests.dll (net10.0|arm64)(50s)

[+1650/x0/?3] Exceptionless.Tests.dll (net10.0|arm64)(53s)

[+1792/x0/?3] Exceptionless.Tests.dll (net10.0|arm64)(56s)

[+1864/x0/?3] Exceptionless.Tests.dll (net10.0|arm64)(59s)

[+1873/x0/?3] Exceptionless.Tests.dll (net10.0|arm64)(1m 02s)

[+1873/x0/?3] Exceptionless.Tests.dll (net10.0|arm64)(1m 05s)

[+1874/x0/?3] Exceptionless.Tests.dll (net10.0|arm64)(1m 08s)

[+1875/x0/?3] Exceptionless.Tests.dll (net10.0|arm64)(1m 11s)

[+1876/x0/?3] Exceptionless.Tests.dll (net10.0|arm64)(1m 14s)

/Users/blake/Projects/Exceptionless/copilot-worktrees/Exceptionless/niemyjski-stunning-robot/tests/Exceptionless.Tests/bin/Debug/net10.0/Exceptionless.Tests.dll (net10.0|arm64) passed (1m 14s 985ms)

Test run summary: Passed!
total: 1880
failed: 0
succeeded: 1877
skipped: 3
duration: 1m 15s 407ms), the PR checks are green, and the open inline threads have been replied to and resolved.

[ejsmith]

The steady-state SSE fanout design should not require sticky sessions: each long-lived stream remains local to the replica that accepted it, Redis pub/sub delivers notifications to every API replica, and only the replica owning a given local connection writes to it. Reconnects can land on any replica.

However, the shared connection metadata currently has two scale-out failure modes that should be addressed before enabling push in production:

1. Authentication-token revocation can be consumed destructively by a non-owning replica, leaving the owning replica's stream connected.
2. Redis connection reservations have no lease or expiry, so an ungraceful pod/node failure can leave permanent entries that eventually cause users with no live streams to receive 429 responses.

The current tests use a single host/manager and cannot expose these cases. Please add a two-replica test with shared connection state/message bus covering cross-replica delivery and token revocation, plus stale-reservation behavior after simulated replica loss.

[niemyjski]

Follow-up feedback audit complete at 5ee86353a9412d2d19394025fccee7b4dea25b01.

• Live inventory before this summary: 37 review threads, 70 inline comments, 42 submitted reviews, and 2 issue comments. All 37 threads were already resolved and none were open, so there was no open thread to reply to or resolve.
• Classification: 4 live-line findings are already fixed/current (EnablePush with legacy EnableWebSockets, SSE resource disposal, and fetch-based SSE auth); 31 old bot/Copilot findings are outdated or superseded by the lifecycle redesign and deleted code; 2 human scale-out findings are already fixed by process-local ownership and expiring Redis leases. Narrow exception-allowlist prescriptions were not safe as written, but their cleanup and backpressure concerns are covered by the current design and tests.
• The one still-actionable item was @ejsmith requested true two-replica shared-message-bus regression coverage. RCA: production fanout/revocation behavior existed, but the test called handlers directly and therefore did not prove bus subscription behavior across replicas. Commit 5ee86353a adds a shared-bus test proving fanout reaches both local replicas and token revocation closes only the owner replica connection while preserving the non-owner connection.
• Focused proof: SSE broker 10/10, manager 7/7, deduplication 9/9, HTTP integration 7/7, WebSocket compatibility 14/14, in-memory leases 2/2, real Redis lease expiry 1/1, and push option compatibility 3/3. dotnet build Exceptionless.slnx --no-restore passed with 0 warnings and 0 errors.
• Full thermo-nuclear branch-vs-main review found no remaining structural, concurrency, authorization, resource-cleanup, compatibility, duplication, atomicity, or file-size blocker; the largest changed file is 770 lines.
• CI on this exact head is terminal green: API coverage, E2E, client, Docker, website, version, and CLA all passed.

GitHub still reports the historical CHANGES_REQUESTED decision; I did not dismiss it. The requested production fixes and combined replica coverage are now present for reviewer re-approval.

[github-actions]

Package	Line Rate	Branch Rate	Complexity	Health
Exceptionless.AppHost	24%	24%	132	❌
Exceptionless.Core	78%	70%	11014	✔
Exceptionless.Insulation	51%	44%	376	➖
Exceptionless.Web	87%	73%	9739	✔
Summary	81% (29383 / 36216)	70% (14652 / 20788)	21261	✔