Skip to content

[Hackathon] Bound Response Buffering (Large Histories/Steps Load Fully into Heap) #83

Description

@crypticsaiyan

Discord Username / User ID

uisupersaiyan3

What does this improvement do?

Summary

Every non-streaming HTTP response is fully materialized in memory via
await response.json(). For most requests this is fine, but test result --history pagination and getRun({ includeSteps: true }) can return large
payloads that are read entirely into memory before any processing or
rendering occurs.


Root Cause

The HTTP client's typed read path treats every response uniformly via
response.json(), which requires the full body to be buffered and parsed
before any part of it is available to the caller. This is appropriate for
small, bounded responses, but history and step data have no inherent upper
bound on size, so the same buffering strategy applies regardless of how
large the underlying payload is.


Affected Code

File Lines Behavior
src/lib/http.ts 497 response.json() used for all typed reads, with no size bound
src/lib/http.ts 368-383 getRun step inclusion path, subject to the same unbounded read
src/lib/pagination.ts (history pagination) Used by test result --history, accumulates pages without a bound

For comparison, test code get already streams its payload via bundle.ts.
The JSON read path used by history and step data does not have an
equivalent streaming or bounding mechanism.


Impact

A run with a long step array, or a test result --history invocation over a
deep history window, causes heap usage to grow in proportion to the size of
the response, with no upper bound. In the worst case, this can mean loading
an entire large history or step set into memory before printing a single
row, which is both slower to first output and more memory-intensive than
necessary.

This is a robustness concern rather than a confirmed incident: the failure
mode is degraded performance and elevated memory use under large N, not a
crash under typical usage.

Details / implementation notes

Proposed Resolution

Choose one of the following approaches for the history and step read paths:

  • Bounded reads: Introduce a documented ceiling on the number of
    accumulated history rows or step entries, combined with a --limit flag
    so callers can explicitly request a smaller window.
  • Incremental rendering: Stream and render history pages as they arrive
    rather than buffering the full result set before printing, so memory use
    stays proportional to a single page rather than the full history.
    Either approach should ensure memory use no longer scales unbounded with
    response size for these two read paths.

Confirmations

  • I have searched existing issues and this is not a duplicate.
  • I have provided my Discord identity above for reward coordination.

Metadata

Metadata

Assignees

Labels

hackathonCLI hackathon submissionsin-progressAssigned and actively being worked on

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions