feat: support with_low_memory TCP storage policy

LOW_MEMORY.md

@@ -0,0 +1,126 @@
+# Low-memory TCP storage
+
+Build with the existing `with_low_memory` tag to select smaller TCP storage
+and cache bounds. No runtime setter or additional feature tag is required.
+Ordinary builds retain the existing values and behavior.
+
+| Storage policy | Ordinary build | `with_low_memory` |
+| --- | ---: | ---: |
+| Minimum first send allocation | 2 KiB | 1 KiB |
+| Largest reusable acknowledged send chunk | 32 KiB | 16 KiB |
+| Retained drained receive metadata | 64 slice slots | 32 slice slots |
+
+Allocation remains demand-driven. The first send allocation can exceed its
+minimum when the application writes more data. Repeated moderate writes still
+reuse one eligible acknowledged chunk; common-MTU receive payloads and short
+metadata arrays remain reusable. Larger unused backing is released after it
+drains. Unread or unacknowledged bytes are never truncated.
+
+An undersized small spare is not reused for a larger first write in the
+low-memory build. For example, a one-byte write can leave a 1 KiB spare. A
+subsequent 1,200-byte write gets one fitted allocation, instead of filling the
+small spare and allocating another 16 KiB chunk for its tail. Later chunks
+retain their existing minimum so packet scatter bounds remain valid.
+
+This changes storage retention, not TCP flow control. Default buffer limits,
+automatic receive/send growth, advertised windows, retransmission handling,
+and explicit socket options keep their existing behavior. The package adds
+no pressure polling, timer, callback, forced GC, or per-connection state.
+It does not provide a whole-stack or whole-process memory cap.
+
+## iOS motivation
+
+Hako already builds its iOS/tvOS core slices with `with_low_memory`; its macOS
+slice does not use that tag. Hako's iOS extension targets a 50 MiB engineering
+budget. Its gVisor TCP profile uses 32 KiB initial and 128 KiB maximum buffers
+in each direction. The tests also express those byte limits as MIPS options;
+this is a comparison workload, not a claim that the two stacks have identical
+semantics. Smaller configured maxima alone do not release idle caches.
+
+## Validation
+
+Run the root test/race and vet suites with and without `with_low_memory`.
+The gVisor interoperability tests are a separate module in `interop/gvisor`.
+`TestLowMemory*` covers cache bounds, useful reuse, payload preservation, and
+the tiny-write-to-larger-write allocation edge case. Existing transport tests
+continue to exercise the normal flow-control and lifecycle paths.
+
+`TestTCPMemoryProfileBaseline` accounts for retained backing in a deterministic
+1,000-connection fixture. It excludes live actors, connection structs, runtime
+memory and physical footprint. Stream benchmarks transfer packets between two
+in-process stacks; their throughput is not Internet speed or device energy.
+Physical-device measurements must separately report their process type,
+workload, compiler, memory, throughput, latency, CPU cost and limitations.
+
+### Latest host review
+
+On Go 1.26.5 darwin/arm64, the root race suite and vet passed for both
+ordinary and low-memory builds. The full gVisor interoperability suite also
+passed for both builds in serial, non-race runs with GOMAXPROCS=2.
+
+Running both complete gVisor race suites concurrently with GOMAXPROCS=4
+produced timeouts in IPv4 MTU-68 and some custom-congestion-control cases;
+both suites eventually reached their three-minute timeout. This is an
+unresolved validation limitation, not a passing race result. Similar MTU-68
+instability was previously observed on the unmodified baseline. These results
+do not establish the cause or attribute it to this change, and this patch
+does not claim to fix it.
+
+## Physical iPad comparison
+
+An iPad Pro (12.9-inch, 6th generation; iPad14,5) running iPadOS 26.6.2 ran
+three fresh-process repetitions per build, with baseline/candidate order
+alternated. Both builds used Go 1.26.5, `with_low_memory`, and GOMAXPROCS=2.
+The baseline MIPS commit was `ba762df4c91d6f9bddf82062afb0d40aa1352687`, which
+does not act on that tag. No runtime pressure API was present or called.
+
+The independent Debug App connected two MIPS stacks through an in-process
+packet link, using 256 connection pairs and the Hako 32/128 KiB socket profile.
+Each connection exchanged a one-byte message, two 1,200-byte messages, and
+three 32 KiB messages. Idle measurements followed a 500 ms pause and explicit
+GC in both builds. One connection then ran 1,000 small request/response cycles
+and a fixed 1 GiB bidirectional payload transfer. Every connection was checked
+again after the stream. All six runs completed without transfer errors.
+
+| Measurement (median of three runs) | Baseline | Low-memory candidate |
+| --- | ---: | ---: |
+| Live Go heap after small messages | 4.68 MiB | 4.31 MiB |
+| Live Go heap after repeated 32 KiB messages | 16.19 MiB | 8.08 MiB |
+| Process physical footprint at that latter boundary | 51.74 MiB | 40.19 MiB |
+| Fixed-byte stream throughput, both directions combined | 2.61 Gbps | 2.71 Gbps |
+| Stream process CPU cost | 2.324 CPU seconds/GB | 2.418 CPU seconds/GB |
+| Stream allocation traffic per payload byte | 2.123 B/B | 2.207 B/B |
+| Small-request p95 round-trip latency | 47.67 microseconds | 43.29 microseconds |
+| Highest sampled process footprint | 66.53 MiB | 49.50 MiB |
+
+Throughput uses decimal Gbps (10^9 bits/s) and counts both echoed payload
+directions. For this symmetric echo workload, each direction therefore
+accounts for approximately 1.31 and 1.35 Gbps, respectively. These are not
+separate upload/download saturation tests and cannot be compared directly
+with one-way benchmark charts.
+
+The 32 KiB workload deliberately exercises backing that the ordinary build
+retains and the low-memory build releases. Its roughly 8.11 MiB live-heap
+saving across 512 TCP endpoints is workload-specific. Small messages saved
+about 0.38 MiB; idle savings do not scale from connection count alone.
+The stream showed no large throughput collapse in these runs, but its small
+throughput/latency differences are not proof of a speed improvement. CPU cost
+and allocation traffic per byte increased by about 4%. Applications dominated
+by repeated medium-sized writes may pay more allocation cost and should test
+that workload separately.
+
+These are standalone App process results, not NetworkExtension, PacketFlow,
+Internet throughput, battery, or iOS termination-risk measurements. The probe
+contains both TCP endpoints. Its footprint cannot be interpreted as Hako's
+extension footprint or compared directly with the 50 MiB engineering budget.
+The 100 ms footprint sampler may miss shorter peaks. The API does not force
+GC; explicit GC is used only by the measurement harness.
+
+## Ordinary build boundary
+
+On Go 1.26.5 darwin/arm64, fifteen affected ordinary-build functions matched
+baseline instructions after address relocation normalization, including the
+send-buffer append/acknowledgement paths, TCP read path, established loop and
+handshakes. No connection or stack fields are added. This check is specific
+to that compiler and architecture; it is not whole-binary identity or a claim
+that every platform has been benchmarked.

README.md

@@ -1164,3 +1164,10 @@ would not benefit existing consumers.
 ## License
 
 MIPS is licensed under the Mozilla Public License 2.0. See `LICENSE`.
+
+### Low-memory builds
+
+The existing `with_low_memory` build tag selects smaller idle TCP cache and
+initial allocation bounds while retaining automatic buffer growth and normal
+buffer reuse. Ordinary builds keep the existing policy. See
+[Low-memory TCP storage](LOW_MEMORY.md) for behavior and validation details.

performance_test.go

@@ -13,19 +13,23 @@ import (
 )
 
 func benchmarkTCPControllerConnection(b *testing.B, algorithm string) (net.Conn, *Stack, *stackBridge) {
+	return benchmarkTCPProfileConnection(b, TCPSocketDefaults{CongestionControl: algorithm})
+}
+
+func benchmarkTCPProfileConnection(b *testing.B, defaults TCPSocketDefaults) (net.Conn, *Stack, *stackBridge) {
 	b.Helper()
 	clientAddress := netip.MustParseAddr("192.0.2.201")
 	serverAddress := netip.MustParseAddr("192.0.2.202")
 	client, err := New(Config{
 		LocalAddresses: []netip.Prefix{netip.PrefixFrom(clientAddress, 32)},
-		TCP:            TCPSocketDefaults{CongestionControl: algorithm},
+		TCP:            defaults,
 	})
 	if err != nil {
 		b.Fatal(err)
 	}
 	server, err := New(Config{
 		LocalAddresses: []netip.Prefix{netip.PrefixFrom(serverAddress, 32)},
-		TCP:            TCPSocketDefaults{CongestionControl: algorithm},
+		TCP:            defaults,
 	})
 	if err != nil {
 		b.Fatal(err)

tcp.go

@@ -86,28 +86,16 @@ const (
 	tcpMaximumReceiveCapacity = 16 * 1024 * 1024
 	// tcpMaximumSendCapacity bounds automatic send-buffer growth under the same policy.
 	tcpMaximumSendCapacity = 16 * 1024 * 1024
-	// tcpReadChunkRetain keeps metadata for a modest receive burst after it
-	// drains without retaining the payload backing or a multi-megabyte array of
-	// slice headers on an idle connection.
-	tcpReadChunkRetain = 64
 	// tcpReusableReceivePayloadLimit keeps one common-MTU receive backing per
 	// connection. Larger packets are released after Read so an occasional jumbo
 	// segment cannot permanently raise the idle memory cost.
 	tcpReusableReceivePayloadLimit = 2048
-	// tcpSendChunkInitial bounds the unused backing retained by an idle
-	// connection after a small write. Later chunks in the same live send window
-	// use tcpSendChunkMinimum so packet construction remains scatter-bounded.
-	tcpSendChunkInitial = 2 * 1024
 	// tcpSendChunkMinimum packs later writes without moving bytes referenced by
 	// retransmission metadata and keeps cross-chunk gathers uncommon on bulk
 	// streams.
 	tcpSendChunkMinimum = 16 * 1024
 	// tcpSendChunkMaximum bounds unused tail capacity in one send chunk.
 	tcpSendChunkMaximum = tcpSendCapacity
-	// tcpReusableSendChunkLimit retains only a modest acknowledged send chunk.
-	// Larger chunks are released so a completed bulk transfer does not pin its
-	// former window.
-	tcpReusableSendChunkLimit = 32 * 1024
 	// tcpMetadataQueueInitial avoids charging every connection for a burst
 	// before one occurs.
 	tcpMetadataQueueInitial = 1
@@ -3473,11 +3461,15 @@ func (b *tcpSendBuffer) append(payload []byte) {
 			var storage []byte
 			// A small retained first chunk cannot be inserted behind a live
 			// chunk: doing so would violate the scatter bound used by view.
-			if b.spare != nil && (len(b.chunks) == 0 || cap(b.spare) >= tcpSendChunkMinimum) {
+			if b.spare != nil && (len(b.chunks) == 0 || cap(b.spare) >= tcpSendChunkMinimum) &&
+				(!tcpFitSmallSendSpare || cap(b.spare) >= tcpSendChunkMinimum || cap(b.spare) >= capacity) {
 				storage = b.spare
 				b.spare = nil
 			}
 			if cap(storage) == 0 {
+				if tcpFitSmallSendSpare {
+					b.spare = nil
+				}
 				storage = make([]byte, capacity)
 				if capacity > tcpSendChunkInitial && capacity <= tcpReusableSendChunkLimit && b.reusableState == tcpSendReusableReleased {
 					b.reusableState = tcpSendReusableConfirmed

tcp_low_memory_test.go

@@ -0,0 +1,99 @@
+//go:build with_low_memory
+
+package mipstack
+
+import (
+	"bytes"
+	"testing"
+)
+
+func TestLowMemorySmallWritesRetainReusableBacking(t *testing.T) {
+	var b tcpSendBuffer
+	payload := bytes.Repeat([]byte{0x57}, 1200)
+	for i := 0; i < 3; i++ {
+		b.append(payload)
+		b.acknowledge(len(payload))
+	}
+	if n := cap(b.spare); n > 1200 {
+		t.Fatalf("small idle send backing=%d, want <=1200", n)
+	}
+	if len(b.spare) == 0 && cap(b.spare) == 0 {
+		t.Fatal("small reusable backing was discarded")
+	}
+	p := &b.spare[:cap(b.spare)][0]
+	b.append(payload)
+	if &b.chunks[0].storage[0] != p {
+		t.Fatal("small repeated write lost reuse")
+	}
+	var view tcpPayloadView
+	b.view(0, len(payload), &view)
+	got := make([]byte, len(payload))
+	view.copyTo(got)
+	if !bytes.Equal(got, payload) {
+		t.Fatal("reused storage corrupted payload")
+	}
+}
+
+func TestLowMemoryCapsIdleSendBacking(t *testing.T) {
+	for _, size := range []int{16 << 10, 32 << 10} {
+		var b tcpSendBuffer
+		payload := bytes.Repeat([]byte{0x6b}, size)
+		for i := 0; i < 3; i++ {
+			b.append(payload)
+			b.acknowledge(size)
+		}
+		if n := cap(b.spare); n > 16<<10 {
+			t.Fatalf("%d-byte bursts retained %d bytes; low-memory maximum is 16 KiB", size, n)
+		}
+		if size == 16<<10 && cap(b.spare) != size {
+			t.Fatal("common send chunk reuse was lost")
+		}
+	}
+}
+
+func TestLowMemoryCapsDrainedReadMetadata(t *testing.T) {
+	b := tcpReadBuffer{chunks: make([][]byte, 0, 48)}
+	for i := 0; i < 48; i++ {
+		b.append([]byte{byte(i)})
+	}
+	got := make([]byte, 48)
+	if b.read(got, len(got), nil) != len(got) {
+		t.Fatal("short read")
+	}
+	for i, v := range got {
+		if v != byte(i) {
+			t.Fatal("read data changed")
+		}
+	}
+	if n := cap(b.chunks); n > 32 {
+		t.Fatalf("drained metadata slots=%d, want <=32", n)
+	}
+	for i := 0; i < 8; i++ {
+		b.append([]byte{1})
+	}
+	b.read(got, 8, nil)
+	if cap(b.chunks) == 0 {
+		t.Fatal("common read metadata reuse was lost")
+	}
+}
+
+func TestLowMemoryGrowingSmallWriteAvoidsLargeTail(t *testing.T) {
+	var b tcpSendBuffer
+	b.append([]byte{1})
+	b.acknowledge(1)
+	payload := bytes.Repeat([]byte{0x43}, 1200)
+	b.append(payload)
+	if len(b.chunks) != 1 {
+		t.Fatalf("small write used %d chunks after a tiny write; want one fitted allocation", len(b.chunks))
+	}
+	if n := cap(b.chunks[0].storage); n > 1200 {
+		t.Fatalf("small write retained %d bytes of backing", n)
+	}
+	var view tcpPayloadView
+	b.view(0, len(payload), &view)
+	got := make([]byte, len(payload))
+	view.copyTo(got)
+	if !bytes.Equal(got, payload) {
+		t.Fatal("fitted write lost data")
+	}
+}

tcp_memory_low.go

@@ -0,0 +1,19 @@
+//go:build with_low_memory
+
+package mipstack
+
+// Low-memory builds retain less unused backing while preserving on-demand
+// allocation, active TCP windows, automatic growth and common buffer reuse.
+// These are storage/cache bounds, not advertised TCP receive-window limits.
+const (
+	// Avoid spending a full later chunk on overflow from an undersized small spare.
+	tcpFitSmallSendSpare = true
+	// Keep ordinary short read bursts reusable, but release larger drained metadata.
+	tcpReadChunkRetain = 32
+	// The first live send chunk can be small. Later chunks keep their normal
+	// minimum to preserve the bound on packet scatter/gather segments.
+	tcpSendChunkInitial = 1024
+	// Repeated moderate writes can retain one 16 KiB chunk. Larger acknowledged
+	// chunks are released; unread and unacknowledged storage is never truncated.
+	tcpReusableSendChunkLimit = 16 * 1024
+)