client: set TCP_NODELAY on accepted sockets - #34
Open
VizzleTF wants to merge 1 commit into
Open
Conversation
uh_accept_client() leaves the accepted socket with Nagle enabled, and a response is emitted as one write() per header line plus one for the body. The kernel therefore holds the small final segment: tcp_nagle_check() allows a partial segment only if TCP_NODELAY is set, TCP_CORK is set, or every small packet sent so far has been acknowledged (Minshall's modification, tcp_minshall_check()). On a connection that has just answered a request, the previous response is still unacknowledged, so the next small body waits for the peer's delayed ACK. Every request after the first on a keep-alive connection pays that. Reproduced without LuCI, ubus, a session or a browser - a 5-byte static file and four sequential requests on one connection, on 25.12 (x86_64) and 24.10.7 (uhttpd 2025.07.06~7e64e8ba-r4): req1 0.4 ms / 1.0 ms (first on the connection) req2 41.5 ms / 42.2 ms req3 44.6 ms / 48.8 ms req4 44.6 ms / - fresh connection each time: 0.3-0.4 ms / 0.24-0.33 ms strace shows uhttpd is not the slow part: it wrote response 2 within 0.4 ms of response 1, while the client's next request only arrived 42.5 ms after that write. The stall window is the peer's delayed ACK timer, so it depends on the client; the figures above are Linux peers. Both builds come from the 25.12.5 release SDK, and the unpatched build is byte-identical to the published uhttpd-2026.06.16~7b1bec45-r1.apk, so the before column is what is shipping. Same box, http_keepalive at its default: unpatched 0.35 / 42.9 / 44.8 / 44.8 ms patched 0.32 / 0.087 / 0.075 / 0.075 ms LuCI is where this is visible, since its ubus JSON-RPC replies run a few hundred bytes and a page issues several: warm in-place navigation over five config pages went from 540 ms to 284 ms with keep-alive untouched. The trade-off is packet count, and it is measured rather than assumed. Disabling Nagle lets each of the eight writes leave as its own segment: the four-request reproducer goes from 13 to 32 server-to-client segments, and a full LuCI page load from 752 to 1211 packets for the same payload. For an admin interface that is a few dozen extra packets per page view against 40 ms stalls, but the real answer is to coalesce the response into fewer writes as well - nginx does both, enabling TCP_NODELAY when a connection enters the keep-alive state. Worth noting for that: ustream already carries a `more` flag from ustream_write() down to the backend, and ustream_fd_write() ignores it, using write() rather than send() with MSG_MORE - while ustream_vprintf() hardcodes more=false, so no printf-based header emission can express it today. Signed-off-by: VizzleTF <vizzlef@gmail.com>
VizzleTF
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
uhttpd does not set
TCP_NODELAYon accepted sockets, and it writes a response as onewrite()per header line plus one for the body. The kernel then holds the small final segment:tcp_nagle_check()lets a partial segment out only ifTCP_NODELAYorTCP_CORKis set, or every small packet sent so far has been acknowledged (tcp_minshall_check()). On a keep-alive connection the previous response is still unacknowledged, so the next small body waits for the peer's delayed ACK. Every request after the first on a connection pays it.Reproducer
A static file and plain curl, nothing else:
strace shows uhttpd is not the slow part: it wrote response 2 within 0.4 ms of response 1, and the client's next request arrived 42.5 ms after that write. The window is the peer's delayed ACK timer, so it depends on the client. These are Linux peers.
With the patch
Both builds come from the 25.12.5 release SDK, and the unpatched one is byte-identical to the published
uhttpd-2026.06.16~7b1bec45-r1.apk. Same box,http_keepaliveat its default 20:LuCI is where it shows, since its ubus replies run a few hundred bytes and a page issues several. Warm in-place navigation over five config pages went from 540 ms to 284 ms with keep-alive untouched.
uci set uhttpd.main.http_keepalive=0clears the stall at http too, but keep-alive matters for TLS, and with it off six HTTPS page loads at 120 ms RTT went from 9.8 s to 29.4 s.Trade-off
Each of the eight writes now leaves as its own segment. Counted with tcpdump: four sequential requests to that static file go from 13 to 32 server-to-client segments, and a full LuCI page load from 752 to 1211 packets for the same payload. A few dozen extra packets per page view against 40 ms stalls.