Skip to content

Keep original content-length when beforeRequest changes headers on a truncated body - #221

Open
rcasazza wants to merge 1 commit into
httptoolkit:mainfrom
casazium:fix/before-request-content-length
Open

rcasazza wants to merge 1 commit into
httptoolkit:mainfrom
casazium:fix/before-request-content-length

Conversation

@rcasazza

Copy link
Copy Markdown
Contributor

When a beforeRequest callback returns new headers but no new body, Mockttp recalculates Content-Length from completedRequest.body.buffer. If the request body was larger than maxBodySize, that buffer is empty, because buffering gave up. The recalculated length is therefore 0, while the original body is still streamed upstream in full. The upstream server receives Content-Length: 0 followed by the whole body, a malformed request. Node's HTTP server, for example, answers 400 Bad Request.

This hits any passthrough rule that only tweaks headers, such as adding an auth header or removing a cookie, on large uploads.

Repro

const server = getLocal({ maxBodySize: 10 });
await server.forPost(remoteUrl).thenPassThrough({
    beforeRequest: (req) => ({ headers: { ...req.headers, 'x-added-header': 'yes' } })
});
// POST a 20-byte body through the proxy: upstream sees content-length: 0, then 20 bytes.

Fix

When there's no body override and the buffered body's size doesn't match the original Content-Length, the body was truncated and can't be measured. In that case the original framing is kept. If the new headers left out content-length (and aren't chunked), the original value is restored on HTTP/1. If the new headers set their own content-length, it's left as they set it.

Requests whose body was fully buffered behave exactly as before. So do requests with a body override, which are still measured from the new body.

Test

The new test sits next to "should still proxy larger request bodies even if beforeRequest is used", in the "when only tiny bodies are allowed" block of http-proxying.spec.ts. It fails before the fix (expected '0' to equal '20') and passes after.

Found while building breakpoints (pause, edit and resume) on top of Mockttp's beforeRequest.

All Node tests pass locally (mocha test/**/*.spec.ts: 846 passing, 2 pending).

…truncated body

If the request body exceeded maxBodySize, the buffered copy passed to beforeRequest
is empty, but the original body is still streamed upstream. Recalculating the
content-length from that empty buffer sent 'content-length: 0' followed by the full
body. Now the original framing is kept in that case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pimterry

Copy link
Copy Markdown
Member

This is a good find! Thanks for the PR.

That said, I'm going to leave this open unmerged for now. I have some buffering changes coming imminently that will change truncation & streaming behaviour significantly, which I think will either fix this bug or change the fix completely. Either way I'll pull in your test here to make sure that gets properly covered.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants