Skip to content

Fix S3CrtClient to use origin-form request-target - #3957

Open
hippi345 wants to merge 2 commits into
aws:mainfrom
hippi345:cursor/fix-s3crt-absolute-url-cd0a
Open

hippi345 wants to merge 2 commits into
aws:mainfrom
hippi345:cursor/fix-s3crt-absolute-url-cd0a

Conversation

@hippi345

Copy link
Copy Markdown

Issue #, if available:

Fixes #3956

Description of changes:

This PR fixes S3CrtClient to send origin-form request-targets (e.g., PUT /bucket/key HTTP/1.1) instead of absolute-form (e.g., PUT http://host/bucket/key HTTP/1.1) when connecting directly to origin servers.

Problem:
HttpRequest::ToCrtHttpRequest() was constructing absolute-form request-targets for all requests by prepending the scheme, authority, and port to the path. This violates RFC 9112 §3.2.1, which requires origin-form (path + query only) for direct connections to origin servers. While S3 and most S3-compatible servers accept absolute-form, security tools like CrowdSec's http-open-proxy scenario treat it as open-proxy probing, causing IP bans after a single failed request.

Solution:
Modified ToCrtHttpRequest() in src/aws-cpp-sdk-core/source/http/HttpRequest.cpp to set only the path and query string, removing the scheme, authority, and port components.

Proxy behavior is preserved:
The CRT HTTP layer (aws-c-http) handles proxy rewriting via aws_http_rewrite_uri_for_proxy_request (file: crt/aws-crt-cpp/crt/aws-c-http/source/proxy_connection.c, lines 931-1000). This function:

  • Parses the origin-form path we provide (line 942-949)
  • Rebuilds it with scheme, host, and port into absolute-form (lines 956-963)
  • Only when the request goes through a forward proxy (called via s_proxy_http_request_transform at line 1010)

This is per RFC 7230 §5.3.2, which requires absolute-form only for requests through forward proxies. Direct connections use origin-form.

Signing is not affected:
The CRT signers (SigV4/SigV4a via ToCrtHttpRequest) compute signatures over the canonical URI, which should be just the path component. The signing configuration includes the region and service name separately (see AWSAuthV4Signer.cpp lines 95-98), and the Host header is already set correctly. All 1088 core tests pass with this change.

Testing:
Added unit test HttpRequestConversionTest.ToCrtHttpRequestOriginForm that verifies:

  • Requests use origin-form (no scheme/authority in path)
  • Paths include query strings correctly
  • Root paths are handled properly

Test is placed outside DISABLE_DNS_REQUIRED_TESTS guard as it requires no network access.

Test output before fix (showing absolute-form bug):

Expected: "/bucket/key?uploadId=abc&partNumber=1"
Actual: "http://example.com:9000/bucket/key?uploadId=abc&partNumber=1"

[  FAILED  ] HttpRequestConversionTest.ToCrtHttpRequestOriginForm
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test

Test output after fix (origin-form, all tests pass):

[ RUN      ] HttpRequestConversionTest.ToCrtHttpRequestOriginForm
[       OK ] HttpRequestConversionTest.ToCrtHttpRequestOriginForm (0 ms)
[  PASSED  ] 1 test.

Full test suite: 1088 tests passed.

Check all that applies:

  • Did a review by yourself.
  • Added proper tests to cover this PR. (If tests are not applicable, explain.)
  • Checked if this PR is a breaking (APIs have been changed) change.
  • Checked if this PR will not introduce cross-platform inconsistent behavior.
  • Checked if this PR would require a ReadMe/Wiki update.

Check which platforms you have built SDK on to verify the correctness of this PR.

  • Linux
  • Windows
  • Android
  • MacOS
  • IOS
  • Other Platforms

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.


Generated with AI assistance and reviewed by hippi345.

HttpRequest::ToCrtHttpRequest() was constructing an absolute-form
request-target (e.g., "PUT http://host/path HTTP/1.1") for every
request. This violates RFC 9112 §3.2.1, which requires origin-form
(path + query only) when a client connects directly to an origin server.

aws-c-http already handles proxy rewriting via
aws_http_rewrite_uri_for_proxy_request, converting to absolute-form
only when necessary. By passing origin-form here, we prevent
security tools like CrowdSec's http-open-proxy scenario from
flagging legitimate S3-compatible storage requests as proxy abuse.

Fixes aws#3956
- Remove .changelog file (upstream has no such directory)
- Move ToCrtHttpRequestOriginForm test outside DISABLE_DNS_REQUIRED_TESTS
  guard as it requires no network access
@hippi345
hippi345 marked this pull request as ready for review October 16, 2026 19:01

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

S3CrtClient sends absolute-form request-target (PUT https://host/... HTTP/1.1) instead of origin-form

1 participant