fix(agent): remove ModSecurity, raise HTTP/2 and HTTP/3 upload buffers#423
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves upload performance through the agent reverse proxy by removing ModSecurity (which was adding request-body buffering/scan overhead in DetectionOnly mode) and by increasing HTTP/2 + HTTP/3 request-body buffering limits to reduce flow-control throttling on large uploads.
Changes:
- Removed ModSecurity/OWASP CRS enablement from the generated nginx config and stopped shipping ModSecurity-related packages/config/log rotation in the agent image.
- Increased HTTP/2 and HTTP/3 request-body buffering limits in the nginx
http {}block to improve throughput for large uploads over multiplexed connections. - Updated the 403 error page wording and developer docs to remove WAF-specific messaging.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| pull-config/.fpm | Drops ModSecurity/CRS package dependencies from the agent pull-config package. |
| create-a-container/views/nginx-conf.ejs | Removes modsecurity directives and adds http2_body_preread_size / http3_stream_buffer_size tuning. |
| images/agent/Dockerfile | Removes ModSecurity default configuration + CRS config copy steps from the agent image build. |
| images/agent/nginx.logrotate | Removes rotation stanza for the ModSecurity audit log. |
| images/agent/crs-setup.conf | Deletes vendored CRS setup configuration (no longer used). |
| images/agent/RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf | Deletes vendored CRS exclusion configuration (no longer used). |
| error-pages/403.html | Rewords 403 page from “WAF blocked” to generic access-denied messaging. |
| mie-opensource-landing/docs/developers/docker-images.md | Removes ModSecurity mentions from the agent image documentation. |
runleveldev
force-pushed
the
395-bug-uploads-are-very-slow-through-the-agent-reverse-proxy-modsecurity-scanning-request-bodies
branch
from
July 23, 2026 17:27
0323bc3 to
29e3cf3
Compare
Comment on lines
161
to
165
| <div class="info"> | ||
| <strong>Think this is a mistake?</strong> | ||
| Provide the request ID above to your server administrator. | ||
| The specific rule that triggered the block can be found in the | ||
| server audit log using this ID. | ||
| The request can be traced in the server logs using this ID. | ||
| </div> |
runleveldev
force-pushed
the
395-bug-uploads-are-very-slow-through-the-agent-reverse-proxy-modsecurity-scanning-request-bodies
branch
2 times, most recently
from
July 23, 2026 17:58
e58eede to
aeec0a8
Compare
ModSecurity ran in DetectionOnly mode, so every request body was buffered and scanned on the agent without blocking anything, defeating proxy_request_buffering off and crippling large-upload throughput. Remove it entirely: - nginx directives in the agent config template - libnginx-mod-http-modsecurity / modsecurity-crs package dependencies - agent image build steps and vendored CRS configuration files - the custom nginx logrotate override, which only existed to keep modsec_audit.log out of the stock glob and copytruncate it around owasp-modsecurity/ModSecurity-nginx#351; the Debian default (/var/log/nginx/*.log, same policy) now covers everything - WAF wording on the 403 error page and in the docs Raise the HTTP/2 and HTTP/3 request-body buffers from their small defaults (64k preread / h3 stream buffer, 256k h2 recv buffer) so multiplexed uploads are not throttled by flow control. The h2 recv buffer is per worker, so it is sized at several prereads to keep concurrent uploads from starving each other: http2_body_preread_size 4m; http2_recv_buffer_size 16m; http3_stream_buffer_size 4m; Fixes #395
runleveldev
force-pushed
the
395-bug-uploads-are-very-slow-through-the-agent-reverse-proxy-modsecurity-scanning-request-bodies
branch
from
July 23, 2026 18:04
aeec0a8 to
afebcd6
Compare
cmyers-mieweb
approved these changes
Jul 24, 2026
runleveldev
deleted the
395-bug-uploads-are-very-slow-through-the-agent-reverse-proxy-modsecurity-scanning-request-bodies
branch
July 24, 2026 14:18
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #395
Stacked on #407 — based on
cmyers_issue357since that lands first; GitHub will retarget this tomainwhen #407 merges.Problem
Uploads through the agent reverse proxy crawl. ModSecurity is enabled globally on the agent and runs in
DetectionOnlymode (see #221), so it buffers and scans every request body while blocking nothing — defeatingproxy_request_buffering offand paying the full scan cost for log entries only. The WAF has already required repeated allowlisting to keep legitimate traffic working (#219, #220, #221).Changes
Remove ModSecurity entirely
agent/templates/nginx.conf.ejs: dropmodsecurity on,modsecurity_rules_file,modsecurity_transaction_id, and themodsecurity offcarve-out in the error-pages serveragent/.fpm: drop thelibnginx-mod-http-modsecurityandmodsecurity-crspackage dependenciesimages/agent/Dockerfile: remove the ModSecurity default-configuration steps and the vendored CRS config copiesimages/agent/crs-setup.confandimages/agent/RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.confimages/agent/nginx.logrotateand itsCOPY: the override only existed to keepmodsec_audit.logout of the stock glob andcopytruncateit around ModSecurity-nginx#351. The Debian default (/var/log/nginx/*.log, identical policy) covers access/error/stream logs on its own.error-pages/403.html: reword from "blocked by the WAF / OWASP CRS" to a generic access-denied page (403s now come from the auth layer)mie-opensource-landing/docs/developers/docker-images.md: drop ModSecurity mentionsRaise HTTP/2 and HTTP/3 request-body buffers (http block; defaults were 64k preread / h3 stream buffer and 256k per-worker h2 recv buffer):
The small defaults cap how much body a client can send before waiting on flow control, which throttles large uploads over multiplexed connections. The h2 recv buffer is shared per worker, so it is sized at several prereads (16m) to keep concurrent uploads from starving each other.
Notes
agent/templates/nginx.conf.ejsrenders with Replace pull-config with TypeScript agent #407's config snapshot shape (empty/bootstrap and populated site) — new directives present, nomodsecurityreferences remain in the tree.