Skip to content

Throttle /activities and /api/v1/activities per IP - #4815

Merged
CloCkWeRX merged 1 commit into
devfrom
throttle-api-activities
Sep 21, 2026
Merged

CloCkWeRX merged 1 commit into
devfrom
throttle-api-activities

Conversation

@Br3nda

@Br3nda Br3nda commented Sep 20, 2026 •

Copy link
Copy Markdown
Member

Why

/api/v1/activities is the single busiest route in production, and the only high-volume one with no rate limit at all.

From a 14 minute window of growstuff-prod router logs:

Route Requests % of all
/api/v1/activities 216 19.6%
/crops/:id 164 14.9%
(assets) 110 10.0%
/plantings 91 8.3%
/plantings/:id 63 5.7%

The throttle matched plantings|harvests|members only, so activities — web and API — was unprotected on a dyno that is currently being OOM-killed every ~9 minutes.

What changed

Added activities to the req/ip/restricted_routes throttle, and collapsed the two alternations into a single regex with an optional /api/v1 prefix so the HTML and API route lists can't drift apart:

restricted_routes = %r{^(/api/v1)?/(activities|plantings|harvests|members)(/|$)}

Please read this before assuming it fixes the load

It does not stop the client currently hammering that endpoint, and I don't want that to be a surprise.

Those 216 requests are only 10 distinct URLs, and 189 of them come from a single IP walking page[offset] 10→80 for one owner on a loop. But it makes those 189 requests over 21 minutes — about 9 requests per minute, comfortably under the 15/min limit. On top of that, Rack::Attack counters live in a per-worker MemoryStore (deliberately, per #4800), so at WEB_CONCURRENCY=2 each worker only ever sees about 4.5/min of it.

So this PR closes the hole; it doesn't reduce today's traffic. Actually throttling that client needs one or more of:

  • WEB_CONCURRENCY=1, which unifies the counters (and is the fix for the OOM crashes anyway)
  • a lower limit on this rule, which would also affect plantings/harvests/members and wants its own discussion
  • caching the endpoint — 10 distinct URLs re-requested every ~40s is close to ideal cache material

Happy to follow up with whichever of those you prefer.

Testing

  • Verified the new regex is exactly equivalent to the old pair plus the five activities cases, across both route families. /plantingsfoo, /memberships, /activitiesfoo, /foo/plantings and /api/v2/activities remain unmatched, so there are no new false positives.
  • rubocop config/initializers/rack_attack.rb — 3 offences, byte-identical to the 3 already on dev (lines 54, 61, 64, none of them mine). No new offences.
  • rspec spec/requests/rack_attack_spec.rb — 2 failures, both pre-existing; confirmed identical on a clean tree with this branch stashed.

No spec added, deliberately: this throttle is registered inside if Rails.env.production?, evaluated once at boot, so it isn't reachable from the test environment without restructuring the initializer. That's also why the existing spec file covers the blocklists but no throttle. Worth fixing separately if we want throttle coverage.

Those two pre-existing failures are the honeypot and excessive-crawling ban tests — i.e. the ban machinery itself. Worth a look on its own, because those bans are held in per-worker memory and production currently restarts every ~9 minutes, so long-window bans never accumulate there either.

🤖 Generated with Claude Code

/api/v1/activities is the single busiest route in production: 216 of 1101
requests in a 14 minute log window, 19.6% of all traffic and 21.7% of
non-asset traffic. It was the only high-volume route with no rate limit,
since the throttle matched plantings, harvests and members only.

Collapses the two alternations into one regex with an optional /api/v1
prefix, so the HTML and API routes stay in step instead of being listed
twice. Verified equivalent to the old pair across both route families
plus their edge cases: /plantingsfoo, /memberships, /foo/plantings and
/api/v2/activities are still unmatched.

Note this does not by itself stop the client currently polling that
endpoint. It makes 189 requests over 21 minutes, about 9 per minute,
which is under the 15 per minute limit, and Rack::Attack counters live
in per-worker memory so at WEB_CONCURRENCY=2 each worker sees only half
of them. This closes the hole rather than fixing today's load.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Br3nda
Br3nda requested a review from CloCkWeRX September 20, 2026 10:50
@CloCkWeRX

Copy link
Copy Markdown
Collaborator

Weird, we shouldn't even have that many activities!

@CloCkWeRX

Copy link
Copy Markdown
Collaborator

TODO: Check https://github.com/Growstuff/homeassistant-growstuff/blob/dev/custom_components/growstuff/sensor.py#L68 or add a user agent so its a bunch easier to see if this is part of the problem

@CloCkWeRX
CloCkWeRX merged commit 96b38eb into dev Sep 21, 2026
16 checks passed
@CloCkWeRX
CloCkWeRX deleted the throttle-api-activities branch September 21, 2026 08:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants