Rate-limit handling to count requests

[tobixen]

Currently the rate-limit throttling is done very simple - if it's allowed, say, to send 30000 requests within a 30000 second window, it will sleep 1s between each request.

Two alternative methods should be considered:

• Send 30000 requests without any throttling, then sleep out the window.
• Send the first request without any throttling, then add a progressingly growing delay so that there will never be a long complete halt when the quota has been reached.

The description below was AI-generated and seems to follow the first
method suggested above.

---

There are two separate pieces of rate-limit machinery today, and neither counts
requests.

The client is purely reactive. BaseDAVClient._rate_limit_sleep_seconds()
only ever runs after the server has answered 429 (or 503 with Retry-After).
It sleeps Retry-After, or default_sleep, capped by max_sleep, and retries.
Nothing tracks how many requests have been sent, so the client walks into the
throttle every time and then waits it out.

The test framework throttles pre-emptively, with a fixed delay per request.
tests/test_caldav.py around line 1412:

foo = self.is_supported("rate-limit", dict)
if foo.get("enable"):
    rate_delay = foo.get("interval", 0) / foo.get("count", 1)
    self.caldav.request = _delay_decorator(self.caldav.request, t=rate_delay)

So a server declaring interval: 300, count: 1500 gets 300 / 1500 = 0.2
seconds of sleep before every request, including the first, when the whole
budget is still unspent. A run that makes a few thousand requests pays minutes
for it. For ecloud, which declares interval: 2, count: 1, it is 2 seconds per
request.

What it should do instead

Count. A token bucket, or simply a deque of the timestamps of requests inside
the window:

• under budget → send immediately, no sleep at all;
• budget spent → sleep exactly long enough for the oldest request in the window
  to age out, not a fixed slice.

With 1500 per 300s that means the first 1500 requests go through at full speed
and only a run that genuinely exceeds the server's budget ever waits.

Where it belongs

Arguably in the client rather than the test framework, so that real users get it
too: a client that knows a server's published limits can stay under them instead
of discovering them through 429s. The test framework's decorator could then go
away. Deciding that is part of the issue - the rate-limit feature already
carries interval and count, and nothing outside the test suite reads them.

Noticed while adding a rate-limit declaration for OX (interval 300, count
1500), which switched that 0.2-second-per-request delay on for the whole OX test
run.