Add HTTP rate limiting detection and handling #23

New issue

Closed

opened 2026-04-12 17:29:09 +00:00 by Grandiras · 1 comment

Grandiras commented

2026-04-12 17:29:09 +00:00

Owner

Summary

ATProto.NET doesn't implement HTTP rate limiting detection and handling. The AT Protocol spec implies rate limiting by services, and proper handling is critical for production applications, especially when doing backfill operations or high-volume API calls.

What's needed

Rate limit header parsing - Parse standard rate limit headers:
- RateLimit-Limit - Max requests per window
- RateLimit-Remaining - Remaining requests
- RateLimit-Reset - Time until reset (Unix timestamp)
- Retry-After - Seconds to wait before retry
Automatic retry with backoff - When receiving HTTP 429 Too Many Requests:
- Respect Retry-After header
- Exponential backoff as fallback
- Configurable max retry count
Rate limit information exposure - Expose rate limit info to consumers:
- Current remaining requests
- Reset time
- Events/callbacks for approaching limits
Client-side rate limiting - Optional pre-emptive throttling:
- Useful for backfill/crawl operations
- Configurable requests-per-second limits

Context

From the AT Protocol backfill guide:

"Be careful not to get rate limited. You will be making one call to getRepo per user. It is recommended to implement client side rate limiting to prevent your requests from getting blocked by firewalls."

The firehose consumer already has reconnection backoff, but HTTP API calls have no rate limit awareness.

## Summary ATProto.NET doesn't implement HTTP rate limiting detection and handling. The AT Protocol spec implies rate limiting by services, and proper handling is critical for production applications, especially when doing backfill operations or high-volume API calls. ## What's needed 1. **Rate limit header parsing** - Parse standard rate limit headers: - `RateLimit-Limit` - Max requests per window - `RateLimit-Remaining` - Remaining requests - `RateLimit-Reset` - Time until reset (Unix timestamp) - `Retry-After` - Seconds to wait before retry 2. **Automatic retry with backoff** - When receiving HTTP 429 Too Many Requests: - Respect `Retry-After` header - Exponential backoff as fallback - Configurable max retry count 3. **Rate limit information exposure** - Expose rate limit info to consumers: - Current remaining requests - Reset time - Events/callbacks for approaching limits 4. **Client-side rate limiting** - Optional pre-emptive throttling: - Useful for backfill/crawl operations - Configurable requests-per-second limits ## Context From the AT Protocol backfill guide: > "Be careful not to get rate limited. You will be making one call to getRepo per user. It is recommended to implement client side rate limiting to prevent your requests from getting blocked by firewalls." The firehose consumer already has reconnection backoff, but HTTP API calls have no rate limit awareness.

Grandiras referenced this issue from a commit

2026-04-12 17:54:56 +00:00

feat: HTTP rate limiting with automatic 429 retry

Grandiras closed this issue

2026-04-12 17:54:56 +00:00

Grandiras referenced this issue from a commit

2026-04-12 17:54:56 +00:00

docs: add changelog entries for issues #20, #23, #30

Grandiras commented

2026-04-12 17:55:11 +00:00

Author

Owner

Implemented in commit f8d7141. Added RateLimitInfo model and automatic retry on HTTP 429 with Retry-After/RateLimit-Reset support and exponential backoff. Configurable via MaxRateLimitRetries.

Implemented in commit f8d7141. Added RateLimitInfo model and automatic retry on HTTP 429 with Retry-After/RateLimit-Reset support and exponential backoff. Configurable via MaxRateLimitRetries.