Struct RateLimiter 

pub struct RateLimiter { /* private fields */ }

Shared concurrent rate-limiter state.

Implementations

impl RateLimiter

pub fn from_env() -> Result<Arc<Self>, String>

Errors

As RateLimitConfig::from_env.

pub fn config(&self) -> RateLimitConfig

pub fn admit(&self, key: &str, estimated_tokens: u64) -> RateDecision

Admission check for a new request. estimated_tokens is the caller’s best guess at how many tokens this call will burn (prompt + max completion). Streaming callers pass their worst-case so the reservation is conservative; on completion they call refund_tokens with the true-up.

pub fn refund_tokens(&self, key: &str, amount: u64)

Called after a streaming request completes with (reserved - actual) so over-estimated reservations don’t burn the token bucket forever.