Rate Limiting

Control request frequency with rate limits

Rate limits help you control how frequently requests can be made, preventing abuse and ensuring fair usage.

Creating a Rate Limit

const rateLimit = await client.rateLimits.create({
  customerId: 'customer_123',
  eventType: 'model_call',
  limit: 100,
  window: 'minute',
});

rate_limit = await client.rate_limits.create(
    customer_id="customer_123",
    event_type="model_call",
    limit=100,
    window="minute",
)

Checking Rate Limits

const result = await client.rateLimits.check({
  customerId: 'customer_123',
  eventType: 'model_call',
});

if (!result.allowed) {
  // Rate limit exceeded, retry after result.retryAfter
}

result = await client.rate_limits.check(
    customer_id="customer_123",
    event_type="model_call",
)

if not result.allowed:
    # Rate limit exceeded, retry after result.retry_after
    pass

Next Steps

Setting Quotas - Set usage limits
Best Practices - Learn best practices

Customer Tracking

Track usage per customer or tenant

Setting Quotas

Create and manage usage quotas for your customers

On this page

Creating a Rate Limit Checking Rate Limits Next Steps