Rate limits#
How many calls an agent makes#
Every guard makes one call to Enkrypt each time it runs. An agent with three guards on the prompt, two on tool results and three on the answer makes eight calls for a run with one tool call, more with more tool calls. Retries run the answer’s guards again.
Enkrypt’s default plan allows 10 calls a minute and 3 a second. An agent like the brokerage agent can use most of a minute’s budget in one run. A saved guardrail runs several detectors in one call, and is the main way to make fewer.
What a guard does when the limit is reached#
When Enkrypt answers that the limit is reached (HTTP 429), the guard waits until Enkrypt will accept another call, then tries again, rather than failing:
it waits as long as Enkrypt’s
Retry-Afterheader asks, if it sends one;otherwise, as long as Enkrypt’s rate-limit headers (
x-enkrypt-ratelimit-limit-*and-remaining-*) say the budget needs to refill;otherwise, it backs off exponentially, up to 30 seconds per wait.
Each wait adds a little random time, so guards limited at the same moment do not all retry at the same moment.
Settings#
All on Enkrypt, for every guard it creates:
max_wait=60.0The most seconds one guard call spends waiting in total. After that, the guard’s
on_errorapplies: by default it blocks, with an error naming how long it waited.Enkrypt(max_wait=0)never waits.max_retries=NoneThe most retries per guard call, if set. By default only
max_waitlimits them.max_concurrency=NoneThe most Enkrypt calls in flight at once, from this object. It limits calls at the same moment, not calls per minute.
enkrypt = Enkrypt(max_wait=10, max_concurrency=4)
The defaults favour guards that work over guards that fail fast: on the default plan, a guard call may wait several seconds. Lower max_wait if latency matters more than every check completing, or ask Enkrypt for a higher limit.
Measured#
A burst of 15 calls at once, on the default plan:
Setting |
Failed |
Time |
|---|---|---|
default ( |
0 |
17 s |
|
4 |
6 s |