Rate limits

Throttling behavior, retries, and how to handle 429s.

Dynoyard doesn’t impose fixed per-key request-rate quotas today. The natural ceilings are upstream provider concurrency and (optionally) a per-key monthly spend cap. When a ceiling is hit you get a standard 429.

When you’ll see a 429

{
  "error": {
    "message": "Rate limit reached. Retry after a short backoff.",
    "type": "rate_limit",
    "code": "rate_limited"
  }
}

• Upstream concurrency — the provider serving the chosen model is momentarily saturated. Usually clears in seconds.
• Monthly spend cap (coming soon) — if you set a per-key USD cap, the key returns 429 once it’s exceeded for the month.

Server-side retries

Transient 429 and 5xx are retried on our side with jittered exponential backoff (total budget ~12s) before the error reaches you. So a 429 in your client means the upstream stayed saturated through that whole budget — back off rather than hammering.

Client-side handling

Honor Retry-After when present; otherwise use exponential backoff with jitter. The OpenAI SDKs already retry 429/5xx by default — keep that on.

from openai import OpenAI

# The OpenAI SDK retries 429/5xx with backoff out of the box.
client = OpenAI(
    base_url="https://<your-org>.dynoyard.app/v1",
    api_key="sk-dyno-XXXX",
    max_retries=4,
)

Reducing throttling

• Spread load across time rather than bursting.
• Use streaming for long generations so a single request holds a connection instead of retrying.
• Talk to us before a launch spike — volume customers can get dedicated upstream capacity (no shared queue). Email hello@dynoyard.app.

See the full error table in the API contract.