Rate limits and retries
How BakaSub paces its requests to OpenRouter and what happens when a request fails. Code: src/main/llm/ (rate-limiter.ts, retry-policy.ts, llm-session.ts, free-quota.ts, translate-document.ts). User guide (in-app, en/pt-BR): openrouter-limits.md.
OpenRouter limits (what we design for)
| Model | Per minute | Per day (UTC) |
|---|---|---|
:free variants (or price 0) |
20 requests | 50 if the account bought < US$ 10 of credits (ever), 1000 if ≥ 10 |
| paid | no platform cap | none; the upstream provider may answer 429 |
Paid requests also hold an in-flight budget on prepaid accounts: each running request reserves its estimated cost (input + max_tokens) from the balance; when the reservations do not fit, OpenRouter answers 402 in_flight_budget_exhausted with Retry-After (weight_exceeds_budget means a single request is too big for the balance). Parallel lanes make this reachable with a low balance.
Limits are per account: more keys do not raise them. GET /api/v1/key reports free_model_daily_requests { used, limit, remaining } (not the per-minute limit). OpenRouter-generated 429s carry X-RateLimit-Limit/Remaining/Reset (reset in epoch ms); a 429 can also arrive inside the stream as an SSE error event with code: 429.
Layers
translateDocument └─ K lanes in parallel, each a chunk loop ── extra rounds (60 s, 120 s…) when retries are exhausted └─ ChunkTranslator ── content attempts (translate + 2 repairs), length/moderation splits └─ LlmSession.request ├─ RateLimiter.acquire(lane) → Lease ← process-wide, shared by all jobs │ (cooldown → rpm window → in-flight slot under the AIMD limit) ├─ OpenRouterClient.chat ├─ lease.release('ok' | 'throttled' | 'other') ← before any wait └─ decide(err, attempt, config) ├─ 429 / 402 in-flight → limiter.cooldown(lane, until) (every job waits) ├─ 503 / 5xx… → sleep(backoff) (this request only) └─ fatal → FatalLlmError (LLM_AUTH, LLM_CREDITS, LLM_DAILY_LIMIT…)- Lanes (
laneOf):free= id ends with:freeor prompt and completion prices are 0;paid= everything else, including dynamic pricing (routers). RateLimiter: one instance in the Main (createJobModule), a 60 s sliding window per lane plus a lane cooldown. Waiters are served in arrival order; a cancelled wait does not consume a slot. The config is read on everyacquire, so changes in Settings apply to running jobs.- In-flight cap (
llm.concurrency.*MaxInFlight): at most N requests of the lane run at once, summing every job.acquirereturns aLease;LlmSessionreleases it as soon as the response or the error arrives, before any backoff, so a waiting request never holds a slot. Waiting for a slot is not reported aswaiting(it is parallelism, not OpenRouter holding back). A cancelled wait does not take a slot. - AIMD below the cap: the effective limit starts at the cap, halves on
throttled(429 or 402in_flight_budget_exhausted), and grows +1 after 5 consecutive successes. One decrease per burst: every lease records the limiter’s epoch; athrottledfrom a lease taken before the last decrease is ignored, so 8 simultaneous 429s halve once (8 → 4), not down to 1. Without a configured cap the decrease halves what was actually in flight. After 5 min idle the lane goes back to the cap. - Cooldown: a 429 pauses the whole lane until
Retry-After, elseX-RateLimit-Reset, else the exponential backoff (capped at 60 s). A later, shorter cooldown never shortens an earlier one. 503 is usually the model or provider, not the account, so it only delays its own request. - 402
in_flight_budget_exhausted(LlmError.kind = 'budget-busy', also a 403 withlimit_source: openrouter_in_flight_budget): retried like a 429 (lane cooldown,maxThrottledattempts) and lowers AIMD. Any other 402 is stillcredits→LLM_CREDITS, fatal. - Daily quota: a 429 with
X-RateLimit-Remaining: 0and a limit ≥ 50 (or a reset more than 10 min away) isdaily-limit→LLM_DAILY_LIMIT, no retry; the message carries the reset time. Lines already accepted are in Job Memory, so retrying the job after the reset continues where it stopped. - Chunk rounds: when an exhausted transport retry ends in
LLM_UNAVAILABLE, the chunk getschunkRoundsextra rounds after 60 s, 120 s…, sending only the lines that are still missing. Network errors never turn lines into “untranslated”: the job fails and can be retried.
Settings (Settings → Advanced, not in the wizard)
| Key | Default | Range | Meaning |
|---|---|---|---|
llm.rateLimit.freeRpm |
20 | 1–60 | requests per minute, free lane |
llm.rateLimit.paidRpm |
null |
1–1000 or null | requests per minute, paid lane (null = no cap) |
llm.rateLimit.freeTier |
auto |
auto / under-10 / over-10 | daily quota tier; auto reads GET /key |
llm.retry.maxThrottled |
5 | 1–10 | attempts on 429/502/503 |
llm.retry.maxTransient |
3 | 1–10 | attempts on 5xx, timeout, stream error, network |
llm.retry.baseDelayMs |
1000 | 250–10000 | first backoff (then ×2 per attempt, jitter 50–100%) |
llm.retry.maxDelayMs |
30000 | 1000–120000 | backoff cap (baseDelayMs > maxDelayMs → SETTINGS_CONFLICT) |
llm.retry.chunkRounds |
2 | 0–5 | extra rounds per chunk after transport retries are exhausted |
llm.concurrency.perJob |
4 | 1–8 (recommended 2–4) | lanes per file (1 = sequential); see llm.md |
llm.concurrency.paidMaxInFlight |
8 | 1–32 (recommended 4–12) | requests in flight at once on the paid lane, all jobs |
llm.concurrency.freeMaxInFlight |
3 | 1–10 (recommended 2–3) | same for the free lane (20 req/min makes more pointless) |
The recommended bands live in CONCURRENCY_RECOMMENDED (shared/ipc-contract.ts). The UI draws them on a small track under each field and warns, without blocking, when a value leaves the band (below: slower; above: more cuts, 429s or 402s). It also warns when jobs.concurrency × perJob exceeds paidMaxInFlight (lanes would wait for a slot).
settings:reset { keys } restores factory defaults (“Restore defaults”), going through the same rules as settings:update.
Free quota (FreeQuota, models:free-quota)
status() → FreeQuotaStatus { tier, source, limit, used, remaining }. auto: from GET /key (cached 30 s; invalidated when the API key or the tier setting changes); without an answer (no key, offline) the tier is unknown and the limit is assumed to be 50. An override fixes the limit and still uses used from /key when available.
The preflight adds freeQuota { …status, requests, enough } for free models, where requests = ceil(chunks × 1.25) (repairs and rounds). enough: false never blocks: it forces needs-confirmation (a free job costs US$ 0 and would otherwise auto-confirm) and adds a job warning. With confirmBeforeTranslate: false (batch) only the warning remains.
Visible waits
JobProgress.waiting?: { reason: 'rate-limit' | 'cooldown' | 'backoff', untilMs } is set while every running lane of a job waits (earliest resume time) and cleared when a request goes out (never kept in a terminal state). rate-limit = per-minute window, cooldown = recent 429 on the lane (all jobs on that lane show it), backoff = retry after an error or before an extra chunk round.