LLM translation layer
src/main/llm/ turns a parsed subtitle document into a translated one through OpenRouter. It is pure Node (no electron import) and fully injectable: translateDocument(input, deps, { signal, onProgress }). The Job Manager (jobs.md) wraps it in the translate step.
translateDocument ├─ models.find(preset.model) → LLM_UNKNOWN_MODEL if absent ├─ memory.lookupDocument(doc) → validated hits + misses (see job-memory.md) ├─ planLanes(misses, perJob) → K contiguous lanes of chunks, timeline order └─ K lanes in parallel (Promise.all); inside a lane, chunk after chunk: ChunkTranslator.run ├─ PromptBuilder (system + <context>/<target> | repair) ├─ LlmSession.request → in-flight slot, transport retry, structured output, usage ├─ parseTranslations (Zod) → validateTranslation per line ├─ accept valid → memory.commitChunk (one transaction, process-wide write queue) └─ repair only the invalid lines (≤ 3 attempts per line) → applyTranslation on every known cue → { doc, failed, usage, cache, status }| Module | Responsibility |
|---|---|
sse.ts |
incremental WHATWG SSE decoder (bytes split anywhere, CR/LF/CRLF, comments, BOM) |
openrouter-client.ts |
Chat Completions over SSE, error classification (LlmError.kind), idle timeout, key redaction |
model-catalog.ts |
/api/v1/models normalized to ModelInfo, cached in settings (24 h, offline fallback) |
retry-policy.ts |
transport decision per error kind and attempt (pure, configurable) |
rate-limiter.ts |
process-wide per lane (free/paid): sliding window, cooldown after a 429, in-flight cap with AIMD |
free-quota.ts |
free-model daily quota from GET /api/v1/key, tier override |
llm-session.ts |
one lane ↔ one model: request + retry + output mode + usage, cost lookup of interrupted generations |
chunker.ts |
chunk planning and the context window |
segmenter.ts |
cuts the misses into lanes for parallel translation (planLanes, countChunks) |
prompt-builder.ts |
glossary parsing, System Prompt, translate/repair messages, strict JSON Schema |
response-parser.ts |
tolerant JSON extraction + Zod shape check |
chunk-translator.ts |
translate → validate → accept → repair loop, length/moderation splits |
translate-document.ts |
orchestration of the lanes, failure threshold, drain/cancellation, progress timeline, final doc |
Chunks and context
- Only Job Memory misses are sent, one per context hash (copies are filled from their representative and stay out of
<context>, seejob-memory.md), in timeline order (HashedUnit.timeIndex:start.ms, file order on ties). Inside a lane chunks run sequentially, so each chunk sees the previous translations; lanes run in parallel (next section). - A chunk holds up to 40 lines or ~2,000 estimated tokens of
<target>(token-estimate.ts: ~4 chars/token, ~1 per CJK character; for limits only, the preflight uses a real tokenizer, seepreflight.md). A line larger than the budget goes alone. <context>: the 5 translatable cues before the chunk (source and translation when known, from the cache or an earlier chunk) and 2 after (source only), as plain text without markers.max_tokens≈ 2.5 × the target estimate + JSON overhead, at least 256, capped bytop_provider.max_completion_tokens.
Lanes (parallel translation)
A file is cut into K contiguous lanes that run at the same time. Each lane is sequential, so every chunk but the head of each lane gets translated <before> lines. Cost is unchanged: same lines, same number of requests.
- K =
min(llm.concurrency.perJob, floor(chunks / 2)), at least 1. Two chunks per lane minimum, otherwise a lane would never have translated context. Small files and mostly cached files fall back to one lane.perJob = 1reproduces the old sequential plan exactly. - Only “free” cuts: a cut at line
iis allowed only ifchunks(prev..i) + chunks(i..end) == chunks(prev..end). A cut in the middle of a chunk would create an extra partial chunk, and every extra chunk repeats the System Prompt and the context (measured: +8% prompt on 400 lines × 4 lanes before this rule).countChunksreplaysplanChunks’ rule on precomputed token counts (O(n) per check; 5,000 lines × 8 lanes plan in well under 200 ms). - Where to cut: the ideal cut splits the estimated tokens into K equal parts. Within ±25% of a segment around it, the free cut with the highest score wins:
gap(next line start − previous line end, i.e. pauses, scene changes, OP/ED) + 10 s per cached line right before the cut (up to 5, the size of<before>): with cache hits there the head of the lane gets translated context anyway. Ties go to the closest to the ideal. With no free cut in the window, the closest free cut (greedy chunk boundaries always are). Without slack (e.g. exactly 400 lines and 40-line chunks), the free cuts are only chunk boundaries and lanes can be uneven: cost wins over balance. - Static lanes, no work stealing: stealing would add cuts at arbitrary points. When AIMD lowers the in-flight limit below K, lanes simply wait for a slot in
RateLimiter(FIFO). - Shared state:
known(translations so far) is one map for all lanes, sobuildContextuses another lane’s translation when it already exists.failed,validationFailedand theMAX_FAILED_RATIOcheck are shared too (checked after every chunk of any lane). - One
LlmSessionper lane (usage summed at the end), so waits are tracked per lane. - Failure drains: the first fatal error (or
LLM_TOO_MANY_FAILURES) is recorded and adrainsignal goes up.LlmSessiontreats it as “no new requests and no more waiting”: the nextacquire, backoff or extra round throwsDrainRequestedand the lane leaves quietly. A stream already in flight is never aborted by a drain (it is being paid for): its valid lines are committed. The job then fails with the first error. - Cancellation by the caller still aborts every stream immediately (
signalreacheschat), as before. - Progress (
TranslateProgress):chunk { completed, count, inFlight },timeline { total, cached, translated, failed, active, lanes }as compact[from, to)ranges of timeline positions (rendered as the episode map in the job detail), andwaitingonly when every running lane waits (the earliest resume time); one lane in backoff while others work is not “waiting”.
Prompt
- System (identical for every chunk of a job: it is the prefix provider prompt caching reuses): the termbase on top (
<glossary>, whole preset glossary, entries{source, target?, rule?}, invalid entries dropped and logged), then the preset style guidance (<style>, seepresets.md), then fixed rules in English that always apply (translate only<target>, keep every<gN>…</gN>/<xN/>/<br/>exactly once and<xN/>in order, no new tags or line breaks, no notes; idiomatic writing: idioms adapted to the natural target-language equivalent with the same tone, imperative mood for commands/battle cries, current official spelling; the rule is written with the target language’s English name and followed by per-language notes fromlanguage-notes.ts, looked up by full BCP 47 tag then base language, e.g. pt-BR: 1990 Orthographic Agreement, imperative examples, idiom example; languages without notes get only the generic rule), then the output format, then<media>(frozen TMDB details + user notes, seemedia-context.md) as reference.<media>lives here, not in each chunk, to make the cacheable prefix longer. - User:
<context>(<before>/<after>),<target>with<line id="N">payload</line>. session_id: every request of one translation carries the samesession_id(a UUID pertranslateDocumentcall). OpenRouter’s sticky routing then keeps the model on the same provider endpoint, so the automatic prompt cache (OpenAI, DeepSeek, Gemini 2.5+, Grok, Moonshot…) keeps hitting across chunks and lanes. Fallback on provider errors still applies.cachedTokenscomes back inusage.- Repair: same System; for each failing line,
<source>, what was<returned>and the concrete<problems>(e.g.<g1>…</g1> is missing,expected exactly 2 <br/>, got 1). - Snapshots in
prompt-builder.test.tspin the exact text.
Output
{ "<cueId>": "<translated payload>" }. Mode chosen per model from the catalog:
structured_outputs→response_format: { type: 'json_schema', json_schema: { name, strict: true, schema } }with exactly the chunk ids as required properties andadditionalProperties: false, plusprovider.require_parameters: true(support is per endpoint, not per model).response_format→{ type: 'json_object' }.- Neither → prompt only.
Always parsed with the same two layers: tolerant extraction (code fences, prose around, first balanced object) + Zod shape (extra ids ignored and logged; missing or non-string ids are failures), then validateTranslation per line.
Retries
| Failure | Policy |
|---|---|
| line fails validation / missing from the answer | accept the valid lines (written to the cache), repair only the failing ones; ≤ 3 attempts per line |
| unparseable JSON | counts as an attempt for every pending line; retried with the translate prompt |
finish_reason: length |
split the chunk in half (no content attempt spent) |
moderation (403 / content_filter) |
split until isolated (≤ 3 levels); blocked lines → UNTRANSLATED_CONTENT_FILTERED |
| 429 | the whole lane cools down (Retry-After, else X-RateLimit-Reset, else backoff; ≤ 60 s); ≤ 5 attempts |
| 429 free-model daily quota | fail the job: LLM_DAILY_LIMIT (reset time in the message) |
| 503 | backoff 1 s…30 s with jitter, this request only; ≤ 5 attempts |
| 408 / 5xx / error inside the stream / 60 s idle / network | backoff; ≤ 3 attempts |
| 401 / 402 (or 403 budget) / 400 | fail the job: LLM_AUTH / LLM_CREDITS / LLM_BAD_REQUEST |
| transport attempts exhausted | extra chunk rounds after 60 s, 120 s (only the missing lines); then LLM_UNAVAILABLE |
Every request first takes a slot from the shared RateLimiter; attempts, delays and rounds come from the llm.retry.* settings (read on every request). Details, defaults and the free-model quota: rate-limits.md.
Lines that exhaust their attempts become UNTRANSLATED_VALIDATION_FAILED, stay in the original language and the job ends completed-with-warnings. If more than 10% of the distinct translatable lines (distinct context hashes) fail validation the job stops right away with LLM_TOO_MANY_FAILURES (moderated lines do not count; a failed representative fails its copies too but counts once). Everything accepted so far is already in the cache.
Term consistency pass (quality/term-consistency.ts)
Runs at the end of the translate step, locally and deterministically (no LLM call). It compares the source and the translated document cue by cue and lists recurring terms rendered in more than one way (e.g. Bro → mano ×40, Bro ×6).
- Candidates: source words capitalized in ≥ 3 cues, at least once mid-sentence (drops sentence-initial “What”, “Hey”), and lowercase in ≤ 25% of their occurrences (drops common words).
I/OKforms are ignored, as are all-caps lines (signs) and cues whose translation equals the source (karaoke, signs, failed lines). - Renderings, per candidate, over the translations of its cues: “kept” when the term itself is there; then, greedily over the cues not covered yet, the target word with the highest Dice coefficient between “cues with the term” and “cues with the word” in the whole file (≥ 0.3, in ≥ 2 cues, not present in the source cue). Common words (“de”, “que”) have a high document frequency and score low; the actual rendering concentrates on the term’s cues. Short inflections are one rendering (
mano/manos). The rendering is shown with its mid-sentence spelling. - Report: only terms with ≥ 2 renderings, most frequent first (≤ 50 terms, 3 examples per rendering), plus how many cues match none (rephrased/omitted) and the glossary target when the preset already pinned the term.
The report goes to artifacts.translate.consistency → JobResult.consistency (it does not turn the job into completed-with-warnings) and one info line to the job log. The renderer lists it in the job result, filtered by the current glossary of the job’s preset (not the one of the job, so pins made afterwards count): a term with an entry (case-insensitive, trimmed; with a target or only a rule) is not suggested again; if the job’s glossary already pinned it (glossaryTarget) and the model still varied, it shows only as a warning (“the AI did not follow the glossary”), with nothing to add. The header counts only the suggested terms and the section is hidden when nothing is left. For the rest, the user picks one rendering per term (the most used is preselected) or the explicit “Do not add to the glossary” option after the variants (same as clicking the chosen rendering again: the term is left out and not counted), and saves the choices as { source: term, target: rendering } glossary entries. Saving only appends terms the glossary does not have yet (addNewTerms): an existing entry, even one added after the suggestions were computed, keeps its target and rule. “Dismiss” (both flows) hides the section for that job without touching the preset: jobs:dismiss-consistency sets result.consistencyDismissed on the terminal job (persisted, so reopening the job or the queue does not bring it back; a retry produces a new report without the flag). A custom preset is updated in place (presets:update). A built-in preset is read-only, so the UI offers to create a new preset right there (presets:create): named after the job’s media (TMDB title, else a free “copy” name; editable, numbered when taken), with the built-in’s media type, style prompt, temperature and glossary plus the chosen terms; further choices then go to that new preset. Linear in the text size: ~6.7k cues in a few tens of ms.
Limits: needs a cased script in the source (Latin, Cyrillic, Greek); a rendering used in a single cue is indistinguishable from noise and is not reported as its own variant. Automatic correction and an LLM review are out of scope.
Streaming, cancellation and cost
- Requests always stream (
stream: true). The client accumulatesdelta.content, ignores: OPENROUTER PROCESSINGkeep-alives (they still reset the idle timer), treats a chunk witherrororfinish_reason: "error"(HTTP 200) asstream-error, and a stream that ends without[DONE]or a finish reason as truncated. usage(tokens, cached tokens,costin credits) comes in the last chunk and is summed inUsageTotals.- Cancellation goes through the
AbortSignal(request and backoff sleep). A response that already arrived and validated is still written to the cache (it was paid for); the abort only prevents new requests. Aborting stops generation and billing only for providers that support it (OpenAI, Anthropic, DeepSeek, Together…); others (Google, Groq, Mistral, Bedrock…) bill the whole generation. For every interrupted generation (abort, idle timeout, stream error) the session queriesGET /api/v1/generation?id=(X-Generation-Id, after 1.5 s, 5 s timeout, best effort); failures setcostIncomplete: true. A cancelled job throwsTranslationCancelled(AppErrorCANCELLED) carrying the usage.
API key and logging
- The key lives encrypted in SQLite (
SecretVault, seedatabase.md) and is read on every request (apiKey: () => Promise<string | null>), so changing it in the UI needs no restart. Missing key →LLM_AUTHwithout any network call. - The client redacts the key (and any
sk-or-…token) from every error message. Logs atinfocarry only metadata (model, ids count, finish reason, tokens, cost, status); responses that fail parsing/validation are logged truncated (2 KB) atwarn; subtitle content otherwise only atdebug. - Request headers:
Authorization,Content-Type,X-OpenRouter-Title: BakaSub. NoHTTP-Referer(attribution only).
Open points
- Chunk limits, context window and the failure ratio are constants; presets may override them later.
- Explicit
cache_controlbreakpoints (Anthropic, Qwen: writes at 1.25×, reads at 0.1×) are not sent yet. Worth it only when the prefix is above the model minimum and the job has enough chunks to amortize the write. A “warm-up” (start one lane, release the others once the first response starts streaming) would make the first K requests cache hits too. - The preflight does not subtract the cache discount (conservative); the job result does report
cachedTokens.