Loading…
The 7-step audio production pipeline transforms scene text into multi-voice audiobook narration. Each step is orchestrated by CharacterOS agents with continuity validation and quality gating.
Segment scene text into dialogue blocks per character. Detect speaker transitions, extract bracketed…
Segment scene text into dialogue blocks per character. Detect speaker transitions, extract bracketed stage directions, identify emotional cues (scared, angry, sad). Each block tagged with character, emotion, and context.
VoiceSelectionAgent maps character traits (age, gender, accent, personality) to voice options from a…
VoiceConfigurationAgent adjusts 300+ voice settings: speed (0.25x–4.0x), pitch variation, emotion in…
The synthesis service batches dialogue blocks in parallel (guarded by Semaphore(5) to prevent OOM). …
pydub composes MP3s in sequence, overlaying stage direction audio at lower volume. Normalizes loudne…
AudioContinuityAgent runs 5-layer QA gate: (1) audio duration matches text timing, (2) voice consist…
Audio blob stored in Cloud blob (production) or local file system (dev). File URL recorded in scene_…
Voice DNA is a numeric vector (embedding) that encodes a character's vocal identity: tone, accent, emotional range, speaking style. Generated by VoiceSelectionAgent from character traits. DNA is immutable within a project but can evolve when audio quality score ≥ 85% (DNALearningEngine updates it based on feedback).
300+ configurable parameters per voice: speed (x0.25–4.0), pitch shift (−12 to +12 semitones), emotion intensity (0–1.0), pause timing (100–500ms), breathiness, vibrato. Configurations are scene-specific — same character can sound different based on scene mood/context injected by DirectorAgent.
Voice DNA is IP-locked to project and organization boundaries via VoiceVault. Cannot be exported or cloned across projects without explicit authorization. Prevents voice theft and ensures creator control. Voice cloning (copying one voice to new character) requires 99%+ similarity score.
AudioContinuityAgent returns quality_score (0–100). Factors: text-audio duration alignment (±5% tolerance), speaker consistency, emotional beats matching character arc, absence of distortion/clipping. Scores <70 trigger revision recommendations or manual review queue.
TTS synthesis runs in parallel with asyncio.Semaphore(5) to prevent memory exhaustion under heavy concurrent load. Each character's dialogue is synthesized in isolation, then mixed sequentially. Prevents OOM crashes that plagued early versions.
Audio cache key = hash(text + voice_id + config). Identical dialogue lines with same voice + config reuse cached audio. Saves inference provider calls and speeds generation. Cache stored in Cloud blob with TTL = 90 days.
Neural TTS engine: 150 requests/minute per organization. Marvox queues overflow requests and retries with exponential backoff. Users see "waiting in queue" UI while synthesis completes. Prevents API 429 errors.
If a dialogue block fails synthesis, AudioSceneAgent marks it "synthesis_failed" and skips to next block. Scene audio still delivers, with silent placeholder for failed lines. Failure logged for debugging. User can retry individual lines or entire scene.