Not a turnkey GPU accelerator
ONNX, OpenVINO, TensorRT, DirectML, llama.cpp and vLLM are registered runtime scaffolds. Their synthesize() methods raise NotImplementedError. Do not expect acceleration from them.
Performance & limits
No benchmark theater.
This project does not publish latency, real-time factor or tokens-per-second numbers, because no benchmark was measured on a reference machine for this release. What follows are the limits and targets that are actually encoded in the source — clearly labelled as such.
Read this before quoting a number.
The “speed multiplier”, “quality rating” and preset “expected RTF” values are
design metadata used for backend selection and UI hints. They are not
measurements and are not comparable across GPUs.
DECLARED F5-TTS VRAM FLOOR1.5 GBcapability metadata
MEMORY-MANAGER BUDGET5.5 GB+ 0.5 GB safety margin
F5 OUTPUT24 kHznative sample rate
BATCH CONCURRENCY1serial worker by default
Presets
VRAM tiers as configuration templates.
Parsed from BUILTIN_PRESETS in core/presets.py. The expected values are the author's targets, not measurements.
| Preset | Target VRAM | Backend | Precision | NFE | Expected quality | Expected RTF (declared) |
|---|---|---|---|---|---|---|
| Ultra Low VRAMFor systems with 3GB or less VRAM. Maximum compression with quality tradeoffs. | 3.0 GB | llama-cpp | gguf-q4_k_m | 8 | 2/5 | 0.5 |
| Low VRAMFor systems with 4GB VRAM. Good balance of quality and memory usage. | 4.0 GB | pytorch | int8-dynamic | 16 | 3/5 | 0.4 |
| BalancedFor systems with 5-6GB VRAM. Default recommended settings. | 5.0 GB | pytorch | int8-dynamic | 16 | 4/5 | 0.3 |
| QualityFor systems with 6GB+ VRAM. Maximum quality, higher memory usage. | 6.0 GB | pytorch | fp16 | 32 | 5/5 | 0.25 |
RTF here means the preset's expected_rtf field. Lower is faster
in the original data. None of these values were produced by a benchmark
harness in this repository.
Memory behaviour
- Lazy loading. Backends load on first synthesis, not at CLI start.
- VRAM budget.
AdaptiveMemoryManagerdefaults to a 5.5 GB budget with a 0.5 GB safety margin and optional LRU eviction. - Device resolution.
--device autopicks CUDA only whentorch.cuda.is_available()is true. - Unload is explicit.
manager.unload_model()drops the backend and callstorch.cuda.empty_cache()when available. - CPU offload helpers exist in
core/memory_manager.py, but the inference manager does not automatically move a loaded backend to CPU under pressure.
Audio path
- Sample rates differ by backend: F5-TTS 24 kHz, GPT-SoVITS 32 kHz, Fish Speech declares 44.1 kHz.
- Output is WAV via
soundfile, using the sample rate returned by the backend. - No loudness normalization is applied by the CLI. The stitching helper can normalize when used directly.
- Long text: the studio exposes chunking modes; the module provides sentence/character/token/paragraph strategies and crossfaded stitching. The CLI does not automatically enable
LongTextSynthesizer.
Honest limits
What this release is not.
Not a quality benchmark
There are no WER, speaker-similarity or MOS results in the repository. The site does not invent them, and neither should your project notes.
Not fully offline on first run
Model weights are fetched from Hugging Face when first needed. Once cached, synthesis runs locally. The app itself sends no telemetry.
Not every quantization flag is real
INT4 and GGUF are accepted by the CLI but have no effective F5-TTS path in this revision. The quantization manager's GGUF/GPTQ/AWQ conversions are placeholders.
Not a multilingual guarantee
The UI offers English, Romanian, Chinese and auto-detect. Auto-detection is a three-way character heuristic. Model-level language coverage depends on the weights and upstream package.
Not a Windows/Linux parity story
The pinned requirements target Windows/CUDA 12.1; the batch helpers are .bat files, DirectML is Windows-only, and vLLM is marked Linux-only.
Scaling notes
Where time goes.
| Lever | Effect in the code | Caution |
|---|---|---|
| NFE steps | F5-TTS diffusion sampling; UI range 4–64, default 16. | More steps generally cost more compute. No measured curve ships with the repo. |
| Text length | Chunking exists, but the CLI sends the whole input to the backend. | Very long inputs are not automatically split on the CLI path. |
| Batch size | The batch worker processes one job at a time (max_concurrent=1). | No parallel synthesis. |
| Precision | FP16 halves parameter memory versus FP32; INT8 on CPU can compress further. | CUDA INT8 is not implemented for F5; it silently uses FP16. |
| GPU memory | Declared floors: F5 1.5 GB, GPT-SoVITS 2 GB, Fish 2 GB; runtime scaffolds declare 0–3 GB. | These are minimums from capability metadata, not a peer-reviewed memory profile. |