gradium/phonon-31m-en
text-to-speechtransformersVerified checksumstext-to-speechttsstreamingon-devicephonontransformers
- License
- apache-2.0
- Size
- 213 MB
- Revision
- main
Mirrored from gradium/phonon-31m-en
All models by gradium →Mirrored from gradium/phonon-31m-en
All models by gradium →High-quality Text-to-Speech, on your device.
phonon-31m is a text-to-speech model from the Phonon family. It turns text into natural 24 kHz speech in English. It streams audio while it is still generating, and it is small enough to run in a browser tab, on a phone, or on a laptop CPU.
Phonon builds on Pocket TTS and the Mimi codec, developed by Kyutai.
Try it in your browser · All Phonon models · Runtime on GitHub · Python · Browser · Discord
model.q8.gguf, 64.5 MB (8-bit quantized)All of them run on the same runtime and use the same file layout, so switching models means changing the repo ID.
Every runtime names this model as hf://gradium/phonon-31m-en, and takes lang, which selects how text is normalized (numbers, symbols, dates). This model speaks English, so use en.
uvx runs the Python package's command without installing it. The standalone ptts binary from GitHub Releases or Homebrew (brew install gradium-ai/tap/ptts) takes the same flags.
Install 🤗 Transformers from source:
This repo's default-voice.safetensors loads the same way. Unlike the runtime, the tokenizer does not spell out numbers, amounts, symbols or addresses, and long text is split with processor.split_text: see the Transformers docs.
Phonon comes with a catalog of ~100 voices available in the gradium/phonon-voices repo. Instant voice cloning and voice design are available through the Gradium studio.
Phonon was compared with other open text-to-speech models on three measures, per language:
In every language, a Phonon model is first on all three measures.
English (US) (Elo from 468 raters, 8,290 votes)
French (Elo from 172 raters, 2,270 votes)
German (Elo from 168 raters, 2,220 votes)
Spanish (Spain) (Elo from 201 raters, 2,290 votes)
Portuguese (Brazil) (Elo from 171 raters, 2,000 votes)
Phonon is made for applications that speak on the user's device: offline and private assistants, screen readers and other accessibility tools, games, and interactive apps that cannot wait for a cloud round trip. It also runs as a local server with an OpenAI-compatible API and as a voice for Home Assistant.
lang to match the text.Do not use Phonon to impersonate a real person, to deceive listeners about who is speaking, or to clone a voice without the speaker's consent. Tell listeners when speech is synthetic where that matters to them.
The weights, tokenizer and voices in this repository are released under Apache 2.0, which allows commercial use. The runtime is a separate project, licensed MIT OR Apache-2.0.
uvx ptts --lang en --quant q8 \ --voice Easton "Hello from Phonon." -o speech.wavpip install pttsimport ptts
tts = ptts.TTS("hf://gradium/phonon-31m-en", lang="en", quant="q8")tts.save("speech.wav", "Hello from Phonon.", voice="Easton")
# Or stream: chunks arrive while the rest is still being generated.with tts.stream("Speech can start playing right away.", voice="Easton") as audio: for pcm in audio: ... # float32 chunks at audio.sample_ratepip install git+https://github.com/huggingface/transformersfrom huggingface_hub import hf_hub_downloadfrom safetensors.torch import load_filefrom transformers import AutoProcessor, GradiumPhononTTSForConditionalGeneration
model_id = "gradium/phonon-31m-en"processor = AutoProcessor.from_pretrained(model_id, device_map="auto")model = GradiumPhononTTSForConditionalGeneration.from_pretrained(model_id, device_map="auto")
# Easton (en-us); every voice and its ID: https://huggingface.co/gradium/phonon-voicesvoice_path = hf_hub_download("gradium/phonon-voices", "voices/LMDIYE0Oa377Qh6Z.safetensors")voice = load_file(voice_path)["speaker_wavs"][0].T.float() # (num_frames, 512)
conversation = [ {"role": "user", "content": [{"type": "audio"}, {"type": "text", "text": "Hello from Phonon."}]}]inputs = processor.apply_chat_template( conversation, tokenize=True, return_dict=True, add_generation_prompt=True, processor_kwargs={"audio_prompt_latents": [voice]},).to(model.device)
outputs = model.generate(**inputs)audio = processor.decode(outputs.audio_latents, outputs.audio_latents_mask)processor.save_audio(audio, "speech.wav")