SingerMaker

Open-source, self-hosted alternative to AutoLyrics: upload a song and its lyrics, get back an xLights .xtiming singing-face track (phrases, words and Preston-Blair mouth-shape phonemes, time-aligned to the actual vocals) that you import straight into your sequence — so your prop's mouth movements hit the words.

song.mp3 + lyrics.txt ──► Demucs (vocals) ──► forced alignment (WhisperX)
   └─► 3-layer timing model ─► snap to 25/50 ms grid ─► out.xtiming / out.pgo

Features

• Automatic word timing from the supplied lyrics — WhisperX forced alignment (primary, ±150 ms class accuracy), faster-whisper word stamps (fallback) and an energy-anchored heuristic that never fails the job.
• Exact mouth shapes: ARPAbet → xLights face labels (AI, E, FV, L, MBP, O, U, WQ, etc, rest) via the CMUdict + Papagayo Preston-Blair tables, the same xLights itself uses.
• LRC support: paste synced lyrics ([mm:ss.xx]) and each line's timestamp becomes an alignment anchor — tighter and faster.
• Vocal-stem upload to skip separation, per-job engine/grid settings, app-level user pronunciation dictionary for made-up words.
• .xtiming and .pgo exports — verified against xLights' own import/export code paths.
• CPU-only by default (a GPU profile is available). Docker Compose deployment, single-user, no accounts, your music never leaves the box.

Quick start (Docker)

git clone <repository-url> && cd singermaker
docker compose -f docker/docker-compose.yml up -d --build
# open http://localhost:8844

Create a project → upload audio + paste lyrics → Generate → Download .xtiming. Then see docs/xlights-import-guide.md for the 60-second xLights side.

How it works

1. ffmpeg decodes your audio to 44.1 kHz WAV.
2. Demucs separates vocals (skipped if you upload your own stem, or set None).
3. A forced-alignment engine times every lyric word against the vocals.
4. Words are converted to phonemes: user dictionary → CMUdict → extended dictionary → espeak-ng G2P; unknown words are reported so you can add them.
5. ARPAbet phonemes map to the ten xLights mouth shapes; word and phoneme spans are distributed inside each word like xLights' own breakdown.
6. Everything snaps to your chosen grid (25/50 ms) and the 3-layer .xtiming XML is written.

Fine-tuning is expected and easy: xLights' timing tracks are fully editable — drag a word or regenerate a word's phonemes there. (See docs/alignment-tuning.md for making the raw alignment better first.)

Development

cd backend && uv sync --extra dev --extra web --extra ml   # CPU torch deps
uv run uvicorn singermaker.api.app:app --reload --port 8000
cd ../frontend && npm install && npm run dev              # http://localhost:5173

Task	Command
Backend unit/API tests	cd backend && uv run pytest tests/unit
Backend lint / types	uv run ruff check . / uv run mypy singermaker
Real-model integration (downloads models once)	uv run pytest tests/real -m real
Frontend unit tests / typecheck / build	cd frontend && npm test && npm run typecheck && npm run build
Browser end-to-end	npm run test:e2e (needs Playwright browsers)
CLI pipeline	uv run singermaker generate song.mp3 lyrics.txt -o out.xtiming --engine heuristic

Architecture

Full details in docs/architecture.md and docs/installation.md.

nginx (SPA + /api proxy) ── FastAPI ── Redis ── Celery workers (CPU torch)
                                 │                    │
                            SQLite + files        model-cache volume

Configuration

Variable	Default	Meaning
SINGERMAKER_DATA_DIR	data/	SQLite DB, uploads, outputs
SINGERMAKER_QUEUE	eager	eager (threads, dev) or celery (Redis)
SINGERMAKER_BROKER_URL	redis://redis:6379/0	Celery broker
HF_HOME / TORCH_HOME	container: /cache/...	model download cache
OMP_NUM_THREADS	host default	torch CPU thread cap per job

License

GPL-3.0-or-later. Bundled data: CMUdict (BSD-style, Carnegie Mellon), ARPAbet→mouth mapping derived from Papagayo's phoneme_mapping (GPL, Preston Blair shapes); xLights is a trademark of its authors — this project is unaffiliated, just compatible. See data/README.md for provenance.