LiteRT is Google's on-device runtime, the new name for TensorFlow Lite (Android: com.google.ai.edge.litert:litert), and litert-torch, the renamed ai-edge-torch, is its PyTorch converter: a PyTorch model converted unmodified with litert_torch.convert matched the original to 4e-7 on a Galaxy S26 (measured, LiteRT 2.2.0, Android 16, 2026-09-05).
Measured on device (edge-compat, tiger_dialog_fp16): Galaxy S26 · LiteRT 2.2.0 · NPU (QNN/HTP) · did not run: process_crashed (2026-08-26); Raspberry Pi 5 · LiteRT 2.2.0.dev20260804 · CPU/XNNPACK, 4 threads · 42421 ms p50 (2026-08-31); browser · Chromium 151 on M4 Max · LiteRT.js 2.5.3 · WebGPU · 484 ms p50 · output differs from CPU (max rel diff 1e+05) (2026-08-11). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/tiger-dnr__tiger_dialog_fp16/CARD.md
Measured on device (edge-compat, tiger_effect_fp16): Galaxy S26 · LiteRT 2.2.0 · NPU (QNN/HTP) · did not run: process_crashed (2026-08-26); Raspberry Pi 5 · LiteRT 2.2.0.dev20260804 · CPU/XNNPACK, 4 threads · 42412 ms p50 (2026-08-31); browser · Chromium 151 on M4 Max · LiteRT.js 2.5.3 · WebGPU · 498 ms p50 · output differs from CPU (max rel diff 3.7e+05) (2026-08-11). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/tiger-dnr__tiger_effect_fp16/CARD.md
Measured on device (edge-compat, tiger_music_fp16): Galaxy S26 · LiteRT 2.2.0 · NPU (QNN/HTP) · did not run: process_crashed (2026-08-26); Raspberry Pi 5 · LiteRT 2.2.0.dev20260804 · CPU/XNNPACK, 4 threads · 42479 ms p50 (2026-08-31); browser · Chromium 151 on M4 Max · LiteRT.js 2.5.3 · WebGPU · 497 ms p50 · output differs from CPU (max rel diff 6.5e+05) (2026-08-11). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/tiger-dnr__tiger_music_fp16/CARD.md
TIGER-DnR — LiteRT (CompiledModel GPU) cinematic sound separation
[Image blocked: on-device result]
Mixture input → separated Dialogue / Sound effects / Music stems, from the on-device model (waveforms).
TIGER (ICASSP 2025) DnR cinematic sound separation re-authored to
GPU-native LiteRT .tflite: split a clip into Dialogue / Sound effects / Music stems fully on-device.
Three sibling band-split TIGER graphs (~1.41 M params each, trained on the openly-built
DnR dataset by the upstream authors) each process a 12.06 s,
44.1 kHz mono chunk; per DnR convention each graph contributes one stem (dialog = source 2 of
tiger_dialog, effect = source 1 of tiger_effect, music = source 0 of tiger_music).
Verified on a Pixel 8a (Tensor G3): full LITERT_CL residency — 23 974 / 23 974 nodes, 1 partition per
graph, GPU output vs PyTorch waveform corr 0.99987, ~4.5 s per chunk per graph. FP16, 16.1 MB per graph.
Files
I/O
- Input
[1, 534016] float32 — a 531 968-sample (12.06 s @ 44.1 kHz) mono chunk, reflect-padded by
1024 samples on both sides by the caller (torch.stft(center=True) equivalent). The STFT
(win 2048 / hop 512, periodic Hann) runs inside the graph as a windowed-DFT Conv1d.
- Outputs
[1, 3, 1025, 1040] × 2 (real, imag) — separated one-sided complex spectrograms for the
3 sources. Host runs iSTFT (win 2048 / hop 512, Hann, trim 1024 center pad) and overlap-adds chunks
(reference inference uses a 12 s window with 4–10 s hop, averaging overlaps).
Minimal usage (Python, desktop parity check)
Kotlin (Android, LiteRT CompiledModel GPU)
Conversion (numerically-equivalent re-authoring)
torch.stft → in-graph DFT-as-Conv1d (host reflect-pad); torch.istft → host; folded-batch Conv1d
((B·T, N, band)) → 4D (1,k)-Conv2d; per-sample GlobLN → per-position chained-single-axis-mean
SafeNorm; chunk length chosen so T=1040 is divisible by 16 → adaptive pools become uniform
AVERAGE_POOL_2D and nearest resizes exact integer repeats; the non-uniform 57-band axis uses constant
averaging/one-hot FULLY_CONNECTED matrices; MHSA → per-head batch-1 3D matmuls with 1/√d folded into Q;
PReLU → relu(x) − w·relu(−x); the 6-D mask-head view → static channel slices. Two fp16-on-GPU fixes,
both exact-equivalent: norm eps raised to 1e-4 (1e-8/1e-5 underflow to 0 in fp16 → 0/0 NaN on silent
bands) and the mask head rewritten without dim-1 broadcast MULs.
Desktop parity: fp16 tflite vs PyTorch waveform corr 0.99991 (per-graph, real 12 s mixture).
Upstream
Please cite the TIGER paper (Xu et al., ICASSP 2025, arXiv:2410.01469) when you use these models.
Performance
Measured on a Pixel 8a (Tensor G3, Android 16) with the standard TFLite benchmark_model tool — 10 warm-up runs then 50 timed runs, reported as the tool's mean.
Any on-device figure recorded when this model shipped came from a different runtime. It was taken through LiteRT's own CompiledModel accelerator (logcat reports it as LITERT_CL), which is the path the Kotlin sample app and the LiteRT API use, and it appears elsewhere on this card. The rows above are the classic TFLite OpenCL delegate, measured with a tool anyone can download and re-run. The two are not comparable, so read the rows above as a reproducible floor rather than as this model's speed on LiteRT.
XNNPACK declines these fp16 graphs — it reports failed to delegate DEPTHWISE_CONV_2D and then fails to allocate tensors — so there is no usable CPU number. Disabling XNNPACK falls back to reference kernels, which measured about 20× slower than the GPU on models of this size and would not represent CPU inference anyone would ship.
Snapdragon NPU (Hexagon)
All 3 files here self-heat faster than the phone sheds it. The sweep only accepts a run that begins and ends at thermal status NONE; every attempt on the Galaxy S26 ended at LIGHT or MODERATE, so this sweep has no publishable NPU or GPU number for them. Recipes for both accelerators: NPU · GPU.
Raspberry Pi 5 (CPU)
Measured on a Raspberry Pi 5 Model B Rev 1.1 (8 GB, Raspberry Pi OS 64-bit) with the LiteRT benchmark_model tool from litert-cli-nightly 0.2.0.dev20260805: CPU inference (XNNPACK, 4 threads), 3 invocations per file of 10 warm-up plus 50 timed runs (the tool caps a phase at 150 s, so very slow graphs run fewer — the Runs column is the actual timed total). The latency is the median across invocations; the spread is the min–max over all timed runs. No thermal throttling occurred during these runs (vcgencmd get_throttled stayed 0x0).