No official torrent yet β requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from litert-community/GLiNER2.5-Small-LiteRT
All models by litert-community βRun GLiNER2.5 Small entity extraction on a phone GPU with the official fastino/gliner2.5-small-v1 weights converted through Google LiteRT Torch. The validated Android configuration is LiteRT 2.2.0 with explicit GPU FP32 computation on a Samsung Galaxy S26 (SM-S942Q, SM8850, Android 16). On 70 English inputs the extracted spans match the official gliner2 fp32 CPU implementation exactly (micro-F1 1.000 by label and character offsets) at every shipped window. Other Android GPU families have not been validated here.
[Image blocked: Entity spans returned by the s128 wfp16 model for one sentence]
The image shows the actual output of gliner25_small_s128_wfp16.tflite on LiteRT
CompiledModel for the sentence in host_assets/example.json. The confidences are
the values the model returned. It is a rendering of model output, not a screenshot.
Three encoded-window sizes are shipped. Pick the smallest window that fits your text: the schema prompt (five labels) plus the text must fit in N encoded tokens and the text must have at most T words.
wfp16 files store the 96 FULLY_CONNECTED weight tensors as float16 with a
DEQUANTIZE to float32; every activation and every other constant stays float32.
The fp32 files are the same graphs with float32 weights.
Every graph is the dense part of the upstream BoundaryExtractor: the
DeBERTa-v3-xsmall encoder, the boundary encoder, the boundary query head and the
per-token projections. The graph takes the word-embedding rows as input and
returns one packed float32 tensor with 17 logical outputs. The host does the
token embedding lookup before the graph and the upstream sparse candidate
pooling, scoring and span decoding after it. host_assets/ holds everything the
host side needs:
FP32 model storage and GPU computation precision are separate settings.
With the runtime's default GPU precision the encoder output is NaN from the
first logical output onward; with explicit FP32 computation all gates pass.
Use the explicit option below. Dynamic-range INT8 variants were built and
evaluated but do not compile on LiteRT 2.2.0 CompiledModel GPU (ML Drift
rejects the quantized FULLY_CONNECTED operators with "Unable to parse bc coord
for BATCH axis"), so no INT8 file is published. NPU execution was not evaluated.
The graphs handle one English text and the fixed label set
person, organization, location, product, date; other schemas, other
languages and texts longer than one window are outside what was validated.
Install requirements-lock.txt into a Python 3.12 environment and run from
the downloaded repository. The host runtime imports the pinned gliner2 2.0.0
package for the sparse decoder; no checkpoint download is needed
(HF_HUB_OFFLINE=1 is set by the script).
The script does, in order: build the encoded inputs for the fixed label set
(HostRuntime.prepare), run the graph through the LiteRT CompiledModel Python
API, and decode the packed output with the upstream sparse decoder
(HostRuntime.decode).
Use implementation("com.google.ai.edge.litert:litert:2.2.0"). Stage the model
file in your application's private model directory. Keep one Environment for
the process and run the whole GPU lifetime on the same worker thread. The five
input buffers follow the signature order args_0 β¦ args_4 =
inputs_embeds [1,N,384], attention_mask [1,N], text_routing [1,T,N],
query_routing [1,5,N], text_mask [1,T]; HOST_CONTRACT.md says how to fill
them from the tokenizer output and the embedding table.
The sparse candidate pooling, pair scoring and span decoding that turn the 17
logical outputs into labelled character spans are shipped as the Python host
runtime (host_assets/runtime/host_decoder.py, which calls the upstream
gliner2 functions). The Kotlin block above covers the on-device graph and the
unpacking contract; it propagates GPU failures and has no implicit CPU retry.
A complete Kotlin host is in android/ [blocked]: the word splitter and
SentencePiece Unigram tokenizer, input construction, the memory-mapped embedding
lookup and a float32 port of the sparse decoder, inside a Jetpack Compose sample
app (type text, tap Extract, see highlighted entities with confidence and
offsets). On the Galaxy S26 it returns the same spans as the official fp32 model
on all 70 validation texts, on GPU FP32 and on CPU. Build and install steps are in
android/README.md [blocked].
HOST_CONTRACT.md is the complete specification. In short:
gliner2 processor does:
schema tokens for the five labels, [SEP_TEXT], then the text tokens. Pad
the token ids on the right with id 0 to N.inputs_embeds = rows of word_embeddings_fp32.bin for every position,
padding included. attention_mask is 1 for real tokens. text_routing has
one 1 per text word at its first sub-word position; query_routing has one 1
per label at its marker position; text_mask is 1 for real text words.[1,1,1,1108*T+4574]. The 17 logical
slices (text_states, query_states, boundary_states, start_logits,
end_logits, inside_logits, β¦ count_log_rates) are listed with offset and
shape in graph_contract_s*.json; unpacking is slicing and reshaping only.host_assets/runtime), which
returns label, text, character start/end and confidence per span.Long documents: the upstream library chunks long text on the host
(extract_long) and remaps spans; the graphs here score one window at a time.
Reference = the official gliner2 2.0.0 fp32 CPU implementation at checkpoint
revision f1e4d8fd, same tokenizer, default threshold. 70 English inputs (50
sentences, 10 short sentences, 10 paragraphs of 90β170 words) with the fixed
five-label schema; every input runs at every window it fits (60 / 65 / 70
inputs β 195 window-input pairs). Micro-F1 counts a span as correct only when
label, start and end all match.
Desktop LiteRT CPU (ai-edge-litert 2.1.6, macOS arm64, 4 threads) gives the same F1 = 1.000 for every file; the fp32 files match the PyTorch forward with a maximum absolute output difference of 3.1e-5 (s128), 1.5e-4 (s256) and 2.8e-4 (s512) over all 17 logical outputs.
Latency is the median of 5 timed runs per input after 2 warm-up runs, measured
from run to the end of output readback with the native CompiledModel C API,
LiteRT 2.2.0, GPU with explicit FP32 computation, one process, phone idle and
between 34 Β°C and 38 Β°C battery temperature. It is a single-device sample, not a
benchmark across devices or thermal states.
fastino/gliner2.5-small-v1, revision
f1e4d8fdd6fe328f45dee6aca3e6a07c9db4296e (architecture BoundaryExtractor,
encoder microsoft/deberta-v3-xsmall), loaded with gliner2 2.0.0 and
transformers 4.57.6.litert-torch 0.9.3 (torch 2.12.1), fixed shapes, fp32. The
dense prefix was re-expressed for the GPU delegate without changing its math:
host-side embedding lookup, one-hot routing as matmul, boolean masks as float
arithmetic, prefix sums as constant triangular matmul, attention kept at rank 4,
the exact DeBERTa logarithmic relative-position buckets and the 128-word boundary
window baked as constants, and one packed output tensor. GELU is the native
LiteRT GELU.ai-edge-quantizer 0.8.0 float16 FLOAT_CASTING on the
FULLY_CONNECTED weights only (wfp16 files).requirements-lock.txt.License: the checkpoint is Apache-2.0 and these converted files are released
under the same license (LICENSE). The DeBERTa-v3 encoder is MIT
(licenses/DeBERTa-MIT.txt); the gliner2 and transformers code used by the
host runtime is Apache-2.0 (licenses/). Upstream: the
GLiNER2 repository and the paper
arXiv:2507.18546.
python examples/run_example.py --model gliner25_small_s128_wfp16.tfliteimport json, os, sysfrom pathlib import Pathimport numpy as npos.environ["HF_HUB_OFFLINE"] = "1"sys.path.insert(0, str(Path("host_assets/runtime").resolve()))from host_runtime import HostRuntimefrom ai_edge_litert.compiled_model import CompiledModel, HardwareAccelerator, Options, CpuOptions
host = HostRuntime(Path("host_assets"))text = "Maya Chen from Orvane Robotics demonstrated the Veltrix 9 in Lisbon on March 12, 2025."inputs, metadata = host.prepare(text, seq=128) # 5 float32 tensors, args_0..args_4
model = CompiledModel.from_file("gliner25_small_s128_wfp16.tflite", options=Options( hardware_accelerators=HardwareAccelerator.CPU, cpu_options=CpuOptions(num_threads=4)))ins, outs = model.create_input_buffers(0), model.create_output_buffers(0)for buf, x in zip(ins, inputs): # signature order is args_0..args_4 buf.write(np.ascontiguousarray(x.numpy(), dtype=np.float32))model.run_by_index(0, ins, outs)packed = outs[0].read(57758, np.float32).reshape(1, 1, 1, 57758) # s128 packed lengthprint(json.dumps(host.decode(metadata, packed, inputs), indent=2))# {"entities": {"person": [{"text": "Maya Chen", "start": 0, "end": 9, "confidence": 0.9995}], ...}}import com.google.ai.edge.litert.Acceleratorimport com.google.ai.edge.litert.CompiledModelimport com.google.ai.edge.litert.Environmentimport java.io.File
// embeds/mask/textRouting/queryRouting/textMask are the five float32 inputs// built exactly as HOST_CONTRACT.md describes (N = 128, T = 48 for this file).fun runDensePrefix(env: Environment, modelDir: File, inputs: List<FloatArray>): FloatArray { val options = CompiledModel.Options(setOf(Accelerator.GPU)).apply { gpuOptions = CompiledModel.GpuOptions( precision = CompiledModel.GpuOptions.Precision.FP32) // default precision returns NaN } CompiledModel.create(File(modelDir, "gliner25_small_s128_wfp16.tflite").path, options, env).use { model -> val inBufs = model.createInputBuffers() val outBufs = model.createOutputBuffers() try { inputs.forEachIndexed { i, x -> inBufs[i].writeFloat(x) } // args_0 .. args_4 model.run(inBufs, outBufs) return outBufs[0].readFloat() // 57,758 floats; includes completion/readback } finally { inBufs.forEach { it.close() } outBufs.forEach { it.close() } } }}
// Unpack one logical slice with the offsets in graph_contract_s128.json,// e.g. start_logits [1,5,49] starts at float 26624:fun startLogits(packed: FloatArray): Array<FloatArray> = Array(5) { q -> packed.copyOfRange(26624 + q * 49, 26624 + (q + 1) * 49) }