No official torrent yet β requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from furiosa-ai/Qwen3-Reranker-4B
All models by furiosa-ai βThis repository contains Qwen/Qwen3-Reranker-4B together with a Furiosa Executable Bundle (FXB) for running it on FuriosaAI RNGD with Furiosa-LLM. The same model also runs on other frameworks (such as Sentence Transformers, vLLM, and Transformers); for usage with those, see the upstream Qwen/Qwen3-Reranker-4B model card.
Qwen3-Reranker-4B is the 4B reranking model in the Qwen3-Reranker series, built on the Qwen3 dense transformer backbone. Given a query and a set of candidate documents, it produces relevance scores used to reorder retrieval results β a common second stage in retrieval-augmented generation (RAG) and search pipelines. Its intended use is the same as the upstream Qwen/Qwen3-Reranker-4B, and it is released under the Apache 2.0 License.
No quantization β the model runs in its native BF16 precision.
On RNGD, Qwen3-Reranker-4B runs with a tensor-parallel size of 8 PEs, which maps to a single RNGD card (8 PEs per card).
To run this model with Furiosa-LLM, follow the examples below after installing Furiosa-LLM and its prerequisites. You can use the model either online through the OpenAI-compatible server or offline through the Furiosa-LLM Python API.
The simplest way to serve the model is:
When the server is ready, you will see:
The server exposes a /v1/rerank endpoint (compatible with the Cohere/Jina
rerank API, also used by vLLM). Send a query and the candidate documents with
curl; the server returns the documents reordered by relevance_score:
You can do the same from Python with the requests library, and pass top_n to
keep only the most relevant documents:
To score query-document pairs directly instead of reranking, the server also
exposes a /v1/score endpoint.
For offline use, load the model with the LLM constructor (the FXB shipped in
the repo is discovered automatically) and call score with a query and the
candidate documents to obtain relevance scores:
furiosa-llm serve) β full OpenAI-compatible API reference, including the Rerank and Score APIs# Launch the server, listening on port 8000 by defaultfuriosa-llm serve furiosa-ai/Qwen3-Reranker-4BINFO: Started server process [27507]INFO: Waiting for application startup.INFO: Application startup complete.INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)curl http://localhost:8000/v1/rerank \ -H "Content-Type: application/json" \ -d '{ "model": "furiosa-ai/Qwen3-Reranker-4B", "query": "What is deep learning?", "documents": [ "Deep learning is a subset of machine learning using neural networks.", "Python is a popular programming language for data science.", "Neural networks are inspired by biological neural networks." ] }' \ | python -m json.toolimport requests
response = requests.post( "http://localhost:8000/v1/rerank", json={ "model": "furiosa-ai/Qwen3-Reranker-4B", "query": "What is deep learning?", "documents": [ "Deep learning is a subset of machine learning using neural networks.", "Python is a popular programming language for data science.", "Neural networks are inspired by biological neural networks.", ], "top_n": 2, },)
for result in response.json()["results"]: print(f"score={result['relevance_score']:.4f} {result['document']['text']}")from furiosa_llm import LLM
with LLM("furiosa-ai/Qwen3-Reranker-4B") as llm: query = "What is deep learning?" documents = [ "Deep learning is a subset of machine learning using neural networks.", "Python is a popular programming language for data science.", ] outputs = llm.score(query, documents) for document, output in zip(documents, outputs): print(f"score={output.outputs.score:.4f} {document}")