No official torrent yet — requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from RedHatAI/Qwen3-Coder-Next-FP8-dynamic
All models by RedHatAI →Quantized version of Qwen/Qwen3-Coder-Next.
This model was obtained by quantizing the weights and activations of Qwen/Qwen3-Coder-Next to FP8 data type. This optimization reduces the number of bits per parameter from 16 to 8, reducing the disk size and GPU memory requirements by approximately 50%. Only the weights and activations of the linear operators within transformers blocks of the language model are quantized.
This model was quantized using the llm-compressor library as shown below.
The model was evaluated on the OpenLLM leaderboard task, using lm-evaluation-harness. vLLM was used for all evaluations.
**Coding Benchmarks **
SWE-Bench
vllm serve inference-optimization/Qwen3-Coder-Next-FP8-dynamic --port 8000 --tensor-parallel-size 2 --enable-auto-tool-choice --tool-call-parser qwen3_coder # Your tool implementationdef square_the_number(num: float) -> dict: return num ** 2
# Define Toolstools=[ { "type":"function", "function":{ "name": "square_the_number", "description": "output the square of the number.", "parameters": { "type": "object", "required": ["input_num"], "properties": { 'input_num': { 'type': 'number', 'description': 'input_num is a number that will be squared' } }, } } }]
from openai import OpenAI# Define LLMclient = OpenAI( # Use a custom endpoint compatible with OpenAI API base_url='http://localhost:8000/v1', # api_base api_key="EMPTY") messages = [{'role': 'user', 'content': 'square the number 1024'}]
completion = client.chat.completions.create( messages=messages, model="RedHatAI/Qwen3-Coder-Next-FP8-dynamic", max_tokens=65536, tools=tools,)
print(completion.choices[0])from datasets import load_datasetfrom transformers import AutoModelForCausalLM, AutoTokenizer
from llmcompressor import oneshotfrom llmcompressor.modifiers.quantization import QuantizationModifierfrom llmcompressor.utils import dispatch_for_generation
# NOTE: Requires a minimum of transformers 4.57.0
MODEL_ID = "Qwen/Qwen3-Coder-Next"
# Load model.model = AutoModelForCausalLM.from_pretrained(MODEL_ID, torch_dtype="auto")tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
# Configure the quantization algorithm and scheme.# In this case, we:# * quantize the weights to fp8 with per channel via ptq# * quantize the activations to fp8 with dynamic per tokenrecipe = QuantizationModifier( targets="Linear", scheme="FP8_DYNAMIC", ignore=[ "re:.*lm_head", "re:.*mlp.gate$", "re:.*mlp.shared_expert_gate$", "re:.*linear_attn.*", ], weight_observer="mse")
# Apply quantization.oneshot(model=model, recipe=recipe)
# Confirm generations of the quantized model look sane.print("========== SAMPLE GENERATION ==============")dispatch_for_generation(model)input_ids = tokenizer("Hello my name is", return_tensors="pt").input_ids.to( model.device)output = model.generate(input_ids, max_new_tokens=20)print(tokenizer.decode(output[0]))print("==========================================")
# Save to disk in compressed-tensors format.SAVE_DIR = MODEL_ID.rstrip("/").split("/")[-1] + "-FP8-dynamic"model.save_pretrained(SAVE_DIR)tokenizer.save_pretrained(SAVE_DIR)python -m swebench.harness.run_evaluation \ --dataset_name princeton-nlp/SWE-bench_Lite \ --predictions_path preds.json \ --run_id validate-preds