No official torrent yet β requesting it tells the flock what to host next. You cannot seed a catalog-only page.
Cataloged from drawais/Qwen2.5-32B-Instruct-AWQ-INT4
All models by drawais βINT4 weight-only quantization of Qwen/Qwen2.5-32B-Instruct.
Qwen 2.5 32B-Instruct in INT4. About 17 GB on disk. Runs on a 24 GB consumer GPU.
~19.2 GB on disk. Recommended VRAM: enough headroom for KV cache.
This artifact is a derivative work of Qwen/Qwen2.5-32B-Instruct,
released by its original authors under the Apache License, Version 2.0.
This artifact is distributed under the same license. The full license text is
included in LICENSE [blocked], and required attribution is in NOTICE [blocked].
License text: https://www.apache.org/licenses/LICENSE-2.0 Source model: https://huggingface.co/Qwen/Qwen2.5-32B-Instruct
vllm serve drawais/Qwen2.5-32B-Instruct-AWQ-INT4 \ --max-model-len 32768 \ --gpu-memory-utilization 0.94from vllm import LLM, SamplingParamsllm = LLM(model="drawais/Qwen2.5-32B-Instruct-AWQ-INT4", max_model_len=32768)print(llm.generate(["Hello!"], SamplingParams(max_tokens=128))[0].outputs[0].text)