embedl/Cosmos-Reason2-2B-W4A16-Edge2-FlashHead
Image-Text-to-Text • 2B • Updated • 926 • 7
Jonna Matthiesen
JonnaMat
AI & ML interests
None yet
Recent Activity
liked a model about 10 hours ago
embedl/Cosmos-Reason2-2B-W4A16-Edge2-FlashHead posted an update about 18 hours ago
⚡ FlashHead benchmarks for Llama 3.2, Gemma 3, and Qwen3 are now on https://huggingface.co/spaces/embedl/Edge-Inference-Benchmarks !
These are some of the models used in the FlashHead paper - now easier to explore and compare interactively.
🚀 Jetson AGX Thor (tok/s, batch=1):
- Llama-3.2-1B: 77 → 285 (FlashHead+W4A16, 3.7x)
- Llama-3.2-3B: 34 → 112 (3.3x)
- Gemma-3-1B: 79 → 153 (1.9x)
- Qwen3-1.7B: 49 → 189 (3.8x)
- Qwen3-0.6B: 140 → 177 (1.3x)
✅ Accuracy matches baseline on MMLU-Pro, IFEval, BBH, TruthfulQA, GSM8K.
updated a model about 19 hours ago
embedl/Qwen3-1.7B-FlashHead-W4A16Organizations
liked a
model about 10 hours ago
posted an update about 18 hours ago
Post
46
⚡ FlashHead benchmarks for Llama 3.2, Gemma 3, and Qwen3 are now on embedl/Edge-Inference-Benchmarks !
These are some of the models used in the FlashHead paper - now easier to explore and compare interactively.
🚀 Jetson AGX Thor (tok/s, batch=1):
- Llama-3.2-1B: 77 → 285 (FlashHead+W4A16, 3.7x)
- Llama-3.2-3B: 34 → 112 (3.3x)
- Gemma-3-1B: 79 → 153 (1.9x)
- Qwen3-1.7B: 49 → 189 (3.8x)
- Qwen3-0.6B: 140 → 177 (1.3x)
✅ Accuracy matches baseline on MMLU-Pro, IFEval, BBH, TruthfulQA, GSM8K.
These are some of the models used in the FlashHead paper - now easier to explore and compare interactively.
🚀 Jetson AGX Thor (tok/s, batch=1):
- Llama-3.2-1B: 77 → 285 (FlashHead+W4A16, 3.7x)
- Llama-3.2-3B: 34 → 112 (3.3x)
- Gemma-3-1B: 79 → 153 (1.9x)
- Qwen3-1.7B: 49 → 189 (3.8x)
- Qwen3-0.6B: 140 → 177 (1.3x)
✅ Accuracy matches baseline on MMLU-Pro, IFEval, BBH, TruthfulQA, GSM8K.
updated 4
models about 19 hours ago
updated 7
models about 20 hours ago
embedl/Cosmos-Reason2-2B-W4A16-Edge2-FlashHead
Image-Text-to-Text • 2B • Updated • 926 • 7
embedl/Qwen3-1.7B-FlashHead
2B • Updated • 102 • 3
embedl/Qwen3-0.6B-FlashHead
0.6B • Updated • 85 • 4
embedl/gemma-3-270m-it-FlashHead
0.3B • Updated • 211 • 3
embedl/gemma-3-1b-it-FlashHead
1.0B • Updated • 146 • 3
embedl/Llama-3.2-3B-Instruct-FlashHead
3B • Updated • 83 • 4
embedl/Llama-3.2-1B-Instruct-FlashHead
1B • Updated • 71 • 4
updated a
dataset about 21 hours ago
updated a
Space about 22 hours ago
repliedto their post 4 days ago
Any plans on releasing flashhead for qwen3.5 models?
Yes! FlashHead-enabled Qwen3.5 models are coming soon. We are currently finalizing accuracy and latency evaluations.
vllm serve
4
#2 opened 6 days ago
by 
ParisXu
updated a
Space 6 days ago
published an article 6 days ago
Article
How to Build a vLLM Plugin: A Guide to the general_plugins Entry Point
•
2
posted an update 6 days ago
Post
1220
⚡ FlashHead: Fast LM Head Inference - Now a Simple vLLM Plugin
flash-head replaces the dense LM head with a two-stage retrieval pipeline - up to 2x inference speedup, training-free. Previously required custom Docker images; now it's just:
✨ The plugin activates automatically via vLLM's
🧩 Supported models (full collection):
Qwen Qwen3, 
meta-llama Llama3, 
google Gemma3, 
nvidia Cosmos-Reason2 - BF16 and W4A16 variants.
https://huggingface.co/collections/embedl/flashhead
📊 embedl/Edge-Inference-Benchmarks
🔧 Benchmark it yourself:
FlashHead shines at low batch sizes; the typical real-time / on-device use case. 🚀
flash-head replaces the dense LM head with a two-stage retrieval pipeline - up to 2x inference speedup, training-free. Previously required custom Docker images; now it's just:
pip install flash-head
vllm serve embedl/Qwen3-1.7B-FlashHead-W4A16✨ The plugin activates automatically via vLLM's
vllm.general_plugins entry point. No source patches, no custom imports.🧩 Supported models (full collection):
https://huggingface.co/collections/embedl/flashhead
📊 embedl/Edge-Inference-Benchmarks
🔧 Benchmark it yourself:
vllm bench latency --model embedl/Qwen3-1.7B-FlashHead-W4A16 --batch-size 1
# Baseline comparison
FLASHHEAD_ENABLED=0 vllm bench latency --model embedl/Qwen3-1.7B-FlashHead-W4A16 --batch-size 1FlashHead shines at low batch sizes; the typical real-time / on-device use case. 🚀
- 2 replies