Post
736
Good news, llama.cpp seems to be close to supporting MTP on qwen models. Bad news, every single gguf will have to be redone when it is.
- 1 reply
Cahlen Humphreys PRO
cahlen
AI & ML interests
โ ๏ธ๐ป
Recent Activity
reacted to BlueNipples's post with ๐ about 10 hours ago
Good news, llama.cpp seems to be close to supporting MTP on qwen models. Bad news, every single gguf will have to be redone when it is. reacted to unmodeled-tyler's post with ๐ about 10 hours ago
Hey Hugging Face!
Repo: https://github.com/unmodeled-tyler/vessel-browser
I wanted to share a cool feature from my open source AI native web browser, Vessel: Persistent highlights!
You can highlight anything on the page and the context is provided to the agent. It's kind of a fun way to learn about new stuff, synthesize info, or just deepen your comprehension/understanding.
Since highlights are persistent, you can close the page, come back later - and your highlights will be exactly where you left them. I've found this particularly useful when reviewing technical blogs, model cards, etc.
Check it out! Organizations
reacted to BlueNipples's post with ๐ about 10 hours ago
reacted to yuriyvnv's post with ๐ฅ about 10 hours ago
Post
1083
๐ The WAVe paper is officially out in the Information Sciences Journal.
You saw the PT and NL model releases earlier this year. This is the peer-reviewed paper behind them, with the full method, ablations, and downstream ASR evaluation.
Quick recap: WAVe is a 1B multimodal embedding model that filters synthetic speech at the word level, not the sentence level. On Portuguese ASR it cuts training steps by 34%, improves cross-domain generalization by 50%, and matches WER with 30% less synthetic data.
๐ฆ Resources
- Paper: https://www.sciencedirect.com/science/article/pii/S0020025526005220
- PT model: yuriyvnv/WAVe-1B-Multimodal-PT
- NL model: yuriyvnv/WAVe-1B-Multimodal-NL
- Collection: https://huggingface.co/collections/yuriyvnv/multi-modal-embeddings-for-synthetic-transcript-filtering
- Code: https://github.com/yuriyvnv/WAVe
If you train ASR on synthetic or back-translated data, would like to see WAVe benchmarked on other languages.
@reach-vb @ylacombe @hf-audio @BramVanroy
#speech #asr #multimodal #syntheticdata #lowresource
You saw the PT and NL model releases earlier this year. This is the peer-reviewed paper behind them, with the full method, ablations, and downstream ASR evaluation.
Quick recap: WAVe is a 1B multimodal embedding model that filters synthetic speech at the word level, not the sentence level. On Portuguese ASR it cuts training steps by 34%, improves cross-domain generalization by 50%, and matches WER with 30% less synthetic data.
๐ฆ Resources
- Paper: https://www.sciencedirect.com/science/article/pii/S0020025526005220
- PT model: yuriyvnv/WAVe-1B-Multimodal-PT
- NL model: yuriyvnv/WAVe-1B-Multimodal-NL
- Collection: https://huggingface.co/collections/yuriyvnv/multi-modal-embeddings-for-synthetic-transcript-filtering
- Code: https://github.com/yuriyvnv/WAVe
If you train ASR on synthetic or back-translated data, would like to see WAVe benchmarked on other languages.
@reach-vb @ylacombe @hf-audio @BramVanroy
#speech #asr #multimodal #syntheticdata #lowresource
reacted to unmodeled-tyler's post with ๐ about 10 hours ago
Post
636
Hey Hugging Face!
Repo: https://github.com/unmodeled-tyler/vessel-browser
I wanted to share a cool feature from my open source AI native web browser, Vessel: Persistent highlights!
You can highlight anything on the page and the context is provided to the agent. It's kind of a fun way to learn about new stuff, synthesize info, or just deepen your comprehension/understanding.
Since highlights are persistent, you can close the page, come back later - and your highlights will be exactly where you left them. I've found this particularly useful when reviewing technical blogs, model cards, etc.
Check it out!
Repo: https://github.com/unmodeled-tyler/vessel-browser
I wanted to share a cool feature from my open source AI native web browser, Vessel: Persistent highlights!
You can highlight anything on the page and the context is provided to the agent. It's kind of a fun way to learn about new stuff, synthesize info, or just deepen your comprehension/understanding.
Since highlights are persistent, you can close the page, come back later - and your highlights will be exactly where you left them. I've found this particularly useful when reviewing technical blogs, model cards, etc.
Check it out!
- 1 reply
posted an update about 11 hours ago
Post
39
So I built a multimodal video annotation pipeline in my spare time, as you do.
corpus-mill turns any long-form video with people on camera into a time-aligned event corpus across audio, vision, OCR, faces, brand observations, music, and clip-worthy moments. Runs entirely on local GPU because โ and I cannot stress this enough โ your footage has no business being on someone else's servers.
The honest origin: I needed real multimodal supervision data, the public corpora are weirdly thin once you need per-frame / per-speaker / per-second labels with provenance, so I built one. Then it grew. Then I looked up and it was 30K LOC and ~30 stages and I thought, ok, maybe other people would want this.
Stack is the usual suspects: Whisper-large-v3 (faster-whisper), pyannote-3.1 (which secretly drags in 433 NeMo modules โ surprise!), Qwen2.5-VL-7B for vision/OCR/shoppable detection, dlib + YuNet for faces, qwen2.5:7b / qwen3:14b via local Ollama for the LLM passes, chromaprint + PDQ for fingerprinting. Outputs as Parquet + SQLite. Apache 2.0.
There's a Docker compose that works, after I spent a day discovering that faster-whisper wants CUDA 12 cuBLAS while pyannote 4 wants CUDA 13, and the answer is "install both, point LD_LIBRARY_PATH at the cu12 wheels, ship it." That's now baked in. You're welcome.
Spare-time project, bugs are real, fixing them for your specific footage is on you. If you're training multimodal models and want a corpus pipeline you fully control on-prem, this might save you months. If not, the README is at least mildly entertaining.
https://github.com/cahlen/corpus-mill
corpus-mill turns any long-form video with people on camera into a time-aligned event corpus across audio, vision, OCR, faces, brand observations, music, and clip-worthy moments. Runs entirely on local GPU because โ and I cannot stress this enough โ your footage has no business being on someone else's servers.
The honest origin: I needed real multimodal supervision data, the public corpora are weirdly thin once you need per-frame / per-speaker / per-second labels with provenance, so I built one. Then it grew. Then I looked up and it was 30K LOC and ~30 stages and I thought, ok, maybe other people would want this.
Stack is the usual suspects: Whisper-large-v3 (faster-whisper), pyannote-3.1 (which secretly drags in 433 NeMo modules โ surprise!), Qwen2.5-VL-7B for vision/OCR/shoppable detection, dlib + YuNet for faces, qwen2.5:7b / qwen3:14b via local Ollama for the LLM passes, chromaprint + PDQ for fingerprinting. Outputs as Parquet + SQLite. Apache 2.0.
There's a Docker compose that works, after I spent a day discovering that faster-whisper wants CUDA 12 cuBLAS while pyannote 4 wants CUDA 13, and the answer is "install both, point LD_LIBRARY_PATH at the cu12 wheels, ship it." That's now baked in. You're welcome.
Spare-time project, bugs are real, fixing them for your specific footage is on you. If you're training multimodal models and want a corpus pipeline you fully control on-prem, this might save you months. If not, the README is at least mildly entertaining.
https://github.com/cahlen/corpus-mill
reacted to prithivMLmods's post with ๐ฅ 5 days ago
Post
3765
Multimodal-Edge Demo, a node-based inference canvas demo, is now live on Spaces. It features node-based Transformers for fast inference across 10+ edge-device multimodal models on the Hub, all within a single space. The series includes models from Qwen3.5, Qwen3-VL, Gemma 4, and the LFM 2.5 VL model series, with support for reasoning and grounding tasks.
๐ค Demo: prithivMLmods/Multimodal-Edge-Node
๐ GitHub: https://github.com/PRITHIVSAKTHIUR/Multimodal-Edge-Node
โ Multimodal Apps Collections: https://huggingface.co/collections/prithivMLmods/hall-of-multimodal-apps
๐ค > To learn more, visit the app page or the respective model pages.
๐ค Demo: prithivMLmods/Multimodal-Edge-Node
๐ GitHub: https://github.com/PRITHIVSAKTHIUR/Multimodal-Edge-Node
โ Multimodal Apps Collections: https://huggingface.co/collections/prithivMLmods/hall-of-multimodal-apps
๐ค > To learn more, visit the app page or the respective model pages.
reacted to SeaWolf-AI's post with ๐ 7 days ago
Post
5027
๐ Introducing Model Galaxy โ a Living, Multimodal Fork of the HF Model Atlas
๐ Try it: FINAL-Bench/model-galaxy
This Space is a fork of the brilliant Eliahu/Model-Atlas, the official demo of "Charting and Navigating Hugging Face's Model Atlas" (Horwitz et al., arXiv 2503.10633). Their pre-computed HF model graph is the foundation of every node and edge you see, and we are deeply grateful for its open release.
The original atlas is a static snapshot of early 2025. Model Galaxy turns it into a living, multimodal map. We injected the 2026 trending originals that did not exist when the atlas was frozen โ DeepSeek-V4, Hy3-preview, GLM-5.1, Kimi-K2, gpt-oss, Nemotron-3 Super / Nano / Omni, Hermes-4.3, Qwen3-Coder-Next, Llama-3.3, Granite-4.1, plus the latest multimodal releases (FLUX.2, ERNIE-Image, HunyuanImage / Video, LTX-2.3, Wan2.2, Kokoro-82M, VoxCPM2, Voxtral-TTS, whisper-v3-turbo, Gemma-4, Qwen3-Omni, Phi-4-mm) โ each with proper base_model lineage edges.
We also added the complete VIDRAFT Darwin family ontology: 120 nodes covering Darwin Core, AETHER, every brand variant (Rogue, AWAXIS, TenOS, Warecube), NOESIS-Darwin multimodal extensions, and 40+ community quantizations โ the most complete Darwin lineage view anywhere.
The name "Galaxy" is now literal: our three injected clusters are re-laid out as logarithmic spiral galaxies, with bigger models near the bright cores and quantizations scattering to the outer arms โ just like real star mass distribution. A top-right toggle switches between Galaxy mode (deep-space gradient with 220 animated stars) and Atlas mode (clean white panels for reports). A 15-second progress bar narrates the render, and per-modality / per-company colors make every cluster legible at a glance.
Final scale: 22,480 nodes in the default Modalities atlas, 137,324 in the Large NLP atlas, and a 277-node compact Darwin + Trending view for instant exploration. Feedback and PRs welcome.
๐ Try it: FINAL-Bench/model-galaxy
This Space is a fork of the brilliant Eliahu/Model-Atlas, the official demo of "Charting and Navigating Hugging Face's Model Atlas" (Horwitz et al., arXiv 2503.10633). Their pre-computed HF model graph is the foundation of every node and edge you see, and we are deeply grateful for its open release.
The original atlas is a static snapshot of early 2025. Model Galaxy turns it into a living, multimodal map. We injected the 2026 trending originals that did not exist when the atlas was frozen โ DeepSeek-V4, Hy3-preview, GLM-5.1, Kimi-K2, gpt-oss, Nemotron-3 Super / Nano / Omni, Hermes-4.3, Qwen3-Coder-Next, Llama-3.3, Granite-4.1, plus the latest multimodal releases (FLUX.2, ERNIE-Image, HunyuanImage / Video, LTX-2.3, Wan2.2, Kokoro-82M, VoxCPM2, Voxtral-TTS, whisper-v3-turbo, Gemma-4, Qwen3-Omni, Phi-4-mm) โ each with proper base_model lineage edges.
We also added the complete VIDRAFT Darwin family ontology: 120 nodes covering Darwin Core, AETHER, every brand variant (Rogue, AWAXIS, TenOS, Warecube), NOESIS-Darwin multimodal extensions, and 40+ community quantizations โ the most complete Darwin lineage view anywhere.
The name "Galaxy" is now literal: our three injected clusters are re-laid out as logarithmic spiral galaxies, with bigger models near the bright cores and quantizations scattering to the outer arms โ just like real star mass distribution. A top-right toggle switches between Galaxy mode (deep-space gradient with 220 animated stars) and Atlas mode (clean white panels for reports). A 15-second progress bar narrates the render, and per-modality / per-company colors make every cluster legible at a glance.
Final scale: 22,480 nodes in the default Modalities atlas, 137,324 in the Large NLP atlas, and a 277-node compact Darwin + Trending view for instant exploration. Feedback and PRs welcome.
reacted to anakin87's post with โค๏ธ๐ฅ 13 days ago
Post
3277
A small model that struggled against a random opponent now beats GPT-5-mini at tic-tac-toe
I took LiquidAI/LFM2-2.6B and trained it through play.
๐งโ๐ณ Here's how:
1๏ธโฃ Build a solid RL env with Verifiers (Prime Intellect)
2๏ธโฃ Generate synthetic data: <200 games sampled from GPT-5-mini playing in the env
3๏ธโฃ SFT warm-up to teach format
4๏ธโฃ Group-based RL (CISPO) against opponents making 20-70% random moves
5๏ธโฃ RL again with stronger opponents (0-25% random moves) + 1.25 temperature to push exploration and shake off suboptimal strategies
Done! Beats GPT-5-mini ๐
---
๐ฎ Play against the model: anakin87/LFM2-2.6B-mr-tictactoe
๐ค Model: anakin87/LFM2-2.6B-mr-tictactoe
๐ Walkthrough/course: https://github.com/anakin87/llm-rl-environments-lil-course
๐ค Dataset and checkpoints: https://huggingface.co/collections/anakin87/lfm2-26b-mr-tic-tac-toe
I took LiquidAI/LFM2-2.6B and trained it through play.
๐งโ๐ณ Here's how:
1๏ธโฃ Build a solid RL env with Verifiers (Prime Intellect)
2๏ธโฃ Generate synthetic data: <200 games sampled from GPT-5-mini playing in the env
3๏ธโฃ SFT warm-up to teach format
4๏ธโฃ Group-based RL (CISPO) against opponents making 20-70% random moves
5๏ธโฃ RL again with stronger opponents (0-25% random moves) + 1.25 temperature to push exploration and shake off suboptimal strategies
Done! Beats GPT-5-mini ๐
---
๐ฎ Play against the model: anakin87/LFM2-2.6B-mr-tictactoe
๐ค Model: anakin87/LFM2-2.6B-mr-tictactoe
๐ Walkthrough/course: https://github.com/anakin87/llm-rl-environments-lil-course
๐ค Dataset and checkpoints: https://huggingface.co/collections/anakin87/lfm2-26b-mr-tic-tac-toe
This sounds really cool, definitely adding it to my weekend experiment list (kudos for providing the github repo as well).
Also... ShadowPEFT? Badass name.
reacted to SeanLee97's post with ๐๐ฅ 15 days ago
Post
8135
Our lab recently released a paper where we introduce ShadowPEFT, a new Parameter-Efficient Fine-Tuning (PEFT) paradigm tailored for edge computing scenarios.
Unlike traditional approaches such as LoRA and its variants, which inject trainable parameters directly into the weights of Transformer, requiring tight coupling with the backbone.
ShadowPEFT instead enhances the frozen large base model by adding a lightweight, centralized, pretrainable, and detachable Shadow network.
This shadow network operates in parallel with the base model, delivering learned corrections to each decoder layer. Because the shadow module is architecturally decoupled from the backbone, it can be independently trained, stored, and deployed, benefiting edge computing scenarios and edge-cloud collaboration computing.
- HF Paper: ShadowPEFT: Shadow Network for Parameter-Efficient Fine-Tuning (2604.19254)
- GitHub: https://github.com/ShadowLLM/shadow-peft
- HF Collection: https://huggingface.co/collections/shadow-llm/shadow-peft-models
Unlike traditional approaches such as LoRA and its variants, which inject trainable parameters directly into the weights of Transformer, requiring tight coupling with the backbone.
ShadowPEFT instead enhances the frozen large base model by adding a lightweight, centralized, pretrainable, and detachable Shadow network.
This shadow network operates in parallel with the base model, delivering learned corrections to each decoder layer. Because the shadow module is architecturally decoupled from the backbone, it can be independently trained, stored, and deployed, benefiting edge computing scenarios and edge-cloud collaboration computing.
- HF Paper: ShadowPEFT: Shadow Network for Parameter-Efficient Fine-Tuning (2604.19254)
- GitHub: https://github.com/ShadowLLM/shadow-peft
- HF Collection: https://huggingface.co/collections/shadow-llm/shadow-peft-models
- 7 replies
posted an update 16 days ago
Post
263
In the spirit of "just making shit"โbecause, frankly, you can these daysโI decided to get back to basics. I built a lightweight RL MLP powered by WebGPU that runs directly in your browser.
The twist? I replaced the standard MLP with a Continued Fractions network. The real win here is interpretability; by applying a Taylor expansion to the continued fractions, we can actually decompose the output and see exactly which features influenced the outcome. It effectively replaces the usual "itโs just black-box magic" with actual visibility into the logic.
It was a fun experiment. Feel free to clone the repo and make it your own!
cahlen/neuron-runner
https://github.com/cahlen/neuron-runner
CoFrGeNet: Continued Fraction Architectures for Language Generation (2601.21766)
The twist? I replaced the standard MLP with a Continued Fractions network. The real win here is interpretability; by applying a Taylor expansion to the continued fractions, we can actually decompose the output and see exactly which features influenced the outcome. It effectively replaces the usual "itโs just black-box magic" with actual visibility into the logic.
It was a fun experiment. Feel free to clone the repo and make it your own!
cahlen/neuron-runner
https://github.com/cahlen/neuron-runner
CoFrGeNet: Continued Fraction Architectures for Language Generation (2601.21766)
reacted to imnotkitty's post with ๐๐๐ค๐โค๏ธ๐๐ฅ 20 days ago
Post
2602
Just tried tencent/HY-World-2.0 โ a multimodal world model that takes in text or a single image and generates editable 3D scenes.
Unlike Google's Genie and HY-World 1.5, v2.0 generates engine-ready 3D content:
๐ฎ Direct import into Unreal Engine and Unity โ no format wrangling
๐ง Supports multiple 3D asset formats: Mesh, 3DGS, point cloud, etc.
โ๏ธ Fully editable โ not a baked video, but actual geometry you can modify
๐ค Also usable for embodied simulation environments
Basically: from "AI generates a world you can look at" โ "AI generates a world you can ship."
Unlike Google's Genie and HY-World 1.5, v2.0 generates engine-ready 3D content:
๐ฎ Direct import into Unreal Engine and Unity โ no format wrangling
๐ง Supports multiple 3D asset formats: Mesh, 3DGS, point cloud, etc.
โ๏ธ Fully editable โ not a baked video, but actual geometry you can modify
๐ค Also usable for embodied simulation environments
Basically: from "AI generates a world you can look at" โ "AI generates a world you can ship."
its a really great model. i just got done evaluating it for the past week. the long term agentic loops really really help in autonomous app building, especially the recursive fixing. sometimes i neglect the fact that a bigger model is just.. .well.. better usually.
very good model. now if we can't just figure out how to squeeze it all down to an rtx 5090 possibly? hehe