Article
Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers
โข
46
red335
srushti335
ยท
AI & ML interests
None yet
Recent Activity
upvoted a paper 10 days ago
Omni-SimpleMem: Autoresearch-Guided Discovery of Lifelong Multimodal Agent MemoryOrganizations
None yet
upvoted an article 1 day ago
upvoted 2
papers 10 days ago
upvoted a paper 13 days ago
upvoted a paper 29 days ago
reacted to prithivMLmods's post with โค๏ธ 8 months ago
Post
3454
Introducing prithivMLmods/DeepCaption-VLA-7B, a multimodal VLM designed for reasoning with long-shot captions (Captioning and Vision-Language Attribution). It focuses on defining visual properties, object attributes, and scene details across a wide spectrum of images and aspect ratios, generating attribute-rich image captions. The model supports creative, artistic, and technical applications that require detailed descriptions. ๐ค๐ฅ
โฆ๏ธ Models: prithivMLmods/DeepCaption-VLA-7B, also includes prithivMLmods/DeepAttriCap-VLA-3B, an experimental model for vision-language attribution.
โฆ๏ธ Try the demo here: prithivMLmods/VisionScope-R2
โฆ๏ธ Try it now on Google Colab, with support for T4 GPUs in 4-bit quant_type: https://github.com/PRITHIVSAKTHIUR/Multimodal-Outpost-Notebooks/blob/main/DeepCaption-VLA-7B%5B4bit%20-%20notebook%20demo%5D/DeepCaption-VLA-7B.ipynb
โฆ๏ธ Collection: prithivMLmods/deepcaption-attr-68b041172ebcb867e45c556a
.
.
.
To know more about it, visit the model card of the respective model. !!
โฆ๏ธ Models: prithivMLmods/DeepCaption-VLA-7B, also includes prithivMLmods/DeepAttriCap-VLA-3B, an experimental model for vision-language attribution.
โฆ๏ธ Try the demo here: prithivMLmods/VisionScope-R2
โฆ๏ธ Try it now on Google Colab, with support for T4 GPUs in 4-bit quant_type: https://github.com/PRITHIVSAKTHIUR/Multimodal-Outpost-Notebooks/blob/main/DeepCaption-VLA-7B%5B4bit%20-%20notebook%20demo%5D/DeepCaption-VLA-7B.ipynb
โฆ๏ธ Collection: prithivMLmods/deepcaption-attr-68b041172ebcb867e45c556a
.
.
.
To know more about it, visit the model card of the respective model. !!
- 4 replies
reacted to hesamation's post with โค๏ธ 12 months ago
Post
2948
this paper has been blowing up
they train an open-source multimodal LLM (InternVL3) that can compete with GPT-4o and Claude 3.5 Sonnet by:
> training text and vision on a single stage
> a novel V2PE positional encoding
> SFT & mixed preference optimization
Paper: InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models (2504.10479)
> test-time scaling
they train an open-source multimodal LLM (InternVL3) that can compete with GPT-4o and Claude 3.5 Sonnet by:
> training text and vision on a single stage
> a novel V2PE positional encoding
> SFT & mixed preference optimization
Paper: InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models (2504.10479)
> test-time scaling
reacted to davidberenstein1957's post with ๐ง 12 months ago
Post
1746
๐งโ๐ซ I wrote a brief blogpost to give An Introduction to AI Model Optimization Techniques!
URL: https://huggingface.co/blog/PrunaAI/introduction-to-ai-model-optimization-techniques
URL: https://huggingface.co/blog/PrunaAI/introduction-to-ai-model-optimization-techniques
reacted to retronic's post with ๐ฅ 12 months ago
Post
4659
Colox, a reasoning AI model. I am currently working on a model smarter than GPT o1 that thinks before it speaks. It is coming tomorrow in the afternoon.
- 7 replies
reacted to hexgrad's post with ๐ about 1 year ago
Post
6238
I wrote an article about G2P: https://hf.co/blog/hexgrad/g2p
G2P is an underrated piece of small TTS models, like offensive linemen who do a bunch of work and get no credit.
Instead of relying on explicit G2P, larger speech models implicitly learn this task by eating many thousands of hours of audio data. They often use a 500M+ parameter LLM at the front to predict latent audio tokens over a learned codebook, then decode these tokens into audio.
Kokoro instead relies on G2P preprocessing, is 82M parameters, and thus needs less audio to learn. Because of this, we can cherrypick high fidelity audio for training data, and deliver solid speech for those voices. In turn, this excellent audio quality & lack of background noise helps explain why Kokoro is very competitive in single-voice TTS Arenas.
G2P is an underrated piece of small TTS models, like offensive linemen who do a bunch of work and get no credit.
Instead of relying on explicit G2P, larger speech models implicitly learn this task by eating many thousands of hours of audio data. They often use a 500M+ parameter LLM at the front to predict latent audio tokens over a learned codebook, then decode these tokens into audio.
Kokoro instead relies on G2P preprocessing, is 82M parameters, and thus needs less audio to learn. Because of this, we can cherrypick high fidelity audio for training data, and deliver solid speech for those voices. In turn, this excellent audio quality & lack of background noise helps explain why Kokoro is very competitive in single-voice TTS Arenas.
reacted to fdaudens's post with ๐ฅ about 1 year ago
Post
2119
๐ข SmolLM2 paper released! Learn how the ๐ค team built one of the best small language models: from data choices to training insights. Check out our findings and share your thoughts! ๐ค๐ก
Check it out: SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model (2502.02737)
Check it out: SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model (2502.02737)
reacted to m-ric's post with ๐ฅ about 1 year ago
Post
10033
Introducing ๐ผ๐ฝ๐ฒ๐ป ๐๐ฒ๐ฒ๐ฝ-๐ฅ๐ฒ๐๐ฒ๐ฎ๐ฟ๐ฐ๐ต by Hugging Face! ๐ฅ
OpenAI's latest agentic app Deep Research seems really good... But it's closed, as usual.
โฑ๏ธ So with a team of cracked colleagues, we set ourselves a 24hours deadline to replicate and open-source Deep Research! โฑ๏ธ
โก๏ธ We built open-Deep-Research, an entirely open agent that can: navigate the web autonomously, scroll and search through pages, download and manipulate files, run calculation on data...
We aimed for the best performance: are the agent's answers really rigorous?
On GAIA benchmark, Deep Research had 67% accuracy on the validation set.
โก๏ธ open Deep Research is at 55% (powered by o1), it is:
- the best pass@1 solution submitted
- the best open solution ๐ช๐ช
And it's only getting started ! Please jump in, drop PRs, and let's bring it to the top !
Read the blog post ๐ https://huggingface.co/blog/open-deep-research
OpenAI's latest agentic app Deep Research seems really good... But it's closed, as usual.
โฑ๏ธ So with a team of cracked colleagues, we set ourselves a 24hours deadline to replicate and open-source Deep Research! โฑ๏ธ
โก๏ธ We built open-Deep-Research, an entirely open agent that can: navigate the web autonomously, scroll and search through pages, download and manipulate files, run calculation on data...
We aimed for the best performance: are the agent's answers really rigorous?
On GAIA benchmark, Deep Research had 67% accuracy on the validation set.
โก๏ธ open Deep Research is at 55% (powered by o1), it is:
- the best pass@1 solution submitted
- the best open solution ๐ช๐ช
And it's only getting started ! Please jump in, drop PRs, and let's bring it to the top !
Read the blog post ๐ https://huggingface.co/blog/open-deep-research