Djuunaa's picture

Djuunaa

djuna

·

AI & ML interests

None yet

Recent Activity

reacted to FlameF0X's post with 🚀 2 days ago

Greetings Hugging Face! I started a new project called **FWKV** (Feed-forward Weighted Key Value, or Floored Weighted Key Value), a RWKV-style LM that uses FFNNs (Feed-Forward Neural Networks) instead of RNN and `floor(W·K·V)`. I'm hoping to make it much more efficient and scalable than RWKV. So far I have: - https://huggingface.co/FlameF0X/FWKV-29M — this one is undertrained and doesn't have a Space yet. In the attached image you can see its speed on a T4 compared to models with the same configuration. The only model that's fully working right now is: - https://huggingface.co/FlameF0X/FWKV-TinyStories — trained on TinyStories for one epoch. The demo Space is https://huggingface.co/spaces/FlameF0X/FWKV-demo.

liked a model 2 days ago

FlameF0X/FWKV-TinyStories

liked a model 9 days ago

Wfloat/wfloat-tts

View all activity

Organizations

djuna 's datasets

None public yet