Howdy, CompactAI-O is launching a tiny Model Golf, and the winner walks away with $50 in RunPod credits. Monthly. Every month. Show up, build, somebody wins.
What it is
Build the best language model you can under 100 million parameters, with at least a 1028-token context window. That's it. Any architecture, any tokenizer, any training scheme you can dream up at 3am. The only catch is it's gotta be open source (MIT, GPL, Apache, AGPL) take your pick.
It scratches the same itch as a Kaggle comp without the dataset\leaderboard nonsense. No fixed benchmark to game. No llama.cpp compatibility hoops. If you wanna train a 50M-param MoE with five experts and a tokenizer built on cookbooks, you can do that. Nothing stopping you.
The rules are listed in the discord and on the organization page if you're interested.
Why $50????
It's symbolic. It ain't gonna make anyone rich. But it's enough to cover a weekend of GPU time, enough to keep enthusiasts coming back, and not so much that it pulls in people who are just there for the money. Enthusiasts build interesting things. Interesting things move the field forward. A little incentive. I'd do it for $50 lol.
Is anybody else willing to put a second mortgage on their house, just to spend 40k USD in compute credits? Just me? k...
I got dreams, man. The datasets I could build with 40k would be insane. Somebody called me a genius the other day, they'd be shocked to find out, that I would put my house on the line for 30 days of runpod usage.
What would you do with it? I would turn arxiv into a dataset. Turn each arxiv paper into a QnA. Or... maybe if I got 40k USD in credit's Id end up like those 16 lost scientists.
Food for thought. Anyways, I think I'm going to make a post once a week. In the meantime you can find me building small llm's in discord here: https://discord.gg/4DdwS9D8x9
