Can we have a smaller version version than the 230B / 10B active MoE e.g 120B / 10B active
#15
by
KingBadger
- opened
Can we have smaller versions or a in-house parent teacher distilled version? A Q6 or Q5 quant at about 50GB in size would be perfect for small home lab setups. Great model, great work.
Cheers J