Show HN: 188M Hindi encoder, 28B tokens, 8K context, 1× RTX 4090
By kkkamur · 2026-09-06 · 2 points · 0 comments
https://github.com/kkkamur07/indic-modernBERT
I wanted to improve Hindi retrieval quality, particularly for longer documents ( as the current architectures don't really have a longer context length ), and was curious how far can I push a 4090 haha :) So I trained a Hindi-first ModernBERT from scratch: 188M parameters ~28.5B…
Open the full discussion on BetterNews