Show HN: Sparse kernel delivering 36% higher training throughput
By jakcwi · 2026-10-24 · 1 points · 0 comments
https://huggingface.co/JakCwi/eugen-sparse
I developed a custom GPU kernel in Triton to accelerate training with a sparse feed-forward network (FFN). With K=256, activating 256 of 4,096 FFN neurons per token (6.25%), the sparse model achieved 36% higher training throughput than its dense counterpart. The kernel has so fa…
Open the full discussion on BetterNews