DeepSeek-V3's auxiliary-loss-free load balancing and multi-token prediction
A look at how DeepSeek-V3 keeps load even across its experts during training, and what its multi-token prediction module is for.
DeepSeek
DeepSeek-V3 的 Multi-head Latent Attention 如何壓縮 KV 快取
我們來看看 DeepSeek-V3 如何利用 MLA 的低維潛在向量壓縮 KV 快取，解決長序列推論時的記憶體瓶頸。