Sparse autoencoders and Claude's Golden Gate Bridge feature
A look at Anthropic's sparse autoencoder work on Claude 3 Sonnet through one feature, the Golden Gate Bridge.
Posts
Browse all KbWen Blog articles about AI systems, machine learning, Python, and practical software development.
金門大橋 Claude 與稀疏自編碼器：把單一特徵調大會發生什麼事
我們來看看 Anthropic 怎麼用稀疏自編碼器在 Claude 3 Sonnet 裡找到「金門大橋」這個特徵。
Copying nested lists in Python
A look at what changes when a copied Python list contains other lists.
Python 淺拷貝與巢狀串列
用 list.copy() 備份巢狀串列之後，修改資料會發生什麼事？從兩種修改方式看看淺拷貝的範圍。
RAG 混合檢索的 Reciprocal Rank Fusion
從官方範例看看文字搜尋和向量搜尋的結果，如何用 RRF 合併排序。
Reciprocal rank fusion with multiple vector fields
A closer look at the result lists that go into reciprocal rank fusion when a hybrid search uses several vector fields.
DeepSeek-V3's auxiliary-loss-free load balancing and multi-token prediction
A look at how DeepSeek-V3 keeps load even across its experts during training, and what its multi-token prediction module is for.
DeepSeek-V3 的 Multi-head Latent Attention 如何壓縮 KV 快取
我們來看看 DeepSeek-V3 如何利用 MLA 的低維潛在向量壓縮 KV 快取，解決長序列推論時的記憶體瓶頸。
CPython's dict layout
A look at how CPython lays out a dict in memory.
Python dict 的優化
從 Python 3.6 dict 的改寫，來看看插入順序如何跟著記憶體優化。