The B in a name like gpt-oss-20b counts parameters, in billions. Multiply that count by the bytes each parameter takes and you get a floor for the memory the weights need. That is also why some models now ship with two numbers instead of one.
A transformer redoes the same attention projections for every past token at each decoding step. The KV cache stores those keys and values so they get reused instead of recomputed, and the one cost it adds is memory that grows with the sequence.
網路上常說：要 AI 每次給一樣的答案，把 temperature 設成 0 就好。但有人拿同一個 prompt、temperature 0 連跑 1000 次，還是冒出 80 種不同輸出。原因除了浮點誤差，更關鍵的是它在 GPU 上跟多少別的請求湊成一批一起算。聊聊為什麼『最確定』不等於『可重現』。