What the parameter count in a model's name means

Plain definition: the B in a name like gpt-oss-20b counts parameters (the numbers inside the model that training produces and that then stay fixed) in billions, with T for trillions. Multiply that count by the bytes each parameter is stored in and you get a floor for how much memory the weights need. When a model card carries two numbers instead of one, the larger is what you have to store and the smaller is what runs for a single token.

A model’s parameters are the numbers it settled on during training and then kept: the weights in every layer, the biases, the embedding table. There are enough of them that the total gets rounded long before anyone writes it into a name. B is billion and T is trillion, so a 7B model holds roughly seven billion of these numbers, and Kimi-K3 holds trillions. OpenAI released gpt-oss-20b and gpt-oss-120b; their model card lists them at 21B and 117B parameters.

Bytes per parameter

The count is useful mainly because it converts into bytes. Every parameter occupies a fixed amount of room, set by the number format the weights are stored in. bf16 and fp16 are both sixteen-bit formats, two bytes per parameter, so a 7B model needs somewhere around 14 GB to hold its weights and a 70B model around 140 GB.

Weights can also be stored in smaller formats. Four bits per parameter is a quarter of that, which brings the same 7B model down near 3.5 GB and the 70B near 35 GB. These are estimates: the arithmetic is 7 × 10⁹ × 2 bytes and nothing more. But they are usually enough to see which models are out of reach on the hardware you already have.

The two gpt-oss sizes

OpenAI states the counts and the hardware in the same sentence. gpt-oss-120b is listed at “117B parameters with 5.1B active parameters” and described as one that will “fit into a single 80GB GPU”. The second figure counts the parameters that run for any one token; for memory it is the first that matters, since all 117 billion have to be loaded whether or not a given token touches them. At two bytes those 117 billion come to 234 GB, near enough three of those cards.

The card explains the gap a few lines further down. The models were post-trained with “MXFP4 quantization of the MoE weights”, four bits apiece. A hundred and seventeen billion parameters at half a byte each land at 58.5 GB. That is the version of the model the 80 GB figure describes.

gpt-oss-20b is listed at “21B parameters with 3.6B active parameters” and stated to “run within 16GB of memory”, with its MoE weights quantized the same way. Its 10.5 GB is what makes a 16 GB claim hold, and it leaves a few gigabytes over for everything that is not weights.

Names with two numbers

Other cards carry the same pair, and they write it down differently. DeepSeek-V3 opens its card with “671B total parameters with 37B activated for each token”, calling itself a Mixture-of-Experts model in the same sentence. Qwen puts both figures in the model name, Qwen3-235B-A22B-Instruct-2507, whose card reads “Number of Parameters: 235B in total and 22B activated”. Kimi K3 is described as “a 2.8T-parameter model”, and its Model Summary table gives “Activated Parameters 104B”, about a twenty-seventh of the total.

The two numbers answer different questions. The activated count tracks the arithmetic a single token costs; the total is the figure that has to fit. Kimi K3’s pair is the widest of these: 104 billion run for a token, and 2.8 trillion have to be resident while they do it.

What the weights leave out

The product of a count and a byte size is a floor. Above it sit the activations a forward pass creates, whatever the runtime holds, and the key-value cache, which grows with every token in the sequence and keeps growing while the model generates. A machine that clears the weights figure by a small margin can still run out partway through a long generation.

DeepSeek’s card notes that “The total size of DeepSeek-V3 models on HuggingFace is 685B, which includes 671B of the Main Model weights and 14B of the Multi-Token Prediction (MTP) Module weights.” The figure quoted for the model is 671B; the files add up to 685B.

This is put together from four model cards on Hugging Face, for gpt-oss, DeepSeek-V3, Qwen3-235B-A22B and Kimi K3. The arithmetic laid on top of them is mine, so the memory numbers here are estimates and any error in them is mine rather than theirs. I have left out how a mixture-of-experts model picks which parameters to run for a given token, and whether a bigger count makes a better model, which the counts on their own do not settle. If I have something wrong, tell me and I will fix it.