Parameters and Model Size · How LLMs Work in Generative AI | Generative AI & LLMs

Parameters and Model Size

What Is a Parameter?

A parameter is one number inside the neural network that was adjusted during training. "Llama 3.1 8B" has about 8 billion of them; frontier models are much larger (exact sizes of closed models like Claude and GPT are not published). More parameters usually means more knowledge and better reasoning, but also more memory, cost and latency.

Memory Needed to Run a Model

Example

To run an open model yourself, the weights must fit in GPU (or CPU) memory. Memory ≈ parameters × bytes per parameter. Quantization stores each number with fewer bits to save memory:

models = {"Llama 3.2 3B": 3e9, "Llama 3.1 8B": 8e9, "Llama 3.3 70B": 70e9}
formats = {"fp16 (2 bytes)": 2, "int8 (1 byte)": 1, "int4 (0.5 byte)": 0.5}

print(f"{'model':15}" + "".join(f"{f:>18}" for f in formats))
for name, params in models.items():
    row = "".join(f"{params * b / 1e9:>15.1f} GB" for b in formats.values())
    print(f"{name:15}{row}")

Output

model              fp16 (2 bytes)     int8 (1 byte)   int4 (0.5 byte)
Llama 3.2 3B               6.0 GB            3.0 GB            1.5 GB
Llama 3.1 8B              16.0 GB            8.0 GB            4.0 GB
Llama 3.3 70B            140.0 GB           70.0 GB           35.0 GB

Note: Add roughly 10–30% on top for the context (KV cache) and runtime. A 4-bit 8B model (~4 GB) runs on a normal laptop with Ollama; a 70B model needs a big GPU server.