Concept

Memory bandwidth

The rate at which data can be moved between memory and the processor, usually given in gigabytes per second; generating text one token at a time is often limited by it rather than by arithmetic.