Neural network models, typically with billions of parameters, trained on very large text corpora to predict the next token.
Narrower concepts
Hybrid architecturesLanguage model designs that interleave attention layers with layers of another kind, typically state-space or linear-attention layers, to lower the cost of long contexts.
Inference servingRunning a trained model as a service that accepts requests and returns outputs, scheduling computation and memory so that many requests can be answered with acceptable latency and throughput.
LLM agentsSystems in which a language model works in a loop, choosing actions such as tool calls, observing their results and deciding the next step towards a goal.
Mixture of expertsA model design in which a router sends each token to a small subset of specialised sub-networks, the experts, so that only a fraction of the total parameters is active for any one token.
Repeated samplingDrawing many independent attempts from a model for the same problem and keeping any that a verifier accepts, so that the share of problems solved grows with the number of attempts.
Sampling temperatureA parameter that rescales a model's output probabilities before a token is drawn; lower values make generation more deterministic, and a temperature of zero always takes the most likely token (greedy decoding).
State-space modelsSequence models, such as Mamba, that carry a fixed-size recurrent state through a sequence instead of attending to every earlier token, so that their memory per sequence does not grow with context length.