Concept

Speculative decoding

A decoding method in which a cheaper drafter proposes several tokens ahead and the target model checks them in one forward pass, accepting a prefix of them without changing the target model's output distribution.