Break the Sequential Dependency of LLM Inference Using Lookahead Decoding | LMSYS Org
lmsys.org

TL;DR: We introduce lookahead decoding, a new, exact, and parallel decoding algorithm to accelerate LLM inference. Look...