Draft Models Give Way to MTP Speculative Decoding on Local Hardware

Draft models are on their way out.

When it comes to speeding up inference with speculative decoding, the old approach was to bolt on a small draft model that guesses future tokens while the main model verifies them in batches. But with the rise of native multi-token prediction (MTP), that process is being pulled back inside the main model.

MTP lets the main model predict and verify subsequent tokens on its own. Compared to the external draft model setup, it eliminates the need for a separate set of model weights and the headache of mismatched tokenizers or vocabularies.

The upstream models have already made the move. DeepSeek-V3's technical report lists MTP as a training objective and notes it can be used for speculative decoding at inference time. DeepSeek-V4's report explicitly continues the same MTP strategy. Qwen3.6 goes a step further, spelling out SGLang and vLLM launch commands directly in its official model card.

Downstream inference frameworks and tools are rapidly adapting:

• llama.cpp merged MTP support on May 16, 2026. Developer tests on Qwen3.6 27B and 35B-A3B show a stable acceptance rate of around 75%, with generation speed doubling in some configurations. But the pull request also warns that enabling MTP slows down prompt processing and that parallel decoding still needs improvement.
• vLLM has added MTP to its official speculative decoding list. Per Qwen3.6's recommendation, you can enable it with the flag --speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}'. However, vLLM's documentation is candid: real-world speed gains still depend heavily on the specific model and deployment environment.
• LM Studio 0.4.14 Beta adds a manual MTP toggle. Once you download an MTP-capable model, you can turn it on in the advanced loading settings.
• Ollama is taking a more cautious approach for now. Its official changelog only confirms Gemma 4 MTP support on Mac's MLX runner, with no mention of default acceleration for Qwen3.6 or general MTP-GGUF yet.