This article was converted by SimpRead. Original source: mp.weixin.qq.com
Edited by: Zenan, Yang Wen
Just now, DeepSeek V4 received an update.
A new speculative decoding framework—DSpark—has been officially launched, and the full-stack speculative decoding framework supporting this version, DeepSpec, has also been open-sourced.
DeepSeek-V4-Pro-DSpark is not a newly designed architecture; rather, it integrates a speculative decoding module into the existing DeepSeek-V4-Pro model. The focus of this update lies in engineering deployment—not in iterative improvements to the model’s intrinsic capabilities.
DSpark has already been deployed on real production traffic for both DeepSeek-V4 (Flash and Pro), significantly accelerating large language model (LLM) inference speed.
- Technical report: DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation
- Technical report link: DeepSpec/DSpark_paper.pdf at main · deepseek-ai/DeepSpec · GitHub
The core motivation behind DSpark is to address latency and throughput bottlenecks faced by LLM inference in production environments—especially under high-concurrency workloads. In short, DSpark successfully combines high-throughput parallel generation with adaptive load-aware verification.
Speculative decoding is a technique that accelerates LLM inference without altering the output distribution of the target model. Its core idea is to introduce a lightweight draft model, which pre-generates several candidate tokens; the target model then verifies and accepts these candidates in batch, transforming serial, token-by-token generation into parallel, batch-wise validation—and thereby substantially reducing end-to-end latency.
Building upon this foundation, DSpark’s innovation lies in its semi-autoregressive generation architecture: it preserves the high-throughput advantage of parallel draft models while incorporating a lightweight serial module to model intra-block token dependencies—mitigating the declining acceptance rate commonly observed at later positions in purely parallel draft models.
Additionally, DSpark introduces hardware-aware confidence-scheduled verification: traditional speculative decoding often blindly submits all generated draft tokens for verification. Under high system load, tail tokens—which are highly likely to be rejected—waste precious batch-processing compute resources. DSpark introduces a confidence head to estimate the survival probability of each token. Combined with a hardware-aware prefix scheduler, the system dynamically determines the optimal verification length per request based on real-time engine throughput characteristics—allocating compute only to tokens expected to yield the highest return.
To ensure robust deployment on real-world infrastructure, DSpark’s scheduler employs an asynchronous mechanism compatible with Zero-Overhead Scheduling (ZOS) and continuous CUDA Graph replay. It leverages predictions from the previous two steps to determine the current dynamic truncation length—effectively hiding scheduling latency, avoiding GPU pipeline stalls, and guaranteeing lossless restoration of the target model’s output distribution.
Across diverse evaluation domains—including mathematical reasoning, code generation, and everyday dialogue—DSpark significantly outperforms state-of-the-art autoregressive models (e.g., Eagle3) and parallel draft models (e.g., DFlash). For example, on Qwen3-series target models (4B, 8B, 14B), DSpark achieves average acceptance lengths 26.7%–30.9% higher than Eagle3 and 16.3%–18.4% higher than DFlash.
Compared to the prior single-token-per-step production baseline (MTP-1), DSpark boosts user generation speed by 60%–85% (for Flash models) and 57%–78% (for Pro models), while maintaining identical overall throughput.
Alongside DSpark, DeepSeek has open-sourced DeepSpec—a full-stack codebase for training and evaluating speculative decoding draft models. DeepSpec serves as the “open-source infrastructure” hosting this solution and other cutting-edge algorithm implementations, including data preparation tools, draft model implementations, training scripts, and evaluation utilities.
DeepSpec decomposes the entire workflow into three sequential stages: data preparation, training, and evaluation—where outputs from one stage serve as inputs to the next.
In the data preparation stage, users must download prompt datasets, re-generate answers using the target model’s inference engine, and construct the target cache. Notably, using the default Qwen/Qwen3-4B configuration as an example, the target cache size reaches approximately 38 TB—requiring careful assessment of storage resources before use.
The training stage can be launched via bash scripts/train/train.sh. This script invokes train.py and launches one worker per visible GPU. Users may specify a config_path to select different algorithms and target model configurations from the config/ directory. The project also supports fine-grained customization—e.g., overriding config_path, target_cache_dir, or modifying individual config fields using the --opts flag.
Hardware-wise, DeepSpec’s default configurations and scripts target an 8-GPU single-node environment. If fewer GPUs are available, users should correspondingly reduce the number of visible GPUs listed in CUDA_VISIBLE_DEVICES.
The evaluation stage is launched via bash scripts/eval/eval.sh. This script evaluates trained draft model checkpoints across multiple speculative decoding benchmark tasks, measuring acceptance behavior. Currently supported evaluation datasets include GSM8K, MATH500, AIME25, HumanEval, MBPP, LiveCodeBench, MT-Bench, Alpaca, and Arena-Hard-v2—covering diverse task types such as mathematical reasoning, code generation, conversational ability, and comprehensive question answering.
Algorithmically, DeepSpec currently ships with three built-in draft models: DSpark, DFlash, and Eagle3. Regarding target model families, the project currently supports Qwen3 and Gemma.
By open-sourcing DeepSpec, DeepSeek consolidates speculative decoding—an engineering practice previously scattered across individual research teams—into a standardized, reproducible, and extensible toolchain. For researchers and engineers aiming to accelerate inference for their own LLMs, this means they can directly train custom draft models atop a mature framework, bypassing extensive redundant infrastructure development.
Reference links:
© THE END
Reprinting requires authorization from this official account.
Submissions or media inquiries: liyazhou@jiqizhixin.com