Red-EAD/FlowBlock

TL;DR: FlowBlock is a training-free decoding framework that breaks the strictly serial block-by-block generation of diffusion LLMs (LLaDA-2.x) by letting downstream blocks start decoding from drafts of their predecessors, relying on the model's token-to-token self-correction to fix errors as the draft firms up. Self-reported results: up to 2.95×/4.01× higher throughput than LLaDA-2.1/2.0 in batched serving (1.57×/3.17× at batch 1), with accuracy actually going up 1.3 points on average across eight math/code benchmarks. Runnable code on unmodified LLaDA-2.1 weights, Apache-2.0.

The idea

Block-wise diffusion LLMs decode a block of tokens via iterative denoising, freeze it, then move to the next block — so inter-block progress is serial even though intra-block decoding is parallel. This is the main serving bottleneck of the model class.

FlowBlock's observation is that LLaDA-2.1's token-to-token (T2T) self-correction — the ability to revise already-decoded tokens — means a downstream block doesn't need a finalized predecessor, just an informative one. Block finality becomes a scheduling resource rather than a hard dependency. This is the same conceptual move as speculative/pipelined execution in CPUs: start work on a draft, correct later.

Two mechanisms implement it:

• Gated Wavefront Decoding (GWD): a bounded wavefront of active blocks (window W, default 2). A new block is admitted only when a readiness gate SPAWN_TH on the predecessor is satisfied (0.6 for math, 0.8 for code — code apparently needs firmer drafts). Active blocks are refined jointly, with T2T editing fixing draft-conditioned mistakes, and blocks commit in order under a windowed block-causal mask. The in-order commit is what preserves exact frozen-prefix KV-cache reuse — no approximate cache invalidation.
• Heterogeneous Wavefront Packing (HWP): in batched serving, each request's wavefront progresses asynchronously, but all active windows get packed into dense, shape-stable batched forwards so the GPU stays busy. This is where the bigger batched-serving speedups come from.

FlowBlock (W = 2) Block 1 Block 2 Block 3

gate: predecessor readiness ≥ SPAWN_TH Block 1 commits → exact KV reuse T2T editing fixes draft-conditioned tokens

What's there and the evidence

The repo is a full evaluation harness built on dInfer/DMax with SGLang as the model runner (compile + CUDA graphs), lm-eval task configs, and a single run_eval.sh entry point that runs FlowBlock and serial baselines back-to-back on seed-fixed subsets, dumping side-by-side summaries. No weights of its own — it runs stock inclusionAI/LLaDA2.1-mini (16B-A1B MoE).

At batch size 1, block length 32, gen length 2048, on a single 80 GB GPU:

Tokens-per-forward rises from 4.62 (LLaDA-2.1) to 7.23 — the speedup comes from genuine parallelism, not smaller forwards. Interestingly, accuracy improves (+1.3 avg, +4.9 on HumanEval+ vs LLaDA-2.1), plausibly because joint refinement over the window gives extra correction passes. Batched-serving numbers (the headline 2.95×/4.01×) live in figures referenced but not tabulated in the README text.

Caveats

• Self-reported, one model family. Everything is on LLaDA-2.x-mini; the method explicitly depends on the T2T self-correction ability of the 2.1 series (2.0 is only a baseline). Whether this transfers to other dLLMs (Dream, Mercury-style models) is untested here.
• Per-task tuning exists: spawn threshold 0.6 vs 0.8 and commit thresholds differ between math and code. Modest, but worth remembering when reading "accuracy preserved."
• Benchmarks are all short-form math/code at 2048 tokens; hard long-form generation is unexamined.
• Paper is cited but no arXiv link in the README; repo is 2 days old, 125 stars, 2 forks.
• Needs ~80 GB GPU, CUDA 12.x, pinned torch 2.8.0 + vLLM 0.10.2 (installed last, deliberately). Code benchmarks execute model-generated Python — sandbox them.

The framing — "block finality is a scheduling resource" — is the transferable insight. If self-correction becomes a standard dLLM property, wavefront scheduling like this is a natural default for serving, analogous to what continuous batching did for AR models.

Try it

git clone https://github.com/Red-EAD/FlowBlock && cd FlowBlock
conda create -n flowblock python=3.11 -y && conda activate flowblock
pip install torch==2.8.0 --index-url https://download.pytorch.org/whl/cu128
pip install -r requirements.txt
pip install vllm==0.10.2   # separate step on purpose

python dInfer/evaluations/download_hf_model.py \
  --repo_id inclusionAI/LLaDA2.1-mini --local_dir ./LLaDA2.1-mini

# FlowBlock vs serial LLaDA-2.1 on GSM8K, same seed-fixed subset
bash run_eval.sh --method flowblock,llada2.1 --task math --model-path ./LLaDA2.1-mini

Results land in outputs/<run>/summary.json with accuracy, TPS, latency, and TPF per method.