arXiv:2607.03236 · Diffusion Language Model Decoding

TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding

A training-free parallel decoder that separates token proposing from position committing — combining candidate-conditioned temporal logit contrast (TILG) with discrete proposal persistence (History Gate) while preserving the base proposal at every write.

Chengcheng Wang1* Tingzhang Luo2* Wenhao Li1 Jianyuan Guo2 Chang Xu1†
1University of Sydney  ·  2City University of Hong Kong  ·  *Equal contribution  ·  †Corresponding author
Read Paper (PDF) arXiv:2607.03236 Code (GitHub) Explore Real Traces
+3.02 pts
51.02%
LLaDA-8B Mean Accuracy
vs. 48.00% Native Confidence across 4 tasks
−11.45% NFE
67.82
Mean Evaluations per Sample
Down from 76.59 calls per completion on LLaDA-8B
+8.27 pts
49.42%
Matched-Budget Replay
vs. 41.16% Confidence-ranked at exact 62.79 NFE
512k / 512k
100%
Local Write Invariant
Every committed token matches base \(\arg\max_v p_t\)
01 · Motivation & Empirical Diagnostic

Why Scalar Confidence Is Incomplete for Parallel Commitment

In non-revising diffusion language models (DLLMs), revealing a masked position permanently conditions all subsequent denoising steps. A scalar confidence threshold \(c_{t,i} = p_t(i, \hat{x}_{t,i})\) treats a newly switched candidate identically to a candidate whose support has steadily accumulated across steps.

One-Step HOLD Revision Diagnostic

Before committing position \(i\) at step \(t\), we keep it masked for one additional forward pass while the surrounding context updates, and record whether its top-1 identity flips:

$$R_{t,i} = \mathbb{1}\!\left[\hat{x}^{\mathrm{hold}}_{t+1,i} \neq \hat{x}_{t,i}\right], \qquad g_{t,i} = \log p_t(i, \hat{x}_{t,i}) - \log p_{t-2}(i, \hat{x}_{t,i})$$
  • Discrete Persistence Matters (Panel a): Within every confidence bin on GSM8K, recently switched proposals revise significantly more often than proposals retaining the same top-1 identity for \(\ge 3\) steps.
  • Continuous Support Trend Matters (Panel b): Even after matching both current confidence and persistence, candidates with declining historical log-support (\(g_{t,i} < 0\)) revise more often than candidates with strengthening support (\(g_{t,i} > 0\)).
  • Operational Takeaway: Proposal history carries actionable signal for when a position is commit-ready—without needing any extra forward pass during inference.
One-step HOLD revision rates on GSM8K
Figure 1 · Revision rates differ at matched confidence on GSM8K. (a) Recently switched proposals revise more often than proposals stable for \(\ge 3\) steps across confidence bins. (b) At matched confidence and persistence, weakening proposals revise more often than strengthening proposals.
02 · Method & Interactive Walkthrough

How TACG Separates Token Proposal from Position Commitment

TACG computes its gate entirely from logits already produced by the current forward pass (\(O(BLV)\) overhead, zero extra model evaluations). Every committed position writes the contemporaneous base proposal \(\hat{x}_i = \arg\max_v p_t(i,v)\).

TACG Animated Architecture Pipeline
Animated Information Flow of TACG. (1) The backbone produces current logits \(z_t\) and base proposal \(\hat{x}_i\). (2) TILG contrasts \(z_t\) with the pre-update exponential moving average \(\bar{z}_{t-1}\) to score candidate-specific probability gain \(b_i\). (3) History Gate (HG) verifies proposal streak \(h_i \ge m\) (or confidence escape \(\tau_{\mathrm{esc}}\)) and promotes up to \(k_t \le K_{\max}\) top-readiness candidates. (4) Accepted positions write the unmodified base proposal \(\hat{x}_i\).
PILLAR 1 · EMA LOGIT REFERENCE

Pre-Update Trajectory Memory

TACG maintains an exponential moving average of logits across denoising steps and reads \(\bar{z}_{t-1}\) before incorporating \(z_t\):

$$\bar{z}_t = \beta \bar{z}_{t-1} + (1 - \beta) z_t, \quad \beta = 0.95$$
PILLAR 2 · TILG CONTRASTIVE SCORE

Candidate-Conditioned Support

TILG amplifies historical-to-current log-odds innovation \(z_t - \bar{z}_{t-1}\) and extracts positive probability gain at \(\hat{x}_i\):

$$q_t = \mathrm{softmax}(z_t + w(z_t - \bar{z}_{t-1}))$$ $$b_i = [q_t(i, \hat{x}_i) - p_t^{\mathrm{ref}}(i, \hat{x}_i)]_+, \;\; s_i = c_i + \lambda b_i$$
PILLAR 3 · HISTORY GATE & QUOTA

Persistence + Capped Promotion

Positions require consecutive agreement \(h_i \ge m_j\) (or saturation \(c_i \ge \tau_{\mathrm{esc}}\)), promoting at most \(k_t \le K_{\max}\) extras above \(\tau_{\mathrm{floor}}\):

$$H_i^j = \mathbb{1}[h_i \ge m_j] \lor \mathbb{1}[c_i \ge \tau_{\mathrm{esc}}]$$ $$\mathcal{E}_t = \mathrm{TopK}_{i \in \mathcal{N}_t}(s_i, k_t)$$
Interactive Gate Simulator · Matched Confidence (\(c_4 = 0.86\)), Divergent Trajectories
Sequence State:
Position A · Volatile / Late-Surge Candidate
Current Base Proposal: 'dream'
HOLD
PROPOSAL IDENTITY TRAJECTORY:
Current Confidence \(c_t\) (Base threshold \(\tau_{\mathrm{conf}} = 0.90\)) 0.86
TILG Temporal Support Gain \(b_t = [q_t(\hat{x}) - p_t^{\mathrm{ref}}(\hat{x})]_+\) +0.00
Composite Readiness Score \(s_t = c_t + \lambda b_t\) 0.86
Position B · Persistent / Reinforced Candidate
Current Base Proposal: 'reality'
COMMIT
PROPOSAL IDENTITY TRAJECTORY:
Current Confidence \(c_t\) (Base threshold \(\tau_{\mathrm{conf}} = 0.90\)) 0.86
TILG Temporal Support Gain \(b_t = [q_t(\hat{x}) - p_t^{\mathrm{ref}}(\hat{x})]_+\) +0.32
Composite Readiness Score \(s_t = c_t + \lambda b_t\) 0.97
Live Gate Hyperparameters Tuning
0.35
0.35
3
Detailed TACG and TILG Numerical Walkthrough
Figure 2 · Step-by-step numerical trace of TACG on a masked slot. (1) Pre-softmax logits \(z_t\) are observed across denoising steps. (2) Earlier logits build the EMA reference \(\bar{z}_{t-1} = z_{\mathrm{ref}}\). (3) TILG contrasts \(z_t\) with \(z_{\mathrm{ref}}\) to compute \(q_t = \mathrm{softmax}(z_t + w(z_t - z_{\mathrm{ref}}))\) and scores positive probability gain at the base proposal. (4) When base confidence is below threshold (\(c_i < \theta\)), TILG support and History Gate persistence promote the candidate while writing the unmodified base argmax token (reality).
03 · Real Experiment Trace Explorer

Watch Parallel Decoding Unfold at Identical Per-Event Commit Budgets

To isolate which positions enter the context from how many tokens are committed, our matched per-event commit-count replay forces a confidence-ranked selector to commit the exact same number of tokens at every sample, block, and step as TACG. Explore real step-by-step traces recorded from LLaDA-8B below.

Real LLaDA-8B Step-by-Step Replay Explorer
Benchmark Prompt:
Evaluation Outcome:
■ Masked Slot ■ Committed Earlier (TACG) ■ Committed Earlier (Confidence) ■ Committed at Current Step (Base) ■ Extra Promoted at Current Step (TILG)
TACG (History-Guided Selector)
Confidence-Ranked Selector (Matched Per-Event Quota)
Per-Token Telemetry Inspector · LLaDA-8B HumanEval/0

Inspect Exact Confidence \(c_i\), TILG Bonus \(b_i\), and Streak \(h_i\) at Every Step

Scrub across the 19 decoding steps of has_close_elements (256 tokens generated in 19 forward calls, 13.5 tokens/step average) to inspect the exact gate state of every committed token:

04 · Causal Mechanism Audits

Why TACG Works: Selection vs. Writing, Matched Budgets, and Reference Interventions

We audit TACG with three controlled interventions to verify that improvements stem from history-guided position selection rather than altered token writing or step-budget differences.

INTERVENTION 1 · 2×2 FACTORIZATION (LLaDA-8B)

Position Selection Drives Both Quality and Efficiency

We decouple which positions are selected from which token identity is written (base \(\arg\max_v p_t\) vs. guided \(\arg\max_v q_t\)) across all four complete benchmarks:

Mechanism Arm HumanEval MBPP GSM8K MATH500 Mean ACC ↑ Mean NFE ↓
Confidence Control 37.20 47.08 77.10 30.60 48.00 76.59
Selection Only (TACG Rule) 39.02 50.19 76.04 32.60 49.47 (+1.47) 67.87 (−11.39%)
Guided Writing Only 37.80 49.03 75.36 31.80 48.50 (+0.50) 76.36 (−0.30%)
Selection + Guided Writing 40.24 49.81 75.51 32.60 49.54 (+1.55) 67.53 (−11.83%)

Placing temporal evidence in the position selector accounts for +1.47 accuracy points and −11.39% NFE, whereas modifying written tokens alone barely alters NFE (−0.30%).

INTERVENTION 2 · MATCHED PER-EVENT COMMIT REPLAY

+8.27 Accuracy Points at Identical Per-Sample NFE

Across 141,707 commit events, we replay the exact committed-token count at every sample, block, and step using current top-confidence ranking versus TACG's history-guided selector:

Selector Policy HumanEval MBPP GSM8K MATH500 Mean ACC ↑ Mean NFE ↓
History-Guided (TACG) 39.02 50.19 76.27 32.20 49.42 (+8.27) 62.79
Confidence-Ranked Replay 34.76 38.52 67.55 23.80 41.16 62.79
Position-Permuted History 34.15 36.19 65.73 23.40 39.87 62.79

With identical per-sample NFE (44.02 on HumanEval, 69.39 on MBPP, 58.33 on GSM8K, 79.43 on MATH500), history-guided position choice outperforms confidence ranking on every benchmark.

INTERVENTION 3 · PAIRED LOGIT-REFERENCE DIAGNOSTIC (DREAM MATH500)

Altering Historical Reference Shifts Commit Trajectories While Local Writes Stay 100% Anchored

On 500 paired MATH500 prompts (Dream-7B), we compare the observed EMA reference against three counterfactual references while holding confidence and History Gate streaks fixed:

  • P: Within-Sample Position-Permuted Reference (Sattolo cycle across slots): alters 480 / 500 commit traces and 101 / 500 final texts.
  • F: Frozen First-Step Reference (\(\bar{z}_0\) held fixed): alters 249 / 500 commit traces and 52 / 500 final texts.
  • N: No Temporal Support (\(b_i \equiv 0\)): alters 417 / 500 commit traces and 68 / 500 final texts.
  • Local Write Invariant: Across all four arms, 512,000 / 512,000 recorded token writes exactly match their contemporaneous base proposal \(\hat{x}_i\).
Dream MATH500 Logit-Reference Intervention Distributions
05 · Main Benchmarks & System Profiles

Quality–Cost Operating Points Across Three Decoder Families

We evaluate complete task splits on LLaDA-8B (len 256), Dream-7B (len 256), and LLaDA2-Mini (block-diffusion at len 1024 and 2048), alongside synchronized H200 latency and batch-scaling profiles.

Method (LLaDA-8B, len 256) HumanEval (n=164) MBPP (n=257) GSM8K (n=1,319) MATH500 (n=500) Four-Task Mean
ACC ↑NFE ↓ ACC ↑NFE ↓ ACC ↑NFE ↓ ACC ↑NFE ↓ ACC ↑NFE ↓
Native Confidence 37.2052.28 47.0883.34 77.1074.26 30.6096.48 48.0076.59
AHD (Anchor History-Stable) 39.6351.89 45.9185.46 75.2873.97 30.4095.28 47.8176.65
DSB (Dynamic Sliding Block) 39.0250.46 48.6481.49 76.3572.98 33.4095.68 49.3575.15
KLASS (Fixed Distribution-KL) 39.6392.13 48.25134.82 77.63134.45 30.60152.42 49.03128.45
TACG (Ours, Task-Specific) 42.0745.71 50.9774.42 77.0367.12 34.0084.04 51.02 (+3.02) 67.82 (−11.45%)
80 Candidate Operating Points Sweep across LLaDA and Dream
Complete 80-Run Candidate Sweep (44,800 Generated Samples). Measured quality–NFE operating points across 10 shared-vector TACG configurations per backbone on HumanEval, MBPP, GSM8K, and MATH500, overlaid with Native Confidence, AHD, DSB, and KLASS. Solid orange lines connect non-dominated TACG points; dashed lines show the controlled \(\lambda \in \{0.15, 0.25, 0.50, 0.75\}\) slice.
Single-parameter sensitivity on LLaDA HumanEval
Single-Parameter Sensitivity (LLaDA HumanEval). Varying support weight \(\lambda \in [0.15, 0.75]\) and confidence floor \(\tau_{\mathrm{floor}} \in [0.735, 0.745]\) one control at a time. All tested slices remain below Native Confidence NFE.
Prompt-level NFE savings and History Gate windows
Prompt-Level NFE Savings Across 6 History Gate Settings (984 Paired Records). Comparing streak windows \(m \in \{2, 3\}\) across Low, Moderate, and Strong support vectors on HumanEval.
Length and Batch Scaling on H200 GPU
Length–Batch Scaling on H200 (bfloat16). Decode latency, output throughput (positions/s), batch model calls (NFE), and peak allocated GPU memory across generation lengths \(\{256, 512, 1024, 2048\}\) and batch sizes \(\{1, 4, 8\}\).
3D Throughput and Memory Surface
Throughput–Memory Surface Across Length & Batch Grid. TACG reaches up to 333.0 output positions/s at batch 8 (len 256) and 261.1 positions/s at length 1024 with bounded memory overhead.
06 · Shareable Animated Visualizations

High-Resolution GIFs for README & Presentations

In addition to the interactive simulators above, we provide standalone looping GIF animations under docs/assets/animations/ ready to embed in GitHub READMEs, slides, or social threads.

TACG Mechanism Animated GIF
Animation 1 · tacg_mechanism.gif — Visualizes how two candidates with identical confidence (\(c_4 = 0.86\)) diverge in TACG: Position A (switched identity, \(h_A = 1\)) is held by the History Gate, while Position B (persistent streak \(h_B = 4\), rising TILG support \(b_B = +0.29\)) is promoted and writes its base proposal.
TACG vs Confidence Replay Animated GIF
Animation 2 · tacg_vs_confidence_replay.gif — Real 43-step LLaDA-8B decoding trajectory on GSM8K (#6) under matched per-event commit counts. TACG preserves coherent multi-step reasoning (\(\boxed{260}\) ✓), whereas confidence-ranked selection suffers premature context corruption (\(800\) ✗) at the exact same NFE.
07 · Citation & Open-Source Artifacts

BibTeX & Reproducibility

All evaluation scripts, task configs, and audited result manifests for LLaDA-8B, Dream-7B, and LLaDA2-Mini are available in our public repository. 

@article{wang2026tacg,
  title   = {TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding},
  author  = {Chengcheng Wang and Tingzhang Luo and Wenhao Li and Jianyuan Guo and Chang Xu},
  journal = {arXiv preprint arXiv:2607.03236},
  year    = {2026},
  url     = {https://arxiv.org/abs/2607.03236}
}