TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding

01 · Motivation & Empirical Diagnostic

Why Scalar Confidence Is Incomplete for Parallel Commitment

In non-revising diffusion language models (DLLMs), revealing a masked position permanently conditions all subsequent denoising steps. A scalar confidence threshold \(c_{t,i} = p_t(i, \hat{x}_{t,i})\) treats a newly switched candidate identically to a candidate whose support has steadily accumulated across steps.

One-Step HOLD Revision Diagnostic

Before committing position \(i\) at step \(t\), we keep it masked for one additional forward pass while the surrounding context updates, and record whether its top-1 identity flips:

R_{t,i} = \mathbb{1}\!\left[\hat{x}^{\mathrm{hold}}_{t+1,i} \neq \hat{x}_{t,i}\right], \qquad g_{t,i} = \log p_t(i, \hat{x}_{t,i}) - \log p_{t-2}(i, \hat{x}_{t,i})

Discrete Persistence Matters (Panel a): Within every confidence bin on GSM8K, recently switched proposals revise significantly more often than proposals retaining the same top-1 identity for \(\ge 3\) steps.
Continuous Support Trend Matters (Panel b): Even after matching both current confidence and persistence, candidates with declining historical log-support (\(g_{t,i} < 0\)) revise more often than candidates with strengthening support (\(g_{t,i} > 0\)).
Operational Takeaway: Proposal history carries actionable signal for when a position is commit-ready—without needing any extra forward pass during inference.

Figure 1 · Revision rates differ at matched confidence on GSM8K. (a) Recently switched proposals revise more often than proposals stable for \(\ge 3\) steps across confidence bins. (b) At matched confidence and persistence, weakening proposals revise more often than strengthening proposals.

02 · Method & Interactive Walkthrough

How TACG Separates Token Proposal from Position Commitment

TACG computes its gate entirely from logits already produced by the current forward pass (\(O(BLV)\) overhead, zero extra model evaluations). Every committed position writes the contemporaneous base proposal \(\hat{x}_i = \arg\max_v p_t(i,v)\).

Animated Information Flow of TACG. (1) The backbone produces current logits \(z_t\) and base proposal \(\hat{x}_i\). (2) TILG contrasts \(z_t\) with the pre-update exponential moving average \(\bar{z}_{t-1}\) to score candidate-specific probability gain \(b_i\). (3) History Gate (HG) verifies proposal streak \(h_i \ge m\) (or confidence escape \(\tau_{\mathrm{esc}}\)) and promotes up to \(k_t \le K_{\max}\) top-readiness candidates. (4) Accepted positions write the unmodified base proposal \(\hat{x}_i\).

PILLAR 1 · EMA LOGIT REFERENCE

Pre-Update Trajectory Memory

TACG maintains an exponential moving average of logits across denoising steps and reads \(\bar{z}_{t-1}\) before incorporating \(z_t\):

\bar{z}_t = \beta \bar{z}_{t-1} + (1 - \beta) z_t, \quad \beta = 0.95

PILLAR 2 · TILG CONTRASTIVE SCORE

Candidate-Conditioned Support

TILG amplifies historical-to-current log-odds innovation \(z_t - \bar{z}_{t-1}\) and extracts positive probability gain at \(\hat{x}_i\):

q_t = \mathrm{softmax}(z_t + w(z_t - \bar{z}_{t-1}))$$ $$b_i = [q_t(i, \hat{x}_i) - p_t^{\mathrm{ref}}(i, \hat{x}_i)]_+, \;\; s_i = c_i + \lambda b_i

PILLAR 3 · HISTORY GATE & QUOTA

Persistence + Capped Promotion

Positions require consecutive agreement \(h_i \ge m_j\) (or saturation \(c_i \ge \tau_{\mathrm{esc}}\)), promoting at most \(k_t \le K_{\max}\) extras above \(\tau_{\mathrm{floor}}\):

H_i^j = \mathbb{1}[h_i \ge m_j] \lor \mathbb{1}[c_i \ge \tau_{\mathrm{esc}}]$$ $$\mathcal{E}_t = \mathrm{TopK}_{i \in \mathcal{N}_t}(s_i, k_t)

Sequence State:

Position A · Volatile / Late-Surge Candidate

Current Base Proposal: 'dream'

HOLD

PROPOSAL IDENTITY TRAJECTORY:

Current Confidence \(c_t\) (Base threshold \(\tau_{\mathrm{conf}} = 0.90\)) 0.86

TILG Temporal Support Gain \(b_t = [q_t(\hat{x}) - p_t^{\mathrm{ref}}(\hat{x})]_+\) +0.00

Composite Readiness Score \(s_t = c_t + \lambda b_t\) 0.86

Position B · Persistent / Reinforced Candidate

Current Base Proposal: 'reality'

COMMIT

PROPOSAL IDENTITY TRAJECTORY:

Current Confidence \(c_t\) (Base threshold \(\tau_{\mathrm{conf}} = 0.90\)) 0.86

TILG Temporal Support Gain \(b_t = [q_t(\hat{x}) - p_t^{\mathrm{ref}}(\hat{x})]_+\) +0.32

Composite Readiness Score \(s_t = c_t + \lambda b_t\) 0.97

Live Gate Hyperparameters Tuning

Contrast Weight: w 0.35

Support Weight: λ 0.35

History Streak: m 3

Detailed TACG and TILG Numerical Walkthrough

Figure 2 · Step-by-step numerical trace of TACG on a masked slot. (1) Pre-softmax logits \(z_t\) are observed across denoising steps. (2) Earlier logits build the EMA reference \(\bar{z}_{t-1} = z_{\mathrm{ref}}\). (3) TILG contrasts \(z_t\) with \(z_{\mathrm{ref}}\) to compute \(q_t = \mathrm{softmax}(z_t + w(z_t - z_{\mathrm{ref}}))\) and scores positive probability gain at the base proposal. (4) When base confidence is below threshold (\(c_i < \theta\)), TILG support and History Gate persistence promote the candidate while writing the unmodified base argmax token (reality).

03 · Real Experiment Trace Explorer

Watch Parallel Decoding Unfold at Identical Per-Event Commit Budgets

To isolate which positions enter the context from how many tokens are committed, our matched per-event commit-count replay forces a confidence-ranked selector to commit the exact same number of tokens at every sample, block, and step as TACG. Explore real step-by-step traces recorded from LLaDA-8B below.

Benchmark Prompt:

Evaluation Outcome:

■ Masked Slot ■ Committed Earlier (TACG) ■ Committed Earlier (Confidence) ■ Committed at Current Step (Base) ■ Extra Promoted at Current Step (TILG)

TACG (History-Guided Selector)

Confidence-Ranked Selector (Matched Per-Event Quota)

Per-Token Telemetry Inspector · LLaDA-8B HumanEval/0

Inspect Exact Confidence \(c_i\), TILG Bonus \(b_i\), and Streak \(h_i\) at Every Step

Scrub across the 19 decoding steps of has_close_elements (256 tokens generated in 19 forward calls, 13.5 tokens/step average) to inspect the exact gate state of every committed token:

04 · Causal Mechanism Audits

Why TACG Works: Selection vs. Writing, Matched Budgets, and Reference Interventions

We audit TACG with three controlled interventions to verify that improvements stem from history-guided position selection rather than altered token writing or step-budget differences.

INTERVENTION 1 · 2×2 FACTORIZATION (LLaDA-8B)

Position Selection Drives Both Quality and Efficiency

We decouple which positions are selected from which token identity is written (base \(\arg\max_v p_t\) vs. guided \(\arg\max_v q_t\)) across all four complete benchmarks:

Mechanism Arm	HumanEval	MBPP	GSM8K	MATH500	Mean ACC ↑	Mean NFE ↓
Confidence Control	37.20	47.08	77.10	30.60	48.00	76.59
Selection Only (TACG Rule)	39.02	50.19	76.04	32.60	49.47 (+1.47)	67.87 (−11.39%)
Guided Writing Only	37.80	49.03	75.36	31.80	48.50 (+0.50)	76.36 (−0.30%)
Selection + Guided Writing	40.24	49.81	75.51	32.60	49.54 (+1.55)	67.53 (−11.83%)

Placing temporal evidence in the position selector accounts for +1.47 accuracy points and −11.39% NFE, whereas modifying written tokens alone barely alters NFE (−0.30%).

INTERVENTION 2 · MATCHED PER-EVENT COMMIT REPLAY

+8.27 Accuracy Points at Identical Per-Sample NFE

Across 141,707 commit events, we replay the exact committed-token count at every sample, block, and step using current top-confidence ranking versus TACG's history-guided selector:

Selector Policy	HumanEval	MBPP	GSM8K	MATH500	Mean ACC ↑	Mean NFE ↓
History-Guided (TACG)	39.02	50.19	76.27	32.20	49.42 (+8.27)	62.79
Confidence-Ranked Replay	34.76	38.52	67.55	23.80	41.16	62.79
Position-Permuted History	34.15	36.19	65.73	23.40	39.87	62.79

With identical per-sample NFE (44.02 on HumanEval, 69.39 on MBPP, 58.33 on GSM8K, 79.43 on MATH500), history-guided position choice outperforms confidence ranking on every benchmark.

INTERVENTION 3 · PAIRED LOGIT-REFERENCE DIAGNOSTIC (DREAM MATH500)

Altering Historical Reference Shifts Commit Trajectories While Local Writes Stay 100% Anchored

On 500 paired MATH500 prompts (Dream-7B), we compare the observed EMA reference against three counterfactual references while holding confidence and History Gate streaks fixed:

P: Within-Sample Position-Permuted Reference (Sattolo cycle across slots): alters 480 / 500 commit traces and 101 / 500 final texts.
F: Frozen First-Step Reference (\(\bar{z}_0\) held fixed): alters 249 / 500 commit traces and 52 / 500 final texts.
N: No Temporal Support (\(b_i \equiv 0\)): alters 417 / 500 commit traces and 68 / 500 final texts.
Local Write Invariant: Across all four arms, 512,000 / 512,000 recorded token writes exactly match their contemporaneous base proposal \(\hat{x}_i\).

Dream MATH500 Logit-Reference Intervention Distributions

05 · Main Benchmarks & System Profiles

Quality–Cost Operating Points Across Three Decoder Families

We evaluate complete task splits on LLaDA-8B (len 256), Dream-7B (len 256), and LLaDA2-Mini (block-diffusion at len 1024 and 2048), alongside synchronized H200 latency and batch-scaling profiles.

Method (LLaDA-8B, len 256)	HumanEval (n=164)		MBPP (n=257)		GSM8K (n=1,319)		MATH500 (n=500)		Four-Task Mean
Method (LLaDA-8B, len 256)	ACC ↑	NFE ↓	ACC ↑	NFE ↓	ACC ↑	NFE ↓	ACC ↑	NFE ↓	ACC ↑	NFE ↓
Native Confidence	37.20	52.28	47.08	83.34	77.10	74.26	30.60	96.48	48.00	76.59
AHD (Anchor History-Stable)	39.63	51.89	45.91	85.46	75.28	73.97	30.40	95.28	47.81	76.65
DSB (Dynamic Sliding Block)	39.02	50.46	48.64	81.49	76.35	72.98	33.40	95.68	49.35	75.15
KLASS (Fixed Distribution-KL)	39.63	92.13	48.25	134.82	77.63	134.45	30.60	152.42	49.03	128.45
TACG (Ours, Task-Specific)	42.07	45.71	50.97	74.42	77.03	67.12	34.00	84.04	51.02 (+3.02)	67.82 (−11.45%)

Backbone & Decoder	HumanEval ACC / NFE	MBPP ACC / NFE	GSM8K ACC / NFE	MATH500 ACC / NFE	Mean ACC ↑	Mean NFE ↓
LLaDA-8B · Native Confidence	37.20% / 52.28	47.08% / 83.34	77.10% / 74.26	30.60% / 96.48	48.00%	76.59
LLaDA-8B · TACG	42.07% / 45.71	50.97% / 74.42	77.03% / 67.12	34.00% / 84.04	51.02%	67.82
Dream-7B · Native Confidence	47.56% / 51.19	63.42% / 82.46	73.16% / 74.63	39.00% / 94.60	55.79%	75.72
Dream-7B · TACG	50.00% / 50.80	63.04% / 93.53	73.62% / 74.26	39.60% / 91.14	56.56%	77.43
LLaDA2-Mini (1024) · Baseline	84.15% / 67.38	80.09% / 77.93	87.64% / 147.90	64.80% / 294.77	79.17%	146.99
LLaDA2-Mini (1024) · TACG	85.98% / 69.66	80.56% / 75.36	88.17% / 148.11	67.40% / 290.79	80.53%	145.98

LLaDA2-Mini Generation Budget	HumanEval (164)	MBPP (427)	GSM8K (1,319)	MATH500 (500)	Mean ACC ↑	Mean NFE ↓
Length 1024 · Baseline	84.15% (67.38)	80.09% (77.93)	87.64% (147.90)	64.80% (294.77)	79.17%	146.99
Length 1024 · TACG	85.98% (69.66)	80.56% (75.36)	88.17% (148.11)	67.40% (290.79)	80.53% (+1.36)	145.98 (−1.01)
Length 2048 · Baseline	84.15% (71.51)	80.09% (78.12)	88.32% (176.97)	75.20% (456.83)	81.94%	195.86
Length 2048 · TACG	85.37% (78.32)	80.33% (79.72)	88.63% (171.18)	76.40% (451.52)	82.68% (+0.74)	195.19 (−0.67)

Method & Operating Setting (H200 GPU)	HumanEval (ms)	MBPP (ms)	GSM8K (ms)	MATH500 (ms)	Mean Decode ms ↓	Speedup ↑	Quality / Mean NFE
Native Confidence (Reference)	1391.3	2073.7	1975.5	2341.0	1945.4 ms	1.000×	48.00% / 76.59
TACG (Shared Across-Task Vector)	1330.5	1949.8	1784.9	2119.7	1796.2 ms	1.083×	49.49% / 67.62
TACG (Timing-Study Vector)	1383.0	1780.1	1836.4	2119.7	1779.8 ms	1.093×	50.47% / 66.82

80 Candidate Operating Points Sweep across LLaDA and Dream

Complete 80-Run Candidate Sweep (44,800 Generated Samples). Measured quality–NFE operating points across 10 shared-vector TACG configurations per backbone on HumanEval, MBPP, GSM8K, and MATH500, overlaid with Native Confidence, AHD, DSB, and KLASS. Solid orange lines connect non-dominated TACG points; dashed lines show the controlled \(\lambda \in \{0.15, 0.25, 0.50, 0.75\}\) slice.

Single-parameter sensitivity on LLaDA HumanEval

Single-Parameter Sensitivity (LLaDA HumanEval). Varying support weight \(\lambda \in [0.15, 0.75]\) and confidence floor \(\tau_{\mathrm{floor}} \in [0.735, 0.745]\) one control at a time. All tested slices remain below Native Confidence NFE.

Prompt-level NFE savings and History Gate windows

Prompt-Level NFE Savings Across 6 History Gate Settings (984 Paired Records). Comparing streak windows \(m \in \{2, 3\}\) across Low, Moderate, and Strong support vectors on HumanEval.

Length–Batch Scaling on H200 (bfloat16). Decode latency, output throughput (positions/s), batch model calls (NFE), and peak allocated GPU memory across generation lengths \(\{256, 512, 1024, 2048\}\) and batch sizes \(\{1, 4, 8\}\).

Throughput–Memory Surface Across Length & Batch Grid. TACG reaches up to 333.0 output positions/s at batch 8 (len 256) and 261.1 positions/s at length 1024 with bounded memory overhead.

06 · Shareable Animated Visualizations

High-Resolution GIFs for README & Presentations

In addition to the interactive simulators above, we provide standalone looping GIF animations under docs/assets/animations/ ready to embed in GitHub READMEs, slides, or social threads.

Animation 1 · tacg_mechanism.gif — Visualizes how two candidates with identical confidence (\(c_4 = 0.86\)) diverge in TACG: Position A (switched identity, \(h_A = 1\)) is held by the History Gate, while Position B (persistent streak \(h_B = 4\), rising TILG support \(b_B = +0.29\)) is promoted and writes its base proposal.

Animation 2 · tacg_vs_confidence_replay.gif — Real 43-step LLaDA-8B decoding trajectory on GSM8K (#6) under matched per-event commit counts. TACG preserves coherent multi-step reasoning (\(\boxed{260}\) ✓), whereas confidence-ranked selection suffers premature context corruption (\(800\) ✗) at the exact same NFE.

07 · Citation & Open-Source Artifacts

BibTeX & Reproducibility

All evaluation scripts, task configs, and audited result manifests for LLaDA-8B, Dream-7B, and LLaDA2-Mini are available in our public repository.

@article{wang2026tacg,
  title   = {TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding},
  author  = {Chengcheng Wang and Tingzhang Luo and Wenhao Li and Jianyuan Guo and Chang Xu},
  journal = {arXiv preprint arXiv:2607.03236},
  year    = {2026},
  url     = {https://arxiv.org/abs/2607.03236}
}