Concertina: Embodying Manipulation and Cognition in One Diffusion Vision-Language Model

SECTION 01 PARADIGM SHIFT · FROM ATTACHED VLA TO NATIVE eVLM 范式演进 · 从外挂式 VLA 到原生具身视觉语言模型 (eVLM)

Why Embodying Control Requires Rethinking the VLM Interface 为什么具身操作需要重新审视视觉语言模型的动作接口？

Existing Vision-Language-Action (VLA) systems force an uncomfortable architectural compromise: either quantize smooth motor trajectories into discrete vocabulary tokens that suffer from severe bit-flip discontinuities, or attach a separate hundred-million-parameter diffusion/flow policy head outside the VLM. Concertina establishes a third paradigm—the Embodied Vision-Language Model (eVLM).

现有视觉-语言-动作（VLA）系统往往陷入两难的架构妥协：要么将平滑的连续机械臂轨迹离散化为词表 Token（导致单 Token 翻转即可破坏整段轨迹几何），要么在 VLM 外部嫁接数亿参数的独立扩散策略网络（割裂高层语义与底层时序耦合）。Concertina 提出了第三种原生范式——具身视觉语言模型（eVLM）。

Bottleneck I · Discrete Tokenization 瓶颈一 · 离散词元化失真

Quantization Cliffs & Non-Local Bit Flips

量化阶梯悬崖与非局部序列崩溃

Uniform 256-bin tokenizers (RT-2, OpenVLA) exhibit a 25.14× single-token perturbation ratio ($p_{95}/p_{05}=0.1972\text{ vs. }0.0078\text{ rad/m}$). Spectral DCT + BPE tokenizers (FAST+, MolmoAct2) turn 82.2% of single-token flips into invalid trajectory shapes and perturb all 30/30 timesteps simultaneously.

256 桶均匀量化（RT-2、OpenVLA）的单 Token 扰动动态范围比高达 25.14×；而频域 DCT+BPE 编码（FAST+、MolmoAct2）在单 Token 替换下会导致 82.2% 的解码序列维度失配，且有效解码序列中 30/30 个时间步全部遭到全局污染。

EGC = 25.14× / 82.2% Invalid

Bottleneck II · Attached Policy Experts 瓶颈二 · 外挂重型动作专家

Decoupled Stacks & 16%–35% Parameter Overhead

双栈割裂与 16%–35% 额外参数开销

Attached continuous policies ($\pi_0$, $\pi_{0.5}$, SmolVLA, X-VLA) allocate 16.5% to 35.1% of total active model capacity (`100M`–`860M` parameters across 16–24 external transformer blocks) to a separate action expert, relegating the VLM to a static feature extractor.

外挂式连续控制模型（$\pi_0$、$\pi_{0.5}$、SmolVLA、X-VLA）需额外部署 16–24 层独立 Transformer 动作专家，消耗全模型 16.5% 至 35.1% 的活跃参数（1 亿至 8.6 亿参数），使 VLM 退化为静态前缀编码器。

ASCR = 16.51% – 35.10%

Concertina Paradigm · Native eVLM Concertina 范式 · 原生单主干 eVLM

Continuous Shared-Backbone Trajectory Synthesis

共享注意力主干内的连续轨迹合成

Concertina maps noisy continuous actions into the 36-layer VLM token stream using only position-wise sinusoidal-affine interfaces (9.49M params, 0.21% ASCR). All cross-timestep coupling (4.10% NLR, peaking at 9.93% in Layer 27) is computed natively inside the shared VLM attention layers.

Concertina 仅通过逐位置正弦-仿射接口（9.49M 参数，0.21% ASCR）将连续噪声动作直接嵌入 36 层 VLM 词元流。接口自身跨步耦合为 0%，所有长程时序协调（4.10% NLR，第 27 层达 9.93% 峰值）均在共享 VLM 内部涌现。

ASCR = 0.21% · EGC = 1.00×

Comparison of VLA Paradigms vs Concertina eVLM

FIGURE 02

Architectural taxonomy across five embodied modeling paradigms. Unlike discrete autoregressive VLAs (RT-2, OpenVLA), parallel discrete decoders ( OpenVLA-OFT, Discrete Diffusion VLA), attached diffusion/flow experts (Octo, $\pi_0$, SmolVLA), or hybrid discrete-continuous stacks (HybridVLA), Concertina embeds continuous masked diffusion trajectory tokens directly inside the shared VLM transformer with blockwise causal-bidirectional attention and reusing KV caches.

五类具身建模范式的架构对比。 不同于离散自回归 VLA（RT-2、OpenVLA）、并行离散解码器（OpenVLA-OFT）、外挂扩散/流匹配专家（Octo、$\pi_0$、SmolVLA）或双套混合系统（HybridVLA），Concertina 将连续掩码扩散动作词元直接置入共享 VLM 主干，通过分块因果-双向注意力与持久化 KV 缓存实现单主干感知-认知-控制闭环。

Table 1 · Architectural Comparison Along Ten Key Design Axes

表 1 · 十维核心架构设计对比（VLA 诸范式 vs. Concertina eVLM）

Full Manuscript Table 1 对应论文正文 Table 1

Method / Family方法 / 家族	Cont. Action连续动作保真	Shared VLM Gen.主干原生生成	No Heavy Head无重型外挂头	Single Objective单一动作目标	Intra-Chunk BiDir块内双向协调	Inter-Block Causal块间因果建模	KV Cache ReuseKV 缓存复用	1-Pass Block单步分块解码	Iter. Refine迭代精化能力	Risk Budget风险自适应预算
RT-2 / OpenVLA / FAST+	✗	✓	✓	✓	✗	✓	✓	✗	✗	✗
OpenVLA-OFT ( Discrete)	✗	✓	✓	✓	✓	✗	✗	✓	✗	✗
Discrete Diffusion VLA	✗	✓	✓	✓	✓	✗	✗	✓	✓	✗
Octo / $\pi_0$ / $\pi_{0.5}$ / SmolVLA / DreamVLA	✓	✗	✗	✓	✓	✗	✗	✗	✓	✗
HybridVLA (Dual AR + Diffusion)	✓	✓	✓	✗	✓	✗	✗	✗	✓	✗
Concertina (Ours · Native eVLM)	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓

SECTION 02 METHODOLOGY · BLOCKWISE CAUSAL-BIDIRECTIONAL DIFFUSION RECONSTRUCTION 核心方法 · 分块因果-双向扩散重建 (BCR) 与风险引导解码

Concertina Architecture: Continuous Trajectory Diffusion Inside One VLM Concertina 架构：单一视觉语言主干内的分块连续扩散重建

Built upon the 4.46B-parameter BARD-VL-B4 backbone (Qwen3-4B language tower with 36 layers, hidden width $d=2560$, and SigLIP2-So400M vision encoder), Concertina partitions an $H$-step horizon into $M$ sequential blocks of $K$ continuous action tokens ($H = M \times K$, with default $K=10$).

Concertina 基于 4.46B 参数的 BARD-VL-B4 多模态主干（36 层 Qwen3-4B 语言塔，隐藏维度 $d=2560$，搭配 SigLIP2-So400M 视觉编码器），将长度为 $H$ 的未来动作轨迹划分为 $M$ 个顺序动作块，每块包含 $K$ 个连续动作词元（$H = M \times K$，默认块大小 $K=10$）。

FIGURE 01

Overview of the Concertina eVLM architecture. Multi-view camera observations, language instructions, and robot proprioception form the multimodal prefix $\mathcal{C}$. During training, clean and noisy action blocks are interleaved in a dual-stream sequence under a custom 4D attention mask $\mathcal{M}_{\mathrm{train}}$. At inference, finalized blocks are committed to the persistent KV cache while the active block is decoded in a single parallel pass or selectively refined via risk-guided re-masking.

Concertina eVLM 整体架构图。 多视角相机图像、自然语言指令与本体状态共同构成多模态前缀 $\mathcal{C}$。训练阶段通过定制 4D 注意力掩码 $\mathcal{M}_{\mathrm{train}}$ 在单次前向中交织干净上下文流与带噪重建流；推理阶段将已生成块固化至持久 KV 缓存，对当前活跃块支持单步并行解码（6.58 ms/步）或基于不确定度触发风险引导迭代精化。

01 Canonical 20D Embodiment-Indexed Space & Affine Interfaces 20 维统一规范动作空间与轻量正弦-仿射接口

Heterogeneous single-arm (`7D`, $m^{(e)}=1_{1:7}$) and bimanual (`14D`, $m^{(e)}=1_{1:14}$) robots are embedded into a unified $D=20$ canonical action vector $\mathbf{a}_{t,i} \in \mathbb{R}^{20}$. Position-wise encoder $E_\phi(\tilde{\mathbf{a}}, \tau) \in \mathbb{R}^{2560}$ and decoder $D_\psi(\mathbf{h}) \in \mathbb{R}^{20}$ add only 9.49M parameters (`0.21%` ASCR) with zero cross-timestep mixing ($0\%$ NLR).

$\tilde{\mathbf{a}}_{t,i} = (1 - \tau_{b,i})\,\mathbf{a}_{t,i} + \tau_{b,i}\,\boldsymbol{\epsilon}_{t,i}, \quad \mathbf{e}^{\mathrm{act}}_{t,i} = E_\phi(\tilde{\mathbf{a}}_{t,i}, \tau_{b,i}) \in \mathbb{R}^{2560}$

通过具身维度掩码 $\mathbf{m}^{(e)} \in \{0,1\}^{20}$ 将单臂（7D）与双臂（14D）机器人统一嵌入 $D=20$ 维规范动作空间。逐位置动作编码器 $E_\phi$ 与解码器 $D_\psi$ 仅含 9.49M 参数（占全模型 0.21%），其自身跨时间步耦合严格为 0%，迫使所有时序规划完全在共享 VLM 主干内完成。

02 Dual-Stream Interleaved 4D Attention Topology 双流交织训练序列与 4D 分块因果-双向掩码

To prevent ground-truth leakage while preserving exact causal conditioning on past blocks, training interleaves clean blocks $\mathcal{B}^{\mathrm{clean}}_b$ and noisy blocks $\mathcal{B}^{\mathrm{noisy}}_b$: $\mathcal{S}_{\mathrm{train}} = [\mathcal{C}, \mathcal{B}^{\mathrm{clean}}_0, \mathcal{B}^{\mathrm{noisy}}_0, \dots, \mathcal{B}^{\mathrm{clean}}_{M-1}, \mathcal{B}^{\mathrm{noisy}}_{M-1}]$. Noisy tokens in block $b$ attend bidirectionally within $\mathcal{B}^{\mathrm{noisy}}_b$ and causally to $\mathcal{C} \cup \mathcal{B}^{\mathrm{clean}}_{ $\mathcal{M}_{\mathrm{train}}(q \in \mathcal{B}^{\mathrm{noisy}}_b, k) = \mathbb{I}\big[k \in \mathcal{C} \;\lor\; k \in \mathcal{B}^{\mathrm{clean}}_{

为防止同块真值泄漏并保持严格的前序因果条件，训练序列将干净动作流与带噪重建流交织排列：$\mathcal{S}_{\mathrm{train}} = [\mathcal{C}, \mathcal{B}^{\mathrm{clean}}_0, \mathcal{B}^{\mathrm{noisy}}_0, \dots]$。第 $b$ 个带噪块内部具备全双向注意力以协调局部短程几何，同时因果地关注前缀 $\mathcal{C}$ 及前序干净块 $\mathcal{B}^{\mathrm{clean}}_{ $\mathcal{M}_{\mathrm{train}}(q \in \mathcal{B}^{\mathrm{noisy}}_b, k) = \mathbb{I}\big[k \in \mathcal{C} \;\lor\; k \in \mathcal{B}^{\mathrm{clean}}_{

03 Physically Weighted Objective & Probability-Valued Gripper BCE 物理坐标加权回归与概率值夹爪交叉熵目标

Cartesian translations ($\sim 10^{-2}$ m), rotations ($\sim 10^{-1}$ rad), and discrete grippers span incompatible physical scales. We apply coordinate-group weights $\omega_d = (500, 10, 1)$ on continuous pose channels (`+1.431` CALVIN AvgLen) and supervise gripper closure with probability-valued Binary Cross-Entropy (`+0.427` AvgLen, `+11.2 pp` SR@5 over Smooth-$L_1$).

$\mathcal{L}_{\mathrm{act}} = \frac{1}{\sum m^{(e)}_d}\sum_{b,i}\Big[\sum_{d \in \mathcal{I}_{\mathrm{cont}}} \omega_d\,\ell_\beta(\hat{a}_d - a_d) + \sum_{d \in \mathcal{I}_{\mathrm{grip}}} \mathrm{BCE}(\sigma(\hat{z}_d), y_d)\Big]$

笛卡尔位移（$\sim 10^{-2}$ m）、旋转欧拉角（$\sim 10^{-1}$ rad）与二值夹爪存在显著物理量纲差异。我们引入物理坐标组权重 $\omega_d = (500, 10, 1)$（在 CALVIN 上带来 `+1.431` AvgLen 跃升），并对夹爪通道采用概率值二元交叉熵（Prob-BCE）替代回归损失（在 8,000 条序列因子实验上带来 `+0.427` AvgLen 与 `+11.2 pp` SR@5 提升）。

04 Single-Objective Shared-Space Discipline (Why Aux Heads Fail) 共享主干单一相对目标准则（反事实梯度冲突分析）

Because action tokens share all 36 VLM layers, auxiliary objectives that are harmless in detached heads cause catastrophic interference here. Adding a $0.5\times$ auxiliary absolute-action head induces an average gradient cosine of $-0.891$ (with 99% of slots negative), collapsing CALVIN AvgLen from 2.970 to 0.237 (`-92.0%`).

$\cos\big(\nabla_{\mathbf{h}} \mathcal{L}_{\mathrm{rel}},\, \nabla_{\mathbf{h}} \mathcal{L}_{\mathrm{abs}}\big) = -0.891 \;\Longrightarrow\; \text{CALVIN AvgLen: } 2.970 \to 0.237$

由于动作词元贯穿全部 36 层共享 VLM，在外挂头上看似无害的辅助损失会在主干内引发剧烈梯度对抗。反事实实验表明，仅引入 $0.5\times$ 绝对动作辅助头，就会在共享隐藏层上产生平均 $-0.891$ 的负余弦相似度（99% 时间槽为负），导致 CALVIN 性能从 2.970 暴跌至 0.237（跌幅达 92%）。

$\cos\big(\nabla_{\mathbf{h}} \mathcal{L}_{\mathrm{rel}},\, \nabla_{\mathbf{h}} \mathcal{L}_{\mathrm{abs}}\big) = -0.891 \;\Longrightarrow\; \text{CALVIN AvgLen: } 2.970 \to 0.237$

05 KV-Cached 1-Pass Decoding & Risk-Guided Refinement KV 缓存单步并行解码与风险引导选择性精化

At inference, the multimodal prefix $\mathcal{C}$ is encoded once into the KV cache. A 10-step block decodes in a single parallel pass ($D=1, R=0$) in 65.8 ms (6.58 ms/step, achieving `100%` on LIBERO-Spatial and `3.92` on CALVIN). When block risk $\mathcal{U}_b = \max_i u_{b,i} > \delta$, top-$R$ uncertain tokens are selectively re-masked and refined (`3.92 → 4.04` CALVIN).

$u_{b,i} = \alpha_{\mathrm{step}}\|\hat{\mathbf{a}}^{(s)}_{t,i} - \hat{\mathbf{a}}^{(s-1)}_{t,i}\|_1 + \alpha_{\mathrm{adj}}\|\Delta^2_i \hat{\mathbf{a}}^{(s)}_t\|_1 + \alpha_{\mathrm{grip}}\big(1 - |2\hat{p}_{t,i}-1|\big)$

推理时多模态前缀 $\mathcal{C}$ 仅编码一次并存入持久 KV 缓存。单步并行解码（$D=1, R=0$）仅需 65.8 ms 即可生成 10 步动作块（折合 6.58 ms/步，在 LIBERO-Spatial 上达 100%，CALVIN 达 3.92）。当块级不确定度 $\mathcal{U}_b > \delta$ 时，仅对前 $R$ 个高风险槽位局部重掩码精化（将 CALVIN 提升至 4.04，并在 25/32 个离线窗口上降低误差）。

Animated Mechanism · Blockwise Causal-Bidirectional Decoding 机制动图 · 分块因果-双向解码与风险引导精化全流程

H = 20 (M = 2, K = 10)

Blockwise Causal-Bidirectional Decoding Animation

ANIMATION 01

Step-by-step trajectory synthesis inside the shared 36-layer VLM. Prefix $\mathcal{C}$ is cached once; Block $b=0$ ($t=1..10$) decodes in 1 parallel pass, triggers selective top-$R$ re-masking on high-curvature gripper transition slots (`t=4,5,6`), commits to the KV cache, and conditions Block $b=1$ ($t=11..20$).

36 层共享 VLM 内部的分块轨迹合成动态演示。 多模态前缀 $\mathcal{C}$ 一次编码入库；块 $b=0$（$t=1..10$）经单步并行预测后，自动定位夹爪闭合转换处的高风险槽位（`t=4,5,6`）执行局部重掩码精化，随后锁定为持久 KV 缓存，并因果引导块 $b=1$（$t=11..20$）生成。

FIGURE 03

Risk-guided blockwise decoding & selective re-masking. Easy blocks exit immediately after 1 parallel pass ($D=1$), while high-risk blocks refine only the top-$R$ uncertain slots.

风险引导分块解码与选择性重掩码机制。 平稳动作块在单次前向（$D=1$）后立即提前退出；高风险接触块仅重掩码精化不确定度最高的前 $R$ 个时间槽。

FIGURE S1

Canonical 20D embodiment-indexed action space. Binary validity masks $\mathbf{m}^{(e)} \in \{0,1\}^{20}$ route single-arm (7D) and bimanual (14D) robots through one shared interface without zero-padding distortion.

20 维具身索引规范动作空间。 二值有效掩码 $\mathbf{m}^{(e)} \in \{0,1\}^{20}$ 在加噪与损失计算中精准路由单臂（7D）与双臂（14D）自由度，消除零填充伪影。

SECTION 03 CLOSED-LOOP EXECUTION · ANIMATED ROLLOUTS & COUNTERFACTUAL COMPARISONS 闭环实机/仿真评测 · 连续五连任务、四大套件与反事实对照视频展厅

Closed-Loop Manipulation Across Long-Horizon Chains & Perturbations 长程连续指令链、多套件操作与受控消融的闭环表现

Explore real closed-loop rollouts from our released checkpoints across zero-shot CALVIN ABC→D 5-subtask chains, all four LIBERO suites, matched Counterfactual & Ablation Pairs on identical initial states, and heterogeneous Single-Arm & Bimanual camera streams.

以下展示由开源权重直接生成的真实闭环评估轨迹，涵盖 CALVIN ABC→D 零样本连续五子任务链、LIBERO 四大评测套件、同一初始状态下的受控消融与反事实对照实验，以及跨具身构型（单臂双视角 + 双臂三视角）同步视频流。

Animated GIF · CALVIN Zero-Shot 5-Task Chain Execution 实时动图 · CALVIN ABC→D 零样本连续五子任务全通演示

5 / 5 Subtasks · AvgLen 4.142

Unseen Environment D (`Seq #0000`): Concertina executes five language instructions back-to-back without state resets: rotate red block right → push red block left → open drawer → turn on green lightbulb → push into drawer.

未见环境 D（序列 #0000）： Concertina 在无状态重置条件下连续完成五条自然语言指令：向右旋转红块 → 向左推红块 → 拉开抽屉 → 点亮绿灯 → 将物体推入抽屉。

Animated GIF · Synchronized 4-Suite LIBERO Rollouts 实时动图 · LIBERO 四大评测套件同步闭环执行

98.15% Avg · 85.9% LIBERO-Plus

LIBERO 4 Suites Synchronized Animated GIF

Single unified checkpoint across all 4 suites: LIBERO-Spatial (99.0%), LIBERO-Object (100.0%), LIBERO-Goal (98.8%), and long-horizon multi-object LIBERO-10 (94.8%) decoded in 10-step blocks ($H=10, K=10$).

单一权重通吃四大套件： 在 LIBERO-Spatial（99.0%）、LIBERO-Object（100.0%）、LIBERO-Goal（98.8%）及长程复合任务 LIBERO-10（94.8%）上实现 98.15% 平均成功率。

Sequence #0000 5 / 5 SUCCESS

rotate red right → push red left → open drawer → green light on → push into drawer

Sequence #0002 5 / 5 SUCCESS

lift blue block → place in slider → push red left → lift red block → stack blocks

Sequence #0007 5 / 5 SUCCESS

turn on LED → open drawer → push blue left → lift blue block → place in drawer

Sequence #0010 5 / 5 SUCCESS

move slider left → lift red from slider → place in slider → green light → lift blue

Sequence #0014 5 / 5 SUCCESS

lift red from table → place in slider → move slider left → lift pink → place in drawer

Sequence #0016 5 / 5 SUCCESS

open drawer → rotate blue right → lift blue block → place in drawer → close drawer

CALVIN ABC→D · Sequence #0000

Zero-Shot 5-Instruction Horizon in Unseen Environment D

未见环境 D 中的零样本五阶段连续语言指令执行

Trained exclusively on environments A, B, and C, Concertina generalizes zero-shot to Environment D (`AvgLen = 4.142`, `98.6%` SR@1, `64.9%` SR@5 across 1,000 chains) using 10-step block execution ($H=10, K=10$).

仅在环境 A、B、C 上训练，零样本直接迁移至未见纹理与布局的环境 D（1,000 条序列平均完成长度 4.142，首任务成功率 98.6%，五连全通率 64.9%）。

1rotate red block right
2push red block left
3open drawer
4turn on green lightbulb
5push into drawer

LIBERO-Spatial · Task 0 99.0% SUITE SR

pick up the black bowl between the plate and the ramekin and place it on the plate

LIBERO-Spatial · Task 4 99.0% SUITE SR

pick up the black bowl on the wooden cabinet and place it on the plate

LIBERO-Object · Task 0 100.0% SUITE SR

pick up the alphabet soup and place it in the basket (500/500 across suite)

LIBERO-Object · Task 8 100.0% SUITE SR

pick up the chocolate pudding and place it in the basket

LIBERO-Goal · Task 3 98.8% SUITE SR

open the middle drawer of the cabinet

LIBERO-Goal · Task 9 98.8% SUITE SR

put the wine bottle on the rack

LIBERO-10 (Long) · Task 2 94.8% SUITE SR

turn on the stove and put the moka pot on it (multi-stage long horizon)

LIBERO-10 (Long) · Task 9 94.8% SUITE SR

put the yellow and white mug in the microwave and close it

LIBERO-Spatial · Task 0 (Suite SR: 99.0%)

Unified 4-Suite Evaluation (2,000 Standard + 10,030 LIBERO-Plus Episodes)

统一权重跨四大套件闭环评测（2,000 标准回合 + 10,030 鲁棒扰动回合）

A single multi-task Concertina checkpoint achieves 98.15% average success across the 4 standard LIBERO suites (`1,963/2,000` episodes) and 85.9% on LIBERO-Plus (`8,615/10,030` perturbed visual, language, camera, and layout tasks).

单一多任务 Concertina 权重在四大标准套件共 2,000 回合中斩获 98.15% 成功率（其中 LIBERO-Object 达 100.0% 满分），并在包含相机偏移、光照干扰、背景替换与语言改写的 10,030 个 LIBERO-Plus 任务上保持 85.9% 的领先鲁棒性。

1Pick up the black bowl between the plate and the ramekin
2Place it accurately on the target plate

Single-Objective Relative Co-Design (Ours · exp26) 5 / 5 Chain · AvgLen 2.970

Dual-Objective + 0.5× Aux Abs Head (Counterfactual · exp27) 0 / 5 Chain · AvgLen 0.237 (-92.0%)

Counterfactual Test on Sequence #0014: Adding a 0.5× auxiliary absolute-action head induces severe negative gradient cosine (-0.891 across 99% of slots) inside the shared VLM hidden states, causing the robot arm to stall before lifting the first red block. 序列 #0014 反事实对照： 在共享主干上附加 0.5× 绝对动作辅助头会引发高达 -0.891 的剧烈负余弦梯度冲突（99% 时间槽冲突），导致机械臂在抓取第一个红块时即发生轨迹死锁（AvgLen 从 2.970 暴跌至 0.237）。

Animated GIF Summary · Single-Objective Co-Design (5/5) vs. Dual-Objective Gradient Conflict (0/5) 核心结论动图 · 共享空间单一相对目标 (5/5 全通) vs. 双目标负余弦梯度冲突 (0/5 锁死)

cos(g_rel, g_abs) = −0.891

14D BIMANUAL

AgiBot Bimanual Embodiment (`14D` active mask $\mathbf{m}^{(e)}=\mathbf{1}_{1:14}$ in canonical `20D` space). Synchronized 3-camera input stream (Left Wrist + Head Center + Right Wrist) used during 90k-step cross-embodiment pretraining (`Traly/concertina-pretrain-90k`).

AgiBot 双臂具身构型（20 维规范空间中激活前 14 维掩码 $\mathbf{m}^{(e)}=\mathbf{1}_{1:14}$）。 90k 步跨具身预训练阶段使用的三视角同步视频流（左腕相机 + 头部主视角 + 右腕相机）。

7D SINGLE-ARM

Franka Panda Single-Arm Embodiment (`7D` active mask $\mathbf{m}^{(e)}=\mathbf{1}_{1:7}$ in canonical `20D` space). Synchronized Static Top-Down + Eye-in-Hand Gripper Wrist views during zero-shot CALVIN ABC→D closed-loop execution.

Franka Panda 单臂具身构型（20 维规范空间中激活前 7 维掩码 $\mathbf{m}^{(e)}=\mathbf{1}_{1:7}$）。 CALVIN ABC→D 零样本闭环执行期间的静态全局相机 + 手眼腕部相机双流同步输入。

SECTION 04 EXPERIMENTAL EVALUATION · BENCHMARKS, ABLATIONS & DECODING BUDGET 实验评测 · 标准基准、鲁棒性压力测试与共享空间消融分析

State-of-the-Art Performance Without Attached Policy Heads 无需外挂动作专家：在长程操作与扰动鲁棒性上全面领先

Across 2,000 standard LIBERO episodes, 10,030 perturbed LIBERO-Plus tasks, and 1,000 zero-shot CALVIN ABC→D 5-task chains (plus 8,000 factorial evaluation chains), Concertina outperforms both discrete token VLAs and attached diffusion/flow experts while using `78×–167×` less action-specific capacity.

在 2,000 个标准 LIBERO 回合、10,030 个 LIBERO-Plus 组合扰动任务以及 1,000 条 CALVIN ABC→D 零样本五连任务链（外加 8,000 条正交因子评测链）上，Concertina 以低 78–167 倍的动作专用参数量，全面超越离散词元 VLA 与外挂扩散专家模型。

LIBERO-Plus Robustness (10,030 Perturbed Episodes)

LIBERO-Plus 鲁棒性对比（10,030 个视觉/语言/相机扰动任务）

Overall success rate (%) under unseen camera poses, lighting, backgrounds, distractors, and paraphrased instructions.

在未见相机视角、光照变化、背景替换、干扰物注入与指令改写下的总体成功率 (%)。

CALVIN ABC→D Zero-Shot 5-Task Chain AvgLen (1,000 Chains)

CALVIN ABC→D 零样本五连任务平均完成长度（1,000 条序列）

Trained on Environments A, B, C and evaluated zero-shot in unseen Environment D across 5 consecutive instructions.

仅在环境 A、B、C 训练，零样本迁移至未见环境 D 连续执行 5 条指令的平均完成步数（满分 5.0）。

Table 2 · Unified Comparison on Standard LIBERO, LIBERO-Plus (10,030 Tasks), and CALVIN ABC→D

表 2 · 标准 LIBERO、LIBERO-Plus（10,030 任务）与 CALVIN ABC→D 权威基准全景对比

Full Manuscript Table 2 对应论文正文 Table 2

Method模型方法	Action Paradigm动作生成范式	LIBERO Spatial	LIBERO Object	LIBERO Goal	LIBERO 10	LIBERO Avg	LIBERO-Plus All	CALVIN SR@1	CALVIN SR@5	CALVIN AvgLen
OpenVLA (7B)	Discrete 256-Bin AR	84.7	88.4	79.2	53.7	76.5	15.6	91.3	43.5	3.27
$\pi_0$-FAST	Spectral DCT+BPE AR	96.4	96.8	88.6	60.2	85.5	45.0	—	—	—
$\pi_0$ (Flow Expert)	Attached 300M Flow Head	96.8	98.8	95.8	85.2	94.2	53.6	—	—	—
UniVLA	Latent Action Tokens	96.5	96.8	95.6	92.0	95.2	61.6	95.5	56.5	3.80
OpenVLA-OFT	Parallel Action Chunking	97.6	98.4	97.9	94.5	97.1	69.6	—	—	—
OpenVLA-OFT+ (w/ wrist)	Parallel Chunking + Wrist	96.9	98.1	95.5	91.1	95.4	70.6	—	—	—
EFFVLA	Efficient VLA Fine-Tuning	98.0	99.0	96.8	93.2	96.8	79.8	—	—	—
Seer-Large / Kosmos-VLA	Foresight / Multi-Modal VLA	—	—	—	—	87.7	—	96.3	62.4	3.83 / 4.03
Concertina (Ours · 4.46B)	Native Shared-VLM BCR (0.21% ASCR)	99.0	100.0	98.8	94.8	98.15	85.9	98.6	64.9	4.142

Blockwise Topology and LIBERO Progression

FIGURE 05A

Block partition $K=10\ (M=5)$ vs. $K=5\ (M=10)$ and training progression. Matching the block size to physical action autocorrelation ($K=10$) boosts LIBERO from 38.5% to 73.0% (+34.5 pp) and CALVIN AvgLen 35× (`0.055 → 1.945`).

分块拓扑 $K=10\ (M=5)$ 对决 $K=5\ (M=10)$ 及训练配方演进。 将块大小与物理动作自相关尺度对齐（$K=10$）使 LIBERO 成功率从 38.5% 跃升至 73.0%（+34.5 个百分点），CALVIN 提升 35 倍（`0.055 → 1.945`）。

Temporal Action Autocorrelation and Block Boundaries

FIGURE 05B

Physical action autocorrelation across 999,830 AgiBot trajectory frames. A 10-step block ($K=10$) captures 21.3% of total trajectory correlation energy inside bidirectional attention ($2.17\times$ the 9.8% captured by $K=5$).

999,830 帧 AgiBot 真实轨迹的物理动作自相关衰减曲线。 10 步动作块（$K=10$）在块内双向注意力中捕获了 21.3% 的相关能量，是 $K=5$（仅 9.8%）的 2.17 倍。

Tables 3 & 4 · Blockwise Partitioning ($H=50$) & Step-by-Step Training Recipe Progression

表 3 & 表 4 · 固定视野 $H=50$ 下的分块粒度消融与完整训练配方阶梯演进

Manuscript Tables 3 & 4 对应论文正文 Table 3 & Table 4

Stage / Configuration实验阶段 / 关键配置演进	Block Topology $(H, M, K)$分块拓扑 $(H, M, K)$	Coherent Mass块内相关能量占比	LIBERO Avg (%)	CALVIN SR@1 (%)	CALVIN SR@5 (%)	CALVIN AvgLen	Key Takeaway核心增益归因
Stage 0 · Fragmented Blocks	$H=50,\; M=10,\; K=5$	9.8%	38.5	4.5	0.0	0.055	Over-fragmented causal boundaries sever short-range motion块过短导致局部连续动作被因果边界频繁切断
Stage 1 · Matched Block Size	$H=50,\; M=5,\; K=10$	21.3% (2.17×)	73.0 (+34.5)	69.5	13.0	1.945 (35×)	Intra-block bidirectional attention spans full sub-movement块内双向注意力完整覆盖 10 步物理子运动基元
Stage 2 · + Physical Weights $\omega_d=(500,10,1)$	$H=10,\; M=1,\; K=10$	21.3%	88.5 (+15.5)	86.4	34.5	2.970 (+1.431)	Balances millimeter translation vs. radian rotation gradients对齐毫米级位移与弧度级旋转的物理梯度量纲
Stage 3 · + State Masking & Visual RRC	$H=10,\; M=1,\; K=10$	21.3%	95.5 (+7.0)	96.8	58.2	3.934 (+0.964)	Prevents proprioceptive shortcutting (+0.744) & view overfitting (+0.982)消除本体状态捷径依赖 (+0.744) 并提升视觉几何泛化 (+0.982)
Stage 4 · + Multi-Block Cont. Pretrain + Prob-BCE	Train $H=20 (2\times 10)$ → Exec $H=10$	21.3%	98.15	98.6	64.9	4.142	+0.208 from multi-block continuation; sharp binary gripper transitions多块续训提升长程规划 (+0.208)，概率 BCE 消除夹爪抖动

Shared-Space Co-Design and Counterfactual Gradient Conflict

FIGURE 04A

Shared-space co-design diagnostics. Left: Dual-objective auxiliary head induces $-0.891$ gradient cosine (`99%` negative slots), collapsing CALVIN from `2.970` to `0.237`. Right: $2\times 2\times 2$ factorial main effects across 8,000 chains.

共享空间协同设计诊断。 左：双目标辅助头在共享层引发 $-0.891$ 负余弦梯度冲突（99% 槽位为负），令 CALVIN 从 `2.970` 崩溃至 `0.237`。右：8,000 条序列上的 $2\times 2\times 2$ 因子主效应。

FIGURE 04B

Probability-valued Gripper BCE vs. Smooth-$L_1$ regression. Across all 4 matched pairs in our 8,000-chain factorial study, Prob-BCE consistently lifts AvgLen by +0.427 and SR@5 by +11.2 pp.

概率值夹爪交叉熵（Prob-BCE）对决平滑 $L_1$ 回归。 在 8,000 条评测链的全部 4 组配对比较中，Prob-BCE 一致带来 +0.427 AvgLen 与 +11.2 个百分点的五连全通率提升。

Tables 5, 9 & 10 · $2\times 2\times 2$ Factorial Study (8,000 Chains) & Inference Decoding Budget

表 5, 9 & 10 · $2\times 2\times 2$ 正交因子实验（8,000 条评测链）与推理时延/精化预算对比

Manuscript Tables 5, 9 & 10 对应论文 Table 5 & Appendix Tables 9–10

Experiment Group / Cell ID实验分组 / 因子单元 ID	Gripper Loss夹爪监督目标	Interface $E_\phi, D_\psi$动作接口状态	Decode Schedule $(D, R)$解码预算 $(D, R)$	Block Latency10步块延迟	Per-Step单步均摊耗时	CALVIN SR@1	CALVIN SR@5	CALVIN AvgLen
Factorial Cell c00 (Baseline)	Smooth-$L_1$	Frozen Pretrained	1-Pass ($D=1, R=0$)	65.8 ms	6.58 ms	88.3%	25.4%	2.726
Factorial Cell c01 (+Trainable Intf.)	Smooth-$L_1$	Trainable ($9.49\text{M}$)	1-Pass ($D=1, R=0$)	65.8 ms	6.58 ms	92.4%	35.3%	3.128 (+0.402)
Factorial Cell c02 (+Prob-BCE)	Prob-Valued BCE	Frozen Pretrained	1-Pass ($D=1, R=0$)	65.8 ms	6.58 ms	92.6%	38.1%	3.241 (+0.515)
Factorial Cell c03 (Full Co-Design)	Prob-Valued BCE	Trainable ($9.49\text{M}$)	1-Pass ($D=1, R=0$)	65.8 ms	6.58 ms	95.6%	48.9%	3.535 (+0.809)
Inference · Fast 1-Pass Default	Prob-Valued BCE	Trainable ($9.49\text{M}$)	$D=1, R=0$ (100% LIBERO-Spatial)	65.8 ms	6.58 ms	96.0%	60.0%	3.920
Inference · Risk-Guided Selective Refine	Prob-Valued BCE	Trainable ($9.49\text{M}$)	$D\le 4, R=5$ (Top-$R$ Uncertain Slots)	168.4 ms	16.84 ms	98.0%	64.0%	4.040 (+0.120)

SECTION 05 MECHANISTIC AUDIT · ACTION INTERFACE GEOMETRY, COUPLING & CAPACITY 机理审计 · 动作接口几何连续性 (EGC)、跨步耦合归因 (TCA) 与容量效率 (ASCR)

Auditing How Continuous Control Lives Inside a Vision-Language Backbone 机理透视：连续控制如何在视觉语言主干内部表征与涌现？

To rigorously explain why Concertina outperforms both discrete token VLAs and attached diffusion heads, we audit three mechanistic properties across our released checkpoints: Embedding Geometry Continuity (EGC), Trajectory Coupling Attribution (TCA), and Action-Specific Capacity Ratio (ASCR).

为从机理层面阐明 Concertina 为何能同时克服离散词元量化失真与外挂策略头冗余，我们在开源权重上对三大核心指标进行了定量审计：嵌入几何连续性（EGC）、轨迹耦合归因（TCA）与动作专用参数占比（ASCR）。

PILLAR I · EMBEDDING GEOMETRY (EGC)

Smooth Full-Rank 20D Manifold vs. 25.14× Token Cliffs

平滑满秩 20 维流形 vs. 25.14 倍离散量化悬崖

1.00×

Uniform 256-bin tokenization suffers a 25.14× perturbation jump ratio ($p_{95}/p_{05}$), while spectral tokenizers (FAST+, MolmoAct2) yield 82.2% / 77.9% invalid trajectory shapes under single-token flips. Concertina’s sinusoidal-affine encoder $E_\phi$ maintains a smooth 1.00× perturbation profile (`0%` invalid), effective Jacobian rank 18.63 / 20, and distance Spearman $\rho = \mathbf{0.855\text{–}0.931}$.

256 桶离散量化的单词元翻转跳跃比高达 25.14×，而频域词元化（FAST+、MolmoAct2）在单 Token 替换下出现 82.2% / 77.9% 的非法轨迹维度崩溃。相比之下，Concertina 连续接口保持 1.00× 平滑响应（0% 非法率），雅可比有效秩达 18.63 / 20，距离保序相关系数达 $\rho = \mathbf{0.855\text{–}0.931}$。

PILLAR II · TRAJECTORY COUPLING (TCA)

0% Interface Mixing → 9.93% Deep VLM Temporal Coupling

0% 接口耦合 → 9.93% 深层 VLM 跨时间步涌现

9.93% Peak NLR

By construction, the position-wise interface $D_\psi \circ E_\phi$ has 0.00% non-local ratio ($\mathrm{NLR}, |t-t_0|\ge 6$). Cross-timestep trajectory coordination emerges inside middle-to-late shared VLM layers—peaking at 9.93% NLR at Layer 27 (`4.10%` at CALVIN output, with `23.4%–34.0%` off-time Jacobian energy).

逐位置仿射接口 $D_\psi \circ E_\phi$ 自身的非局部耦合率（$\mathrm{NLR}, |t-t_0|\ge 6$）严格为 0.00%。跨时间步的轨迹协调完全在共享 VLM 中后层自发涌现——在第 27 层达到 9.93% 峰值（CALVIN 最终输出层为 4.10%，非对角跨时间步能量占比达 23.4%–34.0%）。

PILLAR III · CAPACITY RATIO (ASCR)

78× to 167× Lower Action-Specific Parameter Overhead

较外挂专家降低 78 至 167 倍的动作专用参数占比

0.21% ASCR

Concertina requires only 9.49M action-interface parameters out of 4.45B active parameters (`0` external transformer layers), compared to 16.51% ($\pi_0$), 19.17% ($\pi_{0.5}$), 22.19% (SmolVLA), and 35.10% (X-VLA).

Concertina 在 4.45B 活跃参数中仅需 9.49M 动作接口参数（0 层外部生成网络），动作专用容量占比仅 0.21%；远低于 $\pi_0$（16.51%）、$\pi_{0.5}$（19.17%）、SmolVLA（22.19%）与 X-VLA（35.10%）。

Animated Audit · Layerwise Emergence of Trajectory Coupling (TCA) 机理动图 · 36 层共享 VLM 内部跨时间步耦合能量的逐层涌现

Interface 0% → L27 Peak 9.93%

Layerwise Trajectory Coupling Attribution Animation

ANIMATION 02

Layer-by-layer progression of Non-Local Coupling Ratio ($\mathrm{NLR}$, $|t-t_0|\ge 6$). Starting from a strictly diagonal $30\times 30$ temporal Jacobian in the interface (`0.00%`), off-diagonal horizon coordination grows across `L0 → L9 → L18`, peaks at `L27 (9.93%)`, and projects cleanly to the `Output (4.10%)`.

非局部时间耦合率（$\mathrm{NLR}, |t-t_0|\ge 6$）的逐层演化动图。 从接口层严格对角的 $30\times 30$ 时间雅可比矩阵（`0.00%`）出发，非对角长程协调能量经 `L0 → L9 → L18` 逐步累积，在 `L27` 达到 `9.93%` 峰值，并收敛至输出层（`4.10%`）。

Three-Pillar Action Interface Audit Figure

FIGURE 06

Comprehensive three-pillar diagnostic suite (Manuscript Figure 6). Left: Embedding Geometry Continuity (EGC) & Jacobian spectrum. Center: Trajectory Coupling Attribution (TCA) across layers. Right: Action-Specific Capacity Ratio (ASCR) across VLA families.

动作接口三支柱全景诊断图（对应论文 Figure 6）。 左：嵌入几何连续性（EGC）与奇异值谱；中：跨时间步耦合归因（TCA）层间曲线；右：各主流 VLA 架构的动作专用容量占比（ASCR）对比。

Table 6 · Quantitative Action Interface Audit Across Tokenizers, Attached Heads, and Concertina Checkpoints

表 6 · 离散词元器、外挂扩散专家与 Concertina 各阶段权重的定量接口审计对比

Full Manuscript Table 6 对应论文正文 Table 6

System / Interface Variant系统 / 动作接口类型	Interface Category接口范式分类	Invalid Flip (%)单步翻转失效 (%)	Jump Ratio $p_{95}/p_{05}$跳变比 $p_{95}/p_{05}$	Eff. Rank (/20)有效雅可比秩	Spearman $\rho$距离保序 $\rho$	NLR ($\|t-t_0\|\ge 6$)长程耦合率 NLR	Action Params动作专用参数	ASCR (%)容量占比 ASCR
Uniform 256-Bin (RT-2 / OpenVLA)	Per-Dim Discrete Bins	0.00%	25.14×	—	—	0.00%	Vocab Rows	<0.1%
FAST+ (Universal Spectral Tokenizer)	DCT + BPE Tokenization	82.23%	30/30 Steps Perturbed	—	—	Global Artifact	Vocab Rows	<0.1%
MolmoAct2-FAST	DCT + BPE Tokenization	77.93%	30/30 Steps Perturbed	—	—	Global Artifact	Vocab Rows	<0.1%
$\pi_0$ / $\pi_{0.5}$ / SmolVLA / X-VLA	Attached Flow/Diff Head	0.00%	1.00×	—	—	External Head	100M – 860M	16.51% – 35.10%
Concertina Interface Only ($D_\psi \circ E_\phi$)	Position-Wise Affine	0.00%	1.00×	18.63 / 20	0.931	0.000%	9.49M	0.21%
Concertina Full VLM (Pretrain / CALVIN / LIBERO)	Shared 36-Layer eVLM	0.00%	1.00×	18.56 – 18.63	0.855 – 0.931	4.06% / 4.10% (9.93% L27)	9.49M	0.21% (78×–167× Lower)

SECTION 06 OPEN-SOURCE RELEASE · HUGGING FACE CHECKPOINTS, DATASETS & REPRODUCIBILITY 开源生态 · Hugging Face 预训练/微调模型权重、数据集与一键评测脚本

Released Model Zoo, Datasets & Closed-Loop Evaluation CLI 开源模型库、转换数据集与闭环复现命令

We release the complete training and evaluation codebase (`concertina`), three official model checkpoints (`4.46B` parameters each), and LeRobot-compatible datasets on Hugging Face.

我们完整开源了 `concertina` 训练与评测代码库、三个阶段的官方模型权重（各 4.46B 参数）以及标准化 LeRobot 格式数据集，支持一键复现论文全部指标。

Released Checkpoints & Datasets on Hugging Face

Hugging Face 官方开源模型权重与评测数据集列表

BF16 SafeTensors · 4.46B Params

Artifact Name开源资产名称	Category类型	Horizon / Block Config动作视野与分块配置	Key Performance / Description核心指标与用途说明	Link访问链接
`Traly/concertina-pretrain-90k`	Pretrained eVLM Base	$H=30,\; M=3,\; K=10$ (`20D`)	90k-step cross-embodiment pretrained checkpoint on AgiBot World + multimodal corpora在 AgiBot 双臂真实轨迹与多模态图文语料上完成 90k 步联合预训练的基座权重	Hugging Face ↗
`Traly/concertina-calvin`	CALVIN ABC→D Policy	$H=10,\; M=1,\; K=10$ (`20D`)	Zero-shot Env D: 4.142 reported AvgLen (4.153 standalone re-eval across 1,000 chains)CALVIN ABC→D 零样本评测权重：论文报告 4.142（独立复测达 4.153 AvgLen）	Hugging Face ↗
`Traly/concertina-libero`	LIBERO 4-Suite Policy	$H=10,\; M=1,\; K=10$ (`20D`)	Unified 4-suite model: 98.15% standard LIBERO (`1,963/2,000`) & 85.9% LIBERO-PlusLIBERO 四大套件统一权重：标准评测 98.15% 平均成功率，LIBERO-Plus 达 85.9%	Hugging Face ↗
`Traly/calvin_abc_d-lerobot`	Dataset (LeRobot v2)	7D Relative Pose + Gripper	Standardized CALVIN ABC→D splits formatted for multi-block eVLM training标准化 LeRobot 格式的 CALVIN ABC→D 训练与验证数据集	HF Dataset ↗

Quickstart · Install & Evaluate Released Checkpoints (CALVIN ABC→D & LIBERO)

# 1. 克隆仓库并安装 Concertina 环境
git clone https://github.com/lose4578/Concertina.git
cd Concertina
pip install -e ".[train,eval]"

# 2. 复现实验：CALVIN ABC→D 零样本 1,000 条连续五子任务链评测 (AvgLen = 4.142 / 4.153)
python -m concertina.eval.calvin_eval \
  --checkpoint Traly/concertina-calvin \
  --calvin-root /path/to/calvin/dataset/task_ABC_D \
  --num-sequences 1000 \
  --action-horizon 10 --block-size 10 \
  --decode-steps 1 --save-videos

# 3. 复现实验：LIBERO 四大标准套件 (2,000 回合, 98.15%) 与风险引导精化解码
python -m concertina.eval.libero_eval \
  --checkpoint Traly/concertina-libero \
  --suites libero_spatial libero_object libero_goal libero_10 \
  --episodes-per-task 50 \
  --action-horizon 10 --block-size 10

BibTeX Citation · Concertina (2026)

@article{wang2026concertina,
  title   = {Concertina: Embodying Manipulation and Cognition in One Diffusion Vision-Language Model},
  author  = {Wang, Chengcheng and He, Wei and Li, Wenhao and Li, Hongguang and Han, Kai and Guo, Jianyuan and Xu, Chang},
  journal = {arXiv preprint},
  year    = {2026},
  url     = {https://github.com/lose4578/Concertina}
}