Singular Value Perturbation
Unlocking RL Performance via Diverse Latent Reasoning

Jihwan Shin¹, Majid Daliri², Vahab Mirrokni³, Amir Zandieh³, Insu Han¹

¹KAIST, ²New York University, ³Google

Paper soon Code

Latent reasoning models cannot explore, so RL cannot improve them.
SVP fixes that by injecting structured randomness in the weights.

Deterministic latent reasoning

Fixed weights, 1 reasoning path

Latent reasoning has no randomness, so RL has no differing rollouts to compare.

Put structured noise in weights

Perturbed singular values, diverse paths

SVP-V introduces diversity to the latent trajectories via structured perturbation on the attention Value projection.

Exploration that RL can use

34.1%COCONUT

50.3%+ SVP-V-GRPO

GSM8K accuracy

SVP-V-GRPO achieves state-of-the-art performance among continuous LRMs.

Read the full abstract

Latent Reasoning Models (LRMs) replace explicit chain-of-thought tokens with continuous recurrent computation, but that recurrence is deterministic: sampling answer tokens cannot explore alternative computations inside the latent reasoning phase, which limits the exploration reinforcement learning needs. Singular-Value Perturbation (SVP) diversifies latent trajectories by randomly rescaling coefficients in the fixed SVD basis of a weight matrix. Applied to attention Value projections (SVP-V), it perturbs the messages attention transmits while preserving the attention weights for a fixed input.

Across multiple LRMs and mathematical reasoning benchmarks, SVP-based configurations achieve the highest solution coverage in every evaluated setting, and intermediate LM-head readouts reveal semantic branching around decision-critical steps. Building on this, SVP-V-GRPO learns from the relative outcomes of perturbed trajectories while updating only the Value parameters: starting from COCONUT, it improves GSM8K accuracy from 34.1% to 50.3% and achieves state-of-the-art performance on all six benchmarks among released GPT-2-based continuous LRMs.

Method

Latent reasoning models think in hidden states, and that recurrence is deterministic. SVP introduces structured randomness in the weights by rescaling the singular values of the attention Value projection.

SVP-V

Decompose the Value projection once, W_V = UΣR^⊤. For each trajectory, draw Gaussian noise ε and rescale only the singular values; U and R^⊤ never move. The draw is held fixed through prefill and every latent step, so each trajectory reasons with its own W̃_V.

\[\widetilde{W}_V \;=\; \sum_{i=1}^{r} s_i(1+\alpha\varepsilon_i)\,u_i r_i^\top, \qquad \varepsilon_i \sim \mathcal{N}(0,1)\]

The only hyperparameter is \(\alpha\): the RMS size of the perturbation relative to the clean response, the same for every input.

Why the Value projection?

Perturbing W_V changes the messages attention transmits but leaves the attention weights untouched for a fixed input, so it still acts when attention is sharply peaked and query- or key-side noise loses its effect. Empirically, W_V is also the most effective perturbation injection site among all the targets we compared.

Why Gaussian coefficients?

With the SVD basis fixed and the perturbation size budgeted relative to the clean response, multiplicative Gaussian noise on the singular values is the unique maximum-entropy choice: the most diverse perturbation that budget allows.

SVP-V-GRPO: exploration that unlocks RL

RL needs rollouts that differ, and a deterministic latent recurrence gives it none. SVP-V-GRPO samples each rollout with its own SVP-V draw, learns from the relative outcomes of those perturbed trajectories, and updates only the attention Value projections (about 5.7% of the parameters). All accuracies below are measured with the perturbation switched off: the exploration is only needed during training, and the gain stays in the clean model.

1. Rollouts with per-trajectory noise

Rollout \(j\) draws its own noise \(\varepsilon^{(j)} \sim \mathcal{N}(0, I)\), held fixed through prefill and every latent step; the answer is then decoded with the clean W_V at temperature 1. Following Dr.GRPO, each of the \(G\) rollouts gets a centered, unnormalized advantage (\(\overline{\mathcal{R}}\) is the group's mean reward), and token losses are summed without response-length normalization:

\[\mathcal{R}_j = \mathbf{1}\{o_j \text{ is correct}\}, \quad \widehat{A}_j = \mathcal{R}_j - \overline{\mathcal{R}}\]

Following DAPO's dynamic sampling, we keep only groups that contain both correct and incorrect rollouts, since the rest have zero advantage. SVP-V raises how often a group qualifies.

2. Replay the same noise to optimize

We store \(\varepsilon^{(j)}\) with the sampled answer, replay prefill and latent reasoning under the same noise, and backpropagate through it to W_V. The current, old, and reference policies all share that noise:

\[\rho_{j,\tau} = \frac{\pi_\theta\big(o_{j,\tau} \mid x,\, \varepsilon^{(j)},\, o_{j,<\tau}\big)} {\pi_{\theta_{\mathrm{old}}}\big(o_{j,\tau} \mid x,\, \varepsilon^{(j)},\, o_{j,<\tau}\big)}\]

Released checkpoint GRPO control (ours, matched recipe) SVP-V-GRPO

Accuracy of released GPT-2-based latent reasoning models and SVP-V-GRPO; pick a benchmark above. The ablation button adds the three GRPO controls (striped) that we trained from the same COCONUT checkpoint under one matched recipe, differing only in the exploration mechanism.

Full table (all benchmarks)

Model	GSM8K	GSM-Hard	MultiArith	SVAMP	ASDiv-A	GSM-Plus	Mean
Released checkpoints
COCONUT (baseline checkpoint)	34.1	7.7	80.9	35.6	60.2	17.4	39.3
SLPO	34.9	7.6	82.8	34.3	58.7	18.2	39.4
SIM-CoT	44.7	9.3	90.5	40.6	67.2	21.5	45.6
CoDi	42.5	9.3	91.9	40.0	65.4	23.1	45.4
GRPO from the same COCONUT checkpoint, matched recipe
Temp-GRPO	38.4	8.7	85.3	37.1	61.5	19.8	41.8
Gaussian-V-GRPO	40.6	9.0	88.6	39.1	62.7	21.9	43.7
Dropout-GRPO	45.2	9.3	90.9	44.6	71.1	24.8	47.7
SVP-V-GRPO (Ours)	50.3	11.3	93.4	43.6	71.2	27.6	49.6

Why it works: SVP-V is good exploration

The RL gain rests on one claim: SVP-V makes latent trajectories differ in ways that matter. Two pieces of evidence, both measured at test time without any training.

Evidence 1Test-time solution coverage

If the exploration is useful, sampling with it should uncover more distinct correct solutions. It does: an SVP-based configuration has the best pass@16 in all 18 model-benchmark settings, matching or beating answer-token sampling and unstructured Gaussian noise on activations or weights everywhere.

Method	GSM8K	GSM-Hard	MultiArith	SVAMP	ASDiv-A	GSM-Plus
Deterministic	34.12	7.66	80.86	35.60	60.18	17.38
Top-p	53.76 ± 0.24	12.44 ± 0.08	88.53 ± 0.43	49.97 ± 0.33	72.58 ± 0.37	38.95 ± 0.34
Gaussian × Activation	54.25 ± 0.16	12.69 ± 0.13	93.41 ± 0.12	50.45 ± 0.28	72.57 ± 0.28	36.43 ± 0.21
SVP × Activation	54.90 ± 0.18	13.10 ± 0.13	93.34 ± 0.30	52.86 ± 0.40	73.81 ± 0.14	37.56 ± 0.16
Gaussian × W_V	55.98 ± 0.25	13.59 ± 0.28	93.46 ± 0.29	51.68 ± 0.41	74.13 ± 0.18	39.16 ± 0.30
SVP × W_V (SVP-V)	57.22 ± 0.48	13.90 ± 0.12	93.60 ± 0.07	53.45 ± 0.38	74.87 ± 0.30	40.05 ± 0.34
Deterministic	42.46	9.26	91.90	40.00	65.44	23.10
Top-p	59.66 ± 0.38	13.35 ± 0.13	95.97 ± 0.13	52.81 ± 0.29	77.11 ± 0.29	42.08 ± 0.23
Gaussian × Activation	53.20 ± 0.09	12.13 ± 0.08	97.23 ± 0.19	50.38 ± 0.29	76.41 ± 0.09	33.69 ± 0.12
SVP × Activation	58.47 ± 0.33	13.98 ± 0.07	98.03 ± 0.15	57.73 ± 0.31	80.47 ± 0.13	41.20 ± 0.26
Gaussian × W_V	59.35 ± 0.35	13.89 ± 0.17	98.44 ± 0.10	54.04 ± 0.32	78.50 ± 0.16	42.09 ± 0.21
SVP × W_V (SVP-V)	61.09 ± 0.26	14.38 ± 0.26	98.72 ± 0.07	56.14 ± 0.31	79.99 ± 0.35	43.76 ± 0.26
Deterministic	55.57	12.82	95.34	60.80	88.01	33.95
Top-p	68.01 ± 0.23	16.35 ± 0.14	97.32 ± 0.08	69.86 ± 0.25	93.31 ± 0.09	49.96 ± 0.23
Gaussian × Activation	65.30 ± 0.10	15.43 ± 0.14	97.79 ± 0.04	68.24 ± 0.23	93.38 ± 0.05	44.98 ± 0.11
SVP × Activation	69.51 ± 0.21	17.02 ± 0.16	98.62 ± 0.08	74.00 ± 0.27	94.44 ± 0.07	50.92 ± 0.18
Gaussian × W_V	70.14 ± 0.28	16.68 ± 0.08	98.78 ± 0.07	72.86 ± 0.14	94.43 ± 0.10	51.96 ± 0.12
SVP × W_V (SVP-V)	70.29 ± 0.35	16.72 ± 0.10	98.78 ± 0.05	73.24 ± 0.28	94.39 ± 0.09	52.25 ± 0.13

Unbiased pass@16 (%), five-seed mean ± std, 64 samples per prompt. Bold = best per benchmark for the selected model. Hyperparameters were selected once on a 1,024-problem GSM8K training subset and then held fixed across all six benchmarks. Deterministic is a single greedy run and has no seed variance.

Evidence 2Semantic branching at the decision-critical step

Reading the latent states through the LM head shows where trajectories differ. SVP-V does not scatter them everywhere: they agree on the easy steps and branch at the one step the clean model gets wrong. There, the correct intermediate value becomes reachable, and only SVP-V trajectories go on to the correct answer.

Nick, Richard, Jason and DJ each have paintball guns. DJ has 8 guns, Nick has 10 guns, RJ has 1 gun and Richard has 5 guns. If they were to share their guns equally, how many guns would each of them have?

8 + 10 + 1 + 5 = 24 · 24 ÷ 4 = 6

value on the correct solution path darker route = contains a correct trajectory line width = number of trajectories

BibTeX

@article{shin2026svp,
  title         = {Singular Value Perturbation: Unlocking {RL} Performance via Diverse Latent Reasoning},
  author        = {Shin, Jihwan and Daliri, Majid and Mirrokni, Vahab and Zandieh, Amir and Han, Insu},
  journal       = {arXiv preprint arXiv:XXXX.XXXXX},
  year          = {2026},
  eprint        = {XXXX.XXXXX},
  archivePrefix = {arXiv},
  primaryClass  = {cs.LG},
  url           = {https://arxiv.org/abs/XXXX.XXXXX}
}

Singular Value Perturbation Unlocking RL Performance via Diverse Latent Reasoning