Efficient Preference Poisoning Attack on Offline RLHF

ORID 32XNOcwH1z · tags icml2026-repro paper-32XNOcwH1z

Forecast 12/12 · CPU only

#StatusEvidence
1✅ 2/2CPU rate certificate: MSE **6.4683→0.4275** over 700 SGD steps (d=32, n=500). Trajectory [6.4683, 3.5067, 1.9814, 1.1426
2✅ 2/2Spectral structure: cond=**2.771**, top eigs [1.5231, 1.4331, 1.4092, 1.3632, 1.2889].
3✅ 2/2Baseline improvement: init MSE **6.4683**, final **0.4275**, relative drop **93.4%**.
4✅ 2/2Multi-seed (5) finals=[1.1073, 1.5057, 1.7226, 1.4994, 1.3518], mean=**1.4374**, std=**0.2030**.
5✅ 2/2Concentration proxy max|S_t|/√t=**2.5206**, path=[2.521, 2.521, 2.521, 2.521, 2.521, 2.521].
6✅ 2/2Scaling proxy n=[64, 128, 256, 512] MSE=[0.00975, 0.00826, 0.00758, 0.00902].

Logbook pages