New paper challenges semantic embeddings for collective decision-making

A research paper titled "Embeddings for Preferences, Not Semantics" was submitted to arXiv on 8 May 2026, addressing a fundamental challenge in using AI for collective decision-making ¹. Authored by Carter Blair, Ariel D. Procaccia, and Milind Tambe, the 28-page paper argues that standard text embeddings fail to capture participant preferences in deliberation systems ¹.

The paper identifies a core problem: modern AI enables collective decision-making where participants express views as free-form text rather than voting on fixed candidates ¹. While embedding these opinions in vector space appears natural for applying facility location and fair clustering algorithms, standard text embeddings measure semantic similarity rather than what the authors call preferential similarity ¹.

Preferential similarity, according to the paper, requires that a participant's agreement with text be inversely related to their distance from it ¹. Off-the-shelf embeddings inherit only a coarse preference signal through correlation between semantic and preferential similarity, but fail when this correlation breaks ¹.

The researchers formalise this as an invariance problem ¹. Text embedding models encode both preference-relevant signals—such as stance and values—and semantic nuisance, including style and wording ¹. Because these dimensions are observationally correlated, a geometry relying on nuisance can appear preference-correct even when it is not ¹.

The paper demonstrates that synthetic training data designed to break this correlation provably shifts the optimal scorer away from nuisance-dominated cosine similarity ¹. The approach significantly improves preference prediction across 11 online deliberation datasets ¹.

The research paper is classified under Artificial Intelligence (cs.AI) and carries the identifier arXiv:2605.08360 ¹. It has been assigned DOI 10.48550/arXiv.2605.08360 via DataCite, pending registration ¹. Version 1 was submitted on Friday, 8 May 2026 at 18:15:14 UTC ¹.

The work is supported by the Simons Foundation, member institutions of arXiv, and all contributors ¹.