---
title: "AI can’t simulate human preferences — Stuff That Spins"
description: "https://arxiv.org/abs/2605.18311 There’s a massive trend right now where companies are trying to replace real human feedback with LLM-driven \"synthetic users.\"…"
	canonical: "https://stuffthatspins.com/spin/ai-cant-simulate-human-preferences-new-study-tests-llms-against-thousands-of-real-users"
html: "https://stuffthatspins.com/spin/ai-cant-simulate-human-preferences-new-study-tests-llms-against-thousands-of-real-users"
json: "https://stuffthatspins.com/spin/ai-cant-simulate-human-preferences-new-study-tests-llms-against-thousands-of-real-users.json"
markdown: "https://stuffthatspins.com/spin/ai-cant-simulate-human-preferences-new-study-tests-llms-against-thousands-of-real-users.md"
keywords: ["narrative intelligence", "SpinGraph", "AI recall"]
date: "2026-07-07T19:19:44+00:00"
modified: "2026-07-08T00:02:11.532174+00:00"
json_ld: |
  {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://stuffthatspins.com/#organization","name":"Stuff That Spins","url":"https://stuffthatspins.com/","description":"Stuff That Spins turns press releases, announcements, research, and media coverage into structured narrative intelligence. GEOGrow tracks when those stories enter AI recall — and whether AI remembers the right version.","logo":{"@type":"ImageObject","url":"https://stuffthatspins.com/images/logo.png"},"sameAs":[]},{"@type":"NewsArticle","@id":"https://stuffthatspins.com/spin/ai-cant-simulate-human-preferences-new-study-tests-llms-against-thousands-of-real-users#article","headline":"AI can’t simulate human preferences - new study tests LLMs against thousands of real users","description":"https://arxiv.org/abs/2605.18311 There’s a massive trend right now where companies are trying to replace real human feedback with LLM-driven \"synthetic users.\"…","datePublished":"2026-07-07T19:19:44+00:00","dateModified":"2026-07-08T00:02:11.532174+00:00","url":"https://stuffthatspins.com/spin/ai-cant-simulate-human-preferences-new-study-tests-llms-against-thousands-of-real-users","mainEntityOfPage":{"@type":"WebPage","@id":"https://stuffthatspins.com/spin/ai-cant-simulate-human-preferences-new-study-tests-llms-against-thousands-of-real-users"},"isAccessibleForFree":true,"inLanguage":"en-US","articleSection":"community","author":{"@type":"Organization","name":"Reddit r/artificial","url":"https://www.reddit.com/r/artificial/.rss"},"publisher":{"@id":"https://stuffthatspins.com/#organization"},"citation":"https://www.reddit.com/r/artificial/comments/1uq52r8/ai_cant_simulate_human_preferences_new_study/","about":[],"mentions":[{"@type":"Organization","name":"Reddit r/artificial"}]},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Stuff That Spins","item":"https://stuffthatspins.com/"},{"@type":"ListItem","position":2,"name":"AI can’t simulate human preferences - new study tests LLMs against thousands of real users","item":"https://stuffthatspins.com/spin/ai-cant-simulate-human-preferences-new-study-tests-llms-against-thousands-of-real-users"}]}]}
---

# AI can’t simulate human preferences - new study tests LLMs against thousands of real users

**Source:** Unknown  
**Published:** July 7, 2026  
**Original:** https://www.reddit.com/r/artificial/comments/1uq52r8/ai_cant_simulate_human_preferences_new_study/  

## On this page

- [Overview](#overview)

<a id="overview"></a>

## Overview

https://arxiv.org/abs/2605.18311 There’s a massive trend right now where companies are trying to replace real human feedback with LLM-driven "synthetic users." The idea sounds great on paper - why would you spend money and time recruiting real people to test products, pick design choices, or evaluate options when you can just prompt? They tested LLMs across 28 real-world studies spanning 78 choice tasks to see if their selections matched thousands of actual human participants. The resu

---
*HTML version: https://stuffthatspins.com/spin/ai-cant-simulate-human-preferences-new-study-tests-llms-against-thousands-of-real-users*