If you commission a piece of research, how much does your choice of platform shape the answer you get back? And if a synthetic audience sat in that same line-up, would anyone be able to tell?

Those are the two questions behind one of the largest studies of its kind, run in collaboration with Professor Michael Muthukrishna of the London School of Economics (LSE) and New York University (NYU), and pre-registered for transparency. The short version: the platform you pick matters more than most of us would like to admit, and a synthetic audience held its own against human panels across most of what was tested.

What we actually did

The design of the study was straightforward. Researchers gave a single survey to 7,755 people spread across seven different online sampling services, around 1,000 respondents on each, then looked at whether the answers lined up.

The seven platforms covered the spread of how research actually gets sourced today: two opt-in panels, the microtask platform Prolific, two multi-source aggregators, and two "river" samples recruited straight from Facebook and Instagram. Alongside them sat a synthetic audience benchmark, generated by Electric Twin. This approach combines seed data, large language models and behavioural science modelling to create respondents you can actually survey.

Everyone answered the same questions: their financial outlook, grocery shopping habits, social media use, and where they stood on UK military intervention in Ukraine. Every platform's data was weighted the same way, against the same recruitment targets (age, gender, education and voting history), so the comparison was like-for-like. Low-quality responses, from bots, duplicate accounts or failed attention checks, were stripped out.

Two details matter for trust here. The human data collection was run independently by polling firm Stack Data Strategy, to keep the fieldwork at arm's length. And the whole study was pre-registered before any data came in.

Finding one: your platform is quietly making decisions for you

The same survey, sent against the same recruitment targets, came back with strikingly different answers depending on where it was fielded.

The Facebook and Instagram river samples came out worst on data quality, with nearly double the rate of failed attention checks and duplicate accounts compared with Prolific and the traditional panels. They also skewed older, while Prolific skewed young and highly educated. Voting intention swung by as much as 20 percentage points depending on which platform you asked.

When the results were weighted by demographics, the gaps between platforms shrank by more than 80%. In other words, the variation was driven mostly by who each platform recruited, rather than how people answered.

Once you weighed representativeness, response variability and data quality together, no single platform came out consistently on top, which is an uncomfortable conclusion for an industry that tends to lean on one trusted supplier. It echoes earlier work, including Douglas et al. (2023) and Pew Research Center (2024), flagging concerns about the quality of online panels. But it does so at a scale that lets us say something sharper: rely on a single source and you could be misled about what your audience thinks, purely on the basis of which platform you happened to use.

Finding two: the synthetic audience landed in the same range

This was the first time a synthetic audience platform has been stress-tested alongside traditional online sampling in a study of this size. Electric Twin's simulated respondents fell well within the range of the human results across most questions, close enough to suggest it was broadly capturing real public sentiment rather than approximating it from a distance. You can read more about how we measure and report accuracy, and how accurate synthetic audiences are in practice.

We believe synthetic will play an increasing role in research. A synthetic run is quick and cheap compared to commissioning a human panel, so it fits the early, exploratory stretch of a project well: getting a first read on a question, or narrowing the options, before you spend real budget talking to people.

What the researchers say

Dr Michael Muthukrishna, Professor of Economic Psychology at NYU and LSE, and author of A Theory of Everyone:

"This study offers some of the first solid, large-scale evidence that a synthetic audience can stand in for a human panel and get you to the same place. Across most of the questions we tested, Electric Twin's simulated respondents landed within the same range as the responses of real people.

"That's a significant finding for anyone doing research under time, budget or practical pressure.

"This is an impactful moment because studies of this scale almost never get academic funding. It's one of the largest to stress-test the day-to-day tools of market and behavioural researchers alongside the next generation of AI tools that let us reach sections of the population that might otherwise be hard to get to."

Dr Ben Warner, Visiting Senior Fellow at LSE and co-founder of Electric Twin:

"This study shows that at real scale, with independent oversight, a synthetic audience produced results consistent with human panels across most of what we tested. That's a strong indicator this approach can be trusted for exploratory and iterative audience research.

"It also puts synthetic audiences in useful company. This study didn't set out to crown one method as best. It set out to test methods properly, side by side, at a scale the industry rarely gets to see. Synthetic audiences earn their place by being testable, repeatable and inspectable in ways traditional sampling can't match. This isn't about picking a winner. It's about an industry trying to understand its own methods better, because that's the only honest path to understanding the world better."

The takeaway

Two things sit alongside each other here. The methods we already trust vary more than we tend to assume, which is worth knowing whichever way you research. And a synthetic audience, tested in the open, can land in the same range as a human panel on most questions. It gives you a fast, inspectable first pass to decide where the expensive fieldwork is worth spending, with real people firmly in the loop.

Note: The study was led by Professor Michael Muthukrishna (LSE and NYU) in partnership with Electric Twin, with independent human data collection by Stack Data Strategy, and was pre-registered for transparency. Electric Twin was founded by Alex Cooper and Dr Ben Warner. Michael Muthukrishna is Chief Science Advisor.