Why generic LLMs aren't synthetic audiences

Why generic LLMs aren't synthetic audiences

Ask a generic LLM what your customers think, and you'll get the average opinion, smoothed over. Your customers aren't average. They disagree with each other, hold unpopular views, and contradict themselves. If you want to understand them, you need those messy truths, not the consensus

Blog

Electric Twin

Why not just ask Claude or ChatGPT?

Office workers are adopting AI at a dizzying speed. ChatGPT and Claude have become mainstay tools in everyone's workflow, permanently changing how work gets approached.

'Ask Claude' has become shorthand for solving any problem. To its credit, it's genuinely useful: quick answers on general topics, high-level summaries, deep research. Workers have leaned in hard, and it's changed how a generation works. But can we trust the output? That's the real question.

Plenty of people are quietly treating a generic LLM as a synthetic audience. Most don't know the difference, and the impact is huge. In the age of AI tools, knowing which tool to use, and when, will be the difference between a productive team and a distracted one.

At Electric Twin, we believe that the average represents nobody

State-of-the-art LLMs pull in context from huge amounts of data every time you ask a new question. They've read basically everything we've ever written about brands, values, work and attitudes. In the right context, that's real power. It can reason fairly well about what someone with a given background might say, and give you a plausible answer to almost any query.

It's also the catch. An LLM is built to regress to the mean. Ask for 'a 55-year-old woman in Yorkshire' and it hands you the most statistically central version it can find, flattened and shaped by whatever biases sit in its training data. You don't get a person. You get the average of everyone on the internet who's ever been filed under that label.


If you want to understand your customer, you need real responses from specific segments of your population with their contradictions, outlying opinions and truths. A generic LLM will gloss over these outliers, in favour of the voice of the majority.

What LLMs can't capture

Most LLMs model demographics and stop there. Demographics give you the basic attributes of a group, but they don't explain how those attributes turn into specific beliefs or behaviours. Model the stereotype, claim it's the real thing, and you lose sight of your actual customer.

Real people are shaped by four things at once: their culture, their community, their friends and family, and their own experience. 'A 55-year-old woman in Yorkshire' isn't a category. She's a specific mix of all four.


How Electric Twin turns a model into an audience


The critical piece of proof is also the part that most people skip: validation. At Electric Twin, we run holdout testing on every dataset. Hide some of the real survey data, build the audience from the rest, predict the hidden answers, then compare. We repeat this every time new data arrives, and we've done it thousands of times.

None of this is magic, and we don't pretend otherwise. The results are only ever as good as the data you put in, and anyone claiming a synthetic method beats a real survey deserves suspicion.


A tool for results you can trust, not for distractions that mislead

In today’s world, understanding your customer deeply means really understanding who they are as people beyond their demographics. It means knowing what they think and why they think it. That takes the right tool asking the right question.

Electric Twin was built to put your hardest questions to your audience, and give you richer, evidence-backed responses to make better decisions. Test twenty message variants across eight segments before lunch. Reach audiences you could never get a panel for. Ask questions you’d be nervous to ask real people. Run exploratory work that would never have justified a full study.

So when someone says they’ll ‘just ask the AI’, remember: it matters is what that AI is anchored to, and how you’d know if it were wrong.

A generic LLM answers as everyone, a good synthetic audience answers as someone. That gap is everything.

Stop guessing.

Start predicting

Stop guessing.

Start predicting

Stop guessing.

Start predicting