Most message decisions are made before anyone has evidence. 3 routes go into a meeting, and the one that comes out is often the one the room already preferred.
Ben Warner, co-founder of Electric Twin, describes the problem plainly. There is no Minority Report-style tool that understands human behaviour perfectly. The practical problem is more ordinary: surveys and focus groups are often too slow for the moment when a headline, claim or campaign route has to be chosen.
That leaves teams relying on intuition.
Experience matters, but it cannot represent the lived experience of every group the message needs to reach. And when research arrives after the route has already been approved, it can only validate or reject a decision that is expensive to change.
Message testing with a synthetic audience moves evidence earlier. You can put taglines, claims and campaign routes in front of a model of your audience, compare how segments respond and ask follow-up questions before launch.
The aim is not a magic prediction. It is a faster way to explore, find risk and improve the options while they are still options.
What can you test with a synthetic audience?
Message testing covers more than choosing a tagline. It helps when the decision concerns what you say, what the audience understands and which reaction might create risk.
Message test | What it helps you decide |
|---|---|
Campaign taglines | Which line lands with the audience the campaign is for |
Headlines and subject lines | Which framing earns attention without changing the meaning |
Product and proposition copy | Whether people understand the offer and what would stop them acting |
Claims and proof points | Which claims build confidence and where readers want evidence |
Sponsorships and partnerships | How different fan or customer segments may react before an announcement |
Fundraising and campaigning appeals | Which message is clear and credible to the communities it needs to reach |
Most of these decisions are too small or urgent to justify commissioned fieldwork. A subject line, a proof point or a shortlist of three taglines gets settled by opinion instead.
That gap is where much of day-to-day marketing work lives.
5 steps, 3rd is where they go wrong
Build the audience. From real survey or first-party data about the people you are targeting. The stages are in how synthetic audiences work.
Load every option. Including the ones you expect to lose. They all run against the same audience in one test, so the 5th is no more effort than the first, and the fifth sometimes wins.
Ask a question that forces a decision. "Which do you prefer?" returns a ranking you cannot act on. Ask what the message made the reader expect, what they would do next, and what would hold them back.
Read it by segment. The all-audience average is usually the least useful number in the output. The gap between two segments is where the decision sits.
Turn the response into the next audience. If one group rejects a claim or interprets a headline differently, save that group as a segment and question them directly. Ask what created doubt, which words shaped their interpretation and what evidence would change their view. The first test identifies the split. The follow-up tells you how to rewrite the message.
Ask the question that changes your mind, not the one that flatters the copy
Step 3 deserves its own section, because it is the one teams get wrong.
A good test question is specific about the decision, and open enough to surface an objection you had not thought of. The pattern below works whether you are asking real people or a synthetic audience.
Instead of asking | Ask | Why |
|---|---|---|
Which of these taglines do you prefer? | After reading this, what do you think the product does? | You learn whether it communicates what you intended |
Is this message clear? | What would stop you from acting on this? | It identifies the objection instead of confirming the copy |
Do you like this offer? | What would you expect this to cost, and what would make it worth that price? | It turns a general reaction into a price expectation you can test |
Rate this advert out of ten | Who is this advert for, and does that include you? | It reveals whether the message is aimed at the wrong segment |
Does this sound credible? | Which part of this would you want evidence for? | It identifies the missing proof point |
Would you buy this? | What would you do in the week after seeing this? | A described next action gives you more information than stated intent |
The through-line: questions that invite approval get approval, and approval rarely tells you what to change. Ask what the reader understood, expected, and would do.
Why should you ask the question more than once?
There is rarely one perfect way to phrase a research question. A small wording change can shift an answer, and that shift is useful information rather than a nuisance to hide.
Ask what someone understood from a message. Then ask what they would tell a colleague it meant. Ask what they would do next, then what might stop them. If the answers move around, the audience may not care strongly or the message may be unclear. If differently phrased questions point to the same conclusion, you have a stronger signal.
Traditional fieldwork makes this kind of exploration expensive because every extra question consumes time and sample.
A synthetic audience lets you ask, read and sharpen the question in the same session. Ben describes this as the move research has struggled to make towards an agile way of working: the answer to one question can shape the next question immediately.
Why the all-audience number is the one to distrust.
Reading by segment catches the messages that test well on average and fail with the people you are trying to reach.
A single figure can sit between two groups who want opposite things and represent neither, so the all-audience average is often the number to be most suspicious of. You choose the cuts that matter for the decision: prospects against current customers, price-sensitive against premium, one channel against another, and the audience answers each.
The Times marketing team saw this testing 4 names for a new business podcast. The winner, The Business, did well with a general reader panel and markedly better with the specific group the show was made for.
Average the two together and that edge disappears, and the choice slides back to whichever name the room liked.
The same pattern behind broad-audience generalisations is in why Gen Z generalisations are bad for business, and the full account of the Times work is in the Times case study.
What a test should hand back.
If a tool gives you only the first of these, you cannot act on it.
A ranking, by segment. Which route won overall, and separately which won with each segment you care about. The overall winner and the segment winner are often different, and that difference is frequently the finding.
The reasoning, in the audience's own words. This is where a popular but wrong message shows itself. It scores well because it is being misread.
The objections, named. What would stop each segment acting. These map straight onto the proof points you are missing, and they are often more useful than the ranking itself.
The spread. Where opinion is genuinely divided, the output should say so rather than returning a midpoint no one holds.
Feed the named objections back in as new variants. Two or three rounds of that, at seconds each, gets you further than one traditional test.
What does an iterative message test look like?
Consider an illustrative example. You are launching a savings feature in a banking app, and the team has three taglines for the launch. Rather than argue them out in a room, you load all three into a synthetic audience of your customers, along with the two routes someone had already discarded.
You ask the question from the table above: after reading this, what do you think the feature does, and what would stop you switching it on? Then you read the answers split by the cut that matters here, the customers who already save with you against the non-savers you are trying to convert.
Route A wins overall, and it wins clearly with existing savers.
But the non-savers, the whole reason for the launch, mostly read it as "another account to open" and say they would not bother. Route C, third overall and nearly cut, is the one non-savers understood and would act on. The all-audience ranking pointed you at the group you already had.
The output also hands you the objection in the audience's own words. For non-savers the blocker was effort, the sense that this meant opening something new. That is a proof point you were missing. It tells you what to add. So you write a fourth route that leads with "no new account, switched on in one tap", and retest in seconds. This time it holds up with both groups.
The team now has a clearer route and a reason for changing it. The average alone would have favoured Route A and quietly missed the people the launch was built for.
Match the method to the stakes
The right process depends on the cost of the decision. A social post that can be changed tomorrow needs a different level of evidence from a campaign carrying a major media budget.
Use message testing early to compare more routes, expose likely objections and improve the shortlist.
For a high-stakes launch, regulated claim or decision that requires people on the record, take the surviving route into real research and the appropriate approval process. The fuller comparison is in synthetic versus traditional research.
Consideration | Traditional message test | Synthetic audience test |
|---|---|---|
Best point in the process | Final validation and evidence on the record | Early exploration and shortlist development |
Iteration | New questions usually require more fieldwork | Follow-up questions can shape the next round |
Segment analysis | Needs enough participants in each cell | Segments can be compared in the same run |
Confidential concepts | Shared with recruited participants | Can be explored before public launch |
Best use | Signing off a high-stakes route | Deciding which route deserves further investment |
This is where speed changes the quality of the work. It lets teams test a bolder idea instead of defaulting to the least controversial route. Evidence arrives while the message can still be rewritten.
What if the message is part of an image or video?
Copy rarely appears alone. A headline can change meaning when it sits beside an image, and a claim can feel more or less credible depending on the execution around it.
Test the words on their own when you need to isolate the message. Use creative testing when the decision concerns how the copy and asset work together. Keep the question specific: what did people notice, what do they think is being promised, who does it appear to be for and which part creates doubt?
3 cases, third catches people out
A regulated claim. Wording that carries legal exposure needs real respondents on record.
A final gate on major spend. At serious budget, a synthetic test narrows the shortlist and a real test signs it off. Reading the synthetic result as the sign-off is the most common way teams overreach.
Anything sensory. How a pack feels in the hand, how sound design fills a room, how something tastes. Those depend on a body in the room, not a reported opinion.
The standing limits apply too. Audiences with too little real data behind them need primary research first, and synthetic testing runs alongside a survey programme rather than replacing it. There is more on how the two fit together in the same survey given to seven panels and one AI.
Frequently asked questions
How many variants can you test at once?
Every variant runs against the same audience in a single test, so you can put the whole shortlist in at once, including the routes you would normally cut for time. The only real limit is how many results you want to read back.
Does this replace A/B testing?
No. An A/B test measures real behaviour against real spend, once something is live. A synthetic test comes before that, and decides which routes are worth putting into an A/B test in the first place.
What if the result contradicts what the team believes?
Ask a follow-up. An audience grounded in your real data holds up under questioning and gives you the reasoning behind its answer, which is usually what shifts a view in the room.
Book a walkthrough to test a live message against a synthetic audience and explore the response by segment.
