Directional, and checked in public.
We test simulated customers against what real customers actually did, with the answers sealed before we start. Then we publish every result, including where we fall short.
Better than chance in all three. A model trained on each company’s own data still does better in all three, and we publish that too.
How the tests work.
A test you can only quote in your favour isn’t a test. So each one is set up so that we can’t peek, and so that you can check.
The outcome is sealed first.
Before any simulated customer is built, we lock away what each real customer actually went on to do, and record a fingerprint of it. Nothing that makes predictions can see it.
Predictions come from the past only.
Each simulated customer is built from that customer’s history up to a cut-off date. It predicts what happens next, one prediction per customer, written down before the seal is opened. No tuning afterwards.
Scored against chance, and against a trained model.
Then we open the seal and compare. Not only with a coin toss, but with a model trained on the company’s own data, which is the harder bar. We report both, even where the trained model wins.
Code and results are public.
The code, the seals and every prediction sit in a public repository. Rebuild the fingerprint yourself and confirm nothing changed after the fact.
See the audit repository →How each step works
The outcome is sealed first. Before any simulated customer is built, we lock away what each real customer actually went on to do, and record a fingerprint of it. Nothing that makes predictions can see it.
Predictions come from the past only. Each simulated customer is built from that customer’s history up to a cut-off date. It predicts what happens next, one prediction per customer, written down before the seal is opened. No tuning afterwards.
Scored against chance, and against a trained model. Then we open the seal and compare. Not only with a coin toss, but with a model trained on the company’s own data, which is the harder bar. We report both, even where the trained model wins.
Code and results are public. The code, the seals and every prediction sit in a public repository. Rebuild the fingerprint yourself and confirm nothing changed after the fact.
Example: the sealed outcome record
Each fingerprint is made from the real outcomes before any prediction runs. Recompute it from the published results: sha256(customer → outcome). If one value had changed, the fingerprint wouldn’t match.
How closely the population matches.
A different question from prediction. We put simulated populations through national surveys that real people have already answered, then measure how closely the spread of their answers matches the real one.
Against the Pew American Trends Panel
Average, against Pew surveys in each country
Against Pew surveys
Where these figures come from. Simulatte’s Pew calibration studies, April 2026: US v2, Europe v2 (nine countries) and India v2. Each score is on the survey questions the population was calibrated to; on held-out questions the scores were 81.9% (US), 68.6% (Europe average) and 95.9% (India). They were measured on research populations built with the same persona generator for these studies, not on the ready-made audiences in the product today. Code and results →
This is how closely the simulated population’s makeup matches the real population. It is not prediction accuracy. These are the figures after calibration. On questions held back from calibration the match is lower, and we publish both, country by country.
See the population studies →What it’s good for, and what it isn’t.
Good for
- A directional read, fast. Which way it’s likely to go, before you spend anyone’s time.
- Narrowing options. Three headlines down to one, five price points down to two.
- Finding who disagrees, and why. The groups inside a market and the reasons they give.
- Deciding what to ask real people. So interviews go straight to what matters.
Not a replacement for
- Your real customers on high-stakes calls. A launch, a price change, a bet the business rides on: confirm it with people.
- A model trained on your own data. If you have a long record of who left, a trained model ranks who will leave better. Our tests show that.
- An exact forecast. Read the results as a direction and a reason, not a number to put in the plan.
Real people That’s why you can check with real people side by side: the same questions, with the places they agree and differ called out.
How it works →The method, in plain words.
Four steps, and an example customer
Built from real data about a market.
Who lives there, how they shop, what they earn and what they care about. Traits go together the way they do in real people, rather than being assigned at random.
Each one is a person, not an average.
Every simulated customer has a background, habits and a memory of what they’ve seen. Ready-made audiences cover 12 countries, or one is built from a description of your market.
They react the way people do.
Shown a question, a price or an ad, each one recalls what’s relevant, weighs it and decides, then says why in their own words.
You see the spread, not one answer.
Ask many at once and you get the share on each side, the groups that differ and the reasons behind them. A chatbot gives you one plausible average; this gives you the disagreement.
Simulated customer, modelled on real people. Not a real respondent. Illustrative answer.
Questions about accuracy.
What does a score of 0.75 mean?
Take one customer who cancelled and one who stayed. A score of 0.75 means the simulation picks the one who cancelled as more likely to leave three times out of four. A coin toss gets it right half the time, which is 0.5. It is a ranking score: it doesn’t say how many customers will leave.
Why don’t you give one accuracy number?
Because it depends on the question. We publish each test separately, against real outcomes, so you can see where it works well and where it’s weaker.
Why publish that a trained model does better?
Because it’s true, and you should know it. If you have a long record of who left, use a trained model to rank who will leave. Simulated customers are for the questions that record can’t answer: why, and what would change their mind.
Are the 95.3%, 93.3% and 97.6% figures prediction accuracy?
No. They measure how closely a simulated population’s answers to national surveys match the real population’s answers. They say the population is built well, not that a prediction will come true. They come from our April 2026 Pew calibration studies on research populations, not from the ready-made audiences in the product.
Can I check the results myself?
Yes. The code, the sealed outcomes and every prediction are in a public repository. You can rebuild the seal and recompute the scores.
Are simulated customers real people?
No. They are simulated customers, modelled on real people. When the decision matters, put the same questions to your own customers and compare.
Check it against something you already know.
Run a question you’ve already answered the hard way, and compare.