hello@irrationalagency.com | +44 (0)20 7064 6555
Back to All Posts
hand shake

How Simulated Randomized Control Trials Work and How to Leverage Them

,
Published by

The Nobel Prize in Economics was awarded recently to three dynamic and impressive economists – Esther Duflo, Abhijit Banerjee, and Kremer – working on one of the world's most important problems: relieving global poverty. Duflo is only the second woman to receive the award as well as the youngest-ever winner. All three awardees are notable for going out into the field and spending their whole careers on an issue that is much harder than what many economists do: staying in their office and working out theoretical insights on a blackboard.

Behavioral Economics and RCTs
Two of the approaches used by Duflo, Banerjee, and Kremer are of particular interest to us in market research.
 
The first is behavioral economics. Duflo and Banerjee's book Poor Economics outlines some of the behavioral and psychological traps that can keep people in poverty, and how carefully designed nudges can help them escape. If you keep up with behavioral science you'll recognize many of the terms – commitment devices, hyperbolic discounting, and anchoring for example.
 
The second is the RCT or randomized control trial. In an RCT, a new approach or incentive is randomly assigned to 50% of a population, with the other 50% as a control group. Afterward, the outcomes or behavior of each group are measured. If the test group outperforms the control group, the intervention is judged a success.
 
This is a great way to test new policies for alleviating poverty in Malawi, but not only that. The same principle is used in A/B testing of your website or email marketing. And at Irrational Agency, we have started to use RCTs in market research, most recently to test TV ads on three randomized groups for a US barbecue brand. We measured which one would buy the most product after seeing the ad, allowing us to get a true behavior measure instead of relying on claimed appeal.
 
However, sometimes it isn't possible to run a randomized test on a real population. A recent example from a client of ours in the pharmaceutical industry: they wanted to try some new communications tactics, but it would have taken months and cost millions of dollars to get legal approval to release each version and reissue all their marketing collateral. So we designed a new approach: the Simulated RCT.
 
How a Simulated RCT Works

In a simulated RCT, we recruit a sample of consumers and apply the same randomization principle as a real RCT: divide them into different groups and give a different marketing message to each one.

Then we test the behavior of each group in a simulated shopping test (or in this case, a simulated doctor-patient consultation). The group who most increases purchasing of the test product, or changes their behavior in the desired way (such as prescribing a particular medicine) is deemed the successful group. The communications message seen by that group is the winner of the test.

Not only can we identify the most effective communications message, but we can determine how much it outperforms the other test messages and the control group. Other clients who have used this approach include an energy drinks company testing key drivers of consumption, an insurance company measuring the effectiveness of different marketing touchpoints, and a whisky company exploring how to maximize their returns from e-commerce.

Simulated RCTs are not suitable for every type of research, of course. And there are lots to discuss both RCTs and Simulated RCTs, such as how best to generalize the results to other situations. We designed the Simulated RCT to address these questions as well as possible, for example by measuring not just which stimulus works best, but why it works (using our System 3 analysis method).

How to Leverage Simulated RCT?

If you'd like to take some inspiration from the Nobel Prize winners making an impact against global poverty and hunger, why not try a cost-effective and accurate Simulated RCT on your next research question? We're donating 10% of all our profits from Simulated RCTs to Evidence Action, a charity that uses RCTs to work out the most effective ways to spend its donors' money around the world to save the most lives.

Simulated RCTs accurately predict the future behavior of your consumers in response to the marketing strategies you will introduce. Helping your stakeholders make reliable, informed decisions based on actual evidence.

Related Posts

Back to All Posts