Leigh Caldwell
6.43: The rules require firms to undertake appropriate testing of their products or services. In doing so, they must assess whether the product or service will meet the identified needs, characteristics and objectives of the target market, including customers in the target market who have characteristics of vulnerability.
The FCA’s new consumer duty, with its strong focus on customer outcomes, requires you, among other things, to review all your products to:
- determine the outcomes they provide
- understand customer needs
- check for potential behavioural biases
The guidelines mention several times the need to test your products and communications, but how much testing is needed?Firms with many products are legitimately concerned that this could create an immense workload over the next six months.
6.44: Firms should consider the appropriate level of testing.
- In all cases, they must test products and services in a qualitative manner. For example, they could consider likely changes to the target market’s needs in the future and whether the product or service would continue to meet those needs.
- Where relevant, depending on the type and nature of the product or service and the risk of harm, firms must also conduct quantitative testing. This could include, for example, testing how investments would perform in different market conditions
6.47: Manufacturers could also consider consumer testing. This could be particularly relevant where there are greater risks of consumer harm.
The guidance sends mixed messages, with lines such as “In all cases, they must test products and services” and “Manufacturers could also consider consumer testing.”
Our interpretation is that the testing requirement depends on these two criteria:
- Risk of consumer harm – potentially high-risk products such as high-interest lending, or products sold to vulnerable consumers, are more likely to need testing
- Size of firm – larger firms (who have more consumers, therefore more risk of harm) are more likely to be expected to carry out testing
The FCA may provide more clarity on this question later as the guidance starts to be implemented. But based on conversations with existing regulated firms, we would suggest that small firms, with fewer than around 10 employees or 1,000 customers (especially if the customers are less vulnerable), may not need consumer testing; they can potentially rely on carrying out a desk review of their products. We call this a “behavioural audit” and it can be carried out either by a behavioural scientist or by a firm’s own staff, with the help of a guide to cognitive biases and comprehension.
Slightly larger firms, with more than 1,000 customers should carry out some level of consumer testing. At this scale, the research could be as simple as conducting some face-to-face interviews to understand what consumers need and check that they understand the purchase process. Again you could carry out this research yourself, or have a specialist agency or moderator run the interviews for you and document the results.
Larger firms with 20,000 or more retail customers should consider doing more in-depth tests. The more customers who are affected by a product or firm, the more testing is likely to be expected. The most common addition to the behavioural audit and customer interviews are online comprehension tests, in which a representative sample of consumers reads your documents and answers questions to confirm whether they understand it.
For very large firms or riskier products these methods might be supplemented with:
- Value modelling, where customers take an online survey to quantify the value they get from your products – so you can ensure they are priced fairly
- Behavioural experiments, in which new or redesigned products are A/B tested against old ones in a simulated purchasing environment to ascertain the customer outcomes they deliver
- Mystery shopping, in which researchers buy and use your products and document the process, rating it on comprehension, quality of support and any identified barriers to usage
Please note that these classifications are based on our own judgement of the appropriateness of consumer testing at each level, and do not necessarily correspond with FCA classifications such as the small/medium/large CASS firm types.
If you’d like to talk through these testing options, and find out the time and budget that might be required, just give us a call/email or download our guide here.