How to test boosts

There are at least three ways to empirically test boosts:

1. Proof-of-concept tests: Does the boost achieve the policy goal?
   Case study: Detecting microtargeting

2. Comparative tests of two or more boosts: Which boost is better at reaching the policy goal?
   Case study: Risk communication about opioid use

3. Comparative tests of a boost relative to a nudge or another class of policy intervention: Which kind of intervention is better at reaching the policy goal? Do the interventions complement each other in meaningful ways?
   Case study: Improving hand hygiene

Proof-of-concept test: Detecting microtargeting online

What is the problem?

Online platforms give advertisers the ability to microtarget: to exploit people’s personal vulnerabilities by tailoring different messages for the same thing (e.g., a product or political candidate) to people’s vulnerabilities (e.g., personality). Compared to traditional audience segmentation, microtargeting can be ethically problematic because the personal attribute in question might concern sensitive characteristics (e.g., sexual orientation or a specific anxiety), because it can occur without recipients’ knowledge or consent, and because it allows advertisers to make contradictory promises and pledges to different groups of people, unbeknownst to the general public—this is particularly problematic in political advertising.

What is the boost?

Based on the finding that people can be “inoculated” against misinformation, Lorenz-Spreen et al. (2021) devised and tested a boost that prompted participants to reflect on the type of personality (extraversion–introversion) being microtargeted. They examined whether this intervention would increase people’s competence to detect whether an advertisement was microtargeting them. Participants completed an 8-item extraversion questionnaire, then received truthful personalized feedback on their extraversion score relative to a large sample of online participants. Specifically, participants were told whether their personality tended more towards extraversion or introversion, and their percentile was shown both numerically and visually. Feedback was accompanied by a simple definition of extraversion.

What are the results?

Participants were shown 10 ads for beauty products and had to decide whether or not they were targeted at their personality. Accuracy was measured in terms of whether participants correctly identified microtargeted ads. In the boosting condition the mean accuracy rate was 90%, 26 percentage points higher than in the control condition (64%). This difference translates into a probability of .78 that a randomly selected participant from the boosting condition more accurately detects microtargeting than a randomly selected participant from the control condition.

Reference

Lorenz-Spreen, P., Geers, M., Pachur, T., Hertwig, R., Lewandowsky, S., & Herzog, S. M. (2021). Boosting people’s ability to detect microtargeted advertising. Scientific Reports, 11(1), Article 15541. https://doi.org/10.1038/s41598-021-94796-z

Note

For more on the digital domain, see the digital boosts page.

Comparative tests of two or more boosts: Communicating risks of opioid use

What is the problem?

In Germany, about 80% of patients who are prescribed strong opioids long-term (> 3 months) suffer from chronic noncancer pain, even though national guidelines advise against the long-term use of strong opioids for this patient group. Overprescription of opioids is a problem across Europe and the United States. One possible explanation for this phenomenon is that physicians’ risk perceptions are miscalibrated.

What are the boosts?

In a randomized clinical trial, Wegwarth et al. (2021) studied two boosting interventions for improving physicians’ prescription practices: communicating risk through a fact box and communicating risk through interactive simulated experience (see Figure 1). In different ways, both boosts aimed to educate physicians on the benefit–harm ratio associated with the long-term administration of strong opioids in patients with chronic noncancer pain. Benefits and risks were presented in frequencies, adjusted to the same denominator (per 100 people), and compared with a control group (nonopioids or placebo). Unlike the fact box, the interactive simulation allowed participants to sequentially and over time observe a drug’s benefit and its potential tapering off, as well as the possible emergence of serious harm.

What are the results?

The two risk formats improved the mean risk estimates on strong opioids’ benefits and harms equally well among 300 pain specialists and 300 general practitioners. The simulated experience condition was, however, significantly more effective than the fact box in reducing physicians’ reported actual prescription rates. It also resulted in a higher actual implementation of intended behavior in terms of prescribing other therapies (drug-based and non-drug-based) among both groups of physicians.

Note

Simulated experience as a way to communicate risk information has also been shown to be effective for financial investment decisions (see also this outline of a description-experience framework of the dynamic human response to risk).

For more on risk literacy boosts, see here.

Reference

Wegwarth, O., Wind, S., Goebel, E., Spies, C., Meerpohl, J. J., Schmucker, C., Schulte, E., Neugebauer, E., & Hertwig, R. (2021). Educating pharmacists on the risks of strong opioids with descriptive and simulated experience risk formats: A randomized controlled trial. MDM Policy & Practice. https://doi.org/10.1177/23814683211042832

Comparative tests of a boost and a nudge: Improving hand hygiene

What is the problem?

In hospitals, hand hygiene compliance is crucial to preventing hospital-induced infections. All hospitals have standardized protocols that provide details about when and how to perform hand hygiene. However, many studies report compliance rates under 50% and infections continue to be transmitted between patients and hospital employees.

What is the boost?

The boosting intervention took the form of posters and flyers aimed at helping nurses understand the frequency of hospital infections and the role of their hand hygiene in causing them. The actual magnitude of the risk was presented in a frequency format in order to help nurses better understand the risks of deficient hand hygiene. The tagline used was “prevent infections.” Facts about infection risks were listed in frequencies rather than probabilities, including:

“One in every twenty patients receive a hospital-induced infection.”

“Research shows that in two American hospitals, the number of cases with MRSA (Methicillin-resistant Staphylococcus aureus) infections decreased by half after healthcare employees improved their hand hygiene.”

What is the nudge?

The nudging intervention was designed to reframe the issue of hand hygiene. Reframing aims to shift the perspective of decision makers in ways that change their subjective evaluations of their choices. For example, reframing blood donation as preventing deaths instead of saving lives can increase donations. This nudge, communicated via posters and flyers, reframed the hand hygiene protocol from an extra burden to a moment of care for the patient. The tagline used was “in good hands,” which, accompanied by an image of hands being cleaned, implicitly related caring for patients to having clean hands. Both interventions were highly cost-effective.

What are the results?

A 4-week nonrandomized quasi-experiment found that both interventions led to significantly higher compliance compared to pre-intervention levels. The results suggest that there might be a temporal difference in the effectiveness of nudges and boosts after they are removed: Directly after the intervention, the effect of the nudge was stronger than the effect of the boost. However, after the interventions were removed the effect of the boost remained at a similarly high level, whereas the effect of the nudge faded somewhat.

Reference

van Roekel, H., Reinhard, J., & Grimmelikhuijsen, S. (2021). Improving hand hygiene in hospitals: Comparing the effect of a nudge and a boost on protocol compliance. Behavioural Public Policy, 1–23. https://doi.org/10.1017/bpp.2021.15

See also this popular-science article on the same study:

van Roekel, H., Reinhard, J., & Grimmelikhuijsen, S. (2021, October 18). Building the behavior change toolkit: Designing and testing a nudge and a boost. Behavioral Scientist. https://behavioralscientist.org/building-the-behavior-change-toolkit-designing-and-testing-a-nudge-and-a-boost/