• 
• 

Three Part Question

In [Emergency Medicine physicians or students] does [teaching cognitive debiasing, cognitive forcing strategies or metacognition] lead to [a reduction in error attributable to cognition]?

Clinical Scenario

Very few environments rival the complexity, unpredictability, acuity, time pressures and decision density of the Emergency Department (ED)1,2. Unsurprisingly it has been described as a natural laboratory for human error3. Despite the skills of the Emergency Physician in making decisions, an unacceptable number of decisions made in the process of medical diagnoses are wrong with error or diagnostic failure rate estimated to occur in 10-15% of decisions in the ED4. Expert opinions within Emergency Medicine have highlighted the role of cognitive debiasing strategies5 and cognitive forcing strategies6 to decrease the error attributable to cognition.
The need to take all available steps to prevent error and harm from occurring has been highlighted as a moral and professional obligation in order to honour the ethical principles of beneficence, non-maleficence, fairness and justice7.

Search Strategy

Pubmed (inc. Medline), search strategy A = 106
Embase 1974-2016 via Ovid interface, search strategy A = 289
Cochrane Library, search strategy B = 220

1) Emergency, 2) Error, 3) Cognitive, and 4) Metacognition
Search strategy using search terms above:
A. 1) (All text) AND 2) (All text) AND 3) (All text) OR 4) (All text)
B. 1) (Abstract, Keywords, Titles) AND 2) (Abstract, Keywords, Titles) AND 3) (Abstract, Keywords, Titles) OR 4) (Abstract, Keywords, Titles)

Search Outcome

615 papers were returned, of which 2 were relevant8,9. These are displayed in table two.

Relevant Paper(s)

Author, date and country	Patient group	Study type (level of evidence)	Outcomes	Key results	Study Weaknesses
Sherbino et al, 2014, Canada	191 final year medical students on emergency medicine. 145 attended a 90-minute seminar on cognitive forcing (intervention group) and 46 did not (controls). Tested on six scenarios after 3 weeks.	Non-randomised controlled trial	Proportion of students identifying a subtle second diagnosis (absence of search satisfaction)	52% and 48% of students looked for a second diagnosis in the intervention and control groups respectively (p=0.13)	Smaller control group. No randomisation. Novice Clinicians. Potential contamination between groups Single 90 minute teaching intervention with remote testing interval (3 weeks). Artificial study setting with challenge of transfer to clinical setting.
			Proportion of students correctly identifying the less common explanation for the findings (absence of availability bias)	45% in both the intervention and control groups identified the correct uncommon diagnosis (p=0.98)
			Proportion of students wrongly identifying a second diagnosis (false positives)	64.5% in the intervention group vs. 76.7% in the control group (p=0.12)
Sherbino J et al 2011, Canada	56 final year medical students on emergency medicine at a single University. Each attended a 90-minute seminar on cognitive forcing. Students were then tested on four scenarios similar (near group) or dissimilar (far group) to educational cases they had reviewed, two of which had a subtle second diagnosis to detect and two did not. 47 students were tested immediately; 9 students were tested after 2 weeks	Level 4 Non-randomised experimental study	Proportion of students identifying a subtle second diagnosis (absence of search satisfaction)	64% and 55% of students looked for a second diagnosis in the near and far transfer groups respectively after immediate testing (p=0.129)	Small sample No randomisation, unequal group sizes and very few students in the delayed testing group. Novice Clinicians. No control group. Poor reference standards Artificial study setting with challenge of transfer to clinical setting.
			Proportion of students correctly identifying a less likely explanation for findings (absence of availability bias)	30% and 17% of students identified the correct uncommon diagnosis in the near and far transfer groups respectively after immediate testing (p=0.24)
			Absence of search satisfaction bias and availability bias on delayed testing after2 weeks	Only 22% and 11% of students looked for a second diagnosis in the near and far groups respectively (p<0.05 vs. immediate testing). 0% and 11% identified the correct uncommon diagnosis in each group (p<0.05 vs. immediate testing)
			Prevalence of false positive diagnoses in cases where there was no second diagnosis	53% (near group) and 32% (far group)

Comment(s)

There is currently little evidence that teaching cognitive forcing strategies reduces cognitive error in the practice of Emergency Medicine. The evidence that is available is subject to important limitations. That evidence suggests that the delivery of a single 90-minute teaching intervention to medical students has no effect on search satisfaction bias, availability bias or the prevalence of false positive diagnoses on testing after 3 weeks. No evidence is currently available on the impact of teaching cognitive debiasing, metacognition or cognitive forcing strategies on error attributable to cognition in postgraduate learners of any grade practicing in Emergency Medicine.

Clinical Bottom Line

Despite multiple expert opinions on the role of teaching and implementing training on cognitive debiasing, metacognition or cognitive forcing strategies to reduce error attributable to cognition in the ED, no study evidence can be drawn to support this statement at present.

Regarding future research there is a clear need for further research into cognitive debiasing and cognitive forcing strategies and their role in the reduction of cognitive errors made within the ED. There has been insufficient progress in systematically evaluating and implementing proposed strategies7. It is an ethical imperative to act on the expanding body of expert opinion; continued refinement of this area should be considered integral to medical education and be seen not only as a research priority but a moral and professional duty7.

References

1. Croskerry P. ED cognition: any decision by anyone at any time. CJEM 2014;16:13-9.
2. Croskerry P. The cognitive imperative: thinking about how we think. Acad Emerg Med 2000;7:1223-31.
3. Bogner MS (ED). Human error in medicine. 1st ed. Hillsdale (NJ): Lawrence Erlbaum Associates; 1994. 1994. ISBN-10: 0805813861.
4. Berner ES, Graber ML. Overconfidence as a cause of diagnostic error in medicine. Am J Med 2008;121:S2-23.
5. Croskerry P. When I say... cognitive debiasing Med Educ 2015;49:656-7.
6. Croskerry P, , Singhal G, Mamede S. Cognitive debiasing 2: impediments to and strategies for change. BMJ Qual Saf 2013;22 Suppl 2:ii65-ii72.
7. Stark M, Fins JJ. The ethical imperative to think about thinking - diagnostics, metacognition, and medical professionalism. Camb Q Healthc Ethics 2014;23:386-96.
8. Sherbino J, Kulasegaram K, Howey E et al. Ineffectiveness of cognitive forcing strategies to reduce biases in diagnostic reasoning: a controlled trial. CEJM 2014;16:34-40.
9. Sherbino J, Yip S, Dore KL et al. The effectiveness of cognitive forcing strategies to decrease diagnostic error: an exploratory study. Teach Learn Med 2011;23:78-84.