Objectives and hypotheses
Are the objectives of the study clearly stated?
Yes. To compare post-call performace after heavy call with performance after light call with blood alcohol concentration between .04-.05. As well as evaluate association between self-assessed and actual performance.
Is the study design suitable for the objectives?
Yes, a prospective 2-session within-subject study, with 4 conditions: light call, light call with alcohol, heavy call and heavy call with placebo.
Who / what was studied?
34 pediatric residents(18 female, 16 male) in an academic medical center(Brown University)
Was a control group used if appropriate?
Yes, residents served as thier own control(light call).
Were outcomes defined at the start of the study?
Yes, outcomes were defined as scoring on Stanford Sleepiness Scale, Visual Analog Scale (for assesment of alertness, sleepiness, and overall functioning), Psychomotor Vigilance Task, Continuous Performance Test, Simulated Driving Task, and Posttest Self-assessments.
Was this the right sample to answer the objectives?
Yes, Although participants were not allowed to ingest caffeine after noon, to nap, or to ingest good or drink within 4 hours of testing.
Is the study large enough to achieve its objectives? Have sample size estimates been performed?
Were all subjects accounted for?
Were all appropriate outcomes considered?
Yes, although authors did not perform an intention-to-treat analysis.
Has ethical approval been obtained if appropriate?
Measurement and observation
Is it clear what was measured, how it was measured and what the outcomes were?
Yes, the authors clearly described each measurement modality, as well as how individuals were scored within each method.
Was the assessment of outcomes blinded?
There was incomplete blinding.
Was follow up sufficiently long and complete?
Are the measurements valid?
No, investigators did not randomize or counterbalance the order of test conditions. The lab tests of performance have not been validated against medical tasks. As pointed out by the authors, response times, vigilance, and driving performance are not measures of clinical efficacy or patient safety.
Are the measurements reliable?
Are the measurements reproducible?
Yes, although there may have been a self-selection bias.
Presentation of results
Are the basic data adequately described?
Are the results presented clearly, objectively and in sufficient detail to enable readers to make their own judgement?
Yes, results for each outcome were cleary stated in words, and in graphical(table) format.
How large are the effects within a specified time?
Compared with light call, heavy call reaction times were 7% slower, commision errors were 40% higher, and lane variability and speed variability were 27%, and 71% greater on the driving simulator, respectively.
Are the results internally consistent, i.e. do the numbers add up properly?
Are the data suitable for analysis?
Are the methods appropriate to the data?
Are any statistics correctly performed and interpreted?
Are the results discussed in relation to existing knowledge on the subject and study objectives?
Is the discussion biased?
Are the author's conclusions justified by the data?
Yes. Although the authors acknowledge that these lab tests of performance have not been validated against medical tasks, the indirect implication is that residents working 80-90 hour weeks are at an equivalent or greater risk compared with an intoxicated physician.
What level of evidence has this paper presented? (using CEBM levels)
Does this paper help me to answer my problem?
Yes, studies such as this should motivate clinical managers to reflect on the legal and scientific defensibility of current work practices.
Can any necessary change be implemented in practice?
Yes. Beware of post-call performance--limit significant patient care during post-call hours. Beware of driving home after a long call.
What aids to implementation exist?
80 hr. work week limit. Call rooms available in house. Reporting of duty hour violations.
What barriers to implementation exist?
In some scenarios, limiting working hours may increase risk to patients and physicians. For example, restricted working hours may lead to limited access to health care practitioners through a reduction in labor supply, insufficient clinical preparation for the "real world", increased sleep restriction in a senior physicians, or increases in error rates due to work intensification.
Are the study patients similar to your own?
Emergency medicine or intensive care rotations may have a different workload than elective or selective rotations. Thus, the results need to be interpreted in light of the contribution of the extended work hours as well as the type of work being performed.
Does the paper give any conclusions that will affect what you will offer or tell your patient?
Movement of residencies toward limiting hours worked by residents is neccessary, and should in theory, limit or decrease the medical errors and increase resident safety.