Using BETs to teach EBM
The process of writing and reviewing BETs can be a comprehensive introduction to the principles and practice of Evidence Based Medicine. In Manchester Royal Infirmary's Accident & Emergency Department, BETs have formed part of a structured journal club for clinical staff (full details below), enabling staff first-hand experience of critical appraisal, and the process of applying its results to clinical practice. Departmental policy is often informed and changed by BETs written within the department.
Our programme for medical students includes a significant amount of teaching about EBM, and often uses BETs and the process of creating them as an example of how to apply the principles of EBM and critical appraisal.
Journal Club Paper
Keeping up to date with new research and clinical developments is one of the major challenges in post graduate education 1. In a broadly based speciality such as emergency medicine this is particularly difficult as relevant articles may be published in a wide spectrum of journals. Estimates of reading times amongst clinicians have shown that it is impossible for an active clinician to keep up to date by reading journals on an individual basis 2.
There has been increasing interest about how the principles of evidence based medicine (EBM) can be introduced into clinical practice and, in common with other specialists, practitioners of emergency medicine are struggling with this problem. Emergency medicine is a relatively new speciality with unique features - in particular single patient episodes, short patient consultations, no ward rounds, a wide scope of practice, a limited research base (especially for more minor complaints), and large numbers of junior doctors led (but rarely directly supervised) by a small number of more senior colleagues. These particular factors mean that many of the established methods for introducing EBM (which were developed in a different and less pressured context) are unsuitable for use in an emergency department.
In an attempt to overcome some of these problems an evidence based emergency medicine journal club was set up at the Manchester Royal Infirmary in 1997. This had the triple aims shown in Box 1:
This paper outlines the methods by which the aims were achieved and explores how the findings of an evidence based journal club (the evidence for the practice of emergency medicine) can be disseminated into clinical practice.
The Journal Club
The journal club meets weekly for approximately one hour. It is held within the emergency department and is open to all medical and nursing staff. The authors represent the initial core members of the club which is now open to clinicians from other departments and hospitals. The aims of the club have been achieved as follows:
Teaching critical appraisal of the literature
A key component to practising evidence based medicine is the ability to interpret the literature 3. This ability to critically appraise papers is a key skill for practising emergency physicians and is now a component of the final fellowship in emergency medicine in the United Kingdom 4. To achieve a basic competence in this skill a series of workshops were constructed based around the critical appraisal text by Crombie 5. This was chosen for its brevity and its use of checklists.
The workshop program is outlined in box 2. Each session was led by a nominated member of the group with the assistance of one of the senior physicians experienced in critical appraisal.
The series of tutorials were designed to cover the standard types of paper published in journals 6. From week 4 to week 9 papers relevant to emergency medicine that utilised the particular methodology under discussion were selected in advance. These papers, which are listed in box 3, were distributed to all group members one week before the workshop so as to allow appraisal prior to the meeting. In this way the group were able to develop and exercise critical appraisal skills across a range of paper types and standards.
Keeping abreast of current developments in emergency medicine
There are a large number of journals that could contain useful papers for the practising emergency physicians, yet it would be impossible for any one individual to scan, read and appraise all of them.
Many journal clubs attempt to highlight interesting or relevant papers as a mechanism for changing practice. However, this is often done in an unstructured and haphazard fashion. In order to ensure that important articles were not missed and that time was not wasted on reading irrelevant or poor quality articles a structured approach was taken. A list of 17 peer reviewed journals that were both relevant to the practice of emergency medicine and available for study was formulated. Each journal was assigned a number of times to be reviewed per year (the review rate reflecting both the number of issues in a year and the estimated chances of quality emergency medicine papers appearing). The list of journals and their review rates are shown in Box 4.
Only the best, most relevant or interesting articles from the journal of the week are presented to the other members of the journal club. As was the case during the initial workshops papers are made available one week in advance and are appraised using the checklists from the critical appraisal guide 5, 6, 7. Papers are rated using the categorisation of evidence shown in box 5 18. Compelling evidence from high quality papers is integrated into departmental policy.
This structured approach ensures that important papers are not overlooked by individual members of the journal club. In total 49 sessions per year are allocated to particular journals. Since 2 papers are reviewed each week at least 98 papers are critically appraised by the group per year.
On several occasions no high quality papers have been found following the review of the journal. In this case lower quality papers are reviewed and study design improvements are discussed. This process educates the group in research design as well as in critical appraisal. Furthermore potential research topics have been highlighted in this way.
Introducing evidence based medicine into practice "Making BETs"
It could be argued that evidence based practice can be introduced into emergency medicine merely by scanning and appraising journals as outlined above; it is certainly the case that some changes in practice have come about in this way. However this process of learning is not an accurate reflection of clinical practice since it is patients not journal articles that present to the emergency department. Clinical practice is based around making decisions on real patients, and it is essential that evidence based medicine reflects this. Thus evidence that pertains to questions that arise during clinical encounters should also be addressed. As described elsewhere 19 we have modified the Critically Appraised Topic (CAT) technique for use in emergency medicine. The reports generated, called Best Evidence Topic reports or BETs, are used to introduce and change clinical practice within the Emergency Department.
Topics for BETs are selected by the group and each week one member of the group presents a BET report.
Structured journal clubs are not new to emergency medicine 20, 21, 22 but we believe that this is the first report of a journal club being used to implement evidence based practice in an emergency department. Many emergency departments and training rotations in the UK participate in journals clubs of some kind. Experience has shown these to be unstructured and rather haphazard affairs that do little to establish the practice of evidence based medicine. With EBM becoming an ever more important aspect of training and clinical practice we advocate the structure described here to other emergency medicine journal clubs.
No formal assessment of group satisfaction was undertaken before and after the development of the journal club. Anecdotally we found that participants did not feel that their reading time had significantly increased. However, their reading time was more focused, critical and across a much wider range of journals. Overall the members of the group consider the journal club to be a success, and an effective use of the limited amount of time available for private study in a busy emergency department.
We have not assessed the impact of the journal club on clinical practice and patient care within the department as no data was collected prior to the development of the journal club system. Anecdotally lessons learned from BETs and structured journal scans have been implemented within the department.
Emergency medicine has close links with a large number of other specialities. It is important that evidence used in the emergency department is shared with in hospital specialities and on call teams. We believe that the construction and dissemination of the BET's can aid this inter-speciality liaison and improve clinical practice.
Traditional methods of instituting evidence based medicine are not applicable to the emergency department setting. A structured journal club was designed to teach critical appraisal, keep abreast of developments in the field of emergency medicine and institute evidence based medicine in this setting.
A proactive approach to EBM is necessary in emergency medicine. The use of a journal club to critically appraise journals relevant to emergency medicine and to construct and disseminate Best Evidence Topics reports (BETs), is one mechanism by which this can be achieved.
US task Force on Preventative Health Care in Stevens A, Raftery J (Ed) Health Care Needs Assessment: The epidemiologically based needs assessment reviews. Second Series. Radcliffe Medical Press. 1997.