Survey (including pre-test probabilities)
McColl A, Smith H, Field J.General practitioner' perceptions of the route to evidence based medicine: A questionnaire survey.
BMJ
1998; 316:361-5.
- Submitted by:Joel Dunning - RCS Research Fellow
- Institution:MRI
- Date submitted:17th December 2002
1
Objectives and hypotheses
1.1
Are the objectives of the study clearly stated?
The objectives were stated at the start of the abstract :
Objectives: To determine the attitude of general practitioners towards evidence based medicine and their related educational needs.
This was expanded on by the authors at the end of the introduction ( as is standard practise )
To fulfil the objectives of the study we set out to identify general practitioners' attitude towards evidence based medicine
Awareness and perceived usefulness of relevant extracting journals, review publications, and databases
Ability to access relevant databases and the world wide web
Understanding of technical terms used in evidence based medicine
Views on the perceived major barriers to practising evidence based medicine
Views on how best to move from opinion based to evidence based medicine.
Objectives: To determine the attitude of general practitioners towards evidence based medicine and their related educational needs.
This was expanded on by the authors at the end of the introduction ( as is standard practise )
To fulfil the objectives of the study we set out to identify general practitioners' attitude towards evidence based medicine
Awareness and perceived usefulness of relevant extracting journals, review publications, and databases
Ability to access relevant databases and the world wide web
Understanding of technical terms used in evidence based medicine
Views on the perceived major barriers to practising evidence based medicine
Views on how best to move from opinion based to evidence based medicine.
2
Design
2.1
Is the study design suitable for the objectives
The objective was to determine the attitude of general practitioners towards evidence based medicine and their related educational needs.
A survey describes or investigates the status of a condition or situation at a single point in time.
This paper is a questionnaire study and this seems appropriate
A survey describes or investigates the status of a condition or situation at a single point in time.
This paper is a questionnaire study and this seems appropriate
2.2
Who / what was studied?
Setting: General practice in the former Wessex region, England.
Subjects: Randomly selected sample of 25% of all general practitioners (452), of whom 302 replied
Subjects: Randomly selected sample of 25% of all general practitioners (452), of whom 302 replied
2.3
Was this the right sample to answer the objectives?
This was intended to be a representative sample of GPs in Wessex
2.4
Did the subject represent the full spectrum of the population of interest?
GPs in Wessex may not be representative of all GPs nationally. In particular the paper mentions that there have been several EBM initiatives in this region recently and so Wessex GPs are likely to know more about EBM than the general public.
Also there was only a 67% response rate. Non responders were less likely to have the MRCGP and to attend the Wessex primary care research network. In addition we know nothing of their age, ethnic background, practise details or medical school so it is difficult to see exactly how these people differed from the responders
Also there was only a 67% response rate. Non responders were less likely to have the MRCGP and to attend the Wessex primary care research network. In addition we know nothing of their age, ethnic background, practise details or medical school so it is difficult to see exactly how these people differed from the responders
2.5
Is the study large enough to achieve its objectives? Have sample size estimates been performed?
No sample size estimates done.
2.6
Were all subjects accounted for?
Stated in the paper :
Of the 452 questionnaires we sent out, two were returned because the general practitioners had retired. We received 302 replies (67%) to the remaining 450 questionnaires.
Of the 452 questionnaires we sent out, two were returned because the general practitioners had retired. We received 302 replies (67%) to the remaining 450 questionnaires.
2.7
Were all appropriate outcomes considered?
Yes, it seems that the questionnaire was comprehensive
2.8
Has ethical approval been obtained if appropriate?
Not mentioned.
2.9
What measures were made to contact non-responders?
We sent reminders to non-respondents in June and July 1997, and data on non-respondents were collected by AMcC from teaching and research networks and the 1997 Medical Directory
2.10
What was the response rate?
67%
3
Measurement and observation
3.1
Is it clear what was measured, how it was measured and what the outcomes were?
Yes. The questionnaire was described and is available on line. Of note not all data in the questionnaire was used in the paper, e.g. Age data was collected but not presented
3.2
Are the measurements valid?
Validity of a questionnaire is the issue of whether the questionnaire measures what the investigators set out ot measure.
It may be argued that if you ask someone if they understand a topic but don't test that they understand it , then this is not valid as the GPs may be stretching the truth so as not to feel ignorant
It may be argued that if you ask someone if they understand a topic but don't test that they understand it , then this is not valid as the GPs may be stretching the truth so as not to feel ignorant
3.3
Are the measurements reliable?
Reliablility of a questionnaire is if a GP fills it in twice, whether you will get the same answer. This was not tested
3.4
Are the measurements reproducible?
Reproducibility is usually if two different testers apply the test or measure, whether they get the same result. As this was a self completed questionnaire, this is not relevent
4
Presentation of results
4.1
Are the basic data adequately described?
It is usual practise for table 1 to be a descriptive table of your subject group. This has been done.
The other tables are also clearly presented
The other tables are also clearly presented
4.2
Are the results presented clearly, objectively and in sufficient detail to enable readers to make their own judgement?
Yes
4.3
Are the results internally consistent, i.e. do the numbers add up properly?
Yes
5
Analysis
5.1
Are the data suitable for analysis?
Yes
5.2
Are the methods appropriate to the data?
Paper stated that :
We entered the data into a spreadsheet. We initially identified 38 categories, but these were grouped into broader categories during the analysis. We analysed data from the visual analogue scales using SPSS for Windows 6.1.2 and analysed the other data using Microsoft Excel 5.0. We compared differences between respondents and non-respondents using the Chi-2 test.
The CHi-2 test is the appropriate test for categorical variables
We entered the data into a spreadsheet. We initially identified 38 categories, but these were grouped into broader categories during the analysis. We analysed data from the visual analogue scales using SPSS for Windows 6.1.2 and analysed the other data using Microsoft Excel 5.0. We compared differences between respondents and non-respondents using the Chi-2 test.
The CHi-2 test is the appropriate test for categorical variables
5.3
Are any statistics correctly performed and interpreted?
Only a few simple statistical tests were performed as most data was descriptive.
The statistics done were performed correctly
The statistics done were performed correctly
6
Discussion
6.1
Are the results discussed in relation to existing knowledge on the subject and study objectives?
yes. They discuss their findings in relation to a patient consultation study and and Obs & Gynae paper.
6.2
Is the discussion biased?
Review is descriptive and seems fair.
The finding that GPs were on the whole positive about EBM but don't actually want to do it themselves, preferring to rely on guidelines from other people is perhaps not fully discussed.
The implication might be that if the GPs would actually prefer to read guidelines, as long as they were happy that they were well constructed, they might not actually be interested in EBM at all !!
The finding that GPs were on the whole positive about EBM but don't actually want to do it themselves, preferring to rely on guidelines from other people is perhaps not fully discussed.
The implication might be that if the GPs would actually prefer to read guidelines, as long as they were happy that they were well constructed, they might not actually be interested in EBM at all !!
6.3
Can the results be generalised?
There is a big issue as to whether a self selected group of the most positive GPs in 1997 , from Wessex where interventions such as EBM courses have been run, is applicable to GPs in the UK, GPs internationally, or health-workers as whole now.
7
Interpretation
7.1
Are the authors' conclusions justified by the data?
Yes
7.2
What level of evidence has this paper presented? (using CEBM levels)
See http://minerva.minervation.com/cebm/ for details of these catagories
This is level 3b
This is level 3b
7.3
Does this paper help me answer my problem?
I don't have a problem with EBM, as all my questions have been answered on my BEST BETS course !!
8
Implementation
8.1
Can any necessary change be implemented in practice?
For GPs in your area changes can be made to their access to evidence summaries
8.2
What aids to implementation exist?
The internet, NHS initiatives and lectures attended by GPs would all be great helps to dissemination
8.3
What barriers to implementation exist?
The large amount of data that GPs receive through the post and GP perceptions may slow this seamless dissemination process a little