Cohort
Kapur N, Cooper J, Rodway C, Kelly J, Guthrie E, Mackway-Jones K.Predicting the risk of repetition after self harm: cohort study
BMJ
2005;330:394-395
- Submitted by:Simon Carley - Consultant in Emergency Medicine
- Institution:Manchester Royal Infirmary
- Date submitted:11th March 2005
1
Objectives and hypotheses
1.1
Are the objectives of the study clearly stated?
Yes
"We investigated the predictive value of risk assessments after an episode of self harm and compared assessments made by emergency department staff with those made by psychiatric staff"
The basic idea was to see if, after assessment, doctors were any good at predicting suicide in patients presenting with DSH. The comparison was between ED and Psych docs.
"We investigated the predictive value of risk assessments after an episode of self harm and compared assessments made by emergency department staff with those made by psychiatric staff"
The basic idea was to see if, after assessment, doctors were any good at predicting suicide in patients presenting with DSH. The comparison was between ED and Psych docs.
2
Design
2.1
Is the study design suitable for the objectives?
Yes. A longitudinal prospective cohort is the best way of answering this question
2.2
Who / what was studied?
"Four hospitals provide emergency care in the cities of Manchester and Salford. As part of the Manchester and Salford self harm project (MASSH) we collected data on all people aged at least 16 who presented with self harm in 1997-2001"
This seems an appropriate group of patients relevant to my practice.
This seems an appropriate group of patients relevant to my practice.
2.3
Was a control group used if appropriate?
Not relevant
2.4
Were outcomes defined at the start of the study?
The outcome was repetition of self harm which seems appropriate.
2.5
Was this the right sample to answer the objectives?
Yes
2.6
Is the study large enough to achieve its objectives? Have sample size estimates been performed?
No sample size given in this brief paper. The confidence intervals given suggest that this is a reasonably well powerered study.
2.7
Were all subjects accounted for?
They collected data from the same hospitals. The possibility of people attending other hospitals outwith the MASSH system is not addressed.
2.8
Were all appropriate outcomes considered?
No. The brief psychological assessment may have resulted in referral, cure, drug therapy etc. These are all clinically important but not mentioned here.
2.9
Has ethical approval been obtained if appropriate?
Not stated.`
3
Measurement and observation
3.1
Is it clear what was measured, how it was measured and what the outcomes were?
Yes. Limitations of outcome data only through MASSH.
3.2
Was the assessment of outcomes blinded?
Not relevant
3.3
Was follow up sufficiently long and complete?
Probably. One year is a long time and probably is long enough to define the effects of the individuals current problem/illness./
3.4
Are the measurements valid?
Yes
3.5
Are the measurements reliable?
No as stated above
3.6
Are the measurements reproducible?
Probably
4
Presentation of results
4.1
Are the basic data adequately described?
There is little baseline data in this brief report. It would be useful to know about methods of DSH, intercurrent illness or past psych hx.
4.2
Are the results presented clearly, objectively and in sufficient detail to enable readers to make their own judgement?
Yes
4.3
How large are the effects within a specified time?
1195/8707 patients had a second episode of DSH within a year. There was a good spread of assessments in all groups.
If a "high risk assessment" is defined as a predictor of repeat harm:-
The ED docs had a 32 sensitivity and an 82% specificity
The psych docs had a 17.3% sensitivity and a 91.6% specificity
The positive predictive value for ED docs was 21.3% and 25.7% for psych.
When patients were reviewed by both groups there was little change in the findings (data not give here). There was only moderate agreement kappa=0.17
In other words they are both rubbish at it.
If a "high risk assessment" is defined as a predictor of repeat harm:-
The ED docs had a 32 sensitivity and an 82% specificity
The psych docs had a 17.3% sensitivity and a 91.6% specificity
The positive predictive value for ED docs was 21.3% and 25.7% for psych.
When patients were reviewed by both groups there was little change in the findings (data not give here). There was only moderate agreement kappa=0.17
In other words they are both rubbish at it.
4.4
Are the results internally consistent, i.e. do the numbers add up properly?
Yes
5
Analysis
5.1
Are the data suitable for analysis?
Yes
5.2
Are the methods appropriate to the data?
Yes. Sens, Spec with confidence intervals. PPV also given with confidence intervals.
5.3
Are any statistics correctly performed and interpreted?
Yes
6
Discussion
6.1
Are the results discussed in relation to existing knowledge on the subject and study objectives?
Yes.
6.2
Is the discussion biased?
Heavily. This is clearly written by a psychiatrist. They make light of the lower predictive value of ED assessments but they are both rubbish and statistically there appears to be no difference! The fact that ED docs are more cautious is clinically probably a very good idea!
7
Interpretation
7.1
Are the author's conclusions justified by the data?
I think they are but they are weighted to a psych perspective
7.2
What level of evidence has this paper presented? (using CEBM levels)
1 (I think)
7.3
Does this paper help me to answer my problem?
Yes. I now know that risk assessment by ED or Psych staff in the format practiced in this paper is a poor tool. Psychiatrists are no better than ED docs at it. My practice suggests that psychiatrists take considerably longer to come to the same poor conclusion. Maybe the answer is that it cannot be done in the acute setting, maybe it is because we use the wrong tools to come to decisions.
8
Implementation
8.1
Can any necessary change be implemented in practice?
We need to look for other tools for risk assessment.
8.2
What aids to implementation exist?
None
8.3
What barriers to implementation exist?
None
8.4
Are the study patients similar to your own?
Yes
8.5
Does the paper give any conclusions that will affect what you will offer or tell your patient?
nO