Review or meta-analysis
David Oliver, James B Connelly, Christina R Victor, Fiona E Shaw, Anne Whitehead, Yasemin Genc, Alessandra Vanoli, Finbarr C Martin, Margot A GosneyStrategies to prevent falls and fractures in hospitals and care homes and effect of cognitive impairment: systematic review and meta-analyses
BMJ
2007 Jan 13; 334 (7584): 82-88
- Submitted by:Stefan Tino Kulnik - Physiotherapist
- Institution:Imperial College Healthcare NHS Trust
- Date submitted:24th November 2009
1
Objectives and hypotheses
1.1
Are the objectives of the study clearly stated?
Yes. To synthesise and evaluate the evidence for prevention of falls and fractures for populations who are medically unstable with a high prevalence of cognitive impairment, as is typical in hospitals and care homes; to inform the development of guidance on best practice; to avoid implementation of ineffective or harmful strategies; to identify gaps and controversies in the evidence from research; to investigate the impact of cognitive impairment or dementia on the effectiveness of the identified interventions.
2
Design
2.1
Is the study design suitable for the objectives?
Yes. A sufficient number of studies on the topic in question has been published to warrant a systematic review and meta-analysis.
2.2
Were the search methods used to locate relevant studies comprehensive?
Yes.
In accordance with QUOROM and Cochrane, Medline, CINAHL, Embase, PsychInfo, Cochrane Database of Systematic Reviews, and the Register of Clinical Trials were searched for guidelines. References from 115 guidelines, systematic or expert reviews were hand searched. A wide range of relevant search terms was used. When possible authors of included studies were contacted to ascertain their knowledge of unpublished data or ongoing trials.
The full search strategy is provided via an online link.
In accordance with QUOROM and Cochrane, Medline, CINAHL, Embase, PsychInfo, Cochrane Database of Systematic Reviews, and the Register of Clinical Trials were searched for guidelines. References from 115 guidelines, systematic or expert reviews were hand searched. A wide range of relevant search terms was used. When possible authors of included studies were contacted to ascertain their knowledge of unpublished data or ongoing trials.
The full search strategy is provided via an online link.
2.3
Was this the right sample to answer the objectives?
Yes.
Included were: studies of patients in hospitals or care homes that reported the number or rate of falls or fractures or people who fell as a primary or secondary outcome; randomised controlled trials with individual or cluster randomisation, case-control studies, and observational cohort studies - this variety reflecting the methodological and logistical difficulties of performing RCTs in these settings and with these populations. The authors deliberately did not exclude studies that were not RCTs or studies of lower quality, in order to gather maximum data.
Excluded were: studies that did not report sufficient data for calculation of log rate ratios or log relative risks and their variances.
Included were: studies of patients in hospitals or care homes that reported the number or rate of falls or fractures or people who fell as a primary or secondary outcome; randomised controlled trials with individual or cluster randomisation, case-control studies, and observational cohort studies - this variety reflecting the methodological and logistical difficulties of performing RCTs in these settings and with these populations. The authors deliberately did not exclude studies that were not RCTs or studies of lower quality, in order to gather maximum data.
Excluded were: studies that did not report sufficient data for calculation of log rate ratios or log relative risks and their variances.
2.4
Is the study large enough to achieve its objectives?
Yes. 43 studies accepted for inclusion.
2.5
Were all the studies accounted for?
Yes. Refer to Figure 1 (QUOROM flow diagram of selection of studies).
2.6
Were all appropriate outcomes considered?
Yes: number or rate of falls, number or rate of fractures, number or rate of people who fell.
2.7
Has ethical approval been obtained if appropriate?
Not commented on. Included studies were assessed for their quality using the quality score of Downs and Black (reference provided, scores for included studies made available online), which does not include the question of ethical approval.
3
Measurement and observation
3.1
Is it clear what was measured, how it was measured and what the outcomes were?
Yes.
Quantitative outcome data for the purpose of the systematic review and meta-analyses were: falls per person year; fractures/1000 persons years; percentage of people falling.
The prevalence of dementia in study participants was categorised into four categories: 0=unknown; 1=<40%; 2=40-69%; 3=≥70%.
The quality of the relevant studies included was assessed using the quality score of Downs and Black by one of three pairs of peer assessors. The authors trialled the assessment tool amongst themselves to establish inter-rater reliability, and decided that a statistician had to be involved to score the statistical items on the score independently. Scores for the included studies are made available online. Quality scores were not used to include or exclude studies, but to add to the value of the data by informing the reader of the score of the study in the data presentation.
Quantitative outcome data for the purpose of the systematic review and meta-analyses were: falls per person year; fractures/1000 persons years; percentage of people falling.
The prevalence of dementia in study participants was categorised into four categories: 0=unknown; 1=<40%; 2=40-69%; 3=≥70%.
The quality of the relevant studies included was assessed using the quality score of Downs and Black by one of three pairs of peer assessors. The authors trialled the assessment tool amongst themselves to establish inter-rater reliability, and decided that a statistician had to be involved to score the statistical items on the score independently. Scores for the included studies are made available online. Quality scores were not used to include or exclude studies, but to add to the value of the data by informing the reader of the score of the study in the data presentation.
3.2
Were explicit methods used to determine which studies to include in the review?
Yes. See 2.3. of this appraisal. List of rejected studies and reasons for rejection is made available online.
3.3
Was the selection of primary studies re-producible and free from bias?
Yes. Search strategy well described and full search strategy made available online. List of rejected studies and reasons for rejection available online. Reason for rejection is mainly that falls or fracture data are not reported, or that the setting is other than a hospital or care home.
3.4
Was the methodologic quality of the primary studies assessed?
Yes. Quality score of Downs and Black (reference provided) determined for all included papers. This score was not used to in- or exclude any studies, but is presented in the results to add to the interpretation of the results. Any association between effect size and type or quality of studies was investigated by meta-regression.
3.5
Are the measurements valid?
Yes.
3.6
Are the measurements reliable?
Yes.
3.7
Are the measurements reproducible?
Yes.
4
Presentation of results
4.1
Are the basic data adequately described?
Yes.
Data was analysed according to the three outcomes (falls per person year, fractures/100 person years, percentage of people falling) and according to nine different interventions investigated across the included studies:
1) multifaceted intervention for falls prevention in hospital
2) multifaceted intervention for falls prevention in care homes
3) hip protectors in care homes
4) removal of physical restraint in either setting
5) fall alarm devices in either setting
6) exercise in either setting
7) changes or differences in the physical environment in either setting
8) calcium and vitamin D in care homes
9) medication review in either setting
The effect of the reported prevalence of dementia on each of the three outcomes was analysed with meta-regression - which showed no significant effect.
The effect of study design (RCTs vs non-RCTs) on the three outcomes was analysed with meta-regression - there was a significant effect only for the outcome of fractures, RCTs had a rate ratio closed to 1.
The effect of study quality (stratified into three groups of quality scores <15, 15-19, and ≥20) on the three outcomes was analysed with meta-regression - and showed significant effect for the outcome of fractures only.
The authors describe the main findings in the text and illustrate data with graphs (Figures 2-5). Extensive further data is provided online.
Data was analysed according to the three outcomes (falls per person year, fractures/100 person years, percentage of people falling) and according to nine different interventions investigated across the included studies:
1) multifaceted intervention for falls prevention in hospital
2) multifaceted intervention for falls prevention in care homes
3) hip protectors in care homes
4) removal of physical restraint in either setting
5) fall alarm devices in either setting
6) exercise in either setting
7) changes or differences in the physical environment in either setting
8) calcium and vitamin D in care homes
9) medication review in either setting
The effect of the reported prevalence of dementia on each of the three outcomes was analysed with meta-regression - which showed no significant effect.
The effect of study design (RCTs vs non-RCTs) on the three outcomes was analysed with meta-regression - there was a significant effect only for the outcome of fractures, RCTs had a rate ratio closed to 1.
The effect of study quality (stratified into three groups of quality scores <15, 15-19, and ≥20) on the three outcomes was analysed with meta-regression - and showed significant effect for the outcome of fractures only.
The authors describe the main findings in the text and illustrate data with graphs (Figures 2-5). Extensive further data is provided online.
4.2
Were the differences between studies adequately described?
Yes.
16 individually randomised controlled trials; 12 cluster randomised trials; 9 prospective trials with historical control; 2 retrospective observational cohort studies; 2 prospective observational cohort studies, one prospective case-control study, one quasi-experimental study with a multiple interrupted time series.
Downs and Black quality scores, prevalence of dementia in study participants, and other key features of the individual studies are made available online.
? For those included studies for which the prevalence of subjects with dementia is unknown (7 out of 43) - do we know WHETHER there were subjects with dementia in these studies, or could they have included cognitively normal subjects only? - not commented on by authors
Also, for studies with subjects with dementia, the extent of cognitive impairment appears not to be captured and integrated in analysis. It may well have been a too heterogenous parameter and therefore not feasible to include in data analysis, but this is not commented on by authors.
16 individually randomised controlled trials; 12 cluster randomised trials; 9 prospective trials with historical control; 2 retrospective observational cohort studies; 2 prospective observational cohort studies, one prospective case-control study, one quasi-experimental study with a multiple interrupted time series.
Downs and Black quality scores, prevalence of dementia in study participants, and other key features of the individual studies are made available online.
? For those included studies for which the prevalence of subjects with dementia is unknown (7 out of 43) - do we know WHETHER there were subjects with dementia in these studies, or could they have included cognitively normal subjects only? - not commented on by authors
Also, for studies with subjects with dementia, the extent of cognitive impairment appears not to be captured and integrated in analysis. It may well have been a too heterogenous parameter and therefore not feasible to include in data analysis, but this is not commented on by authors.
4.3
Are the results presented clearly, objectively and in sufficient detail to enable readers to make their own judgement?
Yes. Extensive data provided online.
4.4
Are the results internally consistent, i.e. do the numbers add up properly?
Yes.
5
Analysis
5.1
Were the results of primary studies combined appropriately?
Yes. Combined according to outcome measure, and according to intervention.
5.2
Has a sensitivity analysis been performed?
Yes.
5.3
Were all the important outcomes considered?
Yes.
5.4
Are the data suitable for analysis?
Yes. Only studies reporting data suitable for statistical analysis were included.
5.5
Are the methods appropriate to the data?
Within my professional background as physiotherapist, I believe that the statistical methods used are appropriate. Also, published in a renowned journal (BMI) and two of the authors are statisticians.
5.6
Are any statistics correctly performed and interpreted?
See 5.5. of this appraisal.
6
Discussion
6.1
Are the results discussed in relation to existing knowledge on the subject and study objectives?
Yes.
In their introduction the authors put this systematic review in context with the known prevalence of falls in hospitals and care homes, the detrimental effects a fall and subsequent fracture can have on an older person, the increased risk of falls in older people with dementia compared to cognitively normal older people, and known evidence about successful falls prevention in less frail and more clinically stable people living in their own home.
The results are discussed in relation to the study objectives, key findings are summed up in the discussion, a brief consumer summary is provided, recommendations for further research are made.
In their introduction the authors put this systematic review in context with the known prevalence of falls in hospitals and care homes, the detrimental effects a fall and subsequent fracture can have on an older person, the increased risk of falls in older people with dementia compared to cognitively normal older people, and known evidence about successful falls prevention in less frail and more clinically stable people living in their own home.
The results are discussed in relation to the study objectives, key findings are summed up in the discussion, a brief consumer summary is provided, recommendations for further research are made.
6.2
Is the discussion biased?
No.
Authors openly discuss limitations to their review. Their main criticism is related to the heterogeneity within the included studies. The authors acknowledge the inherent difficulties in performing or interpreting studies on falls prevention in care homes and hospitals, such as logistic difficulties, gaining consent from frail, confused and unwell elderly people, or recording bias for outcome measures such as number of falls and fallers. Choosing and analysing studies according to setting (care home and hospital) theoretically provides a certain degree of homogeneity, but the authors acknowledge that heterogeneity of case mix and clinical practice is still likely to exist, and has not been accounted for in this review.
They acknowledge that they did not exclude studies with a low Downs and Black quality score. They explain that rather than exclude data in an area where there is still lack of robust evidence, they chose to include lower quality studies and provide the quality score to guide interpretation of findings.
Also related to the heterogeneity within the included studies, the authors acknowledge that some studies have shown good results for falls prevention interventions, which is not reflected in the combined analysis of all the studies. It is possible that studies may have high internal validity for the specific setting and case mix they were conducted in.
For the studies investigating multifactorial interventions for falls prevention, the authors acknowledge that differing combinations of interventions between studies make it difficult to ascertain the benefit of each individual component. This is important, since data used for analysing the effect of single interventions was also taken from studies that included the single intervention as part of a multifactorial approach.
My two only question that are not clarified in the paper are:
1) whether the included studies that had an unknown proportion of subjects with dementia had subjects with dementia and we don't know how many, or whether they could have been groups with cognitively normal subjects only.
2) whether the attempt was made to stratify data by severity of dementia/cognitive impairment, and if yes why it was not included in the final data analysis.
Authors openly discuss limitations to their review. Their main criticism is related to the heterogeneity within the included studies. The authors acknowledge the inherent difficulties in performing or interpreting studies on falls prevention in care homes and hospitals, such as logistic difficulties, gaining consent from frail, confused and unwell elderly people, or recording bias for outcome measures such as number of falls and fallers. Choosing and analysing studies according to setting (care home and hospital) theoretically provides a certain degree of homogeneity, but the authors acknowledge that heterogeneity of case mix and clinical practice is still likely to exist, and has not been accounted for in this review.
They acknowledge that they did not exclude studies with a low Downs and Black quality score. They explain that rather than exclude data in an area where there is still lack of robust evidence, they chose to include lower quality studies and provide the quality score to guide interpretation of findings.
Also related to the heterogeneity within the included studies, the authors acknowledge that some studies have shown good results for falls prevention interventions, which is not reflected in the combined analysis of all the studies. It is possible that studies may have high internal validity for the specific setting and case mix they were conducted in.
For the studies investigating multifactorial interventions for falls prevention, the authors acknowledge that differing combinations of interventions between studies make it difficult to ascertain the benefit of each individual component. This is important, since data used for analysing the effect of single interventions was also taken from studies that included the single intervention as part of a multifactorial approach.
My two only question that are not clarified in the paper are:
1) whether the included studies that had an unknown proportion of subjects with dementia had subjects with dementia and we don't know how many, or whether they could have been groups with cognitively normal subjects only.
2) whether the attempt was made to stratify data by severity of dementia/cognitive impairment, and if yes why it was not included in the final data analysis.
7
Interpretation
7.1
Are the author's conclusions justified by the data?
Yes.
7.2
What level of evidence has this paper presented?
CEBM level of evidence: 1a- (minus-sign due to heterogeneity in design and quality of included studies).
7.3
Does this paper help me to answer my problem?
Yes. The paper is an attempt to answer clinically relevant questions by going through a process aimed at providing a high level of evidence. Even though the recommendations that can be drawn from the paper are limited, the work adds considerably to the knowledge base around the topic by pooling relevant publications and highlighting the difficulties experienced when researching interventions within complex situations in 'real life' clinical practice.
8
Implementation
8.1
Can any necessary change be implemented in practice?
From recommendations drawn from this study, no changes need to be implemented in my current area of work (acute hospital ward for acute elderly medicine). The team practices a multidisciplinary approach to falls prevention and rehabilitation including multidisciplinary assessment and individuallised multifactorial interventions.
8.2
What aids to implementation exist?
See 8.1. of this appraisal.
8.3
What barriers to implementation exist?
See 8.1. of this appraisal.