• 
• 

Three Part Question

POPULATION: In a [child admitted to a general paediatric ward] with an acute illness
INTERVENTION: can a score based on measured physiological parameters [a paediatric early warning system or score]
OUTCOME: [predict serious illness and/or clinical deterioration requiring high dependency, paediatric intensive care or cardiac arrest]?

Clinical Scenario

A three year old boy presents to his local district general hospital with a one day history of fever and shortness of breath. He is admitted to the paediatric ward for on-going observation and management. The nursing staff calculate a paediatric early warning score, based on physiological parameters, with each set of nursing observations. The student nurse on the ward notices that your chart is different to the one used by the paediatric ward on her last placement. She asks you how accurately paediatric early warning systems predict serious clinical deterioration, particularly cardiopulmonary arrest, paediatric intensive care admission or paediatric high dependency care admission.

Search Strategy

Medline 1950 to present from PubMed
Cochrane Library Issue 9, September 2014
Search terms used: (("pediatrics"[MeSH Terms] OR "pediatrics"[All Fields] OR "paediatric"[All Fields]) OR ("pediatrics"[MeSH Terms] OR "pediatrics"[All Fields] OR "pediatric"[All Fields])) AND (early[All Fields] AND warning[All Fields]) AND (("Sentinel Event Alert"[Journal] OR "alert"[All Fields]) AND ("standards"[Subheading] OR "standards"[All Fields] OR "criteria"[All Fields]))
No limits were placed on the search.

Search Outcome

One systematic review and twelve papers validating paediatric early warning systems in paediatric in-patients were found. Five of these papers were included in the systematic review. Seven papers post-date the systematic review.

Relevant Paper(s)

Author, date and country	Patient group	Study type (level of evidence)	Outcomes	Key results	Study Weaknesses
Chapman et al, April 2010 UK	5 of the 11 included papers reported diagnostic accuracy. Included •1 retrospective case review with 44 cases (2) •2 retrospective case-control studies: • 128 cases and 87 controls (3) • 360 cases and 180 controls (4) 2 prospective cohort studies: • 1000 patients (5) • 2979 patients (6)	Systematic review (level 1a) Included 11 studies, reporting 10 published paediatric alert criteria; a mixture of original tools and those adapted from existing paediatric and/or adult tools.	Cardiac arrest call	Brilli et al. 2007 2: Rate per 1,000 admissions decreased from 1.54 to 0.62 (risk ratio 0.41; 95% CI 0–0.86; p=0.02). Duncan et al. 2006 3: PEWS score of ≥5 78% sensitivity and 95% specificity	Each study used a different paediatric warning system. The tools varied from single parameter to multiple-parameter systems with or without aggregate weighted scores. The systems also combined different physiological parameters including a mixture of objective and subjective criterion. This resulted in major heterogeneity, specifically wide diversity in the cut off for trigger across the tools and precluded performance of meta-analysis. The systemtic review included two studies of poor methodological quality (2, 4): Brilli et al. 2006 (2): systematic review stated 44 cases were included but there were only 31 cases (25 at baseline and 6 post-implementation); only analysed cases (true positives) so no measures of diagnostic accuracy could be determined. Haines et al.2006 (4): Despite being a case-control study, only the case group was analysed and so the sensitivity and specificity published are erroneous.
			Respiratory arrest, cardiac arrest, PHDU admission, PICU admission or death	Edwards et al. 2009 5:Cardiff & Vale PEWS as a single parameter system 89.0% sensitivity (95% CI 80.5-94.1); 63.9% specificity (95%CI 63.8-63.9); 2.2% PPV (95% CI 2.0-2.3); and 99.8% NPV (95% CI 99.7-99.9); as a multiple parameter system with score of ≥2: 69.5% sensitivity (95% CI 59.0-78.4); 89.9% specificity (95% CI 89.8-90.0); 5.9% PPV (95% CI 5.0-6.7); and 99.7% NPV (95% CI 99.6-99.8) AUC=0.86 (95% CI 0.82-0.91)
			PICU admission only	Tucker et al. 2009 6: Modified Monaghan’s PEWS score7 of ≥3: 90.2% sensitivity; 74.4% specificity; 5.8% PPV and 99.8% NPV. AUC=0.89 (95% CI 0.84–0.94) p<0.001
Akre et al April 2010 USA	186 inpatients on paediatric medical surgical units, excluding intensive care and intensive care step-down units, at a Children’s tertiary and quaternary level hospital.	Retrospective case review (level 4)	Rapid response team (RRT) or cardiac arrest (code blue) call	Sensitivity of having critical Modified Monaghan’s PEWS score7 (total score of ≥4 or a score of 3 in any of the domains)	As controls were not included in this study, no other measures of validity could be given.
Edwards et al February 2011 UK	1000 children aged 0-1 years admitted to any of the paediatric wards at a tertiary level hospital. Patients admitted directly to the paediatric intensive care unit (PICU) and the paediatric high dependency unit (PHDU) and those patients presenting in cardiac or respiratory arrest were excluded.	Prospective cohort study (level 2b)	PHDU admission, PICU admission or death	Single parameter system: 68.3% sensitivity (95% CI 57.7-77.3); 83.2% specificity (95% CI 83.1-83.2); PPV 3.6% (95% CI	Use of previous data set from the Cardiff & Vale PEWS prospective cohort study5, to test the predictability of the Melbourne criteria for activation of a Medical Emergency Team (MAC)(10). Data on pre-existing diagnosis of cyanotic heart disease had not been collected and therefore the hypoxaemia criterion was positive for all patients if the SpO 2 was less than 90% in air or any amount of oxygen.
Parshuram et al, March 2011 Canada	Inpatient care provided on 842 patient- days before implementation and 2350 patient-days after implementation of the Bedside PEWS in a 22 inpatient bed community hospital.	Prospective before- and after- study (level 3)	Primary: significant clinical deterioration event (a composite measure of circulatory and respiratory support before transfer) Secondary: STAT (urgent) and resuscitation team calls, paediatrician workload and perception of frontline staff.	Fewer significant clinical deterioration events (2.4 versus 0.43 per 1000 patient-days; p=0.013). Increase in the overall number of transfers (5.9 versus 8.1 per 1000 patient-days; p=0.041). Fewer stat calls to paediatricians (22.6 versus 5.1 per 1000 patient-days;p<0.0001). Fewer stat calls to respiratory therapists (9.5 versus 3.4 per 1000 patient-days; p<0.0001)	Significant clinical deterioration event was defined as transfer to a hospital with a paediatric intensive care unit, following any of invasive positive pressure ventilation, administration of greater than 60 mL/kg of resuscitation fluid in the 12 h before transfer, administration of inotropes or vasoactive medication, provision of cardiopulmonary resuscitation, or death before transfer. Only study conducted in setting similar to UK district general hospital. However very small number of adverse events so hard to see any true measurable difference following implementation of the Bedside PEWS12. The increase in overall number of transfers suggests an increase workload for transport teams and tertiary centres. Some of the softer measures were self- reported and collected by gathering subjective responses. The study did not specify whether the patients were transferred intubated and ventilated or not and whether transfer was appropriate. There was also no further analysis of this subgroup to determine whether transfer subsequently reduced patient risk of cardiorespiratory arrest and/or death. Study also reported reduced nursing apprehension when calling the doctors and no change in paediatrician workload.
Skaletzky et al May 2012 USA	100 cases – patients who were initially admitted to medical surgical wards in a Children’s tertiary level hospital and subsequently transferred to their intensive care unit. 250 controls – patients who were admitted to the same medical surgical wards but not transferred to the intensive care unit during the same study time period.	Retrospective case control study (level 3b)	Transfer to PICU	Modified Monaghan’s PEWS (7) cut-off score of 2.5: 62% sensitivity and 89% specificity. AUC 0.81 (95% CI 0.75-0.86)	Only 3 of the transfers to PICU followed a rapid response team evaluation, 4 followed a code-blue event and 1 followed both. The remaining 92 admissions were at the physician request. It is unclear how “unwell” these patients were – for example, how many were routine PICU admission following surgery.
Robson MA et al 2013 USA	96 cases: patients aged 0-18years who had triggered an emergency medical response team (EMRT) call due to critical illness with impending or actual cardio-pulmonary arrest during the 4 year study period 96 controls: matched to cases based on age, diagnosis, gender, residing patient care unit and month of occurrence.	Retrospective case control study (level 3b)	EMRT call for impending or actual cardio-pulmonary arrest	Comparison of three PEWS: PEW System Score (3): PEWS score ≥ 5 had 86.6% sensitivity and 72.2% specificity, AUC 0.85. PEW Tool (4): PEWS ≥ 1 had 76.3% sensitivity and 61.5% specificity, AUC 0.75. Bedside PEW System Score (12): PEWS ≥ 7 had 56.3% sensitivity and 78.1% specificity, AUC 0.73. PEW System Score (3) demonstrated a significantly greater amount of accuracy.	Good study validating PEWS tools in a different setting to its original design. The retrospective design limits opportunity to verify accuracy of data or collect missing data (assumed to be normal). Also only 1 outcome measure, EMRT call; other studies also measures transfer to intensive care. Even with these limitations can see that two of the tools faired much worse in a different setting. Important study highlighting importance of finding out what works in your setting before widespread adoption of tool validated elsewhere.
Fuijkschot J et al June 2014 The Netherlands	Cohort 1: 118 of the 199 admissions to the paediatric oncology ward in the 3 month study period at a single tertiary hospital Cohort 2: 24 patients from the general paediatric wards at this hospital with admission to PICU in the 9 month study period Cohort 3: 17 patients who received defined emergency medical intervention on the general paediatric wards over a 4 month period	Cohort 1: Retrospective case review (level 4) Cohort 2: Retrospective case review (level 4) Cohort 3: Prospective cohort study (level 2b)	Cohort 1:Cardio-pulmonary arrest or unplanned PICU admission or “sick” if received emergency intervention	Cohort 1: No Cardio-Pulmonary Arrests (CPA) during study period. 1 unplanned PICU admission with PEWS ≥8. PEWS ≥8 in 15/118 admission. Specificity given as 88%; sensitivity 100%. Dichotomised data “sick” vs “well” with PEWS ≥8 27% false positives, PPV = 73%.	Poor methodological study. This study recognised that common endpoints (CPA and PICU admission) are uncommon. However there were no standards used to determine the emergency interventions which they used to define “sick”. Moreover this list included marked extremes in management e.g. 100% supplemental oxygen and cardio-pulmonary resuscitation. Cohort 1 had a large amount of missing data. With only 1 positive outcome the given sensitivity is not reliable. Also the results for cohort 1 are erroneous: specificity is given as 88% but only 1 of the 15 admissions scoring PEWS ≥8 had unplanned PICU admission (6.7%). The dichotomous data focuses on number of scores ≥8 but there is no data to indicate how many patients are included. Validity measures are limited as there were no controls for cohorts 1 or 2. Authors state the modified PEWS has a high sensitivity for identifying patients who need an emergency intervention. They state that there were no falsely negative warning scores detected which indicates a high sensitivity. However, there is no data given for the other patients during the study period and so no validity measures can be calculated.
			Cohort 2: Cardio-pulmonary arrest or unplanned PICU admission	Cohort 2: Median PEWS 2–6 h prior to PICU admission 8.5 (range 2–15) PEWS ≥8 at 2–6 h prior to PICU admission 67% sensitivity
			Cohort 3: Emergency intervention	Cohort 3: median PEWS at time of intervention 10 (range 8–15)

Comment(s)

AUC = Area under Receiver Operating Characteristic curve CI = Confidence Interval PEWS = Paediatric Early Warning Score PICU = Paediatric intensive care unit PHDU = Paediatric high dependency unit Comments Paediatric early warning systems need to alert health professionals about an at-risk child early enough to allow adequate time for treatments and interventions to improve outcome. There is a large amount of heterogeity in the paediatric early warning scores investigated in the literature making it difficult to determine the best tool. Each study tested a different paediatric early warning system3-5, 11-13 or a modified version of a previously validated tool6,8,14,16. A receiver operating characteristic (ROC) curve is a graphical plot which illustrates the performance of tool at various thresholds. The area under the ROC curve is used as a global measure of a tool’s performance – in this case giving the probability that the tool correctly identifies children at risk of serious or life-threatening deterioration. An AUC of greater than 0.8, good overall predictive value, was reached by many PEWS3, 5, 6,13,14. Only one system3 showed consistently good predictive value when directly compared with other paediatric early warning systems15. An ideal paediatric early warning tool should trigger escalation of care for at-risk patients (sensitivity) without inappropriately triggering those at low risk (specificity). No single system has a high sensitivity and a high specificity. Instead, a trade-off between sensitivity and specificity is seen. In clinical practice, sensitivity is most important. Failure to identify that a child is at-risk before cardiorespiratory arrest or death occurs is unacceptable. At the best given trigger or cut-off score for each paediatric early warning system, these studies reported 70-90% sensitivity5,6,8,9,13,14. All the tools could miss at-risk children. With higher sensitivity, specificity would lower considerably. This would result in larger number of patients triggering the tool inappropriately (false positives), wasting valuable time and resources. There is a trade-off between clinical effectiveness and efficiency. Current practice in the UK is diverse17. There are many different paediatric early warning systems in use but the majority are not based on the published and validated data described. Adult UK practice has moved to using one system18 and projects are on-going to consider one national paediatric PEWS solution19, 20. The evidence described does not show that one system excels in predicting serious clinical deterioration. No paediatric early warning system has yet been validated in a large multi-centre randomised control study. Therefore there is no evidence to recommend the use of any one specific published paediatric early warning system in paediatric inpatients.   References 1. Chapman SM, Grocott MP, Franck LS. Systematic review of paediatric alert criteria for identifying hospitalised children at risk of critical deterioration. Intensive Care Med. 2010 Apr;36(4):600-11 2. Brilli RJ, Gibson R, Luria JW, Wheeler TA, Shaw J, Linam M, Kheir J, McLain P, Lingsch T, Hall-Haering A and McBride M. Implementation of a medical emergency team in a large pediatric teaching hospital prevents respiratory and cardiopulmonary arrests outside the intensive care unit. Pediatr Crit Care Med 2007;8: 236-246 3. Duncan H, Hutchinson J and Parshuram CS. The pediatric early warning system score: A severity of illness score to predict urgent medical need in hospitalized children. J Crit Care 2006;21:271-278 4. Haines C, Perrott M and Weir P. Promoting care for acutely ill children – Development and evaluation of a Paediatric Early Warning Tool. Intensive Crit Care Nursing 2006;22:73-81 5. Edwards ED, Powell CVE, Mason BW and Oliver A. Prospective cohort study to test the predictability of the Cardiff and Vale Paediatric Early Warning System (C&VPEWS). Arch Dis Child 2009;94:602–606 6. Tucker KM, Brewer TL, Baker RB, Demeritt B, Vossmeyer MT. Prospective evaluation of a pediatric inpatient early warning scoring system. J Spec Pediatr Nurs. 2009 Apr;14(2):79-85. 7. Monaghan A. Detecting and managing deterioration in children. Paediatric Nursing 2005;17:32-35 8. Akre M, Finkelstein M, Erickson M, Liu M, Vanderbilt L, Billman G. Sensitivity of the pediatric early warning score to identify patient deterioration. Pediatrics. 2010 Apr;125(4):e763-9 9. Edwards ED, Mason BW, Oliver A, Powell CV. Cohort study to test the predictability of the Melbourne criteria for activation of the medical emergency team. Arch Dis Child. 2011 Feb;96(2):174-9 10. Tibballs J, Kinney S, Duke T, Oakley E, Hennessy M.Reduction of paediatric in-patient cardiac arrest and death with a medical emergency team: preliminary results.Arch Dis Child. 2005; 90(11): 1148–1152 11. Parshuram CS, Bayliss A, Reimer J, Middaugh K and Blanchard N. Implementing the Bedside Paediatric Early Warning System in a community hospital: A prospective observational study. Paediatr Child Health Mar 2011;16(3):e18-e22 12. Parshuram CS, Hutchison J and Middaugh K. Development and initial validation of the Bedside Paediatric Early Warning System score. Critical Care 2009, 13:R135 13. Parshuram CS, Duncan HP, Joffe AR, Farrell CA, Lacroix JR, Middaugh KL, Hutchison JS, Wensley D, Blanchard N, Beyene J and Parkin PC. Multicentre validation of the bedside paediatric early warning system score: a severity of illness score to detect evolving critical illness in hospitalised children. Critical Care Aug 2011, 15:R184 14. Skaletzky SM, Raszynski A, Totapally BR. Validation of a modified pediatric early warning system score: a retrospective case-control study. Clin Pediatr (Phila). 2012 May;51(5):431-5 15. Robson MA, Cooper CL, Medicus LA, Quintero MJ, Zuniga SA. Comparison of three acute care pediatric early warning scoring tools. J Pediatr Nurs. 2013 Nov-Dec;28(6):e33-41. 16. Fuijkschot J, Vernhout B, Lemson J, Draaisma JM, Loeffen JL. Validation of a Paediatric Early Warning Score: first results and implications of usage. Eur J Pediatr. 2014 Jun 20 17. Roland D, Oliver A, Edwards ED, Mason BW and Powell CVE. Use of paediatric early warning systems in Great Britain: has there been a change of practice in the last 7 years? Arch Dis Child 2014;99:26-29 18. Royal College of Physicians. National Early Warning Score (NEWS): Standardising the assessment of acute illness severity in the NHS. Report of a working party. London: RCP, 2012. 19. HS&DR - 12/178/17: PUMA - Paediatric early warning system (PEWS): Utilisation and Mortality Avoidance. A prospective, mixed methods, before and after study identifying the evidence base for the core components of an effective PEWS and the development of an implementation package for implementation and use in the UK http://www.nets.nihr.ac.uk/projects/hsdr/1217817 [Last accessed 12/01/15] 20. Patient Safety First. Recognising and responding to deterioration. http://www.patientsafetyfirst.nhs.uk/Content.aspx?path=/interventions/deterioration/recognising/ [Last accessed 12/01/15]

Clinical Bottom Line

Clinical bottom line There is no inpatient paediatric early warning system that is 100% diagnostic. However, paediatric early warning systems can be useful adjuncts to clinical judgement. These tools can identify deteriorating paediatric patients early enough in their physiological decompensation to help direct resources more appropriately with significant benefits in reducing adverse outcome.

Level of Evidence

Level 2 - Studies considered were neither 1 or 3.

References

1. Chapman SM, Grocott MP, Franck LS. Systematic review of paediatric alert criteria for identifying hospitalised children at risk of critical deterioration. Intensive Care Med 2010 Apr;36(4):600-11
2. Akre M, Finkelstein M, Erickson M, Liu M, Vanderbilt L, Billman G. Sensitivity of the pediatric early warning score to identify patient deterioration. Pediatrics 2010 Apr;125(4):e763-9
3. Edwards ED, Mason BW, Oliver A, Powell CV. Cohort study to test the predictability of the Melbourne criteria for activation of the medical emergency team. Arch Dis Child 2011 Feb;96(2):174-9
4. Parshuram CS, Bayliss A, Reimer J, Middaugh K and Blanchard N. Implementing the Bedside Paediatric Early Warning System in a community hospital: A prospective observational study. Paediatr Child Health Mar 2011;16(3):e18-e22
5. Skaletzky SM, Raszynski A, Totapally BR. Validation of a modified pediatric early warning system score: a retrospective case-control study. Clin Pediatr (Phila). 2012 May;51(5):431-5
6. Robson MA, Cooper CL, Medicus LA, Quintero MJ, Zuniga SA. Comparison of three acute care pediatric early warning scoring tools. J Pediatr Nurs. 2013 Nov-Dec;28(6):e33-41.
7. Fuijkschot J, Vernhout B, Lemson J, Draaisma JM, Loeffen JL. Validation of a Paediatric Early Warning Score: first results and implications of usage. Eur J Pediatr 2014 Jun 20