INTRODUCTION
Falls of inpatient children are an important issue as part of initiatives to prevent inpatient safety accidents. Falls accounted for the second-highest number, with 61 cases (24%), of 254 inpatient children's safety accidents reported to the Korea Patient Safety Education System (KOPS) in 2016-2017 [1]. This is reportedly because 1- to 3-year-old children's physical function and cognitive development are still premature, fore, they have low judgment and coping skills to protect themselves [2,3]. In a survey by 26 US agencies, the fall frequency of pediatric patients within 6 months was high, ranging from 0.4 to 3.8 per 1,000 individuals. Of these, 68% occurred in the presence of medical staff or family members [4-6]. However, the most notable finding of this survey was that fall prevention programs were actually in place in most of these hospitals. Nonetheless, the incidence of falls did not decrease.
The sensitivity of an assessment tool is the proportion of people with a positive test result among those with the relevant disease; thus, sensitivity is an important concept for evaluating the role of tests, symptoms, and signs in the diagnosis of a disease. On the one hand, if certain symptoms, signs, or tests are highly sensitive, a negative result can rule out certain diseases. On the other hand, the specificity of a test refers to the proportion of people whose test results are negative among those without a target disease, and this concept is used to evaluate the role of tests, symptoms, and signs in the diagnosis in conjunction with sensitivity. If a certain symptom, sign, or test shows high specificity and the result is positive, it is highly likely that the diagnosis of a specific disease will be confirmed. Therefore, the sensitivity and specificity of tools are utilized to predict the occurrence of events such as diseases.
Although the Humpty Dumpty Fall Scale (HDFS) is widely used in clinical practice as a tool to assess the risk of falls in pediatric inpatients, its markedly low specificity is problematic and causes 80% of studied children to be classified as high-risk [6]. Although the General Risk Assessment for Pediatric Inpatient Falls (GRAF-PIF) is believed to be useful for pediatric wards, a report suggested that it requires computerization and exhibits accuracy issues [1]. Consequently, issues have been raised regarding the assessment tools that have been widely used in the past [1], as confirmed by the results of a meta-analysis of tools for evaluating fall risk in pediatric inpatients. For example, when testing the combined sensitivity and specificity of existing fall risk assessment tools, the sensitivity was found to be as high as 80%, while the specificity was as low as 36% [7]. This outcome seems to indicate that the widely used fall risk assessment tools lack sufficient discriminant power to predict fall risks.
The HDFS, which is one of the most widely used tools, places a large weight on age and sex; for age, it gives 4 points to those 3 years old or under and 3 points for those 3-7 years old. However, this age classification is not appropriate when considering the large proportion of children who are 3-5 years old. In addition, when assessing environmental aspects, 4 points are given for "in bed," whereas 3 points are given for "in crib" without any scientific basis, creating confusion among nurses who actually use the tool in clinical practice [2]. Therefore, when an aspect of the assessment is ambiguous, nurses tend to give a higher score, which leads to a stricter evaluation [1]. The fall risk assessment tool used by Franck et al. [8], which assessed four items (age, mobility, elimination, and status of medication usage), had a low sensitivity, but a very high specificity of 97%(with a cut-off score of 3 points). The ability to accurately classify high-risk children with just four items seems to be an advantage of using this tool, but its low sensitivity remains a problem. The fall risk assessment tool used for inpatient children at Seoul National University Hospital (Seoul National University Hospital Pediatric Fall-risk Scale, SNUFS), which was developed in Korea in 2014, assesses eight areas: age, cranial nerve disease, hospital stay, dangerous drugs, physical function, use of assistive devices, and physical and cognitive development. The sensitivity of this tool was found to be 76.6%, and its specificity was 30.3%.
To accurately identify high-risk children through fall risk assessment, both sensitivity and specificity should be properly considered. Tools with low specificity must appropriately reflect the characteristics of pediatric inpatients, and those with low sensitivity must subdivide the criteria for classifying scores while minimizing items that could be impacted by nurses' subjective biases. Regarding the sensitivity of a fall risk assessment tool, it would be helpful to objectively evaluate the clinical situations in which fall risk is reportedly high (i.e., involving a male child aged 3 to 5 with a level of cognitive development according to his developmental stage and medical conditions such as surgery or sedation). Meanwhile, it seems possible to complement specificity by including situations such as a history of falls, the use of assistive devices, and elimination status.
Moreover, for such assessments to be objectively and accurately evaluated without the subjective judgments of nurses, a system in which an automatic scoring system is linked to the fall risk assessment tool in the electronic medical record (EMR) system may be considered. Given these points, it is necessary to develop a tool that can increase sensitivity and specificity by analyzing existing tools to improve the predictability of falls. Considering that only falls that led to injury were counted in reports during the development of the existing fall risk assessment tools, it may be helpful to design an objective and accurate assessment tool by including falls that did not lead to injuries in the development process.
Screening with a fall risk measurement tool is a very effective intervention for maintaining and developing a fall prevention program. Fall risk screening for pediatric inpatients should also be discussed in depth to prevent falls, and the effectiveness of the newly developed tool should be tested with a focus on sensitivity and specificity. Therefore, the aim of this study was to develop a new tool for fall prediction by extracting elements of the HDFS from the HDFS, GRAF-PIF, and SNUFS based on electronic nursing records and fall reports of inpatients in the pediatric ward at a general hospital, and to evaluate its specificity and sensitivity in comparison to the HDFS, GRAF-PIF, and SNUFS. The goals of this study were (1) to develop an assessment tool with appropriate sensitivity and specificity that can efficiently predict the fall risk of pediatric inpatients, and (2) to confirm the validity of the newly developed assessment tool by comparing it with existing ones.
METHODS
Ethics statement: This study was approved by the Institutional Review Board of Bundang Jesaeng Hospital (No. DMC 2020-01-003). Only EMR records were used after approval from the Institutional Review Board.
1. Study Design
Using fall reports and electronic nursing records from a general hospital, this study investigated the statistics of fall incidence as well as general and clinical characteristics of inpatients. This retrospective case-control study was conducted to propose a highly valid fall risk assessment tool for pediatric inpatients by comparing patients who experienced falls (hereafter, the "fall" group) to a control group (the "no fall" group).
2. Participants
Participants in this study were below age 19 years and were admitted to a general hospital in G Province during the 3-year period between January 1, 2016 and December 31, 2019. A total of 29 patients reported in the patient safety management system as having experienced a fall during hospitalization were selected as the "fall" group; 93 patients were selected as the control group according to age and sex matching among pediatric inpatients during the same period. The number of controls was matched 1:3 according to the criteria in a prior study [9].
3. Study Tools
The fall risk assessment tools were the HDFS, GRAF-PIF, SNUFS, and New Fall Risk Scale-Pediatric Inpatient (Newfs-PI).
1) Humpty Dumpty Fall Scale
The HDFS was developed in 2009 to predict pediatric inpatients' risk of falls [10]. Pediatric inpatients were classified as being at low or high risk of falls and were separated by age, sex, diagnosis, cognitive impairment, environment, drug use, surgery, anesthesia, and response to sedation and scored accordingly. The total score of the HDFS ranges from 7 to 23 points, and those who score 12 or more are considered to be at high risk of falls. In terms of age, 4 points are given to those under 3, and 3 points are given to those between 3 and 6. Two points are given to those between 7 and 12, and 1 point is given to those aged 13 years or older. Furthermore, 2 points are given to boys, and 1 point is given to girls. Four points are assigned for a neurological diagnosis, and 4 points are given for diagnoses that cause changes in oxygenation, such as respiratory system issues, dehydration, anemia, anorexia, syncope, and dizziness. Two points are given for mental and behavioral abnormalities, and 1 point is given for other diagnoses. In terms of cognitive impairment, 3 points are assigned to those aware of their limitations, 2 points are assigned for those who forget their limitations, and 1 point is assigned for those aware of their own abilities. As for environmental factors, 4 points are given to those with a history of falls or infant-toddlers placed in bed, 3 points are given for patients using assistive devices or an infant-toddler in a crib, 2 points are given for patients in bed, and 1 point is given for outpatients. Moreover, 3, 2, and 1 points are given to those with a history of surgery, sedation, and anesthesia within the last 24 hours, within the last 48 hours, and prior to the last 48 hours, respectively. As for medication use, 3 points are given for the usage of multiple sedatives (e.g., hypnotics, barbiturates, phenothiazines, antidepressants, laxatives, diuretics, and narcotics) except for intensive care unit (ICU) patients who are sedated or paralyzed, 2 points are given for the usage of one medication, and 1 point is given for other medications or not having used medication.
2) General Risk Assessment for Pediatric In-patient Falls
Graf [11] reported a retrospective case-matched control study in hospitalized children, and developed the GRAF-PIF. The fall risk assessment scale of the GRAF-PIF is composed of seven items, and the total score ranges from 0 to 10. The first item is the length of hospital stay: 0 is given for those who have stayed from 1 to 4 days, 1 is given for 5-9 days, and 2 is given for 10 days or longer. The second item is related to an intravenous (IV) heparin lock. One point is given if there is no IV/heparin lock, while 0 points are given if there is one. Next, 1 point is given for participants who had physical therapy or an operation (in the recent past, currently, or expected in the near future), and 0 points are given for those who were not. Additionally, 1 point is given for those prescribed seizure medications (given for any reason), and 0 points are given otherwise. Finally, 1 point is given for acute or chronic orthopedic or musculoskeletal diagnoses, and 0 or 2 points are given for those with a history of falling within the past month. Two points are also given to those who had fallen during hospitalization. A total score of 2 or more is considered high-risk.
3) Seoul National University Hospital Pediatric Fall-risk Scale
The SNUFS was developed to validate a fall-risk assessment tool for hospitalized children by Shin et al. [12]. This scale consists of seven items with a total score ranging from 0 to 23. Three points are given to children under 36 months of age or those with cranial nerve diseases (including epilepsy). One point is given to those with a hospital stay of 7 days or less. Three points are given to those prescribed three or more high-alert medications (e.g., antidepressants, antipsychotics, hypnotic sedatives, analgesics, nerve contractions, laxatives, diuretics, heart medications, hypoglycemic medications, and antihypertensive medications) or for activities and physical functions that require assistance (independent or bedridden). The use of assistive devices (wheelchair, crutches, etc.) is given 1 point, but children with normal physical development are given 7 points, and, finally, 1 point is given for children with normal cognitive development or whose environments (transfusion, catheterization, oxygen, etc.) are also considered normal. In this study, the scores were retrospectively calculated after the doctor's instructions and nursing records were checked via the patients' EMRs.
4) New Fall Risk Scale-Pediatric Inpatient
The Newfs-PI tool was developed in the present study, and data analysis was conducted for five factors that could be extracted from patients' EMRs. The fall risk assessment tool for new pediatric inpatients consisted of seven items: age (3 points for children aged 36 months or under), activity (1 point for children without an IV or heparin lock, 3 points for children receiving rehabilitation therapy), prescription (3 points for children taking three or more high-alert medications), history (3 points for a history of falling), and length of hospital stay (0 points for a hospital stay of 1-4 days, 1 point for 5-9 days, and 2 points for 10 days or more). The total score ranged from 0 to 15 (Table 1).
4. Data Collection
For this study, data were collected from December 15, 2019, to January 30, 2020, after obtaining Institutional Review Board (IRB) approval to access data from hospital EMRs. EMR data were processed by creating a structured data collection tool, and all personally identifying data were coded. The structured data collection tool had birth date, sex, admission date, discharge date, department, state of consciousness, diagnosis, whether the patient had an operation or was receiving rehabilitation therapy, whether the patient used an assistive device, whether the patient had an IV/heparin lock, history of fall, and HDFS score at admission. The scores of the Newfs-PI and other tools were measured by nursing records and physician prescriptions. The following items were identified in the patient safety accident reports to evaluate the characteristics of the fall group: date of fall event, time of fall event, and harm level (no risk, recovery after treatment without complication, temporary damage, long-term damage, death, or unclear).
5. Data Analysis
The collected data were entered and statistically processed using SPSS version 24.0 (IBM Corp., Armonk, NY, USA) and MedCalc version 18.5 (MedCalc Software, Ostend, Belgium). To evaluate the validity of each tool, its sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were analyzed. The cut-off score to distinguish between the fall and non-fall groups was determined as the score that maximized the sum of sensitivity and specificity. The area under the curve (AUC) was compared and analyzed based on a receiver operating characteristic (ROC) curve in order to compare the predictive ability (diagnostic accuracy) for the fall/non-fall group based on the assessment score of each tool.
6. Ethical Considerations
To protect the human rights of the study participants, the study was approved by the Institutional Review Board (No. DMC2020-01-003). The collected data were used for research purposes only. No personal information was collected other than the general data and information related to falls required for the survey. During the data-extraction process, any personally identifying patient information was destroyed, encoded, and entered into the computer by the first researcher. All data were stored and analyzed on one main computer with a set password.
RESULTS
1. Development of the New Fall Risk Scale-Pediatric Inpatient
The tool was developed through a literature review, and data analysis was conducted for the factors that could be extracted by EMR to enhance the validity of the above three inpatient fall risk assessment criteria. A meta-analysis of diagnostic test accuracy was conducted on fall risk measurement studies published as of 2018 to determine the items of those tools [8]. In the 10 studies included in the meta-analysis, there were a total of five children's hospitalization fall risk assessment tools, with 14 items [8]. Our research team selected seven items and the content validity index (CVI) was calculated for all individual items (I-CVI). To determine the CVI, a panel of experts was asked to rate each scale item in terms of its relevance to the underlying construct. The panel consisted of three nursing professors and one clinical nursing professional. A 4-point scale was used to avoid a neutral point. The I-CVI was 0.80. We provided each tool's cut-off score and three different models for scoring each item before checking its specificity and sensitivity. The Newfs-PI consisted of seven items: age (3 points for children aged 36 months or under), activity (1 point for children without an IV or heparin lock, 3 points for children receiving rehabilitation therapy), prescription (3 points for children taking three or more high-alert medications), history (3 points for a history of falling), and length of hospital stay (0 points for a hospital stay of 1-4 days, 1 point for 5-9 days, 2 points for 10 days or more). The total score ranged from 0 to 15. Table 1 shows the sensitivity and specificity according to the cut-off score of the Newfs-PI. A cutoff score of 3 points, which maximized the sum of sensitivity and specificity, was selected as the cut-off score (Table 1).
2. Evaluation of New Fall Risk Scale-Pediatric Inpatient
1) Participants' characteristics
The average age of the fall group was 21.1±15.4 months; 20 out of 29 were boys (69.0%), and 20 were being treated in the pediatric unit (69.0%), while nine were rehabilitation patients (31.0%). The average age of the non-fall group was 24.5±13.6 months old; 66 were male participants (70.9%), and 82 were being treated in the pediatric unit (88.0%) (Table 2).
The mean scores and standard deviation for fall risk in the fall and non-fall groups for each tool were evaluated. The HDFS score was 12.73±1.71 in the non-fall group and 14.17 ±3.73 in the fall group, and the GRAF-PIF score was 0.70±1.44 in the non-fall group and 1.93±1.98 in the fall group. The SNUFS score was 11.33±1.43 in the non-fall group and 11.03±1.61 in the fall group, and the Newfs-PI score was 3.08±1.78 in the non-fall group and 4.31±1.73 in the fall group.
2) Sensitivity, specificity, PPV, NPV, and cut-off score for each fall risk assessment tool
Table 3 shows the sensitivity, specificity, PPV, NPV, and AUC values for each tool. The HDFS cut-off score was 12 points, which is similar to that of the original tool. Its sensitivity was 75.86%, while its specificity was 30.11%, and the AUC was 0.614. The GRAF-PIF cut-off score was 2 points. It had a sensitivity of 37.93%, a specificity of 89.25%, and an AUC of 0.685. The SNUFS cut-off score was 12 points. Its sensitivity was 44.83%, its specificity was 86.02%, and its AUC was 0.676. The Newfs-PI cut-off score was 3 points. Its sensitivity was 62.07%, specificity was 74.19%, and AUC was 0.711 (Table 3). As the AUC approaches 1, higher sensitivity and specificity indicate a stable diagnostic tool. The sensitivity and specificity of the HDFS were not statistically significant (z= 1.90, p=.058), whereas those of the Newfs-PI, GRAF-PIF, and SNUFS were statistically significant (Table 4). The ROC curve for each tool is shown in Figure 1.
DISCUSSION
This study developed the Newfs-PI as a fall risk assessment tool with satisfactory sensitivity and specificity for pediatric inpatients. The Newfs-PI was developed by reviewing existing fall risk assessment tools. Children under 36 months are given 3 points, which is similar to the score given in HDFS [10] and SNUFS [12]. The assessment of whether a heparin lock is used is the same as that of the GRAF-PI [11]. Whether three or more dangerous drugs are administered is an item measured in the HDFS, GRAF-PI, and SNUFS. The number of hospitalization days is a measure used in the GRAF-PI and SNUFS, and fall history is a measure used in the GRAF-PI.
The Newfs-PI also excluded some measurement items used in previous tools, such as sex, diagnosis, and cognitive impairment used in the HDFS, physical therapy and operation used in the GRAF-PI, and physical development and assistive device used in SNUFS. As a result, a simpler and more objective fall risk assessment tool for pediatric inpatients was developed that consists of items that can be measured objectively through the hospital's EMR system and excludes measurement items that may involve subjectivity of the evaluator.
The purpose of developing the Newfs-PI was to address the problem that in several previous studies, including that of Kim et al. [1,7], the sensitivity of an existing fall risk assessment tool for pediatric inpatients was found to be high, while the specificity was relatively low. For example, the HDFS is widely used in clinical settings because it is easy to use, but it has been suggested that more than 80% of the targets are classified as being high-risk for falls, indicating the low specificity of the tool [8]. In other words, this tool forces medical staff, who are in charge of pediatric patients at a high risk of falling, to paradoxically view most pediatric inpatients as high-risk, which may result in losing the opportunity to closely monitor and care for pediatric patients who are actually at a disproportionately high risk of falling and need medical attention.
The sensitivity of the Newfs-PI developed in this study was 62.07%, while its specificity was 74.19%, showing similar sensitivity and significantly higher specificity compared to the HDFS [10]. Compared to the GRAF-PIF [11], it showed similar specificity and much higher sensitivity. Compared to the third tool, the SNUFS, it also showed similar specificity and much higher sensitivity. In other words, the Newfs-PI is satisfactory in terms of both sensitivity and specificity, as it drew upon the strengths of existing tools and supplemented their weaknesses. Therefore, the Newfs-PI developed in this study can be effectively used to create a safer ward environment and to improve fall risk management by overcoming the weaknesses of existing fall risk assessment tools for pediatric inpatients.
The Newfs-PI showed higher specificity than the HDFS because it excludes items with low discriminant power and items that may involve subjective judgments made by evaluators. For example, in the HDFS, 3 points are given for a diagnosis that causes changes in oxygenation. However, since the tool includes a wide range of symptoms in this diagnostic category, such as respiratory symptoms, dehydration, anemia, loss of appetite, fainting, and dizziness, almost all pediatric inpatients are given 3 points, resulting in low discriminant power. Moreover, the scale gives 3 points to those who are not aware of their limitations under the cognitive impairment category. This allows room for subjective assessment by the evaluator, which is significant in that evaluators tend to unintentionally give high scores to pediatric inpatients because they subconsciously classify them as being high-risk for falls, thus resulting in the belief that they need intensive care. Moreover, the category of those under 3 years of age overlaps with infants and toddlers placed in cribs. Each item receives 3 points despite the overlap, which diminishes the specificity of the HDFS.
The Newfs-PI showed higher sensitivity than the GRAFPIF and SNUFS because the existing tools define comparison items by subdividing them, while the Newfs-PI defines them as comprehensively applicable. For example, in the GRAFPIF, scores are given according to specific conditions, such as 1 point for patients receiving physical therapy or have undergone surgery, 0 points for non-physical therapy/surgical patients, and 1 or 0 points according to whether anti-seizure medication was administered. The SNUFS also presents specific conditions such as 3 points for neurological diseases (including epilepsy) and 1 point for the use of assistive devices (wheelchair, crutches, etc.). However, the Newfs-PI is inclusive, such that it embraces all pediatric inpatients at high risk of falls without exclusion (by assigning 3 points for children aged 36 months or under and those taking three or more high-alert medications).
Another benefit of using the Newfs-PI is that, when this tool is linked with a hospital's EMR system, all assessment items consist of indices that can be extracted from the EMR, leaving no room for the subjectivity of the evaluator. Furthermore, the tool allows continuous 24-hour monitoring and streaming. In most hospitals, an assessment tool is to evaluate the fall risk of pediatric inpatients. The nurse-in-charge usually assesses the risks once for each duty shift, writes down the results on a record sheet, and then either attaches a fall risk notice or provides training for fall prevention. However, since it is impossible to fully foresee when, where, and how falls are likely to occur, this monitoring method is imperfect as a preventive management system because of its lack of continuity.
Furthermore, the validity of the fall risk assessment results may be affected by the subjectivity of the nurse performing the evaluation. Since the currently available HDFS, GRAF- PIF, and SNUFS include assessment items such as cognitive impairment or cognitive development, there is a limit to the accuracy of the fall risk assessments of target patients within the system based on information that is regularly accessible through the EMR. Given the high assessment validity of the Newfs-PI and the ability to monitor and manage the fall risk of pediatric inpatients 24 hours a day through the system, in addition to its high specificity and sensitivity, this study proposes the use of the Newfs-PI as a fall risk assessment and fall prevention management tool for pediatric inpatients in the future.
In the process of evaluating the sensitivity and specificity of Newfs-PI, EMR data from one hospital were used. Therefore, a careful interpretation is needed when generalizing the results of this study. In addition, in the development process of the Newfs-PI, the existing fall risk assessment tools were extensively reviewed to take advantage of their strengths; however, it is necessary to regularly conduct multi-hospital application studies to verify the validity of the Newfs-PI measurement items.
CONCLUSION
This study aimed to develop a new fall risk assessment tool that could complement existing fall risk assessment tools, which did not meet the sensitivity and specificity criteria to reduce fall incidence among pediatric inpatients. The Newfs-PI developed in this study showed superior sensitivity and specificity when compared to existing fall risk measurement tools. This means that the efficacy of identifying pediatric patients at high risk of falling was improved by applying the Newfs-PI. Consequently, we propose applying the Newfs-PI to assess the fall risk of pediatric inpatients to compensate for the shortcomings of existing tools and to improve the specificity of screening for high-risk fall groups. This will contribute to preventing fall accidents among pediatric inpatients and lay the foundation for establishing a more effective fall risk management system in clinical practice.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Download Citation 
Print 
