Developing a pediatric nursing simulation scenario template in South Korea: applying real-time Delphi methods

Article information

Child Health Nurs Res. 2024;30(2):142-153

Publication date (electronic) : 2024 April 30

doi : https://doi.org/10.4094/chnr.2024.012

Eun Joo Kim¹, Meen Hye Lee^,2, Bitna Park³

¹Associate Professor, Department of Nursing, Gangneung-Wonju National University, Wonju, Korea

²Assistant Professor, Department of Nursing, Gangneung-Wonju National University, Wonju, Korea

³Lecturer, Department of Nursing, Gangneung-Wonju National University, Wonju, Korea

Corresponding author: Meen Hye Lee Department of Nursing, Gangneung-Wonju National University, 150 Namwon-ro, Heaungup-Myun, Wonju 26403, Korea TEL: +82-33-760-8658 FAX: +82-33-760-8640 E-MAIL: leemh@gwnu.ac.kr

Received 2024 March 19; Revised 2024 April 15; Accepted 2024 April 21.

Abstract

INTRODUCTION

Simulation education enables repeated learning in a safe environment similar to clinical situations, allowing for the experience of serious and rare clinical situations without risking humans [1]. This educational method is effective in improving clinical performance abilities, clinical reasoning capacities, and nursing practitioners' confidence [2]. Especially in situations where direct patient care in clinical settings is limited, simulation education plays an essential role in enhancing knowledge, skills, and decision-making abilities [3]. In simulation education, students repeatedly learn various complex scenarios, and the process of giving and receiving feedback through debriefing can enhance their learning outcomes [4]. This process develops the competencies needed in situations similar to real clinical environments and prepares them for actual patient care [3].

In recent years, owing to the impact of low birth rates, the number of births in 2022 has decreased to 249,000 in South Korea, a 4.4% decrease compared with that in 2021 [5]. This rapid decline in the number of births has affected opportunities in pediatric nursing practice and education. Moreover, the emergence of various new infectious diseases like the coronavirus disease-19 (COVID-19), has increased the risk of infection, limiting pediatric nursing practice [6]. In this context, simulation education has been established as a method that provides students with the opportunity to experience various clinical cases and effectively acquire the skills and knowledge necessary for patient care [7].

Simulation education programs in the fields of medicine and nursing, which began in the 1960s, have significantly evolved since then [8]. During this development, simulators for teaching cardiopulmonary resuscitation became widespread as an example, bringing revolutionary changes to medical and nursing education [8]. In particular, pediatric nurses require a high level of flexibility and rapid response capabilities to address the diverse and complex health conditions and clinical situations that arise during a child's growth and development process [9]. Children may experience physical and mental health issues at different stages of development that demand special attention and a professional approach to pediatric nursing practice [9]. As a key strategy to meet these demands, simulation education supports better preparation for complex challenges encountered in pediatric nursing [7].

The components of simulation education generally include educational goals, reproducibility, complexity, cues, and debriefing [10]. This necessitates clear educational objectives to maximize learners’ educational effects. Reproducibility and complexity are related to the range of education that realistically represents simulation, which should be planned from simple to complex [4]. Cues are information provided by instructors to help students effectively approach simulation scenarios, and debriefing is the process of critically thinking and discussing complex situations with each other, connecting theory, practice, and research for students' positive experiences and learning effects. The most significant learning effects can be obtained in this area, including nursing knowledge and technical aspects, learner satisfaction, and improvement in critical thinking and confidence [11].

A well-designed template is crucial for effective implementation of the core components of simulation education. The International Nursing Association for Clinical Simulation and Learning (INACSL) emphasizes the importance of evidence-based design templates that educators can use to effectively design simulation scenarios in their standard guidelines. Such templates assist in standardizing the simulation process and ensuring consistent quality and evidence-based approaches during scenario composition and implementation by educators [12].

However, owing to the absence of universal and effective tools currently used for scientifically validating and evaluating scenarios [13], it is challenging to identify the essential components of a scenario and determine an ideal template structure. The standards and guidelines provided by the INACSL for clinical simulation and learning offer a crucial framework for developing and evaluating pediatric nursing simulation scenarios. However, they do not completely resolve the differences in components and expressions across various scenarios [14]. In addition, although some studies have emphasized the importance of pediatric simulation-based education [9], there has been no research on the development of standardized simulation scenario templates for pediatric nursing.

Therefore, efforts are needed to ensure the consistency and standardization of scenario templates for effective learning and assessment. This includes setting objectives that consider learners' levels, verifying the authenticity of patient information, and concretizing effective debriefing and feedback. Additionally, according to previous studies, although simulation education is spreading globally, many scenarios published in various countries do not fully comply with the INACSL’s standard guidelines. In particular, the template elements analyzed according to the INACSL standards show differences in the order and expression of components and sub-elements in various scenarios, indicating limitations in ensuring consistency [15].

To address these issues, the real-time Delphi method can play a crucial role in integrating the opinions of various experts to determine the key components and structure of the template. This approach contributes to determining a balance between educational value and practical feasibility by deriving a broad consensus among experts [16]. This process will allow the development of scientifically validated, realistic, and educationally effective pediatric nursing simulation templates by reconciling differences in opinions during the scenario development process.

The real-time Delphi method enables the collection and analysis of panel members’ opinions through rounds of real-time feedback based on the initial responses of the expert panel [17]. This process ultimately leads to more precise decision-making, playing a pivotal role in the development of pediatric nursing simulation templates. Consequently, this study aimed to use the real-time Delphi method to assemble a panel of experts in pediatric nursing to develop a simulation template. Through this process, it is anticipated that a fundamental structure for pediatric nursing simulations can be established by applying an evidence-based process as the basis for scenario development.

This study describes the process of developing a pediatric nursing simulation template using a real-time Delphi technique.

METHODS

2. Development of Questionnaire

To construct a pediatric nursing simulation scenario template, a content analysis of the scenarios developed for current pediatric nursing simulations was conducted [15]. For directed qualitative content analysis, 32 pediatric patient nursing simulation scenarios were collected and analyzed using the PRISMA 2020 method. Additionally, content analysis was performed based on the INACSL protocol to analyze the contents of the scenarios. The study results led us to establish preliminary principles for developing essential components and contents guided by the INACSL Standards of Best Practice: Simulation. The “INACSL Standards of Best Practice: Simulation” is publicly available, providing specific standards for simulation educators [19]. Our template was structured into 9 parts consisting of 59 items. The questionnaire comprised 83 items, including 59 for the template structure, 19 for the scenario theme, and five for the target developmental stage (Table 1).

Table 1.

Contents of the Preliminary Survey Questionnaire for the Pediatric Nursing Simulation Scenario Template

5. Data Analysis

The data collected through the two rounds of Delphi surveys were analyzed by calculating the CVR and CV for each item using Microsoft Excel. The interpretation of the CVR values followed the criteria set by Lawshe [20]. Figure 1 displays the flowchart of the decision-making process based on the analyzed results.

Figure 1.

Flowchart of real time Delphi survey. CVR, content validity ratio; INACSL, International Nursing Association for Clinical Simulation and Learning.

The CVR depends on the number of panel members. For the first Delphi survey with 13 panel members, a CVR value of 0.54 or higher was required to consider the content valid. Therefore, items with a CVR value less than 0.54 in the first survey were considered to have low content validity and were subject to deletion or modification. A CV of less than 0.5 indicates high consistency in expert responses; thus, it is considered stable and reliable [20].

RESULTS

1. General Characteristics of the Participants

The general characteristics of the experts who participated in the real-time Delphi survey are presented in Table 2. The average age was 41.38±10.65 years, with a range of 32 to 63 years old. 12 out of 13 participants (92.3%) were female, and the total career experience as a nurse was 79.23±65.60 months, with a range of 25 to 254 months. The career length as teaching professionals in nursing was 159.85±97.24 months, ranging from 19 to 326 months. The most common nursing specialty among the experts was pediatrics, with 10 (66.7%) responses. When asked about their experience with simulation-related research or projects, 11 participants (84.6%) reported being experienced. All experts had experience in simulation scenario development and operating simulation education.

Table 2.

General Characteristics of Experts Participating in the Survey (N=13)

2. Assessment of Appropriate Themes and Skill Levels for Pediatric Nursing Simulation Scenarios

The results on the appropriate age for developing pediatric nursing simulation scenarios showed that “Preschooler” was considered most valid with CVR of 1 (Table 3). The survey results for the appropriate skill level for each academic year for the freshman, “Level 1 Novice” had a CVR of 0.67 and a CV of 0.11. For sophomores, “Level 1 Novice” and “Level 2 Advanced Beginner” had CVRs of 0.56 and 0.67, and CVs of 0.28 and 0.33, respectively. In the junior, “Level 3 Competent” was indicated with a CVR of 0.67 and a CV of 0.41. For the senior, “Level 3 Competent” and “Level 4 Proficient” showed a CVR of 0.78 and a CV of 0.37.

Table 3.

Assessment of Appropriate Themes and Skill Levels for Pediatric Nursing Simulation Scenarios

3. Real-time Delphi First and Second Round

The final number of respondents in the first round of the real-time Delphi survey was 13. Initially, the CVR and CV values for each item were calculated. The range of content validity index (CVI) ranged from 0.32 to 1, with the “Development of scenario: Is it appropriate to include the ‘Theoretical framework’ item?” having the lowest score of 0.31, and “part 2. Objectives and Expected Outcomes: Do you think it is appropriate to include the following options in the ‘Evaluation’ section?” being the second lowest at 0.46 (Table 4).

Table 4.

Results of the Real-time Delphi for the First Round and Second Round of the Development of Pediatric Nursing Simulation Scenario Template

In the development of pediatric nursing simulation scenarios, the CV values were below 0.5 for all themes, indicating a high level of stability in these assessments (Table 4). However, Kendall’s W value was 0.00072 (χ² value=43.65, p=.319), indicating an overall low consensus among experts in round 1 (Table 4). This may suggest some variability or lack of consistency among the experts’ opinions. Based on the results of the first-round Delphi process, we removed or modified items with a CVR of 0.54 or below. The remaining items were consolidated, reclassified, or newly generated, yielding 41 items. For all 41 items, the CVR ranged from a minimum of 0.85 to a maximum of 1.0, with the lowest average and standard deviation, including the newly generated items, being 3.31±1.11 and highest being 4.00±0.00, indicating a high level of agreement among the experts. Additionally, Kendall’s W value was 0.34986 (χ² value=181.93, p<.001), confirming consistency among the experts’ opinions. Consequently, the Delphi rounds for the template were concluded based on this consensus (Table 4).

DISCUSSION

This study developed a pediatric simulation template reflecting the views and opinions of 13 pediatric nursing experts using the real-time Delphi technique. This process yielded a final template comprising 41 items across 9 sections, deemed suitable for application as a pediatric simulation template (Supplement 1). All 41 items demonstrated high consensus, ranging from 0.85 to 1 in CVR, indicating their essential inclusion. The CV values for all items were also low ranging from 0.07 to 0.32, indicating good agreement between the predictions or evaluations expressed by the experts [21].

In addition to evaluating the CV and CVR values that provided information on whether to maintain or delete the items, qualitative changes were made to the items based on expert feedback, including modifications to the wording or format. For example, some categories were modified in part 1 (initial elements). Reflecting on the first-round comment on this categorization, we combined the two groups into one group (nurses) for simplicity and conciseness. Specific options were added to certain items in the environment and setting from part 3. We added options “pediatric and adolescent ward/pediatric emergency department/NICU/NR/PICU/Others” in the environment part. The provision of specific options was also added in the “Equipment required and embedded patients and roles” in part 3. Through the round 1 Delphi survey, this specification suggested more concrete information on the template, which would guide educators in preparing for the required equipment, as well as how to set up the designated environment and setting. Only one item (objectives and expected outcomes: following options for evaluation) was deleted from the 42 full items based on the first survey, presenting low CVR (0.46) and high CV (0.27) values, which suggest deletion. A low CVR may be explained due to having a very similar item (objectives and expected outcomes: evaluation), which would be deemed as a repeated question.

In addition to the noted changes, two new items were proposed through the first-round survey: self-reflection and discussion in part 7 (Debriefing Planning). This addition is in line with the INACSL Standard, emphasizing the essential point of integrating the debriefing process into simulation-based experiences that enhance learning and increase participants’ self-awareness and self-efficacy [12]. Many studies have reported that high self-reflection predicts higher nursing competence and mediates the relationship between students’ anxiety and nursing competence [22,23]. A previous study also reported that students enjoyed a more meaningful learning experience when they engaged in self-reflective practices by examining and understanding their emotions, attitudes toward self and others, and reactions to everyday professional situations [24]. Even though our study was mainly developed for template development, and did not include a sample question, future studies may consider a guide comment to facilitate students’ self-reflection in debriefing plans based on the developed scenario content.

The themes of these scenarios were categorized into acute stage care, symptom management, and skill training. Acute stage care and skill training themes generally had high CVR values ranging from 0.89 to 1, indicating their significant inclusion. However, symptom management themes showed more varied CVR values, including fever and gastroenteritis (1), fracture (0.78), acute pyelonephritis (0.67), and sickle cell anemia (SCA; 0.44), suggesting the potential for selective inclusion. These findings offer valuable guidance for prioritizing scenario themes in curriculum development. Although we decided to maintain SCA in the template, the results imply that SCA is a subject of controversy and can thus be skipped depending on the educator.

The results on the appropriate themes represent what content should be prioritized, which would offer helpful information when selecting themes for simulation curriculum development. In order to assess learners’ abilities, academic grades were matched to specific skill levels. The results indicated that skill level increased as the academic grade increased. As it is difficult to determine the level of goals and expected outcomes, this type of question could assist educators in establishing suitable objectives and offering assessment tools aligned with learners’ abilities.

The real-time Delphi technique enabled experts to simultaneously refer to other panelists’ responses and comments while completing their surveys. In the real-time Delphi procedure, participants were provided entry into an online survey platform for a defined duration. Upon entering the platform, expert panelists could view their own responses alongside continuous updates, thus providing real-time, anonymized responses from fellow panelists. The primary novelty of real-time Delphi investigation lies in the concurrent computation and provision of feedback. Unlike the traditional approach, participants in a real-time Delphi approach are not restricted to evaluating at fixed intervals (such as rounds), but can alter their viewpoints as frequently as desired within a designated timeframe [17,25].

The Delphi technique has not been widely used for simulation template development, and only a few studies have used it for survey tool development, such as violence and self-reporting triage survey tools [26,27]. In this aspect, our study is important because it is the first to apply the real-time Delphi method for simulation template development, providing significant findings through the Delphi process. A noteworthy strength of this study is the application of INACSL standards and a previous comprehensive finding on pediatric simulation scenarios. For instance, we included the role of the facilitator in our template, as INACSL emphasized the facilitator’s role as a necessary element in achieving the desired goals throughout the process of scenario-based learning, and that a program to train facilitators should be provided [15]. Since previous findings have reported that only half of the examined scenarios addressed facilitation, integrating facilitator guidelines into this template would serve as a reminder of their significance and underscore the necessity for facilitator training programs. Furthermore, a prompt was displayed in the rightmost section, designed to prompt educators to review each section [15]. INACSL emphasizes the importance of a more effective learning experience and achievement of educational objectives. As one of the ways to achieve this goal, INACSL recommends applying SMART (specific, measurable, attainable, relevant, timely) standards when establishing a simulation template. We included the SMART section in the right column to assist educators in formulating scenario objectives. This addition would remind educators to use action verbs to enable the performance of skills and competencies in scenario-based learning processes [15]. Moreover, we added criteria for expected outcomes and evaluation, such as psychomotor function, cognition, and affectation, which are listed in line with the INACSL guidelines. This reminder serves as a crucial element in ensuring the achievement of the desired learning objectives during the completion of the template [26,27].

The limitations of this study are as follows. First, we did not account for the applicability of the template to various cultural, educational, or local healthcare system contexts because our primary aim was to develop a universal template without considering specific contexts. For example, this study did not consider a virtual reality education method, potentially rendering it unsuitable for such applications. Future template users should customize their content to align with their local contexts and specific requirements. Second, although this study reached predetermined consensus levels, this is not always recommended because psychometric criteria are typically utilized in positivist methodologies [25]. In fact, some studies opt to select an appropriate level within the data rather than relying solely on preset levels. While achieving consensus may not entirely eliminate the panel bias associated with self-interest issues, investigators should be mindful of this concern when determining consensus levels [27]. Our high CVR results for most items and the consensus reached in the two rounds suggest that controversial issues did not arise during the survey process. However, future studies should establish their own criteria to enhance the credibility of the analyzed data when employing the Delphi method for survey development.

CONCLUSION

This study used the Delphi technique to create a pediatric simulation scenario aimed at enhancing the quality of pediatric nursing simulation education. As mentioned previously, this study highlights the need for a standardized pediatric scenario template to ensure consistency and effectiveness in learning and assessment. Our study presents significant findings, as it is the first to employ the real-time Delphi technique in crafting a pediatric simulation template following the INACSL guidelines and drawing upon comprehensive prior research. The broader dissemination of this template will result in an enhanced version incorporating updates suggested by other pediatric nursing education experts in the future.

Supplementary material

Article information Continued