Development of a chatbot for school violence prevention among elementary school students in South Korea: a methodological study

Article information

Child Health Nurs Res. 2024;30(1):45-53

Publication date (electronic) : 2024 January 31

doi : https://doi.org/10.4094/chnr.2023.054

Kyung-Ah Kang¹, Shin-Jeong Kim^,2, Byoung-doo Oh³, Yu-Hyeon Kim⁴

¹Professor, College of Nursing, Sahmyook University, Seoul, Korea

²Professor, School of Nursing · Research Institute of Nursing Science, Hallym University, Chuncheon, Korea

³Research Professor, Cerebrovascular Disease Research Center, Hallym University, Chuncheon, Korea

⁴Undergraduate Student, Division of Software, Hallym University, Chuncheon, Korea

Corresponding author: Shin-Jeong Kim School of Nursing, Hallym University, 1 Hallimdaehak-gil, Chuncheon 24252, Korea TEL: +82-33-248-2721 FAX: +82-33-248-2734 E-MAIL: ksj@hallym.ac.kr

Received 2023 November 8; Revised 2023 December 10; Accepted 2024 January 10.

Abstract

INTRODUCTION

The 4th industrial revolution is denoted by technical development. The advent of artificial intelligence (AI) is known to be the greatest innovation for humanity and is therefore expected to have profound impact on our life [1,2]. Furthermore, AI refers to systems or machines that imitate human intelligence to perform tasks and can self-heal based on collected information, especially in learning [2]. They also reported that students’ lack of interest and motivation in learning was influenced by monotonous and less varied learning media.

One application of the AI technology is chatbot technology. Chatbots are natural language processing software programs that use AI [3] and are designed to mimic human conversations through text [4]. Chatbots interact with users [5] for understanding questions faster, provide efficient answers, and extract critical information from large user-defined datasets to improve timing [3,6]. In addition, with the recent changes in educational paradigms, various learning modalities have emerged. This has been further stimulated by the COVID-19 pandemic, and teachers are considering alternatives to conventional learning environments [7]. Thus, they can be used to develop learning media as presenters of interactive and interesting educational areas by increasing user engagement [1,4,8]. With a rapidly growing trend in education, chatbots have been adopted as educational agents [1,5].

Violence, which is prevalent worldwide, refers to intense and severe power that causes injury or destruction [9]. Violence against elementary school students is a serious and persistent social problem in South Korea [10]. According to the Korean Ministry of Education's 2021 School Violence Survey, elementary school kids (ages 9 to 11) were more likely than middle and high school students to suffer violence [11]. A pressing problem is that school violence is not just a one-time incident but is continuous and is increasingly becoming a common behavior in schools [12].

Considering the seriousness of violence, each elementary school runs a school violence prevention education program as part of its formal education [11]. Most programs aimed at preventing school violence are taught through face-to-face instruction and self-report measures. However, it is unclear whether this approach is effective. Additionally, there are other limiting conditions, such as the number of students per teacher, lack of instructors’ pedagogical knowledge of the question, and limited class time [7,11,13]. To overcome this barrier, a reliable and practical educational modality is required to assist teachers in facilitating students’ effective and efficient participation in learning activity [14].

Today's students, known as Generation Z, have grown up in a digital environment surrounded by computers, the internet, and smartphones [4]. Digital tools have become an integral part of students' lives in this generation, resulting in their superior dexterity with technological devices compared to previous generations [3]. New educational methods that suit the preferences of younger generations have distinct advantages over conventional teaching and learning approaches [15]. Interacting with chatbots in a human-like manner appears to positively contribute to providing support and information [2,4]. Furthermore, students who are unable to ask questions can find answers to their queries at any time of the day with the assistance of chatbots, which supplement teachers [3]. Additionally, chatbots have the potential to encourage students to ask questions, even when the topic is difficult, such as school violence [16].

Therefore, research on chatbot applications is considered valuable. However, research on chatbots that applies it to educational programs for elementary school students is in its infancy, and the development of chatbots t is limited [16]. Therefore, it is necessary to develop a chatbot for school violence prevention (C-SVP) to prevent school violence effectively. Thus, it is a practical guideline to help children live a safe school life by preventing and reducing violence among elementary school children.

This study develops a C-SVP that can be used to provide helpful information for elementary school students. The objective of this study is to a) develop a C-SVP, b) test the validity of the developed C-SVP with the help of experts, and c) identify the satisfaction of elementary school students.

METHODS

2. Study Procedure

To achieve study objectives, only the analysis, design, and development (ADD) phase of Seels and Richey’s analysis, design, development, implementing, and evaluation (ADDIE) model [18] was applied as the developmental method (Figure 1). The reason we adopted this model was that it was the most suitable for this study as it helps to develop the chatbot by reviewing existing literature and analyzing the students’ awareness of chatbots. Further, it reflects opinions through formative evaluation and feedback from a group of experts (computer program developers, teachers, and child health nursing professors) and students who will use the developed chatbot.

Figure 1.

The process of develop a chatbot for school violence prevention (C-SVP) among elementary school students according to the ADDIE model. ADDIE, analysis, design, development, implementation, and evaluation.

1) Analysis phase

In this phase, an analysis of the learner’s needs and the learning environment is required. We searched literature related to school violence prevention in children published in the last 10 years using keywords such as “school violence prevention,” and “chatbot” from April to June, 2022. We conducted preliminary research to identify students’ awareness of the use of chatbots for school violence prevention [13]. In total, 215 fifth- to sixth-grade students were surveyed between July 1 to July 22, 2022 at school using a self-reported questionnaire consisting of 22 items. From the results, it was noted that the students perceived chatbots positively as indicated by the mean score of 3.43±0.83 out of 5 points.

To analyze the educational environment, students’ smartphone ownership rate required to use a chatbot was investigated. We identified whether the students had used the KakaoTalk application. As shown by the survey of the use of Internet services in 2023 Youth statistics [19], all elementary school students had smartphones, installed the KakaoTalk app, and used it for text messaging.

2) Design phase

In the design phase, the Attention (A), Relevance (R), Confidence (C), and Satisfaction (S) (ARCS) theory on teaching strategies [20] was used to construct the contents of the C-SVP. The intent was to compose emoticons or letters to attract students’ attention (A), and the content was related to students’ actual experiences of school violence to induce interest (R). In addition, students were provided with information that could be easily understood, thus making them feel confident about preventing and coping with school violence and satisfied with the answers from the chatbot (C and S).

Chatbot design is a core step in its development [21]. Based on natural language processing that uses AI, chatbots answer questions using a database that is already stored and available (Figure 1) [21]. Therefore, chatbots with abundant knowledge can improve their response accuracy [6,21]. Considering the elementary learner’s cognitive level [22], the content should be kid-friendly texts with attractive features, such as a human-like appearance using empathy and humor, small talk with tips, and referrals to websites or other resources.

3) Development phase

The main content was developed using KakaoTalk (Figure 1). A basic conversation structure was created using user-created nodes, where each unique node represented a specific scenario of the dialogue. When designing a specific scenario for each conversation, the data must be built into various sentence structures or formats for questions and answers [23]. The software platform was based on the dialogue flow in which the children were free to ask any questions to a chatbot, and the conversation started by the students (Figure 2). The KakaoTalk screen was designed with clear colors, fonts, and font sizes that are easily noticeable for elementary school students. The text uses simple and easy-to-understand words that match the reading level of elementary school students. The themes were classified into the following: (1) the definition of violence, (2) the type of violence that students may experience, and (3) Questions and Answers (Q&A).

Figure 2.

Flow of chatbot design for school violence prevention.

Formative evaluation includes obtaining adequate information and using it as the groundwork for further development [18]. A group of experts, including three child health nursing professors, three elementary school teachers, two health teachers, and two computer program staff, were requested to verify the validity of the contents regarding work, scope, contents, and infrastructure. The content validity index (CVI) consisted of nine items using a 4-point Likert scale (1=not relevant, 2=somewhat relevant, 3=quite relevant, 4=highly relevant).

The satisfaction questionnaire regarding C-SVP was developed based on Davis [24] technology acceptance model, which explains the behavior of accepting new technologies. It consisted of 10 items in terms of professionalism, usefulness, ease of use, trustworthiness, and interest in the developed program, using a 4-point Likert scale (1=strongly disagree, 2=mildly disagree, 3=mildly agree, and 4=strongly agree). After obtaining permission from the elementary school principal and class teacher, informed consent forms were collected from 10 students and their parents on school letter. We introduced 10 students (six girls and four boys) to the developed C-SVP and allowed them to use it for approximately 30 minutes. Following this, a satisfaction questionnaire was administered to them on October 14 to 15, 2023. Each student’s satisfaction level was assessed by checking their degree of satisfaction. In this study, Cronbach’s alpha for reliability was .76.

RESULTS

1. A Chatbot for School Violence Prevention among Elementary School Students

In this study, a C-SVP was developed following the algorithm in Figure 2. We built a prototypical chatbot, developed C-SVP using social network, and deployed KakaoTalk channel using key words like “School Guardian Angel.” The final C-SVP is shown in Figure 3.

Figure 3.

Screenshot of developed a chatbot for school violence prevention.

▪ Opening mind: Students who were curious about school violence could access C-SVP through KakaoTalk. When the students enter the “School Guardian Angel” on KakaoTalk, the bot asks “Are you curious about the school violence?”

▪ Start: At the same time, “definition of violence,” “types of violence,” and “Q&A” message would appear. By clicking on it, the user’s content is displayed.

▪ Definition of school violence: This section helps students understand school violence by providing an accurate definition, including what school violence is and what behaviors constitute school violence.

▪ Types of violence: Related content was shown with pictures corresponding to each type to increase Attention. Each type of violence includes physical violence, verbal violence, extortion, coercion, bullying, sexual violence, and cyberviolence. If they wanted to know more, they could click on the item and an explanation would be provided separately.

▪ Q&A: In this part, it consists of a question and an appropriate answer. When the students ask a question regarding school violence prevention, a C-SVP (School Guardian Angel) recognizes key words and provides an answer. The number of questions was not limited. The chatbot’s answers came from information already stored, including various cases that are common among children of this age. Cases that frequently occur among elementary school students make them Relevant. Additionally, suggestions on preventing violence could help them gain Confidence in preventing violence. In addition, while experiencing school violence, organizations providing online reporting, phone reporting, and counseling services provided a link, so that if they clicked on it immediate help could be sought.

2. Formative Evaluation

In the validity test of the expert group, the CVI scores ranged from 0.90 to 1.00, with an average of 0.97, indicating that the developed C-SVP was suitable [25]. Thus, the C-SVP was verified to be valid.

The evaluation targeted elementary school students who would use the developed “School Guardian Angel” chatbot and was conducted in September 20, 2023, at school during a class hour (Table 1). Satisfaction with the developed C-SVP showed a total mean score of 3.51 (standard deviation [SD]=0.29) at 5 (Table 1). The mean scores for the five subcategories were professionalism, 3.57 (SD=0.44); usefulness, 3.50 (SD=0.28); ease of use, 3.71 (SD=0.48); trustworthiness, 3.36 (SD=0.47); and interest 3.14 (SD=0.37).

Table 1.

Elementary School Students’ Satisfaction with the Developed Chatbot for School Violence Prevention (N=10)

Based on formative evaluation, the developed C-SVP was considered appropriate for elementary school students.

DISCUSSION

There is no doubt that the younger generation is shifting towards technology and will continue to grow [2]. Chatbots are computer programs designed to conduct automated conversations with users. They are equipped with knowledge of specific domains and are intended to achieve particular goals [16]. Ibrahim et al. [15] reported that the younger generation showed a preference for computer technology in learning. Education using chatbots is personalized and can enhance their satisfaction [26].

This C-SVP was based on an educational modality composed of prevention information regarding school violence that occurs in various situations and assumes that it would contribute to the safety and health of children. In this study, we used the analysis, design, and development phases of the ADDIE model to develop a C-SVP. In each phase, a literature review was conducted and expert opinions were obtained. Keller’s ARCS theory [27] was also applied to help users provide optimal information and stimulate the active exploration of the unknown or curious things. In this theory, Attention was used by focusing C-SVP on the start to achieve a goal, and Relevance was applied to provide the experiences of various cases around elementary school students’, thus encouraging the students to think for themselves. Confidence made users believe that they are successful at preventing school violence when they are prepared with sufficient information. Satisfaction refers to feeling a sense of accomplishment by answering questions, and its importance is that it motivates the use of C-SVPs.

In addition, as Doi et al. [23] pointed out, preparing a question list for creating a chatbot is a labor-intensive task. There was a problem of looking up sentences in the Q&A database. We also generated other related questions from the Q&A database. Chatbots based on user preferences and needs are lacking [4]. Therefore, this study developed a C-SVP that can be directly utilized not only for children but also in the educational field to help teachers educate elementary school students in the context of rapidly increasing childhood school violence.

A chatbot works on recognition and uses natural processing languages to serve user queries [21]. Chatbot algorithms use question and answer trees to handle various student situations and encourage meaningful responses with quick analysis of their inquiries [14]. To be acquainted with the types of questions the students ask in relation to school violence, the researchers consulted a list of 319 anonymized words and put them into the C-SVP.

Formative evaluation is a process of ensuring validity by verifying a developed program on a small number of participants to determine whether it can be applied in the field; it is conducted before the actual implementation of the developed program to verify its effectiveness. Formative evaluation consists of two processes: collecting the opinions of a group of experts, and conducting a preliminary survey on a small number of people to whom the evaluation will be applied [18]. Through this, the applicability of the program was judged, and its validity was secured. In previous studies on the prevention of violence in children, evaluations such as validity and satisfaction during the process were uncommon [28].

This study was conducted through a satisfaction survey targeting 10 elementary school children belonging to the 5th and 6th grade and showed scores above the midpoint score. According to Wilson and Marasoiu [2], chatbots are particularly needed in this sensitive domain given that violence prevention is appropriate. In a study conducted by Topal et al. [3] targeting 5th grade elementary school science students showed that the students reported positively and additionally stated that they would like to use chatbot activities in other courses.

Chatbots increase user engagement, attention, and interest [4]. For Generation Z, chatbot applications can be an effective educational modality to attract them to the learning-teaching environment, including preventing violence. According to the results of this study, it can be concluded that students perceived the chatbot as a guide to help them learn topics outside the classroom. Chatbots have been found to be good assistants for learning outside the classroom [3].

Elementary school teachers can utilize and integrate the developed C-SVP as material in effective media that can stimulate and challenge students to learn about it. If teachers cannot adapt to technical advances in education, the learning process will continue to be carried out conventionally. Teachers need to focus on creating and enhancing program references for more effective and productive education. In this respect, this study contributes to teachers’ efforts in using chatbots as learning media.

Specifically, the students were satisfied that the chatbot was fast, accurate, easy to use, interesting, and trustworthy. Rebelo et al. [29] emphasized that one of the most prominent advantages of chatbots is accessibility. Chatbots could have a more instrumental function to assist elementary school students with ambiguities and difficulties with applications. Furthermore, one of the main features of a chatbot is that it provides a technical link for the user [30]. A study conducted on 5th grade elementary school students [3] reported that the experimental group reported a positive impact on their online learning experience due to the chatbot applications. Students found the chatbot to be useful, enjoyable, and it induced eagerness to use it for other courses. Additionally, it provided helpful assistance for learning outside the classroom, allowed them to repeat the course without referrals, and increased the reliability of the chatbot's responses [4].

The C-SVP developed in this study may be considered a practical tool to prevent violence in children, and efforts should be made to disseminate it in schools. The C-SVP can be expanded to other developmental stages. Additionally, since the contents and methods for school violence prevention can change according to the times, social demand, and emergence of new technologies, they need to be continuously upgraded to meet each need. Thus, the significance of this study is that the developed C-SVP provides a foundation for promoting the health of elementary school students by providing information that is appropriate and interesting to the target population.

This methodological study has several limitations. First, to develop the C-SVP, we used the ADD phase of the ADDIE model instead of the entire phase. Further studies are needed to investigate the effectiveness of C-SVPs as a learning modality, and the outcomes of C-SVPs should be evaluated. Second, this study verified the developed C-SVP, which cannot be generalized because a formative evaluation was performed with a small number of elementary school students. Therefore, verification through continuous interventional research on elementary students across various regions is recommended. Third, C-SVP provides information by finding answers based on keywords asked by users, so it cannot provide answers that the user wants or answer questions about content that are not asked. Finally, it should periodically check whether it adequately answers users’ questions, includes new content, and corrects errors if found.

CONCLUSION

This study developed a chatbot for violence prevention among elementary school students. We expect that this will contribute to the prevention of violence among elementary school students. Further studies to examine the effectiveness of C-SVPs are needed to generalize and expand among children by continuously updating the chatbot based on user feedback. Thus, more acceptable chatbots with increased usability and educational efficacy should be developed.

Article information Continued