# data[].attributes.headline
How does the use of ai impact high school students’ math performance?

# data[].attributes.subject
How does the use of ai impact high school students’ math performance?

# data[].attributes.body
In summary, the combination of AI and high school classrooms has great advantages,However, it also has disadvantages that cannot be ignored

# data[].attributes.attachment_link.readable_text
Integrating Ethics and Career Futures with Technical Learning to Promote AI Literacy for Middle School Students: An Exploratory Study

Abstract

The rapid expansion of artificial intelligence (AI) necessitates promoting AI education at the K-12 level. However, educating young learners to become AI literate citizens poses several challenges. The components of AI literacy are ill-defined and it is unclear to what extent middle school students can engage in learning about AI as a sociotechnical system with socio-political implications. In this paper we posit that students must learn three core domains of AI: technical concepts and processes, ethical and societal implications, and career futures in the AI era. This paper describes the design and implementation of the Developing AI Literacy (DAILy) workshop that aimed to integrate middle school students' learning of the three domains. We found that after the workshop, most students developed a general understanding of AI concepts and processes (e.g., supervised learning and logic systems). More importantly, they were able to identify bias, describe ways to mitigate bias in machine learning, and start to consider how AI may impact their future lives and careers. At exit, nearly half of the students explained AI as not just a technical subject, but one that has personal, career, and societal implications. Overall, this finding suggests that the approach of incorporating ethics and career futures into AI education is age appropriate and effective for developing AI literacy among middle school students. This study contributes to the field of AI Education by presenting a model of integrating ethics into the teaching of AI that is appropriate for middle school students.

Introduction

This paper reports an exploratory study that aims to broaden participation in AI by developing AI literacy among middle school students. Identifying the middle school years as a key phase in development of AI literacy, we developed the "Developing AI Literacy" (DAILy) curriculum. The DAILy curriculum engaged students in integrating their understanding of three domains in order to become prepared for participating in AI-infused fields and industries of the future: (1) an age-appropriate technical knowledge and skills in AI, (2) an understanding of AI's ethical and societal implications, and (3) a knowledge of AI's impact on jobs (AI career futures) and how to adapt to the future of work (career adaptability).

The three interwoven domains were established based on our previous experience of working with students (e.g., DiPaola et al., 2020; Payne, 2020) and the notion that AI is not only a technical field but one that has wide-ranging societal and career impacts. The inclusion of ethics in AI education is needed to raise the public's awareness of AI's burgeoning impact on industry and society, and to prepare students to investigate and address ethical issues in AI as critical consumers and potential future creators of AI technologies (Grosz et al., 2019; Payne, 2020). Users of AI systems need to be aware of the potential for bias in their predictions. Future designers of AI, in particular, need to ensure that their products minimize negative impacts due to bias in datasets, models, and predictions. Members of the general public need to be aware of how these systems might be biased against them so they can evaluate their impact, and seek justice for themselves and others.

In this paper we report the design of the DAILy curriculum, assessment tools developed to examine student learning, and the findings from our implementation of the curriculum among middle school students. The results illuminate what middle school students were capable of learning and doing with AI, what challenges they encountered in making sense of AI, and how the DAILy curriculum impacted their attitudes toward AI and future career ideas.

Theoretical Framework

A Need for Integrating Ethics and Career Futures into K-12 AI Education

In the past decade, several initiatives and projects promoting K-12 AI education have emerged and various AI courses, tools, and tutorials have been launched for teaching AI to students in the USA, China, Europe, Korea and many other countries. For instance, existent curricula aimed at young students, up through 8th grade, include Code.org's AI for Oceans, ReadyAI's AI-in-a-box, AI4kids, and MIT AI Education Initiative's collection of AI curricula and tools including the Media Lab's AI + Ethics curriculum for middle school. Curricula for high school aged students include AI4All's Bytes of AI and full-length Open Learning curriculum, UDC's AI + Curriculum for European High School (Guerreiro-Santalla et al., 2020), ISTE's AI Foundations course, and Reaktor's Elements of AI. Some other programs such as Technovation's AI Family Challenge, are explicitly aimed at families' exploration of concepts in AI and applying AI tools to solve community problems. Meanwhile several K-12 teacher professional development programs in AI curricula have been established such as CSER's Teaching AI in the Classroom, ISTE's AI Explorations and their practical use in schools, and teacher AI-4-All's Open Learning program. These programs are critical to bringing AI into schools as some countries such as China have mandated that all high school students learn about artificial intelligence (Jing, 2018).

Ethics, or moral philosophy, is the study of moral principles that govern a person's behavior. It involves systematizing, defending, and recommending concepts of right and wrong behavior. The inclusion of ethical consideration is crucial in AI because AI has presented substantial ethical and socio-political challenges that call for thorough philosophical and ethical analysis (Coeckelbergh, 2020; Gunkel, 2012; Müller, 2020). The main ethical issues related to the impact of AI on human society are the prevention of existential risks to humanity, the impact of AI technologies on our privacy, the impacts of bias in AI, and the development of AI systems that meet our ethical standards (Gordon & Wrenn, 2020). For instance, algorithmic bias in AI applications such as facial recognition, predictive policing, and credit score assignment has had negative impacts on communities of color (Buolamwini & Gebru, 2018; Kirkpatrick, 2016; Selbst, 2017; Van Brakel, 2016). These systems can be discriminatory if they are trained upon biased datasets. Even worse, experiencing consequences of bias in AI systems may dissuade people of color from participating in the AI. This would hinder attempts to improve AI systems by involving diverse groups of users and developers in identifying and mitigating sources of bias. Thus, ethical analyses and guidelines are necessary to avoid negative repercussions of AI on society. Furthermore, research has raised questions about the extent to which students are aware of AI's impact on their everyday lives and its application in fields and industries of the future (Druga et al., 2017; Hasse et al., 2019). This lack of awareness may limit students' understanding of the relevance of AI and thus their interest in pursuing learning trajectories that lead toward careers in AI. Therefore, in their proposed design framework for K-12 AI education, Zhou et al. called for recognizing the ethical implications of AI and embedding ethical discussions and activities in all AI curricula (Zhou et al., 2020).

Teaching AI Ethics through a Sociotechnical Systems Lens

Another reason that AI education should include ethics topics is that AI is a sociotechnical system. The term socio-technical system (STS) was originally coined by Emery and Trist ( 1960) to describe systems that involve a complex interaction between humans, machines and the environmental aspects of the work system. Nowadays STS are systems that span social, cognitive and information systems (hardware, software, personal and societal spaces) (Badham et al., 2000). Common STS examples include communication systems ranging from email, blogs, social media and news media, to consumer products such as shopping and entertainment systems (e.g., YouTube). However, STS are not always neutral sources of information and serve stakeholder (and sometimes political) agendas. They are created by humans and humans decide the goals of the sociotechnical systems they create. As such, it is important that people are educated to become aware of the goals of these systems and be able to distinguish between the advertised and actual goals of corporations in order to make informed decisions on whether or not to actively participate. For example, the YouTube recommendation algorithm is advertised as a way to provide entertainment for users whereas in actuality its aim is to make profit for the company, and gather user information.

Furthermore, from a complex systems perspective, it is important to understand the distal effects that organizations using these systems may have on individuals, including those that do not directly use the system, and thus society. Algorithmic bias in machine learning is one such example (Buolamwini & Gebru, 2018). In AI systems training data is used to build a model capable of making predictions. The composition of the training data affects the predictions made by the system. Training data can be biased and not be representative of populations. It can also utilize proxies for data it does not include-for example using zip code as a proxy for socioeconomic status and race may pose unintended (and potentially harmful) consequences. Individual users receive predictions made by the systems, regardless of whether they are represented in the system, and even those who do not directly use the system may be impacted by its existence. An example of secondary and tertiary effects of a biased system's existence and use include unfair hiring practices that impacts the economic viability of communities, and potential civil unrest that results from widening economic gaps.

Teaching AI Ethics through a Career Futures Lens

AI is rapidly changing the industry and shaping the future labor market. Exclusion of topics around career futures in K-12 AI education would leave the students unprepared for the future workplace. A 2017 report by McKinsey Global Institute predicted that 75 million to 375 million workers (3-14% of the global workforce) may be required to change occupations and/or upgrade their skills by 2030 because of the adoption of AI (Manyika et al., 2017). All future workers will be required to adapt to working with increasingly intelligent machines, necessitating their preparation to learn in-demand skills and be flexible in setting expectations about work. Therefore, learning how AI will impact jobs (career futures) and developing skills of how to adapt to changes in working environments (career adaptability) is critical to preparing our youth for a future with AI. Further, learning AI's impact on jobs and employment helps further students' understanding of societal changes driven by AI technologies.

Embedding Ethics into CS/AI Education

Research on computer science education has revealed that the traditional approach of teaching ethics as distinct from the subject content often fails to prepare students for real-world work (Boss, 1994; Gardner, 1991). Fiesler et al. ( 2020) analyzed 51 university-level AI and ML courses and found that a majority of the courses cover ethics-related topics within the last two classes. Ethics topics are often considered as a part of these technical courses "if time allows." Such an approach of ethics instruction fails to translate into experiences outside the classroom and leaves students unprepared for the current and future work in technology. Recently many CS educators have realized the potential benefits of ethics education and attempted to integrate ethics across computer science curriculum (Loescher et al., 2005). For instance, Skirpan et al. ( 2018) integrated ethics throughout an undergraduate Human Centered Computing course and demonstrated a meaningful increase in students' consideration of ethics in technology design. Narayanan and Vallor ( 2014) argued that in order to see ethics as an important part of the building process, students must be able to practice ethical decisions during in-class collaborations. Overall, researchers agree that ethics education should be integral to CS education and it can motivate students and prepare them to create ethical designs. Yet there are still many debates about how to best integrate ethics education into CS courses (DuBois & Burkemper, 2002).

Teaching AI Concepts at the K-12 Level through Interactive Approaches

Different from the math-first, theoretical approach (McGovern et al., 2011; Torrey, 2012) that has been shown successful in teaching AI at the undergraduate, graduate, or professional level, AI curricula and tools aimed at K-12 audiences often take a non-mathematical and interactive approach. For example, Narahara and Kobayashi ( 2018) proposed utilizing hands-on educational modules to introduce ideas in AI and Robotics. Students first build a physical prototype of a toy car, then play and test the car in a virtual reality (VR) environment, train an AI model based on the dataset acquired from the virtual testing, and run the toy car using a trained AI model on a physical track. In another study of teaching primary school students AI concepts, Ho et al. ( 2019) engaged students in building and training a self- learning lawn bowling robot.

One reason for the shifted approach is the high barrier presented by the prerequisite mathematical knowledge of the math-first approach. Many K-12 students, particularly those underrepresented in STEM/Computing, may feel unprepared and not be able to persist and succeed in completing these courses. Marques and colleagues, in their systematic mapping study, noted that many curricula teaching Machine Learning at the K-12 level employed the interactive approach where students are exposed to AI concepts through interactions with AI tools as an end-user followed with some degree of knowledge building and reasoning about how AI works. Some of the underlying complex AI processes are black-boxed as hands-on activities to prevent students from being cognitively overwhelmed. Marques et al. ( 2020) highlighted the importance of identifying a balance between the black-boxed processes and uncovered processes as well as a learning sequence based on the complexity of the concepts when developing AI curricula.

Engaging Middle School Aged Students in AI Ethics Education

The major impetus for educating middle school students about AI is driven by their increasing contact with AI in daily life. First, since many students acquire their first mobile device during middle school, they start consuming data on social media websites such as Twitter and Instagram where they are exposed to AI moderated and generated content (Pew Research Center, 2018). Second, middle school students are already creators of media generated with AI through social media apps such as Snapchat and Instagram. According to a survey conducted by the Pew Research Center ( 2018), 45% of teens admit they are on social media almost constantly and YouTube, Instagram and Snapchat are the most often used social media tools by teenagers. The youth view and create content with tools such as photo filters that integrate generative modeling techniques. Thus, they may be using AI-enabled technologies without realizing it. Third, students upload personal data, such as images, videos and text, on social media sites and may unwittingly be contributing to datasets used to train AI models. Finally, students are witness to and could be targeted by fake media that are generated by applications of AI such as Deepfakes, like in the case of FaceApp. This exposure to AI, whether direct or indirect, can impact students. While some impacts can be relatively harmless, such as entertainment or art, other exposure could be harmful. Students may unwittingly be persuaded to.

think that a fake event, image, or text is real, and act accordingly. Because students are vulnerable to these manipulations, they need to be knowledgeable about AI. Their awareness of manipulated media has ramifications for democracy, trust, security, and privacy. Knowledge of the existence of AI-moderated and generated media would empower students to take information that they witness online with essential skepticism. Thus, AI literacy focusing on machine learning techniques used in media curation and generation is imperative for students to be informed citizens and critical consumers of online media.

Another impetus for engaging middle school students in AI education is that young adolescence is an important time for students to form identity in STEM (i.e., considering themselves as a STEM person) and start thinking about their future career interests (Dabney et al., 2012; Maltese & Tai, 2010). Through analyzing data of a longitudinal study that followed 12,000 students from 8 th grade to college, Tai et al. ( 2006) found that students reporting an interest in science careers in eighth grade were three times more likely to obtain a college degree in a science field than those who did not show that interest after controlling for differences in background and academic history. The DAILy curriculum engages students in finding out future STEM jobs that they are interested in and exploring how AI has been and will continue impacting their jobs of interest. This career training leverages students' existing interests to further spark their enthusiasm in AI and at the same time reinforces the notion that AI is a sociotechnical system with enormous societal impacts.

DAILy Curriculum

Experienced middle school teachers and AI researchers were involved in the development of the curriculum and reviewed the initial drafts of the curriculum. We also pilot tested key activities with a group of middle school students in an afterschool setting and asked for their feedback and suggestions to make the curriculum more exciting for middle schoolers.

AI Technical Concepts Addressed in DAILy Curriculum

To make AI learning engaging and accessible, the DAILy activities utilize everyday contexts and interactive activities (e.g., hands-on, kinesthetic) to explicate AI processes and implications and emphasize the relevance of AI to students' lives. For instance, the Pastaland activity in Module 2 is designed to engage students in learning decision trees and experiencing the construction of a decision tree. After they learn the definition of decision tree (a flowchart-like structure that looks sort of like an upside-down tree) and discussed a few examples of decision trees, students are introduced to a problem:

" There is a land of pasta known for most excellent cuisine with a queen who wants to classify all the dry pasta in her land and store them in bins. She wants to be able to quickly find the pasta she needs to cook the dishes she desires.... YOU, as a subject in PastaLand, are tasked with building a classification system that can be used to describe and classify the pasta so the pasta can easily be found when the queen wants a certain dish. Please help the Queen of PastaLand!! "

Further nearly half of the DAILy activities are devoted to generative AI and introduce how AI has been integrated with arts, media, and social media (Module 5). These activities not only leverage students' interest and experience in these fields to engage them but provide a low entry into AI for students who may be more interested in topics other than STEM or CS and those who may not be confident about their technology skills. The technical aspect addressed in Module 5 includes that generative AI algorithms learn to create novel data that could belong to the training dataset; and algorithms such as Generative Adversarial Networks (or GANs) have the ability to produce novel visual art, text, music and even videos. GANs consist of a generator network that creates new data instances, and discriminator network that classifies the generated media based on a training dataset, and provides feedback to the generator network. GANs have found applications in art, education, robotics and healthcare. Module 5 engages students in understanding how GANs work, exploring AI tools that generates text, images, and videos, and discussing the ethical and societal implications of Generative models such as Deepfakes or potentially fueling the spread of misinformation.

AI Ethics and Career Futures

Each module utilizes a similar structure to engage students in interweaving their learning of technical, ethics, and career of AI. Students typically start with learning of the key technical AI concept, investigate potential for bias in the datasets and algorithms (and potential mitigation strategies), discuss the societal and ethical impacts of biased AI systems, and connect AI to their daily lives and future selves by engaging in career exploration activities. For instance, in Module 1, after learning that AI uses algorithm, students learn and experience how different stakeholders make different algorithmic decisions. They first create algorithms for making the best peanut butter jelly sandwiches and compare the algorithms. Then they bring their understanding of stakeholders like doctors, parents, and classmates (e.g., doctors probably care more about nutrients over the taste of the sandwich, classmates may care more about the taste, and parents may care both), to determine what "the best" peanut butter jelly sandwiches would include based on the perspectives of the stakeholders. Afterward, they discuss how the algorithms of "the best" peanut butter jelly sandwiches made by a parent would differ from the algorithms made by a doctor or a teenager. In Module 2, students learn about various bias issues of AI through investigations of technologies (e.g., doing a Google search for images of "outdoor recreation", exploring the face dataset from QuickDraw) and discuss the implications of the bias they found in existing technologies such as who may be impacted and how they may be impacted.

The career exploration activities were designed to reinforce students' ideas of the ethical and societal implications of AI. Drawing upon the Psychology of Working Theory (Duffy et al., 2016) and the Tools for Tomorrow curriculum (Kenny et al., 2004) that fosters career and self-exploration as well as knowledge about resources and barriers, the DAILy career activities seek to facilitate students' exploration of potential career trajectories in the AI age. The core elements of the career component focus on enhancing 1) critical reflection and action (e.g., finding out what students' interests and strengths are, discerning factors that contribute to accessing meaningful and decent work); 2) proactive engagement (e.g., investigating how their desired careers will be impacted by AI through YouTube videos, developing the skills and orientation to take actions to maximize one's volition and initiatives); and 3) social support and community engagement (e.g., examining whether students have access to relational and community support). A set of videos of interviews with AI experts working in different fields (entertainment, law, automotive, etc.) were also included to further inspire students.

Methods

Research Questions

This exploratory study aimed to examine whether and to what extent the DAILy curriculum helped middle school students develop AI literacy. Using a mixed-methods convergent design (Creswell & Plano Clark, 2018), this paper focused on the following research questions:

1. 1.
What are students' conceptions of AI? Do these ideas change after the DAILy curriculum?

2. 2.
Does learning with the DAIly curriculum change students?

1. a)
technical knowledge and skills of AI?

2. b)
attitudes toward AI and AI careers?

3. c)
ideas about AI's ethical and societal implications?

Participants

Twenty-five middle school students in a summer STEM program participated in this study. The STEM program was an urban outreach initiative of a local university, with a focus on recruiting urban students who have a grade of C or lower in science/technology courses, are from low-income families, and may become the first person to go to college in their family. The recruitment was conducted through the partnership between the university and local urban schools: the students were first identified and recommended by their teachers and/counselors. Then the STEM program coordinator reached out to the students and their families and held information sessions to inform them about the program and to encourage them to fill in the application form.

A majority of the participants of the DAILy workshop were from groups underrepresented in STEM/CS, with 32% being African American, 56% Hispanic/Latino, and 56% female. Most of the participating students (92%, n  = 23) were in the grade 7-9 range. Students were asked about their prior experience with using Scratch as a proxy for programming/coding experience: 58% had no prior experience and 42% reported as having some experience.

Virtual Implementation of DAILy Workshop

Due to the pandemic, the DAILy workshop was implemented online. Students met online for ten hours every week for three weeks. All the activities were implemented synchronously via Zoom and all curricular materials were accessible through Google Classroom. The workshop was taught by a team of educators (2 educators and 3 teaching assistants). The sessions typically started with the instructor introducing the unit's topic, followed by a whole-class activity, a small group or individual activity, a discussion relating to ethical implications, and its connections to AI careers. Participants were randomly grouped into three groups of 7 or 8 individuals for small group discussions and hands-on activities. We found that students in this study completed all curricular activities with high attendance rates (85%-95% of all participants attended each day).

The major change for the virtual implementation of the DAILy workshop was that several activities were converted from live-action to virtual formats in order to be accessible during the COVID-19 pandemic. One example is the participatory simulation called the "Artificial Neural Network Game." In the original game, students played the role of nodes in a neural network formed by students sitting in predefined rows representing layers of the neural network. Input nodes are provided with an image and tasked with writing four words on individual cards that describe the image. Each of the words is distributed to each of the four hidden layer nodes. Hidden layer nodes select two words from their set of four to pass on to the output node. The output node creates a caption using four of the eight words it has received from the hidden layer nodes. Subsequently the "unveiling" takes place wherein the original image and its caption are exposed for all to see. Next, we introduce the processes of back propagation and gradient descent to mimic the training process in supervised learning. After the unveiling of the original image and its caption, students come up with an evaluation function to assess how well the network performed on captioning, then feedback is provided to nodes by passing circled words (if the word appeared in the original caption) or uncircled words back to their originators. After a discussion of the feedback and possible adjustments to the node's word selection behavior, the students can play additional rounds with new images and captions to see if/how the neural network learns to get better at captioning. During wrap-up discussions, facilitators reinforce that students are modeling an artificial neural network and review the analogies made between the actions in the game and processes in supervised learning. In this version of the game, back propagation was described as sending information back through the network wherein each node reflected on which nodes passed them "good" information, and using this information in subsequent rounds when picking words to send on.

To port the Artificial Neural Network game to an online activity, the network diagram was laid out in a Google Drawing and layered with data boxes to hold words. Figure 3 shows the Google Drawing for this NN activity with student sample data. Players in an online platform (typically Zoom) were given access to the Google Drawing and assigned the role of nodes as before. The input nodes were moved to a breakout room to privately view the original image to be captioned prior to returning to the main room to generate descriptive words and enter them into the data boxes. In the feed forward phase, words were distributed as before to the nodes in the next layer. A drawback of this instantiation is that all of the selected words were visible to all players during game play thus reducing the element of surprise and possibly impacting the selections made by those acting as hidden layer nodes. An affordance of the digital version is the ability to represent and change the weighting of the links between nodes thus making it easier for players to remember which nodes provided them with "good" information in prior rounds. This online version was tested with students participating in an after-school club before the DAILy workshop.

Data Collection

We collected quantitative and qualitative data to make sense of elements of AI and AI processes that students were able to grasp or not and the impact of DAILy on the affective domains of student AI learning. The data were collected from students who attended at least 25 h of the DAILy workshop (we eliminated data of students who missed two or more days of the workshop). The quantitative data include responses to a pre and posttest, administered before and after the camp to participating students. The pre/posttest included three instruments developed by the research group: the AI Concept Inventory (AI-CI); Attitudes toward AI survey; and AI Career Futures survey. More details of the instruments and scoring are described later.

The qualitative data include students' final presentations completed at the end of the workshop, observation notes, and semi-structured interviews with a group of purposefully selected students ( n = 19) after the workshop. The interviewees were selected to represent a range of variety in terms of age, gender, ethnic background, and prior technology experiences. The interview questions focused on student learning experiences and their understanding of the ethical and social implications of AI. In this paper we focused on student responses to three semi-structured interview questions, (1) What are the benefits of AI?; (2) What are the harms of AI?; and (3) If you are going to build an AI system, what would you do to ensure it's fair?. Audio recordings of interviews were transcribed and coded using grounded theory (Strauss & Corbin, 1994) to reveal student understanding of the AI implications to the society. The findings were triangulated with observation notes and other data collected to generate an in-depth understanding of how students learn about bias issues in AI.

Instruments

Three instruments were developed to investigate students' learning of AI with the DAILy workshops: AI Concept inventory, Attitudes toward AI survey; and AI Career Futures survey. To ensure the validity of these instruments, we first conducted reviews of the items with AI researchers and STEM educators to gather their feedback of the content validity. Afterwards two rounds of cognitive interviews were conducted with six middle school aged students (not included in this study) to determine if the items meet the targeted construct and assessment objectives. Then a pilot testing was conducted with 30 middle school aged students (not part of this study population) to investigate if there were any confusions in the wording of questions and overall language appropriateness of the items in the instruments for middle schoolers. We revised the instruments based on the feedback from the cognitive interviews and the pilot test. In this study, participants completed the identical revised instruments before and after the DAILy workshop.

AI Concept Inventory (AI-CI)

Informed by Porter and colleagues' work on modular concept inventory (Almstrum et al., 2006; Porter et al., 2014; Taylor et al., 2014), AI-CI focuses on measuring student understanding of a set of core concepts and processes covered in the DAILy curriculum. In total, AI-CI consists of six scales. Table 2 shows example questions of AI-CI.

1. 1)
AI general concepts, including 12 multiple-choice (MC) questions asking if students can distinguish between technologies that involve AI and one explanation item asking students what they think AI is.

2. 2)
Logic systems, including 4 MC questions to assess students' understanding of the processes decision trees utilize and their ability to apply the understanding to categorize things and make decisions.

3. 3)
Machine Learning (ML) general concepts, which includes 7 MC questions asking students to distinguish between example technologies of classifying and generative AI, and to discern examples of supervised and unsupervised learning.

4. 4)
Supervised Learning (8 MC questions), which tests student understanding of the processes of supervised learning and ability to apply their understanding to determine how an AI technology would classify things based on the labeled data on which it is trained.

5. 5)
Neural Network (4 MC questions) that examines student understanding of the processes of neural network.

6. 6)
Generative Adversarial Networks (GANs, 4 questions) that focuses on examining students' understanding of generator, discriminator, and processes of GANs through True or false and MC questions.

Attitudes toward AI Survey

The Attitudes toward AI Survey focuses on soliciting participants' interest in AI, anxiety about AI, and relevance of AI to their life. These survey questions are 5-Likert scale questions drawn and modified from validated instruments of the Science Motivation Questionnaire II (Glynn et al., 2011), the Modified Attitudes Towards Science Inventory (Weinburgh & Steele, 2000), and the AI anxiety scale (Wang & Wang, 2019). Each item presents a statement and asks students to indicate how strongly they agree/disagree with the statement (strongly agree to strongly disagree). The Interest in AI scale consists of 9 items (Cronbach's alpha = 0.94) and sample statements include " I will take a class about AI if it is offered in my school" and " I will talk to members of my household about what I know about AI." The relevance scale has 8 items (Cronbach's alpha of 0.71), with sample item statements like " Learning about AI is relevant to my life." The anxiety scale includes 9 items with Cronbach's alpha of 0.91 and sample item statements like " I do not feel comfortable about using or building AI technologies" and " Working with AI would make me nervous. " Each scale was examined for item-total correlations and Cronbach's alpha if deleted to determine that the items were measuring the same construct.

AI Career Futures Survey

The AI Career Futures survey examines students' awareness of AI impact on future jobs and career adaptability because the DAILy workshops engaged students in exploring careers that matched their interest, finding out how AI has been influencing the matched jobs, and becoming knowledgeable of preparations for entering their desired career fields. The career adaptability subscale was developed based on the revised Career Futures Inventory (Rottinghaus et al., 2012), which is built on a theoretically-derived, multidimensional definition of adaptive behavior for future careers such as identifying educational choices to join future jobs and investigating consequences of the actions. Similar to the Attitudes toward AI survey, the AI Career Futures Survey items are 5-Likert scale questions, each of which presents a statement and asks students to indicate how strongly they agree with the statement. Sample statements of the AI career awareness scale include " I know about jobs that use AI" and " I know a role model of my background in fields related to AI." Sample statements of the career adaptability scale include " I learned to prepare for the future," " I learned about educational choices that I must make to get my dream job," and " I became aware of my future job choices. " In total, the AI careers awareness scale consists of 7 items with Cronbach's alpha of 0.822, and the career adaptability scale was composed of 10 items and had a reliability of alpha = 0.934.

Scoring and Data Analysis

All the multiple-choice questions in the AI Concept Inventory were coded as 0 (wrong) or 1 (correct). The 5-Likert scale questions in the Attitudes toward AI survey and the AI Career Futures survey were coded based on student rated agreement with the statements (1: strongly disagree, 5: strongly agree) and reversed coded if needed. Student responses to the items were scored and added to form an aggregate score of the instrument and of each subscale. We conducted statistical analysis such as paired t-test to compare student performance on the pre and posttest. We also examined student responses to each subscale to identify AI concepts and processes with which students had difficulties.

Besides the quantitative analysis, we also analyzed student responses to the open-ended question in the AI Concept Inventory that asked them to explain what they think AI is. We categorized the explanations using emerging categories. The categories reflect the ideas that students put forth in their definitions and are not meant to be treated as a "score." Categories were developed and iteratively revised by two authors of this paper who coded the explanations independently first and then discussed the coding with another author to solve conflicts. We compared the percentage of students with each category of ideas before and after the workshop to capture a nuanced in-depth picture of what ideas students had about AI before the workshop and whether and/or how their ideas changed after the workshop.

Findings and Discussions

Students' Conceptions of AI Before and After the DAILy Workshop

Using emerging coding, we categorized student responses to the question "What do you think AI is?" on the pre and post-test. In total six categories of ideas emerged (see Table 3 for an explanation of the categories and sample student answers). In the "Incorrect" category, students explained wrong or naïve ideas of AI, e.g., " AI is a human intelligence processed by machines." In the "Vague" category, students expressed some ideas of AI but the ideas were unclear, e.g., " I think it's like a lot of computerized things and people make things or robots with AI. That's all I know." In the "General" category, students often referred to human intelligence and explained that AI is different from human intelligence. Students holding the "Societal" category idea of AI explained AI's impact on the society or its ethical implications. For instance, a student explained that " I think it [AI] is technology that can change the world. Also [AI] can be an everyday technology. But it sometimes has malfunctions." Students with the "Technical" category of idea focused their explanation of AI on the technical components such as data, algorithm, and making predictions, e.g., " A program that can process information. The information turns into an action that the AI was told to do." For students who had the "Complex" category of ideas, their explanation included two or more correct ideas about social, general, or technical aspects of AI and reflected a complicated understanding of AI, e.g., " AI means intelligence made by humans in devices and technology. The intelligence is gained by training and testing a data set. It has a whole process which is called an algorithm. AI heavily affects us today and will even more in the future." Students who held the "Complex" ideas started to recognize both the technical and ethical/societal aspects of AI technology and probably consider AI as a sociotechnical system. Students with "Technical" or "Societal" ideas only considered one aspect of AI and did not have a sociotechnical perspective of AI.

We compared the percentages of students who demonstrated each type of ideas in their explanations on the pre and post-test (see Fig. 4). A majority of students started the workshop with a vague (43.5%) or incorrect (34.8%) description of AI. The most common incorrect answers were "a type of human intelligence" or "coding a robot" and vague answers typically defined AI as "intelligent technology" without a clear understanding of what made it intelligent (see Table 3 for examples). Fewer students defined AI in terms of the technical components (4.3%) or with some combination of ethical, general, and technical (4.3%). No students defined AI in terms of societal impact. On the post-test, students most commonly defined AI as incorrect (30.4%). However, several students described AI as a combination of ethical, general, and technical (or complex definitions = 21.7%), indicating that these students have started to develop a sociotechnical perspective of AI. Fewer students defined AI in vague terms (17.4%). The most common incorrect answer in the post test was describing artificial intelligence as "exactly like human intelligence" or "robots".

A closer examination revealed that of the students who started the workshop with an incorrect understanding of AI, 37.5% students developed a complex understanding of AI after the workshop, but 25% continued to stick to the incorrect understanding. For students who started the workshop with a vague understanding, some started to integrate ideas about AI's societal implications, underlying concepts and processes, and general definitions of AI (30%), although many students remained holding vague ideas (30%) or ended up adding non-normative ideas of AI (40%). With regard to students who started with a general understanding of AI, 66.7% incorporated new ideas (e.g., technical, ethical) and developed a complex understanding of AI after the workshop. Overall, we saw 21.7% of students who did not start with a complex understanding of AI develop it by the end of the workshop.

Comparing Students' Technical Knowledge and skills Before and After the Workshop

The paired t-test showed that students on average significantly improved their AI-CI scores after the workshop (pre: Mean = 24.37, SD = 4.83; post: Mean = 26.75, SD = 4.50; p < 0.01). On the post-test students on average achieved a gain of 2.36, which indicates that a majority of the students answered two or more questions correctly after the workshop. Students made the biggest learning gains on four subscales: (1) general concepts of AI, (2) logic systems, (3) general concepts of machine learning, and (4) supervised learning. Table 4 shows the paired t-test results of student performances on AI-CI. Next, we report student performances on each scale in more detail.

AI General Concepts: Recognizing AI and Identifying Features of AI Technology

Recognizing AI

Students made statistically significant gains (Cohen's d = 1.03) on four items that describe different technologies and ask students whether the technology utilizes AI (a software that classifies different types of hats, a drawing app that generates new paintings, a face recognition software, and a phone app that reminds people to turn off lights). On the pretest, most students were able to correctly determine the use of AI in the first three technology examples (average score = 2.76). On the posttest, over half of the students ( n = 13, 52%) answered at least one more question correctly (4 students answered two or more questions correctly). Figure 5 shows the performance of students on these four questions.

One interesting finding is that students improved the most on the LightReminder question, which asks students whether AI is utilized in a smartphone app that sends a reminder for people to turn off light between 9am to 3 pm. On the pretest, only 18 students (72%) answered it incorrectly, i.e., they thought that AI was required in this app. On the posttest, 60% ( n = 15) students realized that AI was not necessary for the app. This suggests that by exposing students to various example technologies that do and do not use AI, the DAILy workshop helped improve students' ability to recognize AI, a critical competency for informed interactions with AI (Long & Magerko, 2020).

Identifying features of AI technology

Four questions were designed to test whether students were able to identify key features of technologies that use machine learning. Each question describes a technology and asks students to determine if the technology uses datasets and makes predictions. We aggregated student responses to the dataset and predication questions respectively and compared the responses from pre to posttest. The results show that students achieved moderate gains in terms of identifying the use of dataset (pretest Mean = 2.44; posttest: Mean = 2.72, p = 0.07, Cohen's d = 0.40), but no gains on the prediction questions after the workshop (pretest: Mean = 2.76, posttest: Mean = 2.44, p = 0.19). Nearly half of the students ( n  = 10, 40%) still answered the prediction questions incorrectly in at least two scenarios on the posttest. Ten students' scores on these questions decreased on the posttest, suggesting a confusion about prediction among students.

Our observations of the "AI or Not" activity (Module 1) resonate with this finding. In this activity students were presented with a list of technologies and asked to distinguish whether it uses AI based on three criteria: algorithm, prediction, and dataset. We noted that students had a hard time determining what prediction in the context of AI means and whether/how it is different from human intelligence. For instance, in discussing whether automatic doors involve AI, some students thought it involves AI because " it automatically opens" and " it makes a prediction to open when you step on the sensor", whereas other students disagreed because automatic doors involve " just a motion sensor that detects when you are in a close radius". In another example of determining whether regular cars use AI, one student argued that " it [regular car] uses AI because the brake predicts to stop when you press on it ". This shows that students tended to reason about prediction based on their everyday experience of human intelligence but may have had different interpretations of prediction. They encountered challenges when discerning technologies that make predictions. Incorporating more concrete examples of predictions and providing a definition of prediction in the context of AI can help contextualize this concept and improve student ability to analyze and determine if the technology involves AI.

Logic Systems

Students achieved significant gains on items that assessed student understanding of logic systems (pretest Mean = 2.6, SD = 1.11, posttest Mean = 2.99, SD = 0.76, p  < 0.05). They started with high pretest scores on this scale, with fifteen students (60%) correctly answering the three questions that ask to categorize things using a decision tree. This suggests that students may have acquired knowledge of how computers categorize things from previous experiences. They, however, had little or no prior knowledge with regard to how to build a decision tree (15 students incorrectly answered the question of choosing steps and sequence involved in building decision trees on the pretest). On the posttest all students answered at least one question correctly and 80% of the students correctly answered three or all four questions. Nine students chose the correct steps and sequences for making decision trees and five students chose the correct steps but put them in a wrong order.

This finding is promising given the short duration of the intervention (the decision tree/Pastaland activity lasted approximately one hour). It was also noteworthy that the implementation of the Pastaland activity did not explicitly teach students the steps or sequence. By building and using decision trees to categorize different types of pasta, students developed ideas about the processes involved and were able to answer the questions correctly.

Machine Learning General Concepts

Supervised vs. unsupervised learning

Students achieved a significant improvement on the set of supervised vs. unsupervised learning questions after the workshop (Pretest: Mean = 1.84, SD = 1.07; Posttest: Mean = 2.2, SD = 0.91; p < 0.05). After the workshop, ten students (40%) improved their scores and correctly determined the use of supervised or unsupervised learning in more technology examples. Nearly half of the students ( n = 11, 44%) correctly identified the use of supervised or unsupervised learning in all the three scenarios on the posttest. This shows that after the workshop, students were able to discern supervised and unsupervised learning by examining whether the technology uses labeled data. Figure 6 shows the percentages of students who correctly identified supervised or unsupervised learning in none, one, two, or three technology scenarios.

Classifying vs. generative AI

We did not find significant gains in students' performance on the set of classifying or generative AI questions (Pretest: Mean = 2.16, SD = 0.55; Posttest: Mean = 2.24, SD = 0.93, p  = 0.70). One reason for this was the ceiling effect: on the pretest 23 students (92%) were able to distinguish between classifying and generative AI correctly in at least two technology examples. This indicates that students may have developed normative ideas about classifying and generative technologies from previous experiences.

Supervised Learning

Students improved their understanding of supervised learning after the workshop: the average score on this scale increased from 4 on the pretest to 4.6 on the posttest (total score of 8). In particular students achieved significant gains on four items asking about the processes of supervised learning (e.g., " What is the difference between the training and testing phases in Supervised Learning?", "What is the purpose of a label on an image in Supervised Learning?"). They on average had little or no prior knowledge (pretest Mean = 1.38, SD = 0.97) before the workshop. On the posttest, over half of the students ( n = 14) were able to tell the purpose of the labelled data in supervised learning. Sixteen students were able to apply their understanding of supervised learning to identify one reason for a flawed machine learning system that can recognize men's shoes accurately but not women's shoes. These findings suggest that the hands-on activities in Module 3 of the DAILy curriculum was effective in promoting students' understanding of supervised learning. The Teachable Machine activity engaged students in experiencing how supervised learning takes place by training and testing their models. Students also experimented how using an unbalanced dataset can result in a biased model. These activities helped demystify supervised learning processes and also reinforced the idea that AI technology can be biased due to the dataset it was trained on.

Neural Networks (NN)

Students made no gains on instruments that examine student understanding of the processes of neural networks (Pretest Mean = 2.89, SD = 1.33; Posttest Mean = 2.87, SD = 1.14). After the workshop, 40% of the students ( n = 10) did not recall any processes involved in neural networks. Over half of the students ( n  = 13) could not tell whether learning takes place during the testing or training phase in neural networks. Three reasons may account for this. First, the intervention was too short. Students in total spent approximately 40 min on the Neural Network activity where they played a participatory simulation game to make sense of how a neural network works (Module 4). The processes (e.g., feeding forward, evaluation, and back propagation) were only mentioned once on a set of 3 slides. Students may need more time and scaffolding to understand the steps of NN. Therefore, although almost all the students we interviewed expressed that they liked "the telephone game" (as we called the NN game), they struggled to make sense of the NN processes. Second, the online learning format limited student interactions and interpretations. Unlike the face-to-face sessions where students can act out and observe how others respond and how the system as a whole behaves, playing the simulation game via videoconferencing greatly limited the interactions between students and made the observations much more challenging, and thereby hindered student interpretation of NN processes. Third, the NN activities in Module 4 did not include real-world applications of NN to foster student understanding of this concept. (Neural network was the most abstract concept discussed in the DAILy curriculum.) Without concrete examples, it may be difficult for young adolescents to grasp the concept of NN. Future revisions of the curricular activity need to provide more scaffolding (e.g., whole group discussions and explicit instructions on what to observe) and utilize examples of the use of NN in our life to help students connect what they did in the game to the NN processes.

GANs

Students on average improved their understanding of how the GANs work, however the learning gains were not statistically significant (pretest: Mean = 2.36 (SD = 0.82), posttest: Mean = 2.64 (SD = 0.99), p  = . 07). The majority of students understood that the generator and discriminator were neural networks and that they followed a back-and-forth process to eventually create new media. However, students did not see the generator and discriminator as working against one another. This could be due to the fact that the short intervention was not sufficient in terms of helping students develop a solid understanding of the complex mechanisms of GANs (students spent 30 min playing the roles of generators and/or discriminators to see how a generator and discriminator network in a GAN works). More scaffoldings are necessary to explain how GANs work. Many of our lessons focused more on applications of GANs rather than reflecting and discussing the underlying mechanisms of GANs.

Comparing Students' Attitudes toward AI and Career Futures Before and After the Workshop

Attitudes toward AI

Students started the workshop with a positive attitude toward AI, which was not surprising given that this was a self-selected program. We did not find significant differences between the performances on the pre and posttest (pretest: Mean = 3.53, SD = 0.46; posttest: Mean = 3.63, SD = 0.29; p = 0.32). Table 5 shows the paired t-test results of each scale of the Attitudes toward AI pre- and post-survey. The largest increase was observed in an item that asked for students' interest in taking class in AI if it is offered at their school. On the pretest, 60% of the students ( n  = 15) indicated vague or no interest (selected "disagree" or "not sure" to the statement), whereas on the posttest, 15 students agreed or strongly agreed with the statement. This shows that the workshop sustained students' interest in AI and students would like to continue with their AI education after this experience.

Students also continued to find relevance of AI to their lives after the DAILy workshop (see Table 5). Most students agreed or strongly agreed with each item statement and felt AI to be highly relevant to their life. The highest increase in scores was observed in an item asking for students' agreement with the statement " It is useful to me right now to know something about AI". This suggests that the DAILy workshop enabled more young adolescents to realize the importance of learning AI at their age. The biggest drop in mean scores was in the item of " I can use what I learn about AI to help my community ". While students see AI as relevant, they may not know how to use it in their capacity to benefit others in their community. One reason for this may be that the DAILy workshop emphasized the bias of AI would have negative impacts on people from underrepresented groups but did not provide any real-life examples of how AI can help people in students' communities.

With regard to students' anxiety about AI, we found a minor increase from pre to posttest. More students agreed to the statements " I think AI can be dangerous or harmful", " The use of AI in everyday life scares me", and " I am afraid that AI will make us lazy" after the workshop. This indicates that more students left the workshop with an increased sense of the negative consequences of AI. Despite this increased anxiety, we also found decreases in agreement with the statements " I think AI can be threatening to society" and " Working with AI would make me nervous ". Altogether this suggests that while students recognized the potential harms of AI, they felt using AI in a correct way would mitigate the damage. Meanwhile the experience of working with AI technologies during the workshop helped reduce their anxiety about AI.

AI Career Futures

Overall, students showed a significant increase in the awareness of AI related careers from the beginning of the program (Mean = 3.15, SD = 0.68) to the end (Mean = 3.48, SD = 0.60; p < 0.001; Cohen's d = 0.52). The biggest positive differences were related to (1) knowing about jobs that use AI, (2) knowing role models in the AI field, and (3) discussing AI with friends and family. The first two increases were likely due to the fact that the workshop exposed students to various jobs that may include the use of AI and people who work in AI related field. The third increase, however, indicates that the workshop has sparked an early interest in AI careers and students tended to discuss with their family and friends to find more resources and opinions for the topic (Maltese & Tai, 2010).

We also found that students reported developing some aspects of career adaptability skills (see Table 5). The biggest increases were observed in students' agreement with statements of " I learned to think about what my future will be like", " I became aware of my future job choices", "I learned to recognize resources available to me and use them", and " I learned about educational choices that I must make to get my dream job". The career training sessions of the workshop engaged students in exploring their matched future job choices and creating a roadmap to enter those fields. Through these activities, students developed not only more concrete ideas of their future jobs but also skills of how to become more adaptive in the AI era. One student spoke about how knowing about AI might be helpful for them in the future, specifically their ability to provide clarity about AI to others who may be confused about its implications, "They keep saying that the more and more in the future we go, the more we use AI. So, I think that if that's going to happen, the fact that I know a lot about AI, and if somebody has questions, or is worrying, or anything, I can reassure them that nothing bad is going to happen. Or if something goes wrong, I can fix it. I have more knowledge of what's going on, so I don't have a bunch of questions. I have a bunch of answers."

Students' Ideas of Ethical and Societal Implications of AI

To explore what kinds of ideas students developed about AI's ethical and societal implications through the DAILy workshop, we examined the interviews conducted with the 19 students after the workshop. The students were selected to represent a wide range of gender, age/grade, and ethnicity. As noted above our analysis focused on student ideas about the benefits and harms of AI and how to make AI technology less biased.

After the workshop almost all the interviewees (except one) articulated both the benefits and harms of AI technologies. They believed that AI can make life easier, help complete dangerous work (" AI is doing some, maybe, risky tasks that humans now don't have to do"), generate more objective conclusions (" it (AI) wouldn't be based on one person's thoughts, because it will collect data from different sources. And this could help the news reporters. I mean the journalists."), and accomplish tasks more efficiently (" A great consequence about it, about AI is that we can shorten, or we can expedite a bunch of jobs or tasks that we do everyday that we can do over 50% faster. ").

With regard to the bad consequences of AI, students expressed three major concerns:

potential for laziness or loss of autonomy: 8 students mentioned that they were afraid that human beings would become lazy and do not work, e.g., " People will definitely get lazy and God forbid if anything were to happen and they would like stop working"; "It could make people not want to do as much work as before, maybe."

discrimination against people of color due to the use of biased AI technology such as facial recognition systems: 4 students explained how the bias in AI would lead to bad consequences. For instance, one student explained this in detail, " A bad use of AI would be how the government or the police, a police station using face recognition to recognize where you are from. Because black people, since you're a darker color and they think you're dangerous or something. Those are things that have happened before. They think that just because you live where lots of darker people live, there's going to be a higher crime rate, even if you don't do anything. Because many conditions can force a person to do something. We just never know about it. Which is why I think it's unfair for people to be labeled based on where they live. Because if you have the money to live in a rich neighborhood, you'll most likely take that chance to..."

harms of the spread of deepfakes: 4 students mentioned the potential harms of generative AI. They worried that this may lead to faking peoples' words, pictures, videos, and identity, e.g., " That sometimes it (AI) can affect the people in the real world. So hackers can do something, or maybe they can make a video with AI of someone's saying something that they never actually said, because of editing and people saying certain words, and then they just put all the words together. So they make it seem like someone actually said something. So can it be used for bad stuff."

Further, 16 students provided constructive solutions when asked how to build an unbiased AI system. They emphasized the importance of using a dataset that includes diverse and unbiased data. For instance, one student talked about setting up a good dataset for the training purpose, " I would use like different types of data sets to program my AI. Like if I was doing on facial recognition system, I would use people of like different ages, genders, ethnicities, and stuff like that to make it a little less biased and have like an equal amount of said people." Another student explained using a diverse dataset for the testing phase, "I would get people of color to come and also different types of people, different cultures too, to test it out and see."

Overall, the interview results demonstrate that most students incorporated ethical and societal implications into their views of AI technology after the workshop. Analysis of students' final presentations suggested similar themes-students frequently referenced anecdotes of bias as memorable learning moments of the workshop, " I learned from this program that artificial intelligence can be biased and that we put information like racism and sexism in our AI without realizing which can make the people using the AI not want to use it. " Further examining student explanations of what AI is yielded further evidence, as more students added ethical and societal impact of AI in their definition of AI on the posttest.

Conclusions

The rise of AI has attracted many researchers' attention and led to calls for creating opportunities to engage young learners in AI education. While the content and pedagogy of AI education at the K-12 level have not been established, recent review studies (Marques et al., 2020; Zhou et al., 2020) pointed out a dearth of AI curriculum and programs that incorporate AI ethics, a topic that is critical to prepare young students to become informed users and developers of AI technology. This paper aims to help fill in this gap by reporting the design and implementation of the DAILy curriculum that interweaves student learning of the technical, ethical or societal, and career aspects of AI. Our results demonstrate that the DAILy workshop was engaging and productive in supporting student learning of general concepts of AI concepts, logic systems, and machine learning. Students enjoyed ethics-related activities embedded in the DAILy curriculum. They were able to internalize what they had experienced, connected to ethical implications of technology design, and developed positive ideas about their future selves with AI after the workshop. Overall, the approach of combining ethics and technical learning of AI is age appropriate and promising in terms of developing AI literacy among middle schoolers. Our work also contributes to the AI education field by providing a working definition of AI literacy grounded in findings of what middle school students are capable of learning and doing with AI.

Further our approach of positioning AI as closely related to society successfully engaged students from groups underrepresented in STEM and computing. We found that female students of color were particularly active in the investigation of bias and ensuing discussion. For instance, upon viewing the disproportionate representation of women of color in commercial datasets used for facial recognition, a female student stated " What I also noticed is that you can see that the women either way ... This is another thing about sexism. Women are also either way still have a lower percentage than the men in their class, I guess you could say. " She further noted how AI systems are biased against people like her while she positions herself as a woman and part of a group who should be represented.

We also found that the students talked with their family members and friends about AI's impact on current and future jobs. A student in his interview described that "I asked her [my mom], "Are you scared if AI takes your job?" She's like, "Yes, but I know I'll have another job." So, it's really cool what AI could do and make new things for other people." This provides further evidence of student engagement as he was active in leveraging existing familial and aspirational capital (Yosso, 2005) to make sense of AI and refine his existing perceptions of AI. Overall, the emphasis on AI's ethical and societal implications offers an effective way to excite underrepresented students, which resonates with previous research that minority students are often more drawn to STEM/CS programs that teaches the content via the context of solving social justice issues (Mark et al., 2013; Vakil, 2018). This offers valuable insights into broadening participation in Computer Science and AI education.

Limitations and Next Steps

There are a few limitations of this study. First, the students were recruited to participate instead of being randomly selected. The students were from a very specific population, low-income families in urban areas and may become the first generation of college students in their family. Further, the workshop was taught by a team of researchers and educators who developed the DAILy curriculum. The findings may differ from situations involving participants, treatments, and settings different from those in the study. Another limitation is that this study measured immediate effects of the intervention using a pre-posttest design. Complex AI processes such as neural networks may require longer exposures and more time for students to reflect and internalize. Conducting a delayed posttest and interviews could clarify the long-term effects of the DAILy experience on student ideas about AI.

As our next step, we plan to revise the DAILy curriculum based on the findings from this implementation. For instance, many students were found to remain confused about what prediction means in the context of AI after the workshop. In Module 1 (introduction of AI), we plan to add more concrete examples of prediction in AI (based on what is learned to make a prediction/inference) and engage students in discussions or debates on whether and why technologies make predictions or not. These activities would help clarify students' misconceptions and enhance their understanding of prediction in the context of AI. Another challenge we found was that activities of NN were engaging yet difficult for students. Students had a hard time connecting their virtual actions (playing of the role of the nodes) with the abstract processes of NN. We plan to include scaffolds to help students make the connections, e.g., asking students to record their actions, make analogies to the NN processes, and discuss with peers. In addition, some students provided the feedback that they hope to apply their knowledge of AI and ethics to help their community. Informed by DiPaola et al. ( 2020) work, we plan to include a capstone project for students to explain how their knowledge of AI ethics can help others. The capstone project could engage students in developing technologies with ethical considerations such as developing a fair local news feed and redesigning the recommendation system of social media. Such learning experience will not only engage students in applying their AI knowledge and skills, but also enable them to recognize their own AI capability to help others and become more confident in working with AI.

Materials Availability

The materials presented in this work can be found on our website https://raise.mit.edu/daily/index.html.

References

Ali, S., Payne, B. H., Williams, R., Park, H. W., & Breazeal, C. (2019). Constructionism, ethics, and creativity: Developing primary and middle school artificial intelligence education. In International workshop on education in artificial intelligence K-12 (EDUAI'19) (pp. 1-4).

Ali, S., DiPaola, D., Lee, I., Hong, J., & Breazeal, C. (2021). Exploring Generative Models with Middle School Students. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1-13).

Almstrum, V. L., Henderson, P. B., Harvey, V., Heeren, C., Marion, W., Riedesel, C., Soh, L.-K., & Tew, A. E. (2006). Concept inventories in computer science for the topic discrete mathematics. In Working group reports on ITiCSE on Innovation and technology in computer science education (pp. 132-145).

Badham, R., Clegg, C., & Wall, T. (2000). Socio-technical theory. Handbook of Ergonomics. John Wiley.

Buolamwini, J., & Gebru, T. (2018). Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency (pp. 77-91). PMLR.

Coeckelbergh, M. (2020). AI ethics. MIT Press.

Book Google Scholar

Crain, W. C. (1985). Theories of development. Prentice-Hall, Inc.

Creswell, J. W., & Plano Clark, V. (2018). Designing and conducting mixed methods research. Sage Publications.

Google Scholar

Danyluk, A., Howe, A., Gini, M., & Anderson, M. (2014). Broadening Participation in AI. Computing Research News, 26(10), Article 10.

DiPaola, D., Payne, B. H., & Breazeal, C. (2020). Decoding design agendas: an ethical design activity for middle school students. In Proceedings of the Interaction Design and Children Conference (pp. 1-10).

Druga, S., Williams, R., Breazeal, C., & Resnick, M. (2017). " Hey Google is it ok if I eat you?" Initial explorations in child-agent interaction. In Proceedings of the 2017 conference on interaction design and children (pp. 595-600).

Duffy, R. D., Blustein, D. L., Diemer, M. A., & Autin, K. L. (2016). The psychology of working theory. Journal of Counseling Psychology, 63(2), 127.

Emery, F. E., & Trist, E. L. (1960). Socio-technical systems. In C. W. Churchman & M. Verhulst (Eds.), Management science models and techniques. (Vol. 2). Pergamon Press.

Google Scholar

Fiesler, C., Garrett, N., & Beard, N. (2020). What do we teach when we teach tech ethics? A syllabi analysis. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education (pp. 289-295).

Guerreiro-Santalla, S., Bellas, F., & Duro, R. J. (2020). Artificial intelligence in pre-university education: what and how to teach. In Multidisciplinary Digital Publishing Institute Proceedings (Vol. 54, No. 1, p. 48).

Gunkel, D. J. (2012). The machine question: Critical perspectives on AI, robots, and ethics. MIT Press.

Book Google Scholar

Hasse, A., Cortesi, S., Lombana, A., & Gasser, U. (2019). Youth and artificial intelligence: Where we stand. In Berkman Klein Center Research Publication. Article 2019-3.

Ho, J. W., Scadding, M., Kong, S. C., Andone, D., Biswas, G., Hoppe, H. U., & Hsu, T. C. (2019). Classroom activities for teaching artificial intelligence to primary school students. In Proceedings of International Conference on Computational Thinking Education (pp. 157-159).

Kenny, M., Bower, M., Perry, J., Blustein, D., & Amtzis, A. (2004). The Tools for Tomorrow Program: Integrating school-to-career psychoeducation into high school curriculum. Chestnut Hill, MA: Boston College.

Long, D., & Magerko, B. (2020). What is AI literacy? Competencies and design considerations. In Proceedings of the 2020 CHI conference on human factors in computing systems (pp. 1-16).

Mark, S., DeBay, D., Zhang, L., Haley, J., Patchen, A., Wong, C., & Barnett, M. (2013). Coupling social justice and out-of-school time learning to provide opportunities to motivate, engage, and interest under-represented populations in STEM fields. Career Planning and Adult Development, 29(2), 93-104.

Google Scholar

McGovern, A., Tidwell, Z., & Rushing, D. (2011). Teaching introductory artificial intelligence through java-based games. AAAI Symposium on Educational Advances in Artificial Intelligence, North America.

Narahara, T., & Kobayashi, Y. (2018). Personalizing homemade bots with plug & play AI for STEAM education. In SIGGRAPH Asia 2018 Technical Briefs (pp. 1-4).

Payne, B. H. (2020). Can my algorithm be my opinion?: An AI+ ethics curriculum for middle school students (Unpublished master's thesis). Massachusetts Institute of Technology, Cambridge.

Porter, L., Taylor, C., & Webb, K. C. (2014). Leveraging open source principles for flexible concept inventory development. In Proceedings of the 2014 conference on Innovation & technology in computer science education (pp. 243-248).

Selbst, A. D. (2017). Disparate impact in big data policing. Ga. l. Rev., 52, 109.

Google Scholar

Skirpan, M., Beard, N., Bhaduri, S., Fiesler, C., & Yeh, T. (2018). Ethics education in context: A case study of novel ethics activities for the CS classroom. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (pp. 940-945).

Strauss, A., & Corbin, J. (1994). Grounded Theory Methodology: An Overview. Handbook of Qualitative Research. 1st Ed., 17, 273-285.

Tai, R. H., Qi Liu, C., Maltese, A. V., & Fan, X. (2006). Planning early for careers in science. Science, 312(5777), 1143-1144.

Torrey, L. (2012). Teaching problem-solving in algorithms and AI. 3rd Symposium on Educational Advances in Artificial Intelligence.

Touretzky, D., Gardner-McCune, C., Martin, F., & Seehorn, D. (2019). Envisioning AI for K-12: What should every child know about AI?. In Proceedings of the AAAI conference on artificial intelligence (Vol. 33, No. 01, pp. 9795-9799).

Van Brakel, R. (2016). Pre-emptive big data surveillance and its (dis) empowering consequences: The case of predictive policing. In B. van der Sloot, D. Broeders & E. Schrijvers (Eds.), Exploring the boundaries of big data (pp. 117-141). Amsterdam University Press.

Wood, C. (1997). Yardsticks: Children in the Classroom Ages 4-14. A Resource for Parents and Teachers. Northeast Foundation for Children Publishing Division.

Zhou, X., Van Brummelen, J., & Lin, P. (2020). Designing AI learning experiences for K-12: emerging works, future opportunities and a design framework. arXiv preprint arXiv:2009.10228.

Download references

Acknowledgements

This work was funded by the National Science Foundation award DRL-2022502. We are grateful to the teachers, facilitators and students of Waltham Public Schools, S.T.E.A.M Ahead program and College Bound program. We thank Grace Kim, Jenna Hong, CC Song, Victor Sindato, Nathaniel Brown and Olivia Szendey for assisting us with data collection and analysis.

Author information

Authors and Affiliations

1. Lynch School of Education and Human Development, Boston College, Chestnut Hill, MA, USA

Helen Zhang & Yihong Cheng

2. MIT Scheller Teacher Education Program, Massachusetts Institute of Technology, Cambridge, MA, USA

Irene Lee

3. MIT Media Lab, Massachusetts Institute of Technology, Cambridge, MA, USA

Safinah Ali, Daniella DiPaola & Cynthia Breazeal

Corresponding author

Correspondence to Helen Zhang.

Ethics declarations

Ethics Approval

The work presented in this paper was approved by MIT's Internal review Board, Committee on the Use of Humans as Experimental Subjects (COUHES) under approval no. 1912000072 and Boston College's Internal Review Board, under approval no. 20.207.01e.

Informed Consent to Participate and Publish

All participants signed an assent form to participate in the study, which indicated that we will be collecting their in-classroom audio, video, chat and activity data. Since participants were minors, all participants' parents provided written consent prior to enrollment in the study. Students were told that they can withdraw from the study at any point, and they may choose to turn off their audio and video if they preferred.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Cite this article

Zhang, H., Lee, I., Ali, S. et al. Integrating Ethics and Career Futures with Technical Learning to Promote AI Literacy for Middle School Students: An Exploratory Study. Int J Artif Intell Educ 33, 290-324 (2023). https://doi.org/10.1007/s40593-022-00293-3
Download citation

Share this article

Anyone you share the following link with will be able to read this content:

Get shareable link

Provided by the Springer Nature SharedIt content-sharing initiative

Keywords

# data[].attributes.attachment_link.title
Integrating Ethics and Career Futures with Technical Learning to Promote AI Literacy for Middle School Students: An Exploratory Study - International Journal of Artificial Intelligence in Education

# data[].attributes.attachment_link.description
The rapid expansion of artificial intelligence (AI) necessitates promoting AI education at the K-12 level. However, educating young learners to become AI literate citizens poses several challenges. The components of AI literacy are ill-defined and it is unclear to what extent middle school students can engage in learning about AI as a sociotechnical system with socio-political implications.

# data[].attributes.body
Using AI in high school studies offers significant benefits to students.

artificial intelligence can formulate learning methods based on students' individual needs and improve students' grades. allows teachers to focus on cultivating students' critical thinking.

# data[].attributes.attachment_link.readable_text
Educational optimists predict that artificial intelligence, or "AI," will soon provide amazing efficiencies and progress in teaching and learning. But are efficiency and machine logic what our students need most?

It's certainly true that AI is a marvelous new tool dramatically transforming human life. AI is exponentially improving in speed and scope to recognize patterns in immensely complex data sets of all types, allowing it to make uncanny predictions about what might come next in a sequence, be it a purchase from a customer, a word in a sentence, a sound in spoken language, or countless other processes. AI can answer specific complex questions or perform intricate calculations at a rate impossible for the human mind to comprehend, let alone compete with. It can also generate images and speech which not only mimic reality but surpass it to meet programed standards of excellence.

However helpful these may be to adults seeking to improve productivity and processes, it is also prudent to follow G.K. Chesterton's advice, that children ought not be subjected to educational projects and ideas younger than they are. Allowing students to dodge traditional learning methods with AI will have uncertain and potentially harmful results. The same could be true of educators' dependence on AI for student assessment and lesson planning. In education at least, it is quite possible that AI will work against natural human development and provide not a shortcut to human formation but a short circuit.
Why Students Might Think of AI As a Shortcut
For students inclined to see homework as burdensome or useless, AI is now a tempting shortcut. AI can instantly answer most calculations and informational homework assignments including "show your work" complex math and science problems, and it can aid the writing of unique history or literature papers. AI is a transformative hack to game the homework system. Even so, some pedagogues celebrate this possible dismantling of conventional homework in the hopes that rote learning or linear thinking will no longer be assigned. Teachers will be forced to focus on developing assignments that are personalized and promote "critical thinking" and "authentic learning."
Why This Thinking Is Really a Short Circuit
While developing creative and effective homework assignments should be encouraged, educators cannot short-circuit the complete learning process by giving up on requiring students to engage in rote learning and writing assignments, even if some students cheat with AI. The hard work of instilling facts and information into the human brain and developing complex neural pathways step-by-step still needs to happen. It is not time-efficient to throw out homework altogether. Teachers should integrate some AI speedbumps into homework such as breaking the writing process into multiple submissions that each require peer or teacher feedback, creating more in-class writing assignments and oral presentations, etc., but they should not simply throw out most homework thereby restricting educational time. Students still need to learn prudent use of time and the development of academic skills and virtue which are part and parcel of traditional homework.

Homework in this sense might be viewed as something akin to practicing a musical instrument: A music teacher assigns practice at home, even if the student ignores or cheats in the practice sessions. There simply is no shortcut (technological or otherwise) to the human need to toil through the learning process, if the goal is freedom to play an instrument or earn a living as a musician. Few are interested in listening to a player-piano, but many will take time to listen to a fully formed musician who, after years of memorization and practice, is now free to creatively contribute their complex and personally unrepeatable humanity to the musical occasion. The world of AI requires us to focus even more on cultivating distinct human genius, compassion, insight, and creativity.

Because homework can be easily falsified, educators must be explicit in showing students and their parents how such deception and sloth short-circuits authentic learning and human development. It can lead to a person's real demise in a world where AI will be in constant and growing competition with their individual livelihood. Self-replacing personal development with AI may fool the teacher but may result in the student's inability to compete against AI with future employers. It is a student's complete human development which will be their competitive advantage against AI job replacement in the future.

The use of generative AI to produce writing, which one then presents fully or even partially as one's own, is particularly problematic. It is plagiarism and lying unless fully disclosed. It also weakens thinking, because writing is a type of explicit thinking. Writing assignments at their best require students to engage in multiple levels of critical thinking including synthesis, evaluation, and creativity. Students need extensive practice to develop these profoundly important human skills. We have students write essays on history and literature, not to generate new knowledge for the human species but to develop their own knowledge, understanding, and cognitive power. If they get initial essays and ideas from generative AI they will stymie their own generative capabilities.

Additionally, the AI homework short-circuit deprives a student of the value of their teacher. A teacher uses homework and, especially writing, as one tool among many to gain clear insight into a student's real thinking and processing to assist the student in properly interacting with, interpreting, and valuing what is before them. Any deception in this process does great harm on multiple levels. There is no short-circuiting of the educational process, especially the writing process, without harm.
Why Educators Might Think of AI As a Shortcut
Some educators can be bedazzled by AI's tremendous capability for analyzing student data. AI assisted by standardized test results can establish a student's reading or math level and decide what instruction, texts, or problem sets should come next. AI can also be programed to give students hints as opposed to the final answer in answering questions. It can even predict where errors in thinking may have occurred and offer tailored intervention and practice in that area. It can seem like the ultimate personalized teacher, who is not distracted by other students and has access to unlimited, perfectly tailored resources. As a bonus, teachers assisted by AI may have even more time to be a "guide on the side," with less time grading and lesson planning.
Why This Thinking Is Really a Short Circuit
This apparent "win-win" for the virtual-headset-wearing members of "Plato's Cave Academy" is really a short circuit. Teaching and learning are such fundamental and intimate human-to-human processes that farming out significant elements of human formation to computers is quite literally inhuman.

Humans are social animals who learn best socially, in person, and in relationships. This was made abundantly and tragically clear during the COVID school shutdowns. The academic, social, and psychological damage of isolating students on screens has lasting negative consequences. A child who is fortunate enough to have access to a healthy functioning classroom and school reaps great rewards in total human development. The fact that these conditions may be increasingly rare does not merit jettisoning the community-schooling paradigm for the cost effective, targeted, and quick efficiency of machine-generated teaching and learning.

Students typically find joy, bonding, and inspiration if surrounded by healthy and supportive classmates and a caring and wise teacher. Doing hard human stuff side-by-side with other humans is what students are wired for and can make the difficult delightful. Leaning conducted as human play and human exchange is also essential to motivation and learning, as titillating as computer-based learning games may be. Isolating students on screens where they work at their own pace deprives students of the benefit of watching peers attempting to do similar things and adjusting or confirming their own efforts in response. They witness approaches that they might not otherwise have imagined and bond with fellows in a common project of discovery or skill mastery. This is tremendously satisfying and makes school a joy!

Additionally, the teacher is tasked with instruction through modeling a human passion for engaging with truth, beauty, and goodness wherever it arises, manifesting a rich spirituality, and showing deep care and compassion for others. This has a profound impact on student learning, happiness, well-being, and a healthy disposition toward all that is being studied and fellow students. AI cannot look a student in the eye, smile, act with integrity, and model lived virtue. And AI cannot love the student. Nor can it teach the student to do these things. A real teacher, absent screens and mediators, can write on a child's soul and inspire them to greatness.

The teacher is also a model and source of real human learning by their encouragement and affirmation for the student, who at times struggles and at times makes spectacular breakthroughs-both of which are access points of human intimacy in the face of despair or joy and therefore demand real human response. Despondency and cynicism result when human experiences are isolated, unrecognized by other humans, or simply materialized in data sets. It is just not right, on a regular rather than occasional basis, to have a young child read to a computer to be analyzed and instructed rather than reading to and with another a human. Perhaps it may save time and provide individualized data, but this computerized "personalization" can actually depersonalize the instruction, as it removes the teacher from the process. And besides, what is time for, if not listening to children read?

And, of course, reading is much more than sounding at words and getting right responses about linear or discrete text-based questions, which are at the heart of computer-based instruction. Students are taught to read because they are human beings who love to share stories and insights with each other. They are also spiritual beings made to seek and know the truth, act upon it, and share it with others. Reading helps them transcend themselves, their time, their experience, and their culture. It helps them discover transcendent meaning of things and view the world from another's perspective.

This being said, there is room for providing drills and problem sets based on student interface with AI at home. The data from such AI-informed and tracked home-drills can be used by the instructor to guide in-person instruction. What needs to be protected is the authentic interaction which the school classroom provides.
The Dispiriting Nature of AI
As the excitement of AI dissipates and its numbing and dispiriting effects take hold of more and more people, jaded souls may fall into despair. The lack of human contact, lack of reality, inability to attain true beauty when left with AI-generated avatars and robots, inability to match AI-generated speed and perfection, and inability to compete with AI for employment will be devastating for many. Added to this, carefully crafted and monetized AI experiences such as immersive games and other entertainment specifically tailored by AI to personal desires and weaknesses will flood human senses and their lives, seeming to cover the pain and lack of true happiness, and worse yet, eliminating the sense that this unreality should cause them pain at all.

Wise parents will understand that radical recent changes in technology mean that now, more than ever, screen time and technology are direct threats to their children and their well-being. There is no benefit to introducing children to AI. The whole purpose of AI is to make interfacing with technology seamless, so there is no danger that children who do not use AI now will somehow not be able to use it productively in the future.

Educators and parents need to prevent children from becoming trapped in addictive, unreal worlds. The response to this threat should be dramatic and counter unreality with reality, at every opportunity. Children need to be at home in the real world. It is a world made for them, for their happiness, and for their work by a loving God. Educators must guide them in seeking and ascribing authentic meaning to those flawed but real experiences that make up the real world. Educators must reject anything which threatens their mind/body/spirit unities, including errant human sexual ideologies. Students need to be re-embodied and re-integrated with themselves and the natural/real world, and re-enchanted with the beauty and meaning present in all things, instilled by and delighted in by God, their creator, and in whose image they are made.
The Quest for the Authentically Human
AI is an adult tool which can unlock immense possibilities and creative resources. It can be of use to educators in their own research, development of materials, analysis of data, and administrative duties, but it does not belong in any direct, sustained way in the teacher-student relationship.

Now is the time to re-embrace the humanities in education. Those human actors with rich literary, historical, philosophical, theological, and psychological training will be best suited to succeed in a competitive AI-controlled market. In the end, most problems in the workforce will remain "people problems," and human creativity in all its uniqueness will be even more prized. Those individuals who are creative and who are best able to understand and help people and people-based challenges will always be needed and appreciated. A goal of educators at this point should be to maximize reality-based experiences for students and their unique human potentialities. It is these fully formed and trained memories, minds, imaginations, and wills which will act upon the new and unpredictable opportunities AI is beginning to create for them. Students need to organically perfect their humanity, as technology de-humanizes it. Un-mediated access to the greatest human accomplishments, presented and discussed by other humans who know and love them-even with their blemishes and misshapenness-is what students love and what will help them to love learning and indeed to love each other.
Daniel Guernsey, Ed.D., is senior fellow and education policy editor at The Cardinal Newman Society and associate professor of education and director of the M.Ed. in Catholic Educational Leadership at Ave Maria University.

# data[].attributes.attachment_link.title
There Is No AI Shortcut to Education - Cardinal Newman Society

# data[].attributes.attachment_link.description
Educational optimists predict that artificial intelligence, or "AI," will soon provide amazing efficiencies and progress in teaching and learning. But are efficiency and machine logic what our students need most? It's certainly true that AI is a marvelous new tool dramatically transforming human life. AI is exponentially improving in [...]

# data[].attributes.body
Despite the clear benefits, the use of AI in education also have several drawbacks

Relying too heavily on AI for simple calculations may weaken students' foundational skills

This will lead to some students mastering very advanced formulas but not being able to do some simple calculations

cause student mental problems

# data[].attributes.attachment_link.readable_text
Abstract

Advancements in machine learning have recently enabled the hyper-realistic synthesis of prose, images, audio and video data, in what is referred to as artificial intelligence (AI)-generated media. These techniques offer novel opportunities for creating interactions with digital portrayals of individuals that can inspire and intrigue us. AI-generated portrayals of characters can feature synthesized faces, bodies and voices of anyone, from a fictional character to a historical figure, or even a deceased family member. Although negative use cases of this technology have dominated the conversation so far, in this Perspective we highlight emerging positive use cases of AI-generated characters, specifically in supporting learning and well-being. We demonstrate an easy-to-use AI character generation pipeline to enable such outcomes and discuss ethical implications as well as the need for including traceability to help maintain trust in the generated media. As we look towards the future, we foresee generative media as a crucial part of the ever growing landscape of human-AI interaction.

Main

The idea of computers generating content has been around since the 1950s. Some of the earliest attempts were focused on replicating human creativity by having computers generate visual art and music1. Unlike today's synthesized media, computer-generated content from the early era was far from realistic and easily distinguishable from that created by humans. It has taken decades and major leaps in artificial intelligence (AI) for generated content to reach a high level of realism.

Generative and discriminative models are two different approaches to machines learning from data. Although discriminative models can identify a person in an image, generative models can produce a new image of a person that has never existed before. Recent leaps in generative models include generative adversarial networks (GANs)2. Since their introduction, models for AI-generated media, such as GANs, have enabled the hyper-realistic synthesis of digital content, including the generation of photorealistic images, cloning of voices, animation of faces and translation of images from one form to another3,4,5,6. The GAN architecture includes two neural networks, a generator and a discriminator. The generator is responsible for generating new content that resembles the input data, while the discriminator's job is to differentiate the generated or fake output from the real data. The two networks compete and try to outperform each other in a closed-feedback loop, resulting in a gradual increase of the realism of the generated output.

GAN architectures can generate images of things that have never existed before, such as human faces3,4. However, StyleGAN is an example of a modifiable GAN that enables intuitive control of the facial details of generated images by separating high-level attributes like the identity of a person from low-level features such as hair or freckles, with few visible artefacts4. Researchers have also proposed an in-domain GAN inversion approach to enable the editing of GAN-generated images, allowing for de-aging or the addition of new facial expressions to existing photographs7. Meanwhile, transformers such as the ones used in the massive generative GPT-3 language model are already being shown to be successful for text-to-image generation8.

In addition to the generation of new digital media, generative models like GANs can also manipulate existing media to enable applications such as video dubbing of foreign films9, animating old images of historical figures10 or, perhaps most notoriously, creating 'deepfake'3 videos of people like world leaders, politicians and celebrities in which they are portrayed as saying or doing things they never did. However, the main goal of this Perspective is not to discuss the potential misuses of the technology, as these have been covered extensively in the literature (for example, refs. 3,11,12). Instead, this Perspective aims to present beneficial applications of the technology, with a discussion of some of their societal implications and concerns ( Social implications section).

The technology has found use for several beneficial applications. Currently, generative models are being leveraged across different industries including entertainment, customer services and marketing. Examples include mobile device apps (for example, Reface13) that enable users to humorously swap their faces in video clips and GIFs to share with friends. Other startups allow users to create AI-generated photorealistic virtual assistants or to perform face replacement in videos or live streams14. Virtual characters like Li'l Miquela15 are already exceedingly popular, with millions of followers on Instagram. Beyond images and videos, a model generating realistic lip movements in videos based on voice recordings has recently been made public, allowing anyone to translate recorded lectures, announcements and movies into another language, without any noticeable discrepancy between speaker lip movement and voice overdubs16.

Given the recent advancements in generative AI, we suggest that AI-generated characters have the potential to achieve promising results in learning and healthcare, but more research is needed to determine valuable opportunities and understand their limitations. In this Perspective we present emerging use cases of AI-generated characters and discuss their potential in education and well-being (Fig. 1). Additionally, we demonstrate and make available an easy-to-use AI character generation pipeline ( https://github.com/mitmedialab/AI-generated-characters) based on publicly available code. We highlight some of the ethical implications of AI-generated characters, as well as the need for including traceability to help maintain trust in digital media. As we look towards the future, we foresee generative media as being part of an ever growing landscape of connections between humans and AI.

Users are shown interacting with an AI-generated character during a health check (left), learning from an AI-generated character resembling an expert in a lecture (middle) and becoming AI-generated characters that resemble historical characters for role playing (right).

AI-generated characters

In this Perspective, we define AI-generated characters as digital representations of a person created by machine learning algorithms that are made to look, sound and behave realistically without necessarily being malicious (like deepfakes). To contextualize and position AI-generated characters in the broader research field of humanoid computational characters, we construct a design space with three axes: realism, interactivity and embodiment (Fig. 2). AI-generated characters are virtual, exhibit varying degrees of interactivity, and can be more realistic than hand-crafted characters (like three-dimensional (3D) or CGI/VFX (computer-generated imagery/visual effects) characters). The increased realism is due to the fact that AI-generated characters can leverage a large dataset of human images and videos to create output that resembles a photorealistic input. This stands in contrast to manually crafting a CGI character, which relies heavily on the creator's expertise in manipulating individual 3D parameters such as eye shape or size to satisfy their subjective evaluation of realism. AI-generated characters are-given their virtual embodiment-distinct from physically embodied characters such as humanoid robots (Fig. 2). In terms of proposed applications, we identify two sub-types of AI-generated character: (1) a user becoming an AI-generated character with a face swap and (2) a user interacting with an AI-generated character, which is controlled by either another human or an intelligent system. We see both of these sub-types as having unique strengths that enable novel applications in a variety of settings, as exemplified in the following.

Within these axes, AI-generated characters can be compared with other domains, such as humanoid robots, CGI/VFX characters in films, and 3D-characters in games.

Both sub-types of AI character can be made accessible in multiple forms (video, 3D, audio) and through multiple devices, such as smartphones, laptops or augmented and virtual-reality headsets, which can provide new opportunities for engagement and interaction with the generated representations. The character portrayals can feature different degrees of fidelity and interactivity, from full bodies to voice- or text-only manifestations.

AI-generated characters for education

Students spend ~15,000 h learning in the first two decades of their lives. Thus, learning environments have an enormous impact on the individual and, in turn, on society22. AI characters can be used to create compelling learning material for all ages, from delivery of content in the classroom to engagement with content at locations like museums, historical monuments or even in nature. By using state-of-the-art algorithms, prominent historical, modern-day or fictional figures can be brought to life to engage learners with 'lived' experiences of scientists making their discoveries, historical figures narrating battles or painters discussing their inspiration and process.

AI-generated characters as virtual instructors

Such immersive opportunities support a variety of novel scenarios for lifelong learning, whether they involve learning about art from famous artists or learning to cook from the best chefs in the world. For example, in the 'Dalí Lives' exhibition10, a generated version of the famous artist talks to visitors and even takes selfies with them in the museum. This interactive installation reportedly helped people to more closely connect with Dalí and his artwork. It is easy to imagine other artists coming to 'life' through generated versions of themselves, showcasing the potential of AI-generated characters to motivate learning both in and outside the classroom.

Students embodying AI-generated characters

AI-generated characters as peers

AI-generated characters to support health and well-being

Personal well-being is defined in the literature as a happy, satisfactory and desirable state of being that includes physical, psychological, emotional, mental, spiritual, social and subjective well-being38. We believe that AI-generated characters, in the future, will support several aspects of well-being by augmenting the assistance provided by coaches, therapists or counsellors, especially in resource-constrained environments.

Digital health counselling

Conversations with agents such as Woebot have been shown to resemble human conversations41 and result in emotional bonds with interactees. Examples of conversational agents include (but are not limited to) text-based chatbots, which can engage in casual conversations; voice-based conversational interfaces like Apple Siri, Google Assistant or Amazon Alexa, which respond to questions and instructions; and embodied conversational agents, which involve a virtual character (for example, avatar, virtual agent or photorealistic character) simulating face-to-face conversation with verbal and nonverbal behaviours42. The potential of text-based conversational agents for supporting mental health and well-being has already been shown to be significant in previous work: randomized control trials of the use of conversational agents (chatbots and/or embodied agents) in psychiatry and mental health counselling have demonstrated significant reductions in psychological distress compared with inactive control groups43. As another form of social support, the potential benefits of conversational agents in mitigating loneliness-a growing public health issue-have been suggested by researchers44. We can envision embodied chatbots or conversational AI agents that may be able to provide support through voice-based conversations and facial expressions as a further evolution of the text-messaging-based conversations of today's chatbots. By combining conversational agents with AI-generated characters, there is potential to enable deeper personalization and increased trust. Earlier research has shown that characteristics of doctors and therapists, such as their gender, can lead to significant differences in treatment outcomes. Indeed, one study found that patient satisfaction with the therapeutic relationship was higher when people were paired with same-gender therapists45, while another study found personal characteristics to be strongly associated with trust in physicians46. Similarly, interacting with computer-generated characters has also been found to establish better levels of trust and support open communication. In a study on people with eating disorders, a cartoon-like avatar of the therapist was shown to facilitate greater openness from participants47.

Empirical evidence has shown the potential of conversational agents to induce positive behavioural and cognitive changes48. By combining conversational agents with AI-generated characters, personalized fitness apps could enable users to interact with and be guided by a personalized AI-generated coach or exercise buddy that resembles someone they find motivating or look up to as a role model20.

Exposure therapy

Based on these results, we believe that AI-generated characters could be useful in exposure therapy. Exposing someone to AI-generated versions of themselves, for example, younger versions of themselves, can have a positive impact on mental health and has already been shown to improve alertness, active participation and a general sense of well-being52.

Living memories

Every human culture has developed practices and rituals associated with responding to death and signifying its importance for the living53. The digital world is increasingly intersecting with these practices. For example, digital platforms like Facebook already enable people to revisit deceased friends' and families' profiles as 'digital graves'. In the last year or so, Zoom has also been streaming memorial services and funerals for family members unable to attend in person due to COVID-1954. Research in human-computer interaction has traditionally explored the full range of the human lifespan, with a new call in the past decade for technologies to support practices such as 'collaborative acts of remembrance, bequeathing of digital data, or group reflection on the digital residual of a life'53,55. We believe that the concept of living memories through AI-generated characters could provide a new form of 'technological heirloom' and a new way for reflection and meaning-making53, adding to existing objects of memorabilia of the deceased, such as photographs, videos and other objects.

A recent example includes an AI developer creating a chatbot of her deceased friend56. After sharing the chatbot with other friends of the deceased, she claimed that, for many people, interacting with the bot had a therapeutic effect and that it deeply impacted the lives of people who knew him. His mother told her, 'I am getting to know him more. This gives the illusion that he is here now.' Another example is The Deep Nostalgia service, offered by a genealogy company MyHeritage, which deploys an AI-driven video reenactment technology to animate the faces of departed family members and ancestors in still photos and creates high-quality, realistic moving videos ( https://www.myheritage.com/deep-nostalgia). The service Eterni.me has a similar goal of creating and preserving memories through interactive conversations with the departed, and has garnered over 30,000 people waiting to be digitally immortalized for friends and family-including patients with terminal diseases such as cancer and Alzheimers57. Photogrammetry was used in another recent recreation of a deceased child58 to allow the parents to remember their son and cope with their traumatic loss. Technology now enables the preservation of a large variety of data, from text and photographs to videos and voice data. Although it might seem controversial to 'digitally preserve' individuals beyond death as was explored in fictional works such as Black Mirror 59, 'these AI systems may support the grieving process more effectively and speed up the stages of the process which include denial, anger, bargaining, depression and acceptance'60.

Assistive technologies

Generative media can enable people with a facial deformity or speech inability resulting from injury or a disorder such as amyotrophic lateral sclerosis (ALS) to restore their verbal or facial expressiveness in digitally mediated interactions. Researchers have recently developed non-invasive, real-time silent speech systems to assist patients with speech disorders such as multiple sclerosis and ALS to communicate in natural language merely by articulating words or sentences in the mouth without producing any sounds61. Although this work enables patients to regain close to natural sounding speech through a standard voice synthesizer, it does so by using a non-personal generic voice. AI-generated face and voice replication could enable them to not only regain speech but also their own voice. This idea was realized by an engineer diagnosed with motor neurone disease. In anticipation of muscle deterioration caused by the disease, he developed a life-like avatar with his facial features and voice, trained on data recorded prior to the progression of his illness62, that he now uses for digital communication.

AI-generated characters from text, audio and video

We present a unified pipeline for enabling educators and individuals to easily generate talking characters that can be useful for educational, health and other applications. The pipeline allows for the easy creation of realistic content based on real and fictional humanoid characters with facial gestures, voice and motion. It combines state-of-the-art generative AI models that convert text to audio, audio to video, and video to video, and can be used to create a variety of audio and video outputs based on the available data.

The following models are used: VOCA63 (Voice Operated Character Animation), FLAME64 (Faces Learned with an Articulated Model and Expressions), Speech-Driven Facial Animation65 and First Order Motion Model66. After the generation process, our unified pipeline marks the generated results with a traceable watermark to help people identify that the data are generated, learn about how the output was generated, and distinguish it from authentic video content. Our pipeline differs from existing consumer apps in allowing any type of input medium for generating videos and real-time facial filters while remaining open-source. This lends educators, therapists, health officials and others a higher degree of autonomy in the usage and creation of AI-generated characters.

Video-to-Video

To generate the video output, the user provides a template video and an image of the target character's face. The algorithm maps features from the video template to the target character to create a realistic video. To map the features, our pipeline uses the state-of-the-art First Order Motion Model generator network66 ( https://github.com/alievk/avatarify-python), which utilizes a self-supervised learning algorithm to determine the local transformation of facial animation to generate realistic complex motion in the output ( https://github.com/alievk/avatarify-python). To generate a facial filter for real-time use, the user's webcam stream is used as input (Fig. 4 and top of Fig. 3).

The output is watermarked for traceability ( Traceability section) for detection both by humans and algorithms. Image of face at bottom centre reproduced with permission from ref. 65, Springer Nature. Image of Albert Einstein taken from Pixabay. The following references are cited in the figure: refs. 63,64,65,66,100.

Voice-to-Video

Voice is a critical element of generated output given that watching a generated video without audio is less likely to be perceived as realistic. To add audio to the generated video, the user provides a recorded voice clip to drive the generated character's speech. As shown in Fig. 3, we explore two pathways for converting the voice clip input into intermediary facial animation before generating the final video and audio output.

Of the two approaches, the first approach creates a 3D intermediary talking head that can automatically lip-sync with the given voice clip. To do this we first use VOCA, a deep neural network trained on 4D scans of human faces and synchronized speech from speakers, to automatically generate lip-sync animation on a neutral 3D face63. Next, we use FLAME, a generalized 3D facial model, to animate the neutral face generated in the previous step64. The FLAME model allows the creator to manually customize the expressions and movements of the generated 3D face, while VOCA automatically synchronizes the mouth with the given voice clip.

The second approach utilizes an end-to-end speech-driven facial animation system based on a temporal GAN65. The temporal GAN relies on a generator and three discriminators (frame, sequence and synchronization discriminators) that drive the generation of an auto-lip-sync talking head using only a still 2D image of a person and a voice clip as input. The advantage of the second approach is its capability in synthesizing natural facial expressions such as blinks and eyebrow movements without manual customization.

Text-to-Video

Instead of using a recorded voice as input, text can be used for the generation of a voice clip and further use of our pipeline. If the user does not want to use their own voice or does not have the character's pre-recorded audio, they can simply enter a sentence, phrase or paragraph to drive the generated character's speech. For example, a generated Einstein could speak in Einstein's own voice or Elsa's voice or the teacher's voice who is puppeteering Einstein's face. To synthesize the generated character's speech, we use a state-of-the-art publicly available real-time voice cloning algorithm5,67 that enables using a few seconds of reference audio to synthesize new speech in that speaker's voice, using the text data. After synthesis, the audio is then used for the intermediary facial animation synthesis step described in the Voice-to-Video section.

Traceability

Our implementation of traceability contrasts with typically used methods such as deep-learning-based detectors that act as post factum detectors after the content has already been created3. Because researchers worry that detection algorithms will always remain a step behind3 given the rapid pace of technological advancements, our approach differs from typical detection methods by ensuring traceability through marking the content during the generation process.

Pipeline in action

To demonstrate the use of our pipeline, we experimented with generative characters in online classes at Massachusetts Institute of Technology (MIT). First, we collaborated with an MIT professor for his musical interfaces class by creating a real-time AI-synthesized version (sub-type 1) of Johann Sebastian Bach (Fig. 4b, bottom row), who had a live conversation with the renowned cellist and guest speaker, Yo-Yo Ma. Seeing their musical hero brought to life enthralled all class attendees, and this was followed by a lively discussion of Bach's work. We then created a video of Einstein from his audio recording of the essay titled The Common Language of Science (Fig. 3) and showed the video in another class at MIT to demonstrate how easy it is to convert information that is available only in audio or written format into a video format that engages more senses and thus may be more engaging for learners.

a, One of the authors of this Perspective (bottom row) becoming an AI-generated George Washington (top row). b, One of the authors of this Perspective embodying Johann Sebastian Bach (bottom row) interacting with Yo-Yo Ma (top row) in the online class 'ARTS@ML' in autumn 2020. Panel a (top), George Washington portrait by Gilbert Stuart, image courtesy of the Clark Art Institute ( clarkart.edu). Panel b (top), image used with permission of Yo-Yo Ma; (bottom) image of Johann Sebastian Bach taken from Wikipedia.

Social implications

With the rapid growth of AI technologies, concerns around privacy and surveillance, manipulation of behaviour, innate biases, as well as responsibility and the delegation of decision making, have all been raised72.

Ethical considerations

Discussion regarding the ethics of AI-generated media is an ongoing, expansive conversation happening across different scales, from personal usage to national policies12,75,76,77. The goal of this discussion is to present potential ways in which the technology can be misused and to promote considerations on which to reflect when designing AI-generated characters. In this section, we acknowledge the complexity of the potential for misuse and highlight some concerns as cautionary items that we encourage people to consider when using AI-generated characters in health and education contexts. We further acknowledge that the concerns presented here might not be an exhaustive list of all potential misuses of AI-generated characters.

Misportrayal of a character or event

Promoting harmful behaviour

Beyond the misportrayal of characters, another potential misuse of AI-generated characters could be having them support and endorse harmful behaviour ( https://www.pnas.org/content/118/47/e2114388118). For example, prominent official health figures could be made to speak against COVID-19 vaccinations. In one case study, researchers followed a Twitter account imitating a Disney princess endorsing self-harm to exemplify the negative effects of fake representations of beloved characters79. Among other things, the account shared and praised pictures of the bodies of people with eating disorders and self-injuries like scars from cutting themselves. With much content on video platforms like YouTube being consumed by children80, recognizable characters promoting harmful behaviour could have severe consequences, such as the development of eating disorders and suicidal ideation. Furthermore, AI-generated characters can unintentionally cause harm. Imagine, for instance, a classroom using synthetic characters to allow students to debate each other as presidential candidates for an upcoming election. If these student depictions somehow find their way into the public domain, they might be perceived as true or real by some individuals.

Causing dependence from overuse

Supplanting human relationships

Although we have presented applications of AI-generated characters for teaching, coaching and therapy, we would like to stress that they should be used for augmenting current practices rather than replacing them. For example, although it might be economically motivating to replace teachers with AI systems, research has shown student-teacher relationships to be a key factor in creating an environment for positive student development and learning23,34. Substituting teachers with AI-generated characters might thus jeopardize the ability of the students to form genuine relationships with their teachers and mentors, which in turn may impact learning motivation and cause disruptive behaviour84. Moreover, current video conferencing systems for online learning have been shown to lack emotional attachment and lead to ineffective learning experiences85. Although human resources like teachers and therapists are not always available in low-resource communities, we do, based on these findings, consider it important that AI-generated characters in their current state be used mainly as a supplementary tool rather than a replacement.

Post-mortem AI-generated characters

Legal considerations

In this section we discuss the legal considerations of AI-generated characters, which involve the rights and consequences for a number of individuals: the target individual whose identity is used (and who may be alive or deceased), the individual(s) whose data are used in training the AI models, the individual(s) building and training the model for AI character generation, the people using the model to create characters and the people who will view or interact with the AI-generated characters made by someone else.

Violation of an individual's rights of privacy and publicity

Actor Kirsten Bell, a victim of deepfake pornography, has said 'We're having this gigantic conversation about consent, and I don't consent, so that's why it's not okay... even if it's labelled as, "This is not actually her", it's hard to think about that'86. Using AI-generated media to put someone's face in a pornography video or images is a sexual-privacy invasion, which can have profound effects. Victims report experiencing deep psychological distress, including anxiety, depression, loss of appetite and suicidal ideation87. In a recent study that assesses public attitudes towards deepfakes, subjects viewed non-consensually created pornographic deepfake videos as extremely harmful and overwhelmingly wanted to criminalize them. Labeling pornographic deepfakes as fictional did not mitigate the videos' perceived wrongfulness. By contrast, non-pornographic deepfakes were viewed as substantially less wrongful when they were labeled as fictional or did not depict inherently defamatory conduct like illegal drug use88. Furthermore, labelling the generated pornographic videos as fiction does not meaningfully remove this harm, as the target's identity is still being appropriated88. We clearly see that using AI-generated media without consent is unethical for negative applications. However, there is still ongoing debate about the necessity of consent when it comes to positive use cases, such as the use of David Beckham for a malaria information campaign89 or the use of a historical character such as Dalí to interact with museum-goers10. Does consent change if an individual passes away? Does their legal identity evaporate or does it lie with their estate? Who provides consent for an historical figure? As these considerations may vary across context and applications, they will most likely continue to evolve as AI-generated characters enter mainstream modes of communication.

Liability

If an AI-generated character misportrays or causes harm, who is to blame? Is it the person interacting with the AI-generated character? Is it the programmer who build the AI? Or is it the person who is portrayed? Determining the liability of AI-generated characters is an important topic, as teachers, health workers, agencies and others need to spread awareness of potential liability for claims related to AI-generated content. If an AI-generated character in some way violates someone's privacy, defames them or spreads misinformation, it is essential to determine who should be held responsible. Although some current laws apply, other laws and regulations are unclear and inadequate to deal with such situations90.

In a recently published article on the ownership of AI-generated content, it is argued that, to reach guiding principles of liability, we must distinguish the contributions of different individuals in the creation process with regards to 'creativity, labour or intellectual effort'90. For example, in cases where control over the AI-generated content is strictly bounded by the programmer who created the algorithm, the programmer is-more often than the user-considered liable for the content (with variations existing between jurisdictions)90. Conversely, in cases where a great amount of freedom in content generation is given, users might be considered more liable than programmers.

AI-generated characters add an extra layer to the challenge of liability for generated media, as they are often very realistic (in contrast to artistic), unlabelled and illusory by nature, making it hard to even determine if the character has been programmatically created at all. In this Perspective we have argued that the labelling of AI-generated media could enable some traceability regarding who is involved in the generation process, to what extent and what sources were used to generate the output. Other researchers have made similar arguments, stating that such labelling can greatly help mitigate liability issues by clearly marking something as AI-generated91.

But can liability be resolved that easily? Numerous psychological studies of people's susceptibility to misinformation have, for example, shown that continuous exposure to clearly labelled misinformation can still have adverse effects on an individual's beliefs92. Similarly, using AI-generated media in learning and health, one should be encouraged to go beyond traceability and also think about the social and individual impact a particular piece of generated content might have.

Conclusion

In this Perspective, we have discussed beneficial applications of AI-generated characters in education and well-being, an easy-to-use pipeline for integrating them into these fields, as well as a discussion on the societal implications of using AI-generated characters. We foresee a future where generative AI will become ubiquitous in our daily lives, as part of the ever growing human-AI interaction landscape, but this is not without challenges93. Beyond the education and well-being domains, these technologies are likely to be used for a positive impact in many other areas, including entertainment, creativity and security. Although current algorithms are far from perfect, they open up new creative opportunities for artists and filmmakers without requiring Hollywood-sized budgets and timelines. Although CGI is commonly used in Hollywood to bring deceased actors back to life or to de-age stars, fans have been experimenting with replacing the CGI characters in movies (for example, Star Wars) with convincing AI-generated versions. With these fan versions gaining popularity, and Disney Studios themselves exploring neural face swapping94 technologies, big movie productions may shift from manual and time-intensive CGI to algorithmic film making. As AI-generated characters become more familiar to the public, we also anticipate them to integrate with innovations in robotics95, synthetic biology96 and manufacturing97 to support the presence of generative AI in the physical world. Companies are already working towards that vision by making ubiquitous characters that are deeply integrated with our lives98,99. However, with these opportunities come potential misuses. Challenges around AI-generated synthetic media in general involve multidimensional issues that touch on the notions of privacy, freedom of speech and the fundamental idea of human identity. AI-generated characters present the potential for misportraying people and events, motivating harmful behaviour, causing dependence from overuse, replacing social connections and promoting the digital existence of the deceased. Integrating this technology into social practices requires careful and critical consideration of issues such as privacy, liability and traceability. This will require cooperation between governments, industries, researchers and academic institutions to develop new regulatory mechanisms and safeguards that can address the technical, ethical and legal challenges. Our inevitable future with AI-generated characters will require us to rethink the fundamentals of human identity, its formation, its safeguarding and its role in society.

Author information

Authors and Affiliations

1. Massachusetts Institute of Technology, Cambridge, MA, USA

Pat Pataranutaporn, Valdemar Danry, Joanne Leong, Dan Novy & Pattie Maes

2. Osaka University, Toyonaka, Japan

Parinya Punpongsanon

3. University of California, Santa Barbara, CA, USA

Misha Sra

Contributions

P. Pataranutaporn developed the pipeline, assisted by V.D. The literature review was conducted by P. Pataranutaporn, V.D., J.L., P. Punpongsanon and M.S., who also contributed to the writing and editing of the manuscript. All other authors reviewed the manuscript. P. Pataranutaporn designed the figures. The pipeline was tested by D.N. The work was supervised by P.M. and M.S.

Corresponding author

Correspondence to Misha Sra.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Cite this article

Pataranutaporn, P., Danry, V., Leong, J. et al. AI-generated characters for supporting personalized learning and well-being. Nat Mach Intell 3, 1013-1022 (2021). https://doi.org/10.1038/s42256-021-00417-9
Download citation

Received:

Accepted:

Published:

Issue Date:

DOI: https://doi.org/10.1038/s42256-021-00417-9

# data[].attributes.attachment_link.title
AI-generated characters for supporting personalized learning and well-being - Nature Machine Intelligence

# data[].attributes.attachment_link.description
Digitally recreating the likeness of a person used to be a costly and complex process. Through the use of generative models, AI-generated characters can now be made with relative ease. Pataranutaporn et al. discuss in this Perspective how this technology can be used for positive applications in education and well-being.

# data[].attributes.attachment_link.readable_text
Driving the news: AT&T's cellular network and Google's Gemini chatbot both went on the fritz last week.

In AT&T's breakdown, a "software configuration error" left thousands of customers without wireless service during their morning commute.

It was an annoying but familiar case of the kind of network outage that both the public and the tech industry learned to deal with long ago.

Google's bug was very different. Its Gemini image generator created a variety of ahistorical images: When asked to depict Nazi soldiers, it included illustrations of Black people in uniform; when asked to draw a pope, it produced an image of a woman in papal robes.

This was a more complex sort of error than AT&T's, at the boundary between engineering and politics, where it looked like a diversity policy had gone haywire. Google paused all AI generation of images of people until it could fix the problem.

Google CEO Sundar Pichai Tuesday sent a memo to staff saying of Gemini, "I know that some of its responses have offended our users and shown bias - to be clear, that's completely unacceptable and we got it wrong."

Conservatives shouted, " Woke!"But Google was taking precautions against bias much like those every AI company has learned to apply after years of embarrassing missteps - particularly in the realm of human images, where all AI terrorists seemed to have dark skins and every AI doctor was a white male.

Those problems have dogged the field because the data used to train generative AI models, typically culled from mountains of web pages, is full of the biases of the humans that created it.

AI firms could have reduced these problems by spending a lot of time and money up front to curate their models' training data. Instead, they are now applying spotty Band-Aid patches to the problems after the fact.

The big picture: In both the AT&T and the Google incidents, systems failed because of what people asked computers to do.

AT&T's wireless service crashed when it tried to follow new instructions containing an error or contradiction that caused the system to stop responding.

That's how most computer crashes have happened since computers were invented.

Yes, but: Most AI systems don't operate by commands and instructions - they use "weights" (probabilities) to shape output.

Developers can put their fingers on the scales - and clearly they did with Gemini. They just didn't get the results they sought.

Zoom in: Gemini isn't able to "think" clearly about the difference between historical research and "what if?" creativity.

A smarter system - or any reasonably well-informed human - would understand that the Roman Catholic Church has never had a pope who wasn't male. (There are lots of tales - all believed to be legend rather than fact - of a medieval "Pope Joan," and they likely pop up in the AI's training data, as they do in Wikipedia.)

Zoom out: Making things up is at the heart of what generative AI does.

Traditional software looks at its code base for the next instruction to execute; generative AI programs "guess" the next word or pixel in a sequence based on the guidelines people give them.

You can tune a model's "temperature" up or down, making its output more or less random. But you can't just stop it from being creative, or it won't do anything at all.

Between the lines: The choices AI models make are often opaque and even their creators don't fully understand how they work.

So when developers try to add "guardrails" to, say, diversify image results or limit political propaganda and hate speech, their interventions have unpredictable results and can backfire.

But not intervening will lead to biased and troublesome results, too.

What's next: The more we try to use generative AI systems to accurately represent reality and serve as knowledge tools, the more urgently the industry will try to place boundaries around their "hallucinations" and errors.

Generative AI is, as one expert put it soon after ChatGPT's debut, the greatest BS machine ever.

Over time, as the field learns from experience, AI programmers could succeed at taming and tuning their models to be more factual and less biased.

But it could turn out that using generative AI as the fundamental interface for knowledge work is just a very bad idea that will continue to bedevil us.

Editor's note: This story has been updated with Sundar Pichai's comments.

# data[].attributes.attachment_link.title
AI is making software bugs weirder and harder to fix

# data[].attributes.attachment_link.description
As Google's "Black Nazi" breakdown showed, AI's failures are tricky to predict - and potentially explosive.

# data[].attributes.attachment_link.readable_text
Gina Dimitropoulos 1,2,

Emma Cullen 2,

Olivia Cullen 1,

Chris Pawluk 5,

Alan McLuckie 1,2,

Scott Patten 2,3,

Andrew Bulloch 2,3,

Gabrielle Wilcox 4 na1 &

...

Paul D. Arnold 2,3 na1

Show authors
School Mental Health 14, 402-415 (2022) Cite this article

1634 Accesses

4 Citations

288 Altmetric

Metrics

Abstract

To date, minimal research has explored the perceptions of secondary school staff regarding their realized and potential contributions to the redress of mental health stigma and support of students with mental health concerns within school environments. The aim of this study was to identify and describe the practicable roles school staff perceive of themselves to hold with respect to promoting mental health, challenging stigma, and responding to student mental health problems. Using an interpretive qualitative approach, semi-structured interviews were conducted following purposive and convenience sampling of school staff ( n = 48) including classroom teachers, school counsellors, psychologists, administrators, and various support staff members employed with two secondary schools in southern Alberta, Canada. A structured interview guide was utilized to assess the roles, tasks, and processes school staff undertake to assist students with their mental health concerns. Four main themes inductively emerged from the data: (1) Relationships matter: Establishing and maintaining strong relationships with students, their caregivers, and other staff members are critical to addressing student mental health issues; (2) Empathetic and receptive communication is an antidote to stigma: Various communication processes contribute to disclosures of mental health concerns and challenges; (3) Connecting and facilitating timely access to the right person is key for students experiencing emotional crises; and (4) Facilitators and barriers to addressing student mental health concerns. Clinical implications and policy recommendations are provided to inform directions for administrators, educators, and caregivers regarding student mental health supports.

Levels of Evidence: Qualitative study.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

Atkins, M., Shernoff, E., Frazier, S., Schoenwald, S., Cappella, E., Maríñez-Lora, A., Mehta, T., Lakind, D., Cua, G., Bhaumik, R., & Bhaumik, D. (2015). Redesigning community mental health services for urban children: Supporting schooling to promote mental health. Journal of Consulting and Clinical Psychology, 83(5), 839-852.

Beames, J. R., Johnston, L., O'Dea, B., Torok, M., Boydell, K., Christensen, H., & Werner-Seidler, A. (2020). Addressing the mental health of school students: Perspectives of secondary school teachers and counselors. International Journal of School and Educational Psychology. https://doi.org/10.1080/21683603.2020.1838367

Article Google Scholar

Briner, R., & Dewberry, C. (2007). Staff wellbeing is key to school success: A research study into the links between staff wellbeing and school performance. Worklife Support.

Carr, W., Kutcher, S., Wei, Y., & Heffernan, A. (2017). Preparing for the classroom: Mental health knowledge improvement, stigma reduction, and enhanced help-seeking efficacy in Canadian preservice teacher education. Canadian Journal of School Psychology. https://doi.org/10.1177/0829573516688596

Child and Youth Mental Health: Signs and Symptoms. (n.d) Canadian Mental Health Association. https://ontario.cmha.ca/documents/child-and-youth-mental-health-signs-and-symptoms/

Clark, C., & Teravainen-Goff, A. (2018). Mental wellbeing, reading and writing: How children and young people's mental wellbeing is related to their reading and writing experiences. National Literacy Trust Research Report.

Froese-Germain, B., & Riel, R. (2012). Understanding teachers' perspectives on student mental health (pp. 1-22, Rep.). Canadian Teachers' Federation. (ERIC Document Reproduction Service No. ED544259).

Kutcher, S., Venn, D., & Szumilas, M. (2010). Mental health: The next frontier of health education. Education Canada, 49(2), 44-45.

Google Scholar

Leggio, J., & Terras, K. (2019). An investigation of the qualities, knowledge, and skills of effective teachers for students with emotional/behavioural disorders: The teachers' perspective. The Journal of Special Education Apprenticeship, 8(10), 1-15.

Google Scholar

Manvell, E. C. (2012). The violence continuum: Creating a safe school climate. London: R&L Education.

Google Scholar

Reschly, A. L., & Christenson, S. L. (2012). Jingle jangle and conceptual haziness: Evolution and future directions of academic engagement. In S. L. Christenson, A. L. Reschly, & C. Wylie (Eds.), Handbook of research on student engagement (pp. 3-20). Springer.

Chapter Google Scholar

Rowling, L., Whitman, V., Biewener, C., & Biewener, M. (2009). International survey of principals concerning emotional and mental health and well-being. International Association of Child and Adolescent Mental Health and Schools (Intercamhs)/International Confederation of Principals (ICP). http://www.hhd.org/sites/hhd.org/files/mental-health-booklet.pdf

Srivastava, A., & Thomson, S. B. (2009). Framework Analysis: A qualitative methodology for applied policy research. Journal of Administration and Governance, 4(2), 72-79.

Google Scholar

US Public Health Service. (2000). Report of the Surgeon General's conference on mental health: A national action agenda. Department of Health and Human Services.

Waddell, C., McEwan, K., Shepherd, C. A., Offord, D. R., & Hua, J. M. (2005). A public health strategy to improve the mental health of Canadian children. The Canadian Journal of Psychiatry, 50(4), 226-233.

Download references

Funding

This study is funded by Brain and Mental Health Strategic Research Fund, University of Calgary, and the Alberta Innovates Translational Health Chair in Child and Youth Mental Health.

Author information

Author notes

Gabrielle Wilcox and Paul D. Arnold are co-senior authors.

Authors and Affiliations

1. Faculty of Social Work, University of Calgary, Professional Faculties, MLT 301, 2500 University Dr NW, Calgary, AB, T2N 1N4, Canada

Gina Dimitropoulos, Olivia Cullen & Alan McLuckie

2. The Mathison Centre for Mental Health Research and Education, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada

Gina Dimitropoulos, Emma Cullen, Alan McLuckie, Scott Patten, Andrew Bulloch & Paul D. Arnold

3. Department of Psychiatry, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada

Scott Patten, Andrew Bulloch & Paul D. Arnold

4. Werklund School of Education, University of Calgary, Calgary, AB, Canada

Gabrielle Wilcox

5. Rocky View Schools, Rocky View, AB, Canada

Chris Pawluk

Corresponding author

Correspondence to Gina Dimitropoulos.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests.

Availability of data and material

Data are available from the corresponding author on reasonable request.

Ethics approval

Approval for this study was obtained from the University of Calgary's Conjoint Health Research Ethics Board (REB 16-1352).

Consent to participate

Written informed consent was obtained from all participants prior to participation.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

About this article

Cite this article

Dimitropoulos, G., Cullen, E., Cullen, O. et al. "Teachers Often See the Red Flags First": Perceptions of School Staff Regarding Their Roles in Supporting Students with Mental Health Concerns. School Mental Health 14, 402-415 (2022). https://doi.org/10.1007/s12310-021-09475-1
Download citation

Accepted:

Published:

Issue Date:

DOI: https://doi.org/10.1007/s12310-021-09475-1

Share this article

Anyone you share the following link with will be able to read this content:

Get shareable link

Provided by the Springer Nature SharedIt content-sharing initiative

Keywords

Educator roles

Qualitative research

Student mental health

Stigma

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

# data[].attributes.attachment_link.title
"Teachers Often See the Red Flags First": Perceptions of School Staff Regarding Their Roles in Supporting Students with Mental Health Concerns - School Mental Health

# data[].attributes.attachment_link.description
To date, minimal research has explored the perceptions of secondary school staff regarding their realized and potential contributions to the redress of mental health stigma and support of students with mental health concerns within school environments.