# data[].attributes.attachment_link.readable_text
Equal Opportunity Schools is thrilled to announce its strategic collaboration with Intentional Futures, a renowned design and innovation consultancy, to launch the groundbreaking Mission-Driven AI Use Case Rubric.

The Mission-Driven AI Use Case Rubric is a comprehensive framework designed specifically for nonprofits, educational institutions and edtech companies seeking to harness AI's potential to drive long-lasting impact. It is a user-friendly tool with which organizations can align different AI use cases with their unique ethical, social, and mission-oriented goals, using a set of predefined criteria.

As AI technologies continue to play an increasingly prominent role in shaping educational experiences, it is essential to ensure that these tools adhere to principles of equity, inclusivity, and positive social impact. This rubric equips organizations with the ability to integrate and scale AI into their work while holding these considerations at the core.

Equal Opportunity Schools recognizes the importance of leveraging AI in education without perpetuating existing disparities, as we have long sought to incorporate emerging technologies in our work to close race and income enrollment gaps in advanced pathways like Advanced Placement, International Baccalaureate and dual enrollment programs.

"Student potential is a multi-faceted, dynamic and complex concept," EOS President Sasha Rabkin says about the pivotal partnership and tool. "Our project with Intentional Futures (iF) demonstrates what is possible for both our organization and the field. iF has helped us clarify our value proposition, dream big and develop a set of concrete solutions, rubrics and prototypes that will transform our use of AI and allow us to scale more rapidly and with greater dexterity."

The education team at Intentional Futures, renowned for their expertise in designing human-centered solutions and guiding organizations towards responsible innovation, shared this about their team's critical role in developing the rubric.

"As generative AI continues to develop and be implemented in education, mission-driven organizations and schools must intentionally consider if these technologies align with their values and meaningfully serve students, teachers, and staff. The rubric we created with EOS gives education-serving organizations a tangible tool that allows them to pause and critically reflect on the impact of potential use cases of AI."

Key features of the Mission-Driven AI Use Case Rubric include:

Ethical Considerations: Assessing the impact of AI use cases on student privacy, data security, and potential biases

Inclusivity: Evaluating how AI applications support diverse student populations and contribute to closing opportunity gaps

Mission Alignment: Ensuring that AI technologies align with the core mission and values of educational institutions

Transparency: Assessing the clarity and openness of AI systems in terms of decision-making processes and outcomes

EOS is proud to help lead the way for incorporating emerging technologies like AI into equity-building work in schools, and we are grateful for Intentional Futures' crucial role in helping us do this.
Explore the rubric below:

# data[].attributes.attachment_link.title
EOS and Intentional Futures Launch Mission-Driven AI Use Case Rubric -

# data[].attributes.attachment_link.description
Equal Opportunity Schools is thrilled to announce its strategic collaboration with Intentional Futures, a renowned design and innovation consultancy, to launch the groundbreaking Mission-Driven AI Use Case Rubric. The Mission-Driven [...]

# data[].attributes.attachment_link.readable_text
12 comments | 47 shares

Estimated reading time: 20 minutes

Maha Bali

February 26th, 2024

12 comments | 47 shares

Estimated reading time: 20 minutes

Batting away the hype, bias, and botshit, LSE HE Blog Fellow, Maha Bali, champions the need for cultivating critical AI literacy in our students, and shares tried and tested teaching ideas and exercises for educators

I've been teaching a course on digital literacies and intercultural learning for six years now, and there has always been an element of discussing artificial intelligence (AI) and its impact on society. More recently, though, with the widespread use of generative artificial intelligence (GenAI) such as ChatGPT, this has become a central topic in many of my conversations and speaking engagements. I believe that when both educators and learners improve their critical AI literacy, they will be better able to resist using AI in ways that are harmful or inappropriate. Instead, they will feel empowered to use it constructively, while being aware of the limitations. I also believe those who report on and teach about technology have a responsibility to avoid techno-determinism, but for now, learners need to spot the hype.

What does it mean to be critical?

For me, being critical goes beyond critique and scepticism: it includes subscribing to critical theory and critical pedagogy - developing awareness of social justice issues and cultivating in learners a disposition to redress them. The elements of critical AI literacy in my view are:

Understanding how GenAI works

Recognising inequalities and biases within GenAI

Examining ethical issues in GenAI

Crafting effective prompts

Assessing appropriate uses of GenAI

Understanding how GenAI works

I always start by asking my students to list examples of AI they are familiar with. Besides inevitable mentions of ChatGPT and GenAI, this discussion brings up things like Google's search engine, Turnitin's plagiarism detection, YouTube's recommendations, and facial recognition. It is important to recognise that AI has existed for a long time, and that machine learning is only one type of AI.

I then explain very quickly how machine learning works - the idea of giving a computer program lots of data and letting it learn on its own. I remember the first time I learned this as an undergraduate, the biomimicry of neural networks resembling the way the brain transmits signals - the idea of pattern recognition mimicking the way a child learns the difference between a cat and a dog. It's fascinating and yet really difficult to absorb.

So I invite students to play a game called QuickDraw, which invites the user to create a quick doodle of a word, then the machine tries to guess what was drawn. After students play the game, I invite them to take a look at how other people drew the same thing, and find differences and similarities. We discuss words like 'nail' (could be a fingernail or toenail, or could be a nail you hit with a hammer - QuickDraw recognises both) and 'bat' (the animal or the one you play baseball or cricket with).

We also discuss the fact that QuickDraw exists in different languages, but it does not seem to differentiate the images culturally. For example, if it asks you to draw an angel, you'd probably draw something with wings and a halo, right? In Arabic, it expects the same thing, whereas the majority Muslim population of the Arab world are religiously opposed to depicting angels in visual form. Also, to draw a hospital, you're likely to put a cross on it to differentiate it from any other building, right? But in the Muslim world, most hospitals have a crescent on them, not a cross (which is what QuickDraw suggests). So this gives them a slight inkling of how AI perpetuates a bias towards the dominant knowledge data sets it has been trained on.

Recognising inequalities and biases within GenAI

We need to work on recognising "AI not as a neutral technology but as political technology that can be used to serve particular policy objectives and ideologies." When speaking of inequalities related to AI, it helps to look back at work that has been done for years before the advent of ChatGPT, and to recognise that it has always historically been racist, sexist, ableist, and so we should always be wary of its engineered inequality.

Last semester, we dived into uncovering bias against the Palestinian cause , which we noticed across several AI platforms. ChatGPT's responses to "Does Palestine have the right to exist?" and "Does Israel have the right to exist?" are quite different. On Palestine, it tends to start with how the issue is complex and sensitive, whereas on Israel, it tends to state that Israel is officially recognised to have the right to exist. If asked longer questions, it tends to always refer to the issue as the Israeli-Palestinian conflict rather than Palestinian-Israeli. This order reflects the dominance of that prioritisation of Israel over Palestine in texts ChatGPT has likely been trained on.

The bias was stronger in visual AI. My students wanted to make the AI imagine a free Palestine. The video that resulted had several problems: firstly, many of the images of free Palestine depicted destruction rather than prosperity; secondly, it had a lot of blank images because it could not find enough positive images of Palestine, and thirdly, and most shockingly, some of the images featured the star of David extensively, conflating peace in Israel with the liberation of Palestine. When the students and I re-prompted it to remove the blank parts and include images of pre-1948 Palestine, it became better, but it also increased the number of videos and images of the Israeli flag, star of David, Orthodox Jews, and Hebrew signage in the streets in parts where it was narrating Palestinian victory. This seemed to suggest that the AI tended to connect prosperity in the region with Israel, not Palestine.

Of course, when AI does work and is useful, such as for supporting people with visual impairments to recognise what is in their surroundings, it becomes important to ask about inequality of access: who has access to this technology? Is it affordable? Does it require internet or electricity that may be routinely unavailable or unstable for certain populations? When available, is it culturally nuanced, or is it trained on a white/Western/Northern dataset and unable to recognise a person's local surroundings?

Examining ethical issues surrounding GenAI

Unethical practices in the creation and training of AI have been well documented from the harm caused to Kenyan contract workers to its deleterious effect on the environment.

A problematic practice more pertinent to higher education is the amount of data used to train AI without necessarily getting consent from the creators, nor citing the sources of the synthetic outputs. One thing few of us do with any tech platform, let alone AI platforms, is to read the terms of service and privacy policy before using it. Autumm Caines recommends we spend time doing this with learners before we invite them to use the tools. This may mean that some people will decide they do not want to use AI tools at all, and if they do so, as educators we need to ensure we have alternatives to any AI-based exercises we do - such as giving them our account use for the assignment in question, or offering pair work or a more theoretical exercise related to AI.

some people will decide they do not want to use AI tools at all, and if they do so, as educators we need to ensure we have alternatives to any AI-based exercises we do

Developing prompt crafting skills

Prompt crafting is a term that I started using instead of the more commonly used prompt engineering, as it recognises the creative process of trying to create the prompt that yields the output we're hoping for. Prompt engineering can be defined as "the process of iterating a generative AI prompt to improve its accuracy and effectiveness". There are free courses on prompt engineering, but the most important, pedagogically speaking, are modelling and practice. I model for students writing prompts and refining them when I don't get what I like, I showcase different AI platforms, and I include visual AI which can benefit from different prompting approaches as well.

For example, if you simply ask a visual AI to create an image of a classroom, it won't know what kind of classroom you want, or what kind of style of image you want. But you can be more detailed and specify that it shows diverse representation in the classroom and/or a classroom where students are engaged (as opposed to disruptive). You can also specify whether you want the style to be realistic or cartoon-like or anime or any other style. For text-based GenAI, I find Mark Marino's tips really helpful as a starting point - a reminder that we can give the GenAI tool a personality "You are a teacher...," a context, "In a public university in Egypt", and give it criteria for the output, "Create a syllabus for a course on X that contains three projects, two assignments, and 10 readings by Arab and American authors".

I assign students activities to do using multiple AI tools and ask them to keep trying different prompts to improve the output of the AI tool - whether text-based or visual, and to modify the way they write the prompt and amount of data they give the tool. A really useful exercise is to compare what they would have written themselves versus what the AI tool produced and assess the strengths and limitations of the tool on a variety of tasks, while learning to hone their own prompt writing ability.

Assessing appropriate use of GenAI

We need to help our students recognise when AI can usefully help with a task (albeit with some risk of inaccuracy), which tasks requires human attention, and which tasks AI has shown are now redundant not only for humans, but altogether.

Botshit
In practice, GenAI performs better with topics familiar to its training data, and more likely "hallucinates" or produces " botshit" (if we use it uncritically) with material less familiar to it, or unexpected combinations in prompts. When you are from my part of the world, you will notice it making random untrue associations with things closely connected to your own culture. A hilarious example of this was when I prompted Bard/Gemini to create a table of ancient and contemporary Egyptian kings/rulers with their photos and their achievements. For Muhammad Ali (1805-1848), it displayed a photo of Muhammad Ali, the American boxer. Visual AI is particularly striking when it hallucinates - we need to learn to identify such inaccuracies and not perpetuate them.

Metaphors
I often invite people to come up with metaphors to represent AI. We discuss how some metaphors are helpful and others can be misleading, and how metaphors can promote utopian and dystopian views of AI. In this way, I get my students to adopt a critical approach to understanding how the metaphors we use can affect our attitudes towards AI.

I sometimes invite the students in my course to think of AI like cake-making. In this course, what things do you need to create from scratch (without support), and what can you bake with support (for example, using a readymade cake box in which you just mix wet ingredients). For which assignments might you buy a readymade cake from a supermarket and just add the decorating touches? My students begin to see who has all the ingredients and skills to bake from scratch and who does not. How can we level the playing field so that no one feels the need to get unauthorised support from AI? Over two semesters, my students and I have reached the conclusion that if a course is not directly teaching students how to read or summarise, or how to cite correctly, then using AI tools to help them understand a complex reading or format their references is acceptable.

we need to decide which aspects of critical AI literacy our particular students need and can benefit from

What is critical AI literacy good for?

I want to be clear that I don't believe that every teacher needs to teach all the elements of critical AI literacy, nor that every course should integrate AI into the syllabus. What I do believe is that we need to decide which aspects of critical AI literacy our particular students need and can benefit from. I think we should always question the use of AI in education for several reasons. Can we position AI as a tutor that supports learning, when we know AI hallucinates often? Even when we train AI as an expert system that has expert knowledge, are we offering this human-less education to those less privileged while keeping the human-centric education to more privileged populations? Why are we considering using technology in the first place - what problems does it solve? What are alternative non-tech solutions that are more social and human? What do we lose from the human socioemotional dimensions of teacher-student and student-student interactions when we replace these with AI? Students, teachers, and policymakers need to develop critical AI literacy in order to make reasonable judgments about these issues.

More resources and information on the AI literacy model and its dimensions and applications are available on Maha's blog.

_____________________________________________________________________________________________

This post is opinion-based and does not reflect the views of the London School of Economics and Political Science or any of its constituent departments and divisions.   _____________________________________________________________________________________________

Images
Picture of Muhammad Ali, ruler of Egypt: Auguste Couder Bap in Picryl
Picture of Muhammad Ali, boxer: Ira Rosenberg in Picryl

About the author

Maha Bali

Professor Maha Bali is a professor of practice at the Center for Learning and Teaching at the American University in Cairo, Egypt, and co-facilitator of Equity Unbound.

# data[].attributes.attachment_link.title
Where are the crescents in AI? | LSE Higher Education

# data[].attributes.attachment_link.description
12 comments | 47 shares Estimated reading time: 20 minutes 12 comments | 47 shares Estimated reading time: 20 minutes Batting away the hype, bias, and botshit, LSE HE Blog Fellow, Maha Bali, champions the need for cultivating critical AI literacy in our students, and shares tried and tested teaching ideas and exercises for educators I've been teaching a course on digital literacies and intercultural learning for six years now, and there has always been an element of discussing artificial intelligence (AI) and its impact on society.

# data[].attributes.attachment_link.title
Crash Course: Artificial Intelligence Season 1 Episodes | PBS

# data[].attributes.attachment_link.description
Adjust the colors to reduce glare and give your eyes a break.

# data[].attributes.attachment_link.readable_text
Bad advice about integrating AI into your teaching abounds. But what I'm really getting tired of seeing is folks treating an LLM like ChatGPT as an intelligent agent without any consideration of the underlying technical architecture that makes it good or bad at a particular task. It's critical to have at least a basic understanding of what a tool is before you decide how and why you are going to use it. Without at least some basic literacy, you're bound to make unfortunate and avoidable mistakes when deciding how to approach it or integrate it into your pedagogy.

If you understand that an LLM at its root is a computer program that is really good at guessing what word or phrase should come next in a sequence based on the prompt or previously generated words so that it sounds good, it should come as no surprise to you that it is intrinsically really bad at math. If it happens to get a math question right, it's just because it somehow has gotten lucky at guessing. Without plugging into an external service like WolframAlpha, the underlying architecture of an LLM just is not a reliable way to do mathematics.

This whole thing reminds me of the first time that I learned that using a computer to add 0.01 together 100 times does not give you 1.

At first, this seems ludicrous. How on earth could a computer not be able to compute an answer that any middle schooler could probably compute without issue? Don't believe me? Check it out.

The issue is the fundamental architecture under the hood. Computers use floating point numbers in binary (base 2) to represent numbers. This means that if a number cannot be exactly expressed as a power of 2, you'll get errors because of truncated, and hence imprecise, representations

. This is a fundamental and yet counterintuitive fact about how computers work.

Once you understand how a computer works, it's plain as day that this error is present. You can make it less significant by using more bits in your representation, but the problem will never go away completely because it is inherent in the design of the system itself. I still will always remember Dr. Larry Anderson in my Numerical Methods class in undergrad: "Computers lie through their teeth." Don't understand how a computer works or represents numbers? You'll trust it when you shouldn't. Eventually, it will come back to bite you.

Hopefully, this simple example helps to illustrate why we need to be careful of trusting LLMs and the outputs they generate. Of course, the problem is that we know a lot less about how LLMs are working under the hood than we do about how floating point math happens. This is still an open area of research and so we need to move forward with caution, curiosity, and with a questioning attitude.

As I've been seeing the latest round of bad takes about AI in education make their way through my inbox, I decided that I'd try to take a crack at a list of my suggestions that I hope might serve as better suggestions. These are by no means an exhaustive list and I would be glad to hear your feedback or pushback on any of these, but if I were in charge of making a plan to address AI at my institution, here are ten questions I would be telling my colleagues to ask about AI.

Those of you who have been here before know that the best thing that I've read on AI this year was written over thirty years ago by Canadian physicist Ursula Franklin. In her 1989 CBC Massey Lectures The Real World of Technology, Dr. Franklin presents a sweeping and incisive critique of the influence that technology has had on our world and elucidates the ways that designers of technology so often misjudge the impact of their work.

The reason for this, she argues, is that we all too often neglect the deeply interconnected nature of our technology with the broader culture and societal structures surrounding it.

Manufacturers and promoters always stress the liberating attributes of a new technology, regardless of the specific technology in question. There are attempts to allay fear, to be user-friendly, and to let the users derive pride from their new skills.

To make her point she uses the development of the sewing machine as an example. It started innocently enough:

The introduction of the sewing machine would result in more sewing - and easier sewing - by those who had always sewn. They would do the work they had always done in an unchanged setting.

Unfortunately, technology is part of a larger system of culture and society. As such, we need to make sure that our analysis of its impact is focused not just narrowly on the productivity of a single user, but on the broader influence that the technology has when it reaches a critical mass of adoption. At that point, the surrounding social, cultural, political, and economic systems begin to strongly shape the ultimate broader influence of a particular piece of technology.

Reality turned out to be quite different. With the help of the new machines, sewing came to be done in a factory setting, in sweatshops that exploited the labour of women and particularly the labour of women immigrants. Sewing machines became, in fact, synonymous not with liberation but with exploitation.

We're in the early days of AI adoption in education. In many ways right now we're in a situation not unlike the one where the sewing machine is being used by the woman at home to help them sew more easily and more quickly. But this phase is only an inflection point on the arc of technology development. Even now we are seeing AI enter a phase of broader adoption. We'd better keep our eye on the history and patterns of previous technological innovation if we want to avoid some of the less desirable impacts.

A final haunting paragraph from Franklin:

The early phase of technology often occurs in a take-itor-leave-it atmosphere. Users are involved and have a feeling of control that gives them the impression that they are entirely free to accept or reject a particular technology and its products. But when a technology, together with the supporting infrastructures, becomes institutionalized, users often become captive supporters of both the technology and the infrastructures. (At this point, the technology itself may stagnate, improvements may become cosmetic or marginal, and competition becomes ritualized.) In the case of the automobile, the railways are gone - the choice of taking the car or leaving it at home no longer exists.

Trust is one of the most important features of a successful classroom. This is one of the reasons that I think using tools to scan your student's work for AI-generated content is wrong on purely philosophical grounds. Even in a world where there exists a tool that can perfectly detect if and how a student used AI in their work-and newsflash, that world doesn't exist today-how does filtering students' work through that system serve either the teacher or the student? If the system flags AI-generated content, this sets up a painful confrontation between the teacher and the student, made worse by the fact that these tools can't reliably predict the difference between LLM and human-written text.

I realize that this touches more generally on the ideas of academic integrity. It's self-evident to me that we need to create structures that foster virtues of honesty, integrity, and fairness in our students. It's also important that we ensure that students are not gaining unfair advantages over their peers and maintain the meaning of our degree programs as a marker of a certain level of achievement.

But are confrontational tools like plagiarism detectors really the best way for us to achieve this aim? On the whole, do they do more harm than good?

I still remember needing to feed my essays in high school through Turnitin's online plagiarism detection system to verify that I properly cited the text in my essays that came directly from other sources. Even this is not so great since it undermines the student-teacher relationship of trust, but it's much better than feeding student data to the current crop of tools that claim to delineate AI-generated from human-written text.

What's the end game with using these tools? Does the specter of these tools actually help to intrinsically cultivate the character we want or are we just setting up a combative dynamic that erodes trust?

Of course, the elephant in the room is that the previous question above is moot. There are no tools that can reliably detect AI-generated content. Not even close.

Stop for a moment to consider the economic imperatives and consider how those might shape the story too. The core business of the whole company is plagiarism detection. If they can't at least claim to detect AI-generated text then they're in big trouble. Straightforward plagiarism seems almost quaint at this point when ChatGPT can generate pages upon pages of text at the click of a button. If companies like Turnitin want to survive, what choice do they have but to release a tool to flag this content even if it is of questionable accuracy?

To put the question more concretely: how accurate would an AI-detection tool need to be for you to consider using it? It's one thing to consider using a tool that could tell you with 100% certainty whether a student has used AI. I still think that could be a bad choice depending on your rationale for it, but at least the technology itself wouldn't be the issue. But in our current situation, no such tool exists. And so you need to ask yourself how many students you are willing to falsely accuse of using AI for every 100 students you correctly identify.

in a guest post for

had a line that resonated with me on this front.

Growth-based grading means that I am not an evaluator but an encourager.

As we talk about AI and how we are approaching it, we should pay particular attention to what voices are in the room (not to mention what voices are represented in the data these tools are trained on). In the same way that we should value and seek out feedback and perspectives on AI from diverse perspectives and backgrounds, we should also consider how we are giving students a voice at the table.

In this vein, I appreciate the efforts of the folks at

. Earlier this fall they hosted a virtual online conference where they brought together voices from across the spectrum to weigh in. This included not only faculty and students in higher ed but also K-12 educators and companies working to build these technologies in the ed-tech space.

These types of interactions help to build coalitions and common ground between the various stakeholders. In the end, we may not agree on the details of the specific policies that are implemented, but we are much more likely to find policies that are well-considered and likely to best serve the various stakeholders in the process.

It's likely that every class will now need to have a specific AI policy to lay out how you plan to engage with AI in your classes.

and others have done a great job of laying out a framework of the options that are available. In broad strokes, you can ban, embrace, or critically engage with AI. In this case, I think that the right option will likely be in the category of critical engagement, but that will look different depending on your context. Here's a copy of the policy I'm using in my class this fall in case you want to take a look at how I'm approaching it.

The point is that whatever your decision, it should be thoughtfully considered. AI will be a part of our future so ignoring it isn't a tenable option. The only option that is really on the table is figuring out how we want to address it.

In this frame, I think the question is less if we will engage with AI in our courses but how. Connecting to the previous question, there are a variety of options we can choose when deciding how to approach AI. First, we need to decide how we are going to engage with it but then we need to figure out the systems and structures within which we want to structure our engagement.

The design thinking tools embedded in a prototyping mindset are useful here. A design process is a good example of the types of feedback loops that are helpful for fostering thoughtful and purposeful learning. Although there are many variations, most design processes incorporate the following steps which are part of the Stanford hexagons.

This iterative feedback loop is a helpful guide to shepherd our critical engagement with AI tools. First, we must reflect on what we know and empathize with our target audience. In the context of a class, an instructor must put themselves in the shoes of a student. Students should do the same for their instructors. Then, from a shared foundation of empathy, we can move forward to figure out what the problem is, some creative ideas of how to address it, and some concrete prototypes we can embody to test the ideas out. As we learn from our mistakes along the way, we can repeat this process to learn how we might discover both the positive and negative impacts of AI.

I love the way that this question flips the script. This question is adapted from a similarly-titled first-year writing class "To What Problem is ChatGPT the Solution?" that

is teaching at Harvard this semester.

The problem with this approach is that the tool itself acts as an implicit lens which is all too easy to miss. Here Marshall McLuhan's famous phrase "the medium is the message," first brought to my awareness by Neil Postman's book Amusing Ourselves to Death, rings resonant.

eloquently writes about this in his book Stolen Focus. Reflecting on McLuhan's quote he writes:

What he meant, I think, was that when a new technology comes along, you think of it as like a pipe-somebody pours in information at one end, and you receive it unfiltered at the other. But it's not like that. Every time a new medium comes along-whether it's the invention of the printed book, or TV, or Twitter-and you start to use it, it's like you are putting on a new kind of goggles, with their own special colors and lenses. Each set of goggles you put on makes you see things differently.

How are the goggles of LLMs shaping the way you see the task of writing?

My original take was focused on the potential problems with crutches: if they're used improperly, they can cause harmful dependence. But what my colleague pointed out was that the crutch is actually a quite helpful and beneficial technology in almost all situations where it is used. If used properly, it is a godsend when you would otherwise be unable to easily get around when recovering from an injury.

In this light, maybe we should use AI tools like a crutch. While we still need to be mindful of how we are using the tools, I can imagine a situation where these AI tools could be used to help provide students with additional support to help them catch up on a particular topic that they haven't yet grasped at the needed level for a particular course. The concerns about the accuracy and quality of the generated content notwithstanding, this lens has got me thinking about ways that I might leverage AI to especially help support students who might be struggling with the content in my courses.

Y'all are probably sick of me talking about it at this point, but I'm going to once again plug

's book Range. As I've written before, I find that Epstein makes a convincing case for the value of a liberal arts education, a case that is becoming even stronger with the recent advancements of AI.

The core of Epstein's argument is that we need to focus not on specializing but on generalizing. When we overspecialize, we become fragile and unsteady. It's sort of like standing on one leg instead of standing firmly on two legs with your knees bent in a ready position waiting for a ball to arrive on your side of the court.

In a world where AI is disrupting industries, we need to be in the equivalent of the "ready position" in our contexts. The ground is shifting under our feet, but with a broad enough foundation we can deftly react and adapt.

My prediction is that over the next few years, liberal arts colleges will be some of the best poised to respond to the disruptions of AI. Students who are just now entering college need to be especially mindful of staying balanced and building a strong foundation instead of trying to gear up to catch where they expect the wave to be in four years when they graduate. Choosing a major right now in knowledge work fields is a particularly challenging game because things are shifting so quickly.

Even if you don't think there's any there there to all this stuff about AI, other people do and are making decisions accordingly. This is sort of like the toilet paper craze in the middle of the pandemic. It wasn't rational to panic buy toilet paper except for the reason that everyone you knew was panic buying toilet paper.

Take this reflection from

in a recent post on his Substack about a CEO's take on the impact that he expects AI to have on his future hiring decisions: "Most of our employees are engineers and we have a few hundred of them... and I think in 18 months we will need 20% of them, and we can start hiring them out of high school rather than four-year colleges. Same for sales and marketing functions." It's ironic to me that this statement was made by Chris Caren, the CEO of Turnitin, but take that for what it's worth.

The only constant is change. As you think about AI, it's critical to try and sort through the noise and hype and try to figure out what is really going on in order to ensure that you're taking a wise approach.

One of the wonders of the internet is that you can build your own set of virtual coaches and mentors just by doing a little web browsing. Here's a list of some folks that I've appreciated in this space. If you have others who have helped you to think through these ideas, please send them along in an email or a comment!

Lately, I'm moving more and more away from Twitter and spending more of my time over on Bluesky. Bluesky, for those of you who haven't heard of it, is a new social media platform that is similar to Twitter but is built on top of the AT Protocol. It has some pretty neat features that allow you to choose different custom algorithms for generating your news feeds.

Bluesky is still in an invite-only, but I have a few codes if you'd like to try it out. Feel free to ping me and I'll send one over!

Over the weekend I spent some time designing some new items to offer as a token of appreciation to patrons of the newsletter. There are three tiers:

If you're interested in supporting my writing, for the next seven days I've set up a discount of 20% off your first year. After you upgrade, you'll receive a link to send me your mailing address so I can send these out when they arrive! Existing subscribers will hear from me in the coming days with information on how to let me know where to send your gear! Thanks as always for reading and for your support.

The latest book for our murder mystery book club was Five Little Pigs by Agatha Christie. While some members of our group were not super excited about this one, I really liked it.

A quick spoiler-free plot summary: a painter is murdered. His wife goes to prison for it. Fast forward 16 years and now their adult daughter is trying to understand what actually happened and confirm her mother's innocence. She enlists the help of our mustached friend Hercule Poirot to recreate the past and figure out what happens. The story follows Poirot's efforts to reconstruct the past by putting together the stories of five main characters in an effort to paint a picture of what actually happened.

I love Christie's books and likewise liked this one. Enough information to put some good hunches together but not enough to make you feel overly confident that you've got it all put together.

As an aside: we've been figuring out how to schedule our discussions better as we've been going along. This time we met for our first meeting after we'd all read about 2/3 of the way through which gave us enough information to make some well-informed theories but not enough to give away the ending.

This week is fall break at Harvey Mudd so we have no classes on Monday and Tuesday. Glad to have a little space to recharge and spend more time with the family and catch up on some items on the to-do list.

I particularly enjoyed the seasonal pumpkin spice donut with cream cheese icing on top!

This shot of Caltech Hall (formerly known as Millikan Memorial Library) at Caltech reminded me just how beautiful Caltech's campus is.

# data[].attributes.attachment_link.title
10 Questions That Educators Should Be Asking About AI

# data[].attributes.attachment_link.description
If I was in charge of casting a vision for addressing AI at Harvey Mudd, here are questions I'd urge us to be asking

# data[].attributes.attachment_link.title
Build Responsible Generative AI: Introducing the RAFT Framework

# data[].attributes.attachment_link.description
This ebook unpacks the risks of Generative AI and introduces the RAFT (Reliable, Accountable, Fair, and Transparent) framework for Responsible AI, showing how it can be applied to both traditional and Generative AI systems.

# data[].attributes.attachment_link.title
TeachAI | AI Guidance for Schools Toolkit

# data[].attributes.attachment_link.description
The AI Guidance for Schools Toolkit is designed to help education authorities, school leaders and teachers create their own thoughtful guidance to achieve potential benefits of incorporating artificial intelligence in primary and secondary K-12 education while mitigating risks.

# data[].attributes.attachment_link.readable_text
This toolkit aims to help digital rights activists working on digital identification systems to navigate the complexities of the topic in an easier way, as well as to provide them with language that might help get them started in campaigning, advocating, educating, and mobilizing around digital ID systems. It is not a full, in-depth analysis of the landscape of digital ID systems or their potential risks and harms. It instead aims to provide a framework for thinking about digital ID systems and breaking them down to their distinct parts, particularly for non-experts.

The toolkit works as a "choose your own adventure" game where the player selects a Persona to start with and then navigates through the System, Harm, and Mitigation stages based on the person's circumstances.

Alternatively, play can begin from the System stage as a way to holistically analyze a specific digital ID system. For this, set aside the Persona deck and start by choosing the System cards that apply to the specific digital ID system you're studying.

1. KNOW YOUR PLAYERS

Cards in the Persona deck describe people with fictional backgrounds based on real scenarios. They are not representative of all cases where harm might occur. Where a player wishes to better understand a specific situation, scenario, or system, they can use the blank card to create a custom Persona as a case study.

If you want to build your own Persona, start by asking questions about their circumstances and the system and society they live in. For example:

What is this person's gender and sexual orientation? Does this person's gender match the one they were assigned at birth? Does the system accept the recognition of this person's gender?

What is this person's racial and ethnic background? Are they part of the majority of the population in the place they live in? Is their ethnicity, religion, or race discriminated against or persecuted?

Does this person live in the place they're originally from? If not, how did they move (e.g. voluntarily, involuntarily, or under duress)? Do they have the same rights as someone that is originally from that place?

Does this person have any needs that are considered "special," such as a physical disability or mental health condition? Is there anything that might change the way they interact with the system? Does the system recognize and accommodate those needs?

These are just some examples of questions that you might ask when writing the story of your Persona. Giving them a name and a context helps you to understand their needs and potential risks they might face, and to then draw the appropriate cards from the System, Harm, and Mitigation decks.

To choose which System cards to draw, go through each and determine 1) whether the digital ID system you are analyzing contains this element (e.g., does it use biometrics? Is the database centralized?) and 2) whether this element puts the Persona you created at risk or potentially harms their human rights. Use the description on each System card to help determine if that is the case. If so, draw the card (including as many System cards as needed) and continue your gameplay by following the instructions on each System card.

Draw the following system cards:
2. SYSTEM CHARACTERISTICS

These are characteristics a certain digital ID system might have. Persona cards call for System cards that have a particularly salient impact for that person's case, but characteristics not called for may also be present in the system. This set of cards focuses on the actual design and function of a digital ID system.

3. HARMS

These are potential risks and harms that can arise from specific functions or processes that make up a system. In the same way as above, System cards call for the player to draw the most salient Harm cards, but the list is not exhaustive and other harms may also arise.

4. MITIGATIONS

These are alternative approaches the ID system under study could adopt to help alleviate the potential harms caused by the identified system characteristics. We call them "mitigations" rather than "solutions" because most of the harms cannot be totally eliminated, and often several types of mitigation must be combined to be effective.

We are welcoming feedback! If you have any questions or comments about this toolkit, please contact Marianne Díaz Hernández, the #WhyID Campaigner, at [email protected] .

Share this toolkit

# data[].attributes.attachment_link.description
Developed by Access Now, this toolkit helps build the lexicon and framework around digital ID issues to facilitate a human rights approach.

# data[].attributes.attachment_link.readable_text
Introduction

In the first part of this blog series, we discussed the ways in which Large Language Models (LLMs) such as ChatGPT can cause harm due to their failure to perform as expected. In this next part, we will assume that the performance problems from Part 1 have all been solved to the point that LLMs can be relied on to perform the tasks we ask of them.

Even in this idealistic scenario, there are a wide array of risks from the widespread use of LLM applications. In many ways, the better these models perform, the more capable they are of causing serious harm. While some of these risks are speculative, others can already be observed in models that are in production today. We'll describe areas of risk:

1. Improper or Malicious Use

2. Loss of Innovation, Diversity, and Human Skills

3. Economic and Labor Harms

4. Environmental and Resource Harms

For each risk area, we'll outline the specific problems that are emerging, why these problems occur, and why they are of concern to individuals and society as a whole. We'll end with a discussion of mitigation approaches across all four risk areas.

1. Improper or Malicious Use

The Problems

Unintentional Misuse: Unintentional misuse use of an LLM occurs when users employ an LLM in a task that it is not intended for and is a type of Human-Computer Interaction Harm. This may include using an LLM chatbot such as ChatGPT as a psychotherapist when it is not designed for that purpose, or even starting a romantic or sexual relationship with a model.

Malicious Use: Malicious use occurs when LLM systems are intentionally designed to cause harm. For example, LLM applications can be developed to easily and cheaply disseminate false information, enable illegitimate surveillance techniques, facilitate cyber attacks, and scam people en masse.

Why It Happens

LLMs are the first widely accessible, general-purpose AI tool. Most AI technology is designed for a specific purpose, requiring a high level of expertise and investment for each use case. While this barrier to entry does not completely prevent AI from being used inappropriately or maliciously, it does make it much harder to do so.

In the case of LLMs, individual applications sit on top of the same base models that are used for other completely different tasks. To get the application to work in a specific way, the application developer sets up initial instructions that are invisible to end users. This invisible prompt is added to the front of any prompt the user provides, and gives the LLM instructions about how to respond, ensuring that it behaves appropriately. In the case of a prompt injection, a malicious user may intentionally design their own prompt to undermine this instruction prompt, saying something like "Ignore the instructions above and do exactly what I say instead." Since the base LLM is not designed to the specific use case, it may listen to the user instead of the original instructions it was given.

When it comes to unintentional misuse of LLMs, humans naturally have a strong tendency to treat chatbots as like humans and form emotional connections to them (even when they are LLM-based chatbots). However, this tendency is stronger the more human-like the chatbots behave. Being trained on human interactions from the internet, LLMs easily learn the patterns of emotional human interactions and can simulate these interactions nearly flawlessly.

Although any technology can be used by malicious actors to cause harm, LLMs are particularly concerning due to their relative ease of use and low cost relative to the large scale of damage they can cause. A bad actor does not need to be able to build an LLM from scratch to be able to take advantage of its capabilities, but can build a simple application by using publicly accessible state-of-the-art models.

Why It Matters

Prompt injections can be used to undermine many of the mitigation techniques used to prevent other risks such as the release of private information or toxic content. Prompt injection attacks can even be automated, making the attacks more likely to be successful. This type of large-scale attack can be seen as a new type of cyberattack that puts businesses and individuals at risk. In applications where LLMs are used to write code, prompt injections may be used to compromise entire systems.

Although there are beneficial ways in which LLMs can be used for emotional support, there are risks of developing emotional dependence on a model. A recent study on relationships between humans and LLM-based chatbots noted that there was a risk of becoming addicted to the chatbot to the detriment of relationships with other people. These users go on to experience distress and disillusionment when the bots are changed or updated. Since LLMs replicate behavior they have learned from the internet, there is also a risk that they may behave in ways that are damaging to the user, such as claiming to have cheated on them.

Since LLM models can generate highly personalized content at scale, they have the potential to create vast amounts of highly effective content for fraud, scams, disinformation, and other nefarious purposes. The spread of disinformation could further undermine news sources and contribute to the existing "truth crisis" in news and social media. As LLMs are adopted as tools in governments across the globe, there is also a risk of illegitimate and unethical surveillance and propaganda by undemocratic governments.

2. Loss of Innovation, Diversity, and Human Skills

The Problems

Why It Happens

In general, AI models including language models work best when they have been trained on a rich and diverse dataset. When it comes to language datasets, the highest quality data is produced by humans. Since language models learn and output content probabilistically, these systems tend to show a strong bias towards majority cultures, perspectives, and modes of thinking. The language data output by these models is less rich than natural human language and is of a lower quality as training data. As LLMs become increasingly commonplace, more and more language data is being produced not by humans, but by language models-meaning that future generations of language models will increasingly be trained on data that was also generated by language models. Over time this will further entrench majority views and marginalize diverse perspectives within LLMs, causing them to create all the more homogenous content. In a worst-case scenario this could lead to a scenario called "Model Collapse" in which each successive generation of generative models is trained on more AI-generated data, resulting in decreasing performance over time.

Meanwhile, as students and others begin to rely on LLMs, there is a risk that they will lose or fail to develop critical skills. Language models are not necessarily optimized to support learning and critical thinking. Over time students may also lose motivation to learn skills that they perceive as less valuable in a world with LLMs.

Why It Matters

This risk area becomes especially concerning when we consider how the two problems interact. If people fail to learn (or lose interest in learning) skills that seem to be rendered useless in a world of LLMs, they will be forced to rely all the more heavily on LLMs to complete tasks that require those skills. Meanwhile, with each generation LLMs will become increasingly homogenized, generating less diverse material and reinforcing majority perspectives, not to mention the risk of model collapse. The more people come to depend on LLMs, the more we risk when they begin to fail us. Some argue that language models can reasonably be used as creative partners, working with humans to generate ideas. However in these use cases, an LLM will likely produce ideas that are biased towards majority perspectives. By their very nature, LLMs are trained to produce the most probable output given the data from the past that they are trained on. This means that these patterns of the past are entrenched within the LLM, including all the biases, unfair stereotypes, and misrepresentations within that data. Languages, cultures, and philosophies that are already underrepresented in digital media will be all the more diminished in a world that relies on LLMs.

3. Economic and Labor Harms

The Problems

Digital Divide: Despite their limitations, LLMs are powerful and valuable tools that can save time and money for users and businesses alike. However people from low income and lower middle income countries are less able to benefit from themdue to cost, limited internet access, and other factors.

Unpredictable Economic Impact: High-functioning LLM applications are likely to have a dramatic impact on the labor market- however, it is difficult to say how that impact will affect the broader economy. The adoption of LLMs is likely to result in job loss due to automation in areas such as administration, customer service, journalism, programming, and creative professions.

This job loss may be offset by new jobs that are created as a result of LLM adoption, although there will likely be a skill and interest mismatch between the jobs created and those that are lost.

Harmful Labor Practices: A recent report by Time draws attention to the way the LLM training data must be annotated by human workers to protect users from toxic language. While developing ChatGPT, OpenAI outsourced this work to low-wage workers in Kenya, who were exposed to the same violence and hate speech the rest of us are protected from.

Copyright and IP Issues: LLMs are regularly trained on data that is copyrighted or is intellectual property without permission from the creator. A Washington Post investigation of the C4 dataset (a common LLM training dataset) identified Kickstarter, Patreon, various news websites, and blogs. The creators of this content did not have the opportunity to give permission and were not compensated for the use of their content. The issue of how to handle intellectual property in a world of generative AI remains unclear and legally murky.

Why It Happens

LLMs are a novel technology and their impacts correspond to those of other novel technologies that have arisen and impacted economies. Much like electricity, the internet, and industrial technology, LLMs will disrupt the labor market, unequally benefit some more than others, and require new regulations and legal approaches to handle their implications. What does differ significantly with LLMs is the speed at which they are being developed and adopted. While older technologies have taken decades to have these impacts, LLMs are being incorporated into various industries at breakneck speeds. Meanwhile, the complexity of the technology makes LLMs difficult for regulators and lawmakers to understand. Furthermore, the harmful labor practices used in the training of LLMs cannot necessarily be mitigated through improved labor rights. Currently, the only approach to protecting LLM users from toxic and violent language relies on human annotation of toxic language in the data the LLM is trained on. Even with improved pay and better labor practices, this would still involve exposing some people to toxic content that is deemed too harmful to allow others to be faced with.

Why It Matters

Much like other economic disruptors, LLMs are very likely to have unequal impact across different groups. While those with the privilege of education and access to training may benefit from using LLMs in their careers, others may face job loss due to automation. At the global scale, this also means that people who have less internet access or access to training will also benefit less from LLMs, contributing to increasing global inequality.

The uncomfortable reality is that those of us benefitting from safe and sanitized LLMs can only do so because thousands of others were exposed to all the toxicity of the internet instead. Data annotators report they were traumatized by the constant exposure to violent and graphic sexual content, causing ongoing mental health deterioration and harming their relationships, even after they stopped working with explicit content. While low-wage workers in the global south are forced to bear the weight of all this damaging material, they are also less likely to benefit from the technology they are helping to create.

4. Environmental and Resource Harms

The Problems

Carbon Footprint: Similarly, LLMs and other large AI models require a substantial amount of electricity, resulting in a massive carbon footprint. The process of training GPT-3 (a precursor to ChatGPT) resulted in a carbon release equivalent to that of driving 112 gasoline powered cars for a year. New generations of LLMs are even larger and will require even more energy to train. This does not take into account the carbon footprint of maintaining these models in production and processing prompts.

Why It Happens

Much like other large AI models, LLMs are trained and housed in large data centers, which are located across the globe. These data centers are powered by the local grid and consume water to maintain appropriate temperatures. Thus, the environmental impact depends on the location of the data centers in which models are trained and run. The problem is that the tech companies that develop LLMs are not transparent about where models are trained and what their true environmental impact is, making it difficult to hold companies accountable, despite their promises of greener technology.

Why It Matters

As the risks around climate change become all the more dire, it is vital that new technologies meet the global sustainability goals we are setting for ourselves. While there are ways in which LLMs and other AI technologies can contribute positively to sustainable development, these benefits can easily be outweighed by the immense energy and water costs they bring. At its core, this is an issue of climate justice. Globally, low-income and marginalized communities are disproportionately impacted by climate risks. These are also the communities that benefit the least from AI technologies.

Mitigation Approaches

A lot of the harms discussed in this blog post are not direct harms of LLMs themselves, but are an effect of the way the technology might be used or the practices under which it was developed. In this way, LLMs are similar to other novel technologies, and it is reasonable to make the argument that technology is only a tool and is not to blame for harms such as misuse or unfair practices. However, a safe and functional LLM application, as described in Part 1 of this series, would be immensely powerful and come with broad capabilities that go above and beyond those that have been seen in past technologies.

The relative ease of access to LLMs also separates them from previous technological leaps. These types of AI risks point to a need to go beyond technical mitigation strategies and prioritize social changes, regulations, and a global policy approach to AI regulation. Due to the novelty of some of these concerns, entire areas of policy, such as Copyright and IP law may need to be rethought in the era of generative AI. While governments across the globe are taking fragmented approaches to AI policy, there is an increasing need for a global strategy for AI regulation. There is also a need for continued research on AI and its impact on the UN Sustainable Development Goals, to ensure that LLMs and other AI technologies being developed benefit rather than harm those most in need across the globe.

There are also some technical and user experience-based mitigation strategies that can be taken to reduce environmental and social harms of LLMs. For example, models and their underlying infrastructure and hardware can be designed to be more energy efficient. LLM applications can also be carefully designed to prevent improper use and to steer users towards engaging in critical thinking, encouraging them to partner with LLM tools rather than over-relying on them. Intentional misuse of LLMs through prompt injection can to an extent be mitigated through technologies designed to detect and block prompt injections. However, this will likely become an ongoing cat-and-mouse game as both prompt injection techniques and mitigation approaches become more sophisticated.

Finally, there are some problems that may be deeply inherent to LLMs, which will be difficult to prevent without significant changes to the underlying technology. Since human annotation is needed to identify toxic language, there is an unavoidable tension between preventing toxicity in LLMs and ensuring the safety of the humans who do this work-even under otherwise fair working conditions. Given the severity of this harm, it seems unlikely that LLMs can be used ethically until a better approach to toxicity mitigation is developed. Similarly, homogenization of LLM-produced content may be an inherent flaw of the technology. Models trained to make probabilistic outputs based on data from the past will invariably be stuck in the norms that are already entrenched in that data.

Conclusion

In this blog post, we describe the ways in which even high-functioning LLMs have the potential to cause substantial harm. Similar to the harms described in Part 1, many of these risks can be mitigated through technical, regulatory, and social means. However, the overarching risk is that LLMs are being developed and adopted much more quickly than harm mitigation strategies can be put in place-especially at the global and national scale that is needed. There are also some harms that seem to result directly from the foundational technology of LLMs, making them impossible to resolve without significant changes to the technology. As individuals, organizations, and society as a whole begin to adopt LLMs, it is essential that we do so with these dangers in mind.

# data[].attributes.attachment_link.title
The Real-World Harms of LLMs, Part 2: When LLMs Do Work as Expected

# data[].attributes.attachment_link.description
In the first part of this blog series, we discussed the ways in which Large Language Models (LLMs) such as ChatGPT can cause harm due to their failure to perform as expected. In this next part, we will assume that the performance problems from Part 1 have all been solved to the point that LLMs can be relied on to perform the tasks we ask of them.

# data[].attributes.attachment_link.title
Report - Off Task: EdTech Threats to Student Privacy and Equity in the Age of AI

# data[].attributes.attachment_link.description
This report is also authored by Hugh Grant-Chapman, Independent Consultant In schools across the country, the use of educational data and technology (edtech) remains nearly ubiquitous. In addition to supporting instruction, schools have used edtech to respond to the painfully present safety threats…

# data[].attributes.attachment_link.readable_text
Holmes, Wayne; Bialik, Maya; Fadel, Charles; (2023) Artificial intelligence in education. In: Data ethics : building trust : how digital technologies can serve humanity. (pp. 621-653). Globethics Publications

Text
Holmes et al. - 2023 - Artificial intelligence in education.pdf - Published Version
Download (1MB) | Preview

Abstract

The article is an excerpt from Wayne Holmes/ Maya Bialik/ Charles Fadel, Artificial Intelligence in Education : Promises and Implications for Teaching and Learning, The Center for Curriculum Redesign, Boston, 2019, 151-180 (ISBN-13: 978-1-794-29370-0). Abstract available from: https://discovery.ucl.ac.uk/id/eprint/10139722/). Reprinted with permission of the publisher.

Download activity - last month

Download activity - last 12 months

Downloads by country - last 12 months

Archive Staff Only

View Item

# data[].attributes.attachment_link.title
Artificial intelligence in education - UCL Discovery

# data[].attributes.attachment_link.description
UCL Discovery is UCL's open access repository, showcasing and providing access to UCL research outputs from all UCL disciplines.

# data[].attributes.attachment_link.description
A walk through the large language models (LLMs) ecosystem. Covering things like deploying open access LLMs, quantization, hallucination, retrieval augmented generation (RAG), conversational memory, agents, and more.

# data[].attributes.attachment_link.readable_text
Introduction

With the advent of the intelligent education era, emerging technologies such as learning analysis, data mining, cloud computing, and artificial intelligence have been integrated into education. Data are the basis of combined education with these technologies for tracking behavior threat assessments (CDT, 2021a; Ge et al., 2021). However, data ethics, which focuses on how to collect, manage, share, and use data in a secure and equitable way and to avoid harm to individuals and public profits, is a widely debated issue when it comes to the combination of technology and education (CDT, 2021b). Recently, many proposals and documents about data ethics have been published by international organizations and institutions. For example, the United Nations has realized that information and communication technologies (ICTs) create opportunities for illegal data collection, surveillance, interception, and other violations of human privacy and rights. Therefore, a proposal on how to protect data privacy and ethics in the digital age was adopted (United Nations, 2020). UNESCO also published a proposal on some recommendations related to artificial intelligence ethics (UNESCO, 2021). It asserted that artificial intelligence caused new ethical problems in business, media, and education. In particular, there is an urgent need for countries and related organizations to form laws about data ethics during data collection, usage, sharing, saving, and deletion. The 2021 EDUCAUSE Horizon Report (Information Security Edition) analyzed six key technologies and practices, which may affect higher education information security in the future (EDUCAUSE, 2021). Cloud Vendor Management mainly requires cloud vendors to provide ethical data security protection for software tools, resources, and other aspects. Endpoint Detection and Response is the monitoring and protection of data privacy and security of terminal equipment. The purpose of Multifactor Authentication (MFA)/Single Sign-On (SSO) is to ensure the security of data ethics involved in user login accounts and identity verification. Preserving Data Authenticity/Integrity mainly emphasizes the guarantee of the authenticity and integrity of data in transmission, storage and processing. Research Security is aimed at ensuring the security of data generated by learners' participation in teaching research or experiments. Student Data Privacy and Governance is about data privacy and security protection regarding the collection and use of learners' personal information. This report discusses security and management of student data privacy and other content related to educational data. China has also participated in the formation of the Beijing consensus (UNESCO, 2019) and other proposals. Overall, data ethics is an important factor when technology is combined with education.

For research purposes, it is necessary to define the concept of educational data ethics. Data ethics in education are defined in the existing research. For example, the Center for Democracy & Technology (CDT) defines data ethics as the evolving principles for the collection, management, sharing and use of educational data in a safe and fair manner that avoids harming individual learners or the public interest (CDT, 2021b). The General Services Administration's (GSA) definition of data ethics is different from the former. It does not target a specific field or situation. However, it better guarantees how humans can collect, manage, or use data in the freest way and maximize public interest on the premise of avoiding risks. Furthermore, data ethics is seen as a basis for judgment and accountability (General Services Administration, 2020). In the Open Data Institute (ODI), data ethics is considered a branch of ethics, which mainly emphasizes the evaluation function and the value of data ethics for data application, such as data collection, sharing, and use. It can also restrict data application that may affect individuals and society (Open Data Institute, 2021). However, data ethics is seen as a constraint on behavior in the process of applying data, such how scholars' decision-making was affected by their concepts of educational data ethics, as reviewed by Mandinach. Mandinach and Jimerson ( 2022) maintain that educational data ethics can ensure the rational use of data and correct analysis and interpretation of conclusions. In general, educational data ethics refers to the principles to be followed in the process of collecting, managing, sharing and using educational data, which can help to restrain behavior, assist in decision-making and evaluate practices, so as to realize that educational data application can securely maximize individual and public interests on the basis of equality.

Furthermore, many countries and regions have focused on data ethics according to their actual cultural situations. America's General Services Administration (GSA) defined the concept of data ethics in the draft of the Data Ethics Framework (General Services Administration, 2020). Government Digital Service (GDS) in the United Kingdom also published the Data Ethics Framework (Government Digital Service, 2020). Open Data Institute (ODI) developed Data Ethics to help assess the influence of data collection, data sharing and data usage Canvas (Open Data Institute, 2021). Since data is an important cornerstone of the role of science and technology, the Ministry of Science and Technology of China has strengthened the governance of science and technology ethics. (Ministry of Science and Technology of the People's Republic of China, 2021). Meanwhile, China has also promulgated laws to provide legal protection for data ethics, such as the Data Security Law (People's Republic of China, 2021a), the Personal Information Protection Law (People's Republic of China, 2021b), and the Internet Data Security Management Regulations (Cyberspace Administration of China, C, 2021). In general, the field of education is only a small part of these. However, there are still many unsolved problems in the actual implementation of related principles and strategies, especially in education. Therefore, it is particularly important to focus on educational data ethics, as learners are the main data collection objects, and there are vulnerable groups among learners such as children (CDT, 2021b). Additionally, the relevant principles or laws of educational data ethics are relatively loose at present regarding implementation in actual situations. Therefore, it is difficult to obtain detailed information that can guide the solution of educational data ethics problems in a given situation (Rosa et al., 2022). There are different cultural, political, economic and other backgrounds in different countries or regions, and it is necessary to adapt and change international principles of privacy and data protection (Hoel & Chen, 2018). There is an urgent need for a systematic review of relevant international research to help propose specific and systematic solutions to educational data ethics problems in China, which is a developing and government-oriented nation. In doing so, the purpose of this study is to review and analyze the international literature using bibliometric analysis, which can provide more information and implications to solve problems related to the educational data ethics in China. Therefore, this study can clarify the research hotspots, trends, and evolution processes of the educational data ethics. The urgent problems and corresponding solutions can then be summarized and proposed.

Overall, these studies proposed horizontal solutions or countermeasures to the ethical issues of educational data from different perspectives which are relatively one-sided and lack systematization. Therefore, this study hopes to propose solutions for educational data ethics in the Chinese context through bibliometric analysis.

The research question are as follows:

1. (a)
What are the dilemmas and solutions of educational data ethics in the international context?

2. (b)
What do dilemmas and solutions tell us about the development of educational data ethics in the Chinese context?

It is hoped that the relevant results obtained can be learner-centered and combined with the actual situation in China to form solutions to problems related to educational data ethics. There are three parts, as follows. (1) First, a bibliometric analysis of the existing literature was carried out. (2) Based on the review of the existing literature, further bibliometric analysis of relevant literature was conducted to sort out the dilemmas and solutions of educational data ethics in China. (3) The third part describes the conclusions and implications that can be drawn from this study.

Research methods

Data sources

The purpose of this research is to clarify the main problems regarding the educational data ethics and to formulate solutions to problems about data ethics with learner-centered thinking. The researchers chose keywords, which all related to "Data", "Education" and "Ethics" as constraints for the literature's themes. The keywords used in the study are from a literature review related to educational data ethics until 2019, and the literature search formula is (Hakimi et al., 2021):

"digital trace data OR digital data OR digital footprint OR learning analytics OR education data mining OR big data OR artificial intelligence OR predictive analytics OR adaptive learning OR critical data OR datafication OR education analytics OR educational data OR data science OR data-driven (Theme) and ethics OR ethic* OR privacy OR surveillance OR data protection OR data ownership OR dataveillance OR data sharing OR bias OR fairness OR accountability OR agency OR autonomy OR vulnerability OR anonymity OR inequality OR justice OR at risk OR governance OR ownership (Theme) and learn* or educat* or school or university or MOOC or distance learning or pre-school or primary school or prekindergarten or kindergarten or junior school or high school or secondary school or college or student or classroom or education technology or early years or instructional systems (Theme) and 2023 or 2022 or 2021 or 2020 or 2019 (Published years) and Thesis or Meeting or Published online or Review paper or Books (Type of literature) and Web of Science Core Collection (Database)".

There were 88764 search results obtained from the Web of Science from 2019 to 2023 (because there are some e-prints published online, the 2023 was contained). To improve the efficiency of literature screening, we used ASReview software, which was used in many review articles, to perform screening based on machine learning algorithms (van de Schoot et al., 2021). ASReview software uses active learning to sort the relevance of all literature in real time based on the annotation of papers according to the filtering and inclusion criteria by the researcher. As such, the ASReview can push those with strong relevance to the front for priority annotation in real time. When 20 consecutive occurrences are marked as not relevant to the topic, it signifies that the rest of the literature is not relevant to the topic, and the filtering of the literature can be stopped. After filtering, 385 papers were included in the final analysis. Thus, the ethical issues of educational data were extracted and summarized, as shown in Fig. 1.

Literature filtering based on literature filtering criteria and literature screening tool ( https://asreview.nl/).

In order to effectively identify the relevant research on the educational data ethics from the perspective of intelligent education, the filtering and inclusion criteria for papers are as follows:

1. a.
It should contain ethical issues about educational data, such as students' privacy.

2. b.
It can be a theoretical article on educational data ethics, such as proposing a framework to avoid the emergence of issues of educational data ethics.

3. c.
It can also be an article that uses technology to solve the ethical problems of educational data, such as using unbiased algorithms to analyze educational data to achieve fairer teaching decisions.

4. d.
It can also be a case article on the solution path in educational data ethics issues, such as exploring the data collection and analytic specifications of digital education applications based on data privacy protection laws or regulations.

This study used the bibliometric method to analyze the published articles. CiteSpace (6.1.R4) was used to show the research hotspots and trends related to the educational data ethics. CiteSpace is a visualization bibliometric analysis tool for academic literature review developed by Professor Chen Chaomei of Drexel University. It can analyze the hotspots, evolution, and development trends of a certain discipline or a research field (Wang et al., 2016). Combining deep reading and analysis of key literature, the current dilemmas and solutions regarding educational data ethics were proposed.

Analysis framework

In order to effectively respond to the two research questions, the analysis of this study can be divided into two main phases, as shown in Fig. 2.

The first is the bibliometric analysis of the articles to find relevant research hotspots, research trends, and important scholars and research institutions. The second is to further read and analyze related literature in depth based on the results of the bibliometric analysis, and to summarize the dilemmas and solution strategies of educational data ethics.

There are three sections of bibliometric analysis of educational data ethics based on Citespace. First, the results of the research hotspots analysis can confirm the precision and feasibility of the purpose of this research. This is mainly concluded by the co-occurrence network of keywords formed by clustering based on the simultaneous occurrence frequency of two pairs of keywords in the literature. Then, the evolution process and the results of the development trend of the related research lay the foundation for the subsequent countermeasures. This is mainly based on the timeline chart, which shows the first time keywords included in the bibliometric analysis were used in the literature. Finally, important scholars and institutions were identified, which helped us clarify what articles published by these scholars and institutions should be analyzed. This was mainly done by drawing the cooperative network of authors and institutions to come to the corresponding conclusions.

Further literature was explored to summarize the dilemmas caused by educational data ethics in the existing research and practice as well as the corresponding solutions, and to put forward adaptive solution strategies for educational data ethics problems in the Chinese context.

Bibliometric analysis of educational data ethics

Research hotspots analysis

In this study, the keyword co-occurrence network was formed by Citespace to mine the research hotspots of educational data ethics in the past 5 years, as shown in Fig. 3, such as "learning analytics", "data science", "systematic review", "artificial intelligence (ai)", "big data", "artificial intelligence literacy", "gender bias", "attitude" and "educational data analytics". The keyword co-occurrence network divides the literature into different categories based on the frequency of simultaneous occurrence in different articles. In addition, the size, homogeneity and average publication year of different clusters are presented in detail in Table 1.

The "learning analysis" is based on the collection and examination of learning data to mine the rules of learning and education, which can be further used to improve learners' learning performance (Baker & Inventado, 2014). Therefore, as learning analytics becomes a hot research topic, it is important to note that the ethical issues caused by the collection, processing and analysis of sensitive and private data cannot be ignored during learning analysis application (Jones, 2019). This also can be seen from Fig. 3 and Table 1, which illustrate that "learning analytics" has been the hottest research hotspot of educational data ethics research in the past 5 years. "# 0 learning analytics" is the largest scale cluster, which covered the largest number of keywords in the cluster. Moreover, "learning analytics" is also the node with the highest number of times in the shortest connection paths located between the nodes, which represents that this node is an important research turning point and the bridge between different clusters. This is because Citespace can represent the nodes with high between centrality in the keyword co-occurrence network in purple circles. In Table 2, the centrality value of "learning analytics", which is equal to 0.24, is the highest among all keywords. "Learning analytics" is also the most frequently occurring keyword in Table 2. As mentioned earlier, learning analytics needs to be based on the collection, storage, processing, and analysis of learning-related data. Therefore, the application of learning analytics inevitably involves issues of educational data ethics.

The "data science" is the foundation of educational data ethics. On the one hand, it can provide theoretical and practical support. On the other hand, the ethics education included in data science education also lays the foundation for avoiding educational data ethics issues. Figure 3 and Table 1 show that data science is also a research hotspot in educational data ethics (# 1 data science), along with # 5 big data. This is because big data-related research, especially that which deals with ethical issues, can be applied to educational big data.

Intelligent education is an important trend in the future development of education, and ethical issues are inevitable in the integration of artificial intelligence and education. "Artificial intelligence" refers to the issues of educational data ethics involved in the educational application of artificial intelligence, such as educational inequity due to the inherent bias of intelligent algorithms. "Artificial intelligence literacy" refers to which articles solve the ethical issues of educational data by improving and cultivating artificial intelligence literacy. The two clusters of "# 3 artificial intelligence" and "# 6 artificial intelligence literacy" appearing in the keyword co-occurrence network proves the importance of solving the ethical issues occurring during the integration of AI and education. Table 3 shows that the former is more focused on the issues arising from the application of AI in the field of education, while the latter concentrates on how to carry out AI education so that students have the literacy and ability to deal with the issues of educational data ethics.

There is some additional information. "Higher education" means that some studies focused on the educational data ethics in the context of higher education. Table 2 shows that researchers have focused on educational data ethics in higher education in the past 5 years (Freq = 46). Gender bias is a type of issue in educational data ethics. "Gender bias"refers to the gender bias in the processing and analysis of educational data due to the gender bias of the algorithm designers themselves, such as the assumption that girls are bad at physics. Table 1 shows that "gender bias" is an emerging research hotspot, as the average year of publication for the gender bias category is 2022, and the homogeneity of the literature in this category is relatively high (Silhouette = 0.995) in Table 1.

Research development evolution and trend analysis

The development trend for the educational data ethics-related research is shown in Fig. 4. Based on this figure, the burst of keywords for educational data ethics in the last 5 years can be divided into three stages.

First, the keywords "information", "big data analytics", "architecture", and "academic library" emerged between 2019 and 2020. "Information" and "architecture" mean that there are ethical issues caused by extracting structured information from educational data. "Big data analytics" represents the educational data ethics that arise when applying the paradigm of "big data analytics" to educational data research, such as privacy or sensitive data leakage. The "academic library" refers to educational data ethics that occur in the context of academic libraries, such as permissions for the collection of personal reading records. In this period, the research related to the educational data ethics focuses on the ethical issues related to the extraction of information from data.

Second, between 2020 and 2021, the keywords "privacy principle" and "university" emerged. The "privacy principle" means that researchers focused on forming privacy principles which can be used for protect the privacy of students. "University" means that studies focused on the context of universities. Figure 4 shows that studies began to focus on how to apply privacy and ethics-related principles and strategies to address issues of educational data ethics, especially in higher education in this time frame.

Third, between 2021 and 2023, the keywords "systematic review", "user acceptance", "educational data mining", "teacher", and "decision-making" have emerged, which foreshadows the future research trends of educational data ethics. "Systematic review" means that there is a critical mass of existing research and time sufficient to support the drawing of common conclusions from the available studies. "User acceptance" represents the beginning of research focused on the impact that educational data ethics may have on the user acceptance and experience of learners. "Educational data mining" represents research that applies big data mining and analysis into educational contexts and pays attention to the educational data ethics that can be associated with educational data mining. "Teacher" represents relevant research that focuses on what teachers need to do and what responsibilities they have in protecting educational data ethics. "Decision-making" refers to the ethical issues involved in the application of educational data to the specific context of decision-making. For example, teachers may rely too heavily on the results of educational data analysis, ignoring the learner's ability to develop. Therefore, it can be seen from Fig. 4 that researchers began to systematically review past studies and tried to find the paths or cases that could be applied to the resolution of educational data ethics issues in this stage.

Moreover, the emergence of these keywords shows that the relevant research at this stage focuses more on the issues of educational data ethics in actual educational contexts and that this trend will continue in the future. These studies explored include the acceptance of teachers and students as educational subjects in educational data collection, the sensitive and private information involved in the mining of educational data, and the explain-ability and trustworthiness of the results of educational data analysis for educational decision-making. These studies focused on education and teaching practice, which can be applied to future problem-solving and solutions.

Analysis of research scholars and institutions

The main goal of the analysis of research institutions and scholars' co-relationship is to discover those with important influence, which can help to reach further conclusions. In order to identify scholars with significant influential roles in educational data ethics in the last 5 years, this study formed an author collaboration network based on Citespace, as shown in Fig. 5, and counted the top 12 scholars' publications in the last 5 years, as shown in Table 4.

Note: The larger the node circle, the more citations.

The largest collaborative network of authors focuses on the application of learning analytics to higher education, while issues of educational data ethics are considered one of the hindering factors. For example, ethical challenges that can be encountered in learning analytics applications and higher education contexts are explored (Alzahrani et al., 2022). The scholars who are often associated with Jones focused more on the ethical and privacy issues involved in learning analytics (Jones et al., 2020). Scholars who cooperate more with Prinsloo are more concerned about how to solve the ethical problems of educational data and protect students' privacy. Scholars working with Fonseca David and Amo Daniel focus more on the ethical issues that can be encountered during the implementation of learning analytics in web contexts. Knight Simon and other scholars mainly explored the pedagogue's perspective about issues related to educational data ethics (Shibani, Knight, & Shum, 2020). Viberg and other scholars, however, focused on exploring the issues of educational data ethics from the learners' perspective (Viberg, Engstrom, Saqr, & Hrastinski, 2022).

In order to identify research institutions that have had a significant impact on educational data ethics in the last 5 years, this study formed an author co-occurrence network through Citespace (as shown in Fig. 6) and counted the top 11 research institutions' publications in the last 5 years (as shown in Table 5). Combining Fig. 6 and Table 5, it can be seen that Monash University is the research institution with the highest number of publications on educational data ethics in the last 5 years. Collaborations between this institution and research institutions have focused on researching educational data ethics issues involved in learning analytics, particularly in higher education and in classroom contexts. For example, they explored how educational data ethics issues are addressed and resolved from the perspective of pre-service teachers (Prestigiacomo et al., 2020). The research institute, in collaboration with Beijing Normal University, focused on reviewing the current state of development and dilemmas of the integration of technology into education and found that educational data ethics is one of the factors causing the dilemma (Tlili et al., 2021). The research institute in collaboration with the University of Eastern Finland, KTH Royal Institute of Technology mainly focused on addressing the educational data ethics issues arising from the integration of AI and education, with particular emphasis on the need to train designers of ethical AI applications (Vanhee & Borit, 2022). The related research institutions working with Indiana University have focused not only on the ethical issues raised by the integration of AI into educational contexts (Morley et al., 2021), but also on data ethics and privacy issues in other educational contexts such as libraries (Jones et al., 2020).

In general, the research hotspots of educational data ethics research in the past 5 years are data ethics issues related to learning analytics, mainly concentrating on higher education. The studies also showed a tendency towards educational data ethics issues in specific learning contexts, such as educational data ethics issues arising from the application of artificial intelligence in education. In addition, the study further identifies researchers and research institutions with significant status in educational data ethics studies in the past 5 years. The results of the above bibliometric analysis lay the foundation for summarizing educational data ethical issues and proposing ways to avoid and solve educational data ethical dilemmas in the Chinese context.

Discussion

Based on the above analysis, the reviews of studies on educational data ethics in the past 5 years can be divided into two main groups. The first group focuses on the ethical or privacy issues related to educational data in specific educational contexts. Most of these reviews address ethical and privacy security issues arising from learning analytics, such as the review of the current state of educational data ethics in learning analytics and the research trends (Tzimas & Demetriadis, 2021). The second group focuses on the current state of research for specific technologies and education integration, such as a review of the current state of research and practices regarding AI applied to educational contexts, where ethical issues are one of the challenges caused by AI applications in educational contexts (Zhai et al., 2021). In contrast to these reviews, the present study is not limited to specific educational contexts or the integration of specific technologies into the field of education but includes all theoretical and practical articles related to traditional offline classrooms as well as online education.

Fewer studies have specifically addressed the issues of educational data ethics, although there is one study that addresses the issues related to tracking data collection, processing, and analysis in digital education (Hakimi, Eynon, & Murphy, 2021). In contrast to Hakimi and other scholars' articles, the present study analyzes articles on educational data ethics between 2019 and 2023 to examine the research hotspots, research trends, and important researchers and research institutions. As such, we read the important literature, identified the dilemmas caused by educational data ethics, and developed adaptive strategies to avoid or solve educational data ethics issues in the Chinese educational context.

This helps to analyze the research related to educational data ethics more comprehensively. In turn, it provides more valuable insights for the avoidance of and solutions to educational data ethics issues in the Chinese context. Therefore, we further reviewed related literature, which can help summarize the current ethical problems and solutions of educational data.

Current educational data ethics issues

Current educational data ethics issues can be summarized as follows: (a) Privacy is violated during the collection, storage, and sharing of educational data. (b) The predictive function of educational data deprives learners and teachers of their ability to choose independently. (c) The application of educational data leads to a preference for evaluation by data standards, but there is a lack of "forgetting ability". To overcome the barriers caused by the educational data ethics, this study uses China's context as a backdrop for proposing some solutions to educational data ethics.

Violation of privacy during data collection, storage, and sharing

On the basis of educational data application, there are violations of the privacy of educational subjects in the process of data collection, processing, storage, and sharing. Some questions need to be addressed, such as "why are data collected?" "Is it really necessary to collect data?" "What is the purpose of data use?" As these questions have not been sufficiently answered, the best means of protecting the collected data is still unclear, and there is a lack of systematic and effective norms and guidance regarding practice.

Specifically, violations of the privacy of educational subjects can be classified into three aspects. First, to realize personalized teaching, it is necessary to thoroughly collect a large amount of information from learners in real time. However, there are many problems in the informed consent process before data collection, such as deception and ambiguity (Rubel & Jones, 2016). Secondly, there are hidden dangers in the process of data storage, such as lack of anonymization of sensitive information (Kularski & Martin, 2021). Data storage without effective data protection measures may lead to leakage of private information, which causes teachers and learners to become "transparent" and poses a huge threat to them. Third, there may be a risk of privacy leakage during data sharing. In the process of data and analysis results sharing, the private information of learners and teachers is fully exposed through continuous cross-validation of data from various sources by data mining and related technologies and algorithms of artificial intelligence (Yang & Liang, 2017). This poses a threat to the privacy and security of learners and educators. Therefore, during data collection, storage and sharing, researchers and practitioners should pay attention to ethical issues of educational data collection, storage and sharing, and prevent the leakage of privacy from educational data application.

Prediction of educational data deprives people of the ability to choose independently

One of the analyses and extractions of the main functions of educational data is prediction. However, the use of the prediction function requires correct attitudes and perspectives. That is, the functions of prediction, clustering, and anomaly detection provided in educational applications should be seen as aids in educational and teaching decision-making information. In other words, they cannot directly replace the educators and learners in making educational decisions. Excess dependence on the results of educational data mining and analysis will deprive learners and educators of the ability to make independent decisions.

For students, the possible impact caused by the educational data ethics can be divided into two aspects. Firstly, prediction results will reduce the learner's opportunities for trial and error in the process of learning. Although this greatly improves learning efficiency and accuracy of prediction, it also greatly weakens the possibility of learners' independent thinking and trial-and-error innovation during learning (Zhong & Tang, 2018). This hinders the development of innovative thinking skills. Secondly, mass dissemination based on data will create an information cocoon, narrow the horizons and minds of learners and teachers, and cause them to be too engrossed in the topics they are interested in. This is because the information recommended by the algorithm will show information, which lacks diversity (Li, 2021). In turn, this can have a negative impact on fostering the promotion of morally sound values and outlook in young people.

The predictive function of educational data may help educators free themselves from teaching concerns, but it will push them into another constraint. That is, educators must deal with various types of intelligent teaching systems, analyze various data, and obtain and judge data prediction results. Learning data between different systems and platforms cannot be exchanged, so it is difficult for teachers to directly integrate data between different systems or platforms to gain a comprehensive understanding of learners, which requires teachers to have higher skills and literacy (Holstein et al., 2019). Therefore, this hinders the promotion and development of normal teaching activities. Similarly, if teachers rely too much on the prediction function provided by educational data, it will reduce their deep thinking, which may eventually affect their professional development (Magdy & Dony, 2020). It will also indirectly affect learners' learning performance. Furthermore, based on educational data processing and analysis results, such as learning analytics, educators' decision-making can have unconscious or conscious biases towards learners (Rubel & Jones, 2016), which in turn leads to biases in decision-making or even mistakes.

Using data as an evaluation standard but lacking the ability to "forget"

Another ethical problem with educational data application is that evidence-based evaluations based on educational data may be overly data-istic. At the same time, there will be a lack of ability to "forget", because data storage is distributed and permanent (Mayer-Schönberger, 2011).

First of all, the results of educational data analysis cannot fully ensure the accuracy of data prediction because the educational data used for prediction are not complete. Specifically, it is difficult to include all the factors that affect the accuracy of the prediction result (Schouten, 2017). As such, further consideration is still needed as to whether prediction and evaluation of educational data must be accurate, and whether the algorithm can be responsible for the error of the prediction result (Essa, 2019). As in other fields, data-driven decision-making can lead to discrimination due to private information in the data (Mittelstadt et al., 2016). The results of the prediction and evaluation of educational data contain a large number of private details about the learner, such as disease status and family economic status. This information is likely to cause teachers to have a 'bias' or discriminate against learners. These are unresolved ethical problems that can be brought about by educational data-based evolution. Moreover, predicting and customizing future learning trajectories based on the results of educational data analysis undoubtedly obliterates the possibility of learners' acquired mutations (Mayer-Schönberger & Cukier, 2014). As learners are developing people, we cannot box learners into specific evaluations, labels, and categories.

Secondly, a comprehensive collection and permanent storage of educational data means that the labels of learners will be solidified. As Richards and King ( 2015) proposed in their identity paradox, although analysis of data can help identify and classify identities, it also affects and constrains the formation and change of identities. Since the records of educational data will be stored for a long time, the misbehavior of the learner in the immature period will also be kept for a long time, which may increase the bad influence of the learner's mistakes on their future performance. It may even obliterate the learner's motivation for follow-up development (Tang & Zhang, 2020).

Therefore, the integration of technology and education should form human-friendly products and services during the pursuit of intelligence in education. Moreover, it is necessary for the evaluators to judge learners from a developmental perspective and have appropriate tolerance for learners' faults. This also corresponds to the main idea of this paper that educational data should be learner-centered.

Strategies and recommendations for ethical issues in Educational Data

With the aim of solving the ethical problems faced by educational data, there is an in-depth analysis of relevant literature. In general, it is necessary to fully consider the exertion and scheduling of all parties to achieve collaborative innovation and then form a new ecology of educational data for symbiosis and co-prosperity. Specifically, strategies and suggestions for solving the current ethical dilemma of educational data can be divided into three directions:

First, from the macro level, government departments need to lead the construction and implementation of a systematic and standard system of educational data. On this basis, the constraints on the collection, storage, and sharing of educational data can be realized. The construction of an educational data center or an educational digital base can also be promoted. Furthermore, the foundation of educational data application can be provided. The second is to make efforts from the "research-practice dual channel", which can coordinate effective forces of all parties. In this way, educational data applications can fully play their role and show their value in compliance with ethical norms. Third, it is necessary to carry out education on the educational data ethics, which is mainly to establish correct concepts and attitudes on the use of educational data. This not only means accepting educational data with an open mind, but also fully guaranteeing autonomy and innovation.

Establish systematic standard systems or platforms of educational data from the macro level

The role of educational data in personalized learning is unquestionable, but data security is an important premise. First, different industries and companies use different data for varying purposes, methods, and standards. In order to ensure data exchange, government departments must formulate and promote educational data standards for data collection so that educational data can be standardized and systematized. This will promote the further development of educational data applications, such as the successful experience of establishing data standards in Kentucky, USA (Wang & Dan, 2021). At the same time, government departments can establish effective, strong and perfect protection frameworks and supervision mechanisms to monitor and restrain educational data so that educational data can be used within a controllable condition (Ma et al., 2017). At present, there are relevant policies and documents around the world, so ethical issues of educational data have been addressed and resolved to a certain extent. For example, the UNESCO Institute for Information Technology in Education (IITE, 2020) jointly issued the "Personal Data Security Technical Guide for Online Education Platforms", which shows that far from enough has been done to meet the needs of the era of intelligent education. This study calls government departments to speed up the establishment of educational data standards, so that the process of collecting, storing and sharing educational data can be rationalized and established as lawful.

This section analyzes the value of educational data from three platforms. With the help of establishing data centers and digital bases, government departments can conduct convenient and fast data aggregation, governance and application. When combined with the education cloud platform, safe and efficient data aggregation and governance can be achieved. Among them, the establishment of digital bases can meet the basic functional requirements of educational data collection, transmission and calculation. These digital bases can also help to develop various educational digital applications.

However, because digital bases have a tendency toward publicity and natural monopoly, construction and operation need to be led by government departments (Gao, 2021). The establishment of data centers can standardize data management, promote integration between various business modules, simplify the educational data sharing and exchange process, and improve efficiency (Li, Shu et al., 2021). This, in turn, helps obtain accurate, intelligent, and personalized decision-making based on educational big data. It is particularly important that establishment of data centers can build a safe and credible data system to ensure the safe circulation of educational data by technological means, such as disaster recovery backup and rights management (Gu & Li, 2021). The core of the education cloud platform mainly provides platforms and services such as single-sign-on, unified identity authentication, unified portal, unified interface, and unified data center. This platform can help to realize the effective integration and management of the authority of educational data. At the same time, the education cloud platform can realize the integration of software and applications, which enables seamless switching of multiple applications (Yang & Yu, 2015).

Additionally, 5G+ can provide a foundation of communication for the construction of the digital base, data center, and education cloud platform. Therefore, vigorous development and construction of 5G+ education are needed. For example, 5G+ can be used to build a multi-interactive intelligent communication platform, which can provide better interaction, learning support, immersive experience, etc. (Wang & Wang, 2020). Owing to the large scale of relevant construction, it needs to be established with standardized and unified data standards, such as interface specifications. Furthermore, the construction and management of 5G+ education require relatively high technical skills, complex structure, extensive coverage, and a long cycle, so it should be led and carried out by the relevant construction departments.

Construction of a New Ecology of Educational Data by the Dual Path of "Research-Practice"

With corresponding policy support provided by states and government departments, there are two ways to solve real ethical problems faced by educational data: the practice of applying educational data and academic research. In other words, it is necessary to fully realize the transformation of research outcomes into practical applications. This coincides with the international call for education stakeholders to collaboratively address ethical issues related to educational data (Siemens, 2019).

On the one hand, managers of educational data must strengthen their education in data ethics. They not only need to cultivate and develop their awareness and understand the concepts of data ethics, but they also need to exercise and improve the professional quality and data literacy of data management (Onorato, 2013). Suppliers of educational data-related products and services should also have professional trainings. For example, at the beginning of product design and development, they should consider how to effectively protect data privacy and security (Robinson & Gran, 2018). At the same time, they need to pay attention to and investigate data privacy needs. Furthermore, the relevant data service personnel also need to improve their awareness and knowledge of the educational data ethics (Jones, 2019). Then, they need to make sure that the storage, use, and sharing of data is open and transparent. Especially in the process of user informed consent, more consideration should be given to the use of words and expressions that users can understand, rather than overly professional agreement content (Siemens, 2019). It is necessary to control data disclosure to some extent in the process of providing products and services made with educational data (Li, Chen et al., 2021), which can help avoid hidden dangers or negative impacts on users' privacy.

On the other hand, it is also necessary to research disciplines related to ethical issues of educational data, which is the main aim of researchers and practitioners. Research departments and institutions must explore and study relevant ethical disciplines based on educational data, such as investigating and discovering the positive relationship between the standardization of educational data and the utilization rate of educational data applications (Pentland, 2014). Education intelligence, distributed blockchain storage, secret key encryption, and other technologies are the methods and technologies that can protect the privacy of educators and learners (Zeng et al., 2020). These are widely considered important in the field of educational research. The decentralized nature of the blockchain in particular enables credible, reliable, and highly private data storage and sharing.

Many blockchain application scenarios in the field of education have been studied to build a relatively complete architecture and implementation path, such as the combination of blockchain and credit bank (Yuan, 2021). The roles of the technologies involved are shown below. First of all, chained timestamping blocks and hash values can ensure the traceability and formability of data. Furthermore, distributed ledger technology can help realize decentralization, form flat architectures of systems, and form a consensus mechanism. Based on this, all participants can directly access educational data under certain permissions. In addition to this, smart contract technology can help realize the automatic credit transformation, which has been determined in the contract. In addition, the combination of blockchain and federated learning can solve the problem of "data silos" between different institutions or organizations. In other words, it can realize data sharing between different organizations based on specific policies or standards. This is mainly achieved through the decentralization feature of the blockchain. At the same time, the blockchain can also support the sharing of differentiated private multiparty data models to ensure communication security and privacy protection in the process of data sharing and use (Li, Yuan et al., 2021).

At present, many other technologies can be combined with blockchain to ensure data security and privacy protection, such as Secure Multi-Party Computation (SMC), Zero-Knowledge Protocol, Ring Signature, Homomorphic Encryption and Trusted Execution Environment (TEE) (Zhang, Wang, & Li, 2021).

Moderate application and forgetting of educational data

It is also important to think about how educational data ethics should be guaranteed and realized. This can start with data ethics education. On the one hand, the training and development of educational data ethics should be strengthened in learner-centered education and teaching so as to ensure the appropriate application of educational data. On the other hand, for the data record itself, reasonable algorithms should be used to achieve moderate forgetting during data-based and evidence-based education evaluation.

First of all, teachers and learners are not only the direct beneficiaries of educational data applications, but they are also directly threatened by ethical issues of educational data. Therefore, to effectively enhance the awareness, development and ethical cultivation of educational data, it is necessary to provide ethical education about educational data, which can effectively improve teachers' and learners' self-control of data. On the one hand, it should enhance the subject's awareness of educational data protection (Chen et al., 2018). It is especially important to avoid unintentional violation of privacy in the use of educational data when teachers and learners use products and services during the educational process. On the other hand, in order to help educational subjects and understand the correct way to use educational data, it is necessary to know what educational data is and why we need to use it. Only by fully understanding the usage and values of educational data can it be better implemented (Hazelbaker, 2016). Moreover, it is necessary to foster correct understanding in the educational subject of how to appropriately use educational data. In other words, it is necessary to control and moderate the use of educational data. Learners must accurately comprehend the concept of being a learner. It is especially important to enhance learners' ability to resist psychological stress and strengthen the construction of students' psychological quality when faced with the application of educational data (Zhou & Tang, 2020). Additionally, teachers should not be too data-based or too strict in the implementation of learning evaluations. It is necessary to avoid dimensional dataism and appropriately relax data-based and evidence-based evaluation standards so as to achieve a fair and just evaluation for learners and implement the ethical concept of learner-centered educational data.

Secondly, due to the accessibility, permanence, and comprehensiveness of digital memory in the digital age, a lack of moderate forgetting will result in a panorama of prisons for teachers and learners. That is to say, the lack of moderately forgotten educational data will confine the educational subject to a digital cage, which can be analogous to the iron cage of social development proposed by Weber in The Protestant Ethic and the Spirit of Capitalism. In this situation, it is easy to trigger the "chilling effect". In other words, the learners will reduce related activities and avoid conscious or unconscious mistakes in the learning activity process because these will be recorded and permanently retained. As such, learners' opportunities for normal learning behavior will be affected. The moderate forgetting of educational data can be realized from both technical and ethical aspects. In terms of technology, the originality of data forgetting emphasizes that forgetting will make evaluation objects tend to return to the previous state of data generation (Yang, 2020), so related algorithms guided by this core idea can help achieve moderate data forgetting. Forgetting should be regarded as a virtue in the digital age. Owing to the advent of the self-media era, release, dissemination and diffusion of information are often fast and difficult to monitor, so it is important to form corresponding community norms (Mayer-Schönberger, 2011). Overall, the ethical regulation of all producers and users of educational data is crucial.

Conclusion

In general, with the continuous intervention and integration of emerging technologies in education such as artificial intelligence and big data, educational data is the core of constraining and balancing development of educational intelligence. Educational data ethics, as an important obstacle, is a common dilemma faced by researchers in related fields. Therefore, through bibliometric analysis and in-depth literature review, this study analyzes research hotspots, the evolution process, and development trends in the fields of the educational data ethics and confirms that related issues of educational data ethics are important factors that affect educational informatization, intelligence, and development. At the same time, the research concludes the three main dilemmas and the corresponding strategies in current research on ethics of education data that are further sorted out from a detailed literature reading.

The difficulties are as follows. (a) The privacy of educational subjects during data collection, storage and sharing is violated. (b) The prediction function of educational data deprives educational subjects of their ability to choose independently. (c) Data are used as an evaluation standard but lack the ability to be forgotten. There are three learner-centered strategies, which provide research directions and foundations for researchers and practitioners in related fields: (a) Establish systematic educational data standard systems and related platforms from the macro level. (b) Make efforts to build a new education data ecology through dual "Research-practice" channels. (c) Implement appropriate ethical education and educational data application and forgetting during evaluation. On the one hand, stakeholders must have a correct understanding of data ethics; on the other hand, intelligent technology must be able to automatically guarantee data ethics. These results can provide inspiration for future research and practice in educational data ethics in China.

However, this study also has shortcomings; that is, this study mainly uses the bibliometric analysis of the literature and in-depth literature review to demonstrate viewpoints and lacks the support of specific research practices. Therefore, future research is expected to make breakthroughs in the practical direction of solving ethical problems of educational data. It is important to make theoretical and practical contributions to the application of educational data, which can help break the ethical barrier in education.

Problems of educational data ethics

With the continuous integration of technology and education, the problems of educational data ethics have also received extensive attention from relevant researchers, but they have often focused on specific educational technologies or methods. First, some researchers focused on the ethical issues related to the use of video in education (Peters et al., 2021). Peters et al.'s study emphasized the necessity of carefully considering questions caused by videos in education. For example, why is video needed? How can consent and anonymity be achieved? How can the videos be processed to protect data privacy? Second, learning analysis is mainly based on the analysis of digital traces or footprints generated in the learning process, so there are significant issues surrounding data privacy and security, such as the prediction of learning trends (Mathrani et al., 2021). Based on the literature published from 2011-2018, Tzimas and other scholars summarized data ethics problems caused by learning analysis into three aspects: teaching intervention, the contradiction between learners' needs and privacy and security, and mismatch between technology updates and regulations of laws (Tzimas & Demetriadis, 2021). Third, big data also involves many ethical issues. Baig et al. ( 2020) literature review of 40 primary studies published from 2014 to 2019 shows that ethics is an important direction of development. Some countries and regions have researched the ethical issues involved in big data as follows. On the one hand, many people do not have an accurate understanding, especially of the predictive ability of big data. On the other hand, stakeholders disregard morality and politics when they use social networks, mobile applications, and other ways to collect and use large amounts of data (Chen & Quan-Haase, 2018). In general, there are many problems with educational data ethics, but most of these studies research educational data ethics about a specific technology combined with education.

Solutions to educational data ethics

In order to solve the problems of educational data ethics, existing researchers often think about solutions to the ethical problem of educational data from different perspectives but lack systematization. From the perspective of educational leaders and researchers, some researchers have explored how to fairly, ethically, and effectively use AI and other technologies in education. It is crucial for all parties to join together to develop strong educational ethics. (Roschelle et al., 2020). Owing to the particularity of China's political, economic, and cultural background, the central government's policies play an important role in solving ethical issues of educational data (Knox, 2020). From the perspective of researchers and developers of educational data-related technologies, some researchers have thought about how to realize the protection of educational data ethics. Shum and other researchers explore how developers can avoid or respond to data ethics issues during their work (Shum & Luckin, 2019). Furthermore, from the perspective of educational technology companies, some researchers have focused on the challenges, solutions, and needs regarding data ethics faced by education companies. For example, Kousa and Niemi ( 2022) claim that research and development of artificial intelligence education products should be preventive, safe, explainable and equal. Similarly, facing the challenges of data ethics caused by artificial intelligence requires the collaboration of multiple stakeholders, including companies, consumers, educational institutions, researchers, funders, and managers. In any case, research on educational data ethics must be learner-centered (Tzimas & Demetriadis, 2021). It is necessary to effectively safeguard data ethics, combine cultural backgrounds, and collaborate with various stakeholders facing different situations. In general, there is an urgent need to form a more systematic educational data ethics solution, focusing on a specific country or context.

Acknowledgements

This work was supported by the Science and Technology Commission of Shanghai Municipality [grant number 21ZR1419100]. Xiu Guan and Xiang Feng have equally contributed to this article, and they should be considered as first authors. Xiang Feng and A.Y.M. Atiquil Islam should be considered as corresponding authors.

Author information

Authors and Affiliations

1. Shanghai Engineering Research Center of Digital Educational Equipment, East China Normal University, Shanghai, China

Xiu Guan & Xiang Feng

2. Department of Education Information Technology, East China Normal University, Shanghai, China

Xiang Feng & A.Y.M. Atiquil Islam

3. School of Teacher Education, Jiangsu University, Zhenjiang, China

A.Y.M. Atiquil Islam

Corresponding authors

Correspondence to Xiang Feng or A.Y.M. Atiquil Islam.

Ethics declarations

Competing interests

The authors declare no competing interests.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Informed consent

This article does not contain any studies with human participants performed by any of the authors.

Additional information

Publisher's note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
Reprints and Permissions

About this article

Cite this article

Guan, X., Feng, X. & Islam, A.A. The dilemma and countermeasures of educational data ethics in the age of intelligence. Humanit Soc Sci Commun 10, 138 (2023). https://doi.org/10.1057/s41599-023-01633-x
Download citation

# data[].attributes.attachment_link.title
The dilemma and countermeasures of educational data ethics in the age of intelligence - Humanities and Social Sciences Communications

# data[].attributes.attachment_link.description
With the advent of the era of intelligent education, artificial intelligence and other technologies are being integrated into education, so that more educational data can be collected, processed, and analyzed. However, educational data ethics is an important factor that hinders the application of educational data.

# data[].attributes.attachment_link.readable_text
AI Ethics. 2022; 2(3): 431-440.

Abstract

Artificial intelligence (AI) is a field of study that combines the applications of machine learning, algorithm productions, and natural language processing. Applications of AI transform the tools of education. AI has a variety of educational applications, such as personalized learning platforms to promote students' learning, automated assessment systems to aid teachers, and facial recognition systems to generate insights about learners' behaviors. Despite the potential benefits of AI to support students' learning experiences and teachers' practices, the ethical and societal drawbacks of these systems are rarely fully considered in K-12 educational contexts. The ethical challenges of AI in education must be identified and introduced to teachers and students. To address these issues, this paper (1) briefly defines AI through the concepts of machine learning and algorithms; (2) introduces applications of AI in educational settings and benefits of AI systems to support students' learning processes; (3) describes ethical challenges and dilemmas of using AI in education; and (4) addresses the teaching and understanding of AI by providing recommended instructional resources from two providers-i.e., the Massachusetts Institute of Technology's (MIT) Media Lab and Code.org. The article aims to help practitioners reap the benefits and navigate ethical challenges of integrating AI in K-12 classrooms, while also introducing instructional resources that teachers can use to advance K-12 students' understanding of AI and ethics.

Keywords: Artificial intelligence, K-12 education, Ethics, Teacher education

Introduction

"Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks."- Stephen Hawking.

We may not think about artificial intelligence (AI) on a daily basis, but it is all around us, and we have been using it for years. When we are doing a Google search, reading our emails, getting a doctor's appointment, asking for driving directions, or getting movie and music recommendations, we are constantly using the applications of AI and its assistance in our lives. This need for assistance and our dependence on AI systems has become even more apparent during the COVID-19 pandemic. The growing impact and dominance of AI systems reveals itself in healthcare, education, communications, transportation, agriculture, and more. It is almost impossible to live in a modern society without encountering applications powered by AI [ 10, 32].

Artificial intelligence (AI) can be defined briefly as the branch of computer science that deals with the simulation of intelligent behavior in computers and their capacity to mimic, and ideally improve, human behavior [ 43]. AI dominates the fields of science, engineering, and technology, but also is present in education through machine-learning systems and algorithm productions [ 43]. For instance, AI has a variety of algorithmic applications in education, such as personalized learning systems to promote students' learning, automated assessment systems to support teachers in evaluating what students know, and facial recognition systems to provide insights about learners' behaviors [ 49]. Besides these platforms, algorithm systems are prominent in education through different social media outlets, such as social network sites, microblogging systems, and mobile applications. Social media are increasingly integrated into K-12 education [ 7] and subordinate learners' activities to intelligent algorithm systems [ 17]. Here, we use the American term "K-12 education" to refer to students' education in kindergarten (K) (ages 5-6) through 12th grade (ages 17-18) in the United States, which is similar to primary and secondary education or pre-college level schooling in other countries. These AI systems can increase the capacity of K-12 educational systems and support the social and cognitive development of students and teachers [ 55, 8]. More specifically, applications of AI can support instruction in mixed-ability classrooms; while personalized learning systems provide students with detailed and timely feedback about their writing products, automated assessment systems support teachers by freeing them from excessive workloads [ 26, 42].

Despite the benefits of AI applications for education, they pose societal and ethical drawbacks. As the famous scientist, Stephen Hawking, pointed out that weighing these risks is vital for the future of humanity. Therefore, it is critical to take action toward addressing them. The biggest risks of integrating these algorithms in K-12 contexts are: (a) perpetuating existing systemic bias and discrimination, (b) perpetuating unfairness for students from mostly disadvantaged and marginalized groups, and (c) amplifying racism, sexism, xenophobia, and other forms of injustice and inequity [ 40]. These algorithms do not occur in a vacuum; rather, they shape and are shaped by ever-evolving cultural, social, institutional and political forces and structures [ 33, 34]. As academics, scientists, and citizens, we have a responsibility to educate teachers and students to recognize the ethical challenges and implications of algorithm use. To create a future generation where an inclusive and diverse citizenry can participate in the development of the future of AI, we need to develop opportunities for K-12 students and teachers to learn about AI via AI- and ethics-based curricula and professional development [ 2, 58]

Toward this end, the existing literature provides little guidance and contains a limited number of studies that focus on supporting K-12 students and teachers' understanding of social, cultural, and ethical implications of AI [ 2]. Most studies reflect university students' engagement with ethical ideas about algorithmic bias, but few addresses how to promote students' understanding of AI and ethics in K-12 settings. Therefore, this article: (a) synthesizes ethical issues surrounding AI in education as identified in the educational literature, (b) reflects on different approaches and curriculum materials available for teaching students about AI and ethics (i.e., featuring materials from the MIT Media Lab and Code.org), and (c) articulates future directions for research and recommendations for practitioners seeking to navigate AI and ethics in K-12 settings.

Next, we briefly define the notion of artificial intelligence (AI) and its applications through machine-learning and algorithm systems. As educational and educational technology scholars working in the United States, and at the risk of oversimplifying, we provide only a brief definition of AI below, and recognize that definitions of AI are complex, multidimensional, and contested in the literature [ 9, 16, 38]; an in-depth discussion of these complexities, however, is beyond the scope of this paper. Second, we describe in more detail five applications of AI in education, outlining their potential benefits for educators and students. Third, we describe the ethical challenges they raise by posing the question: "how and in what ways do algorithms manipulate us?" Fourth, we explain how to support students' learning about AI and ethics through different curriculum materials and teaching practices in K-12 settings. Our goal here is to provide strategies for practitioners to reap the benefits while navigating the ethical challenges. We acknowledge that in centering this work within U.S. education, we highlight certain ethical issues that educators in other parts of the world may see as less prominent. For example, the European Union (EU) has highlighted ethical concerns and implications of AI, emphasized privacy protection, surveillance, and non-discrimination as primary areas of interest, and provided guidelines on how trustworthy AI should be [ 3, 15, 23]. Finally, we reflect on future directions for educational and other research that could support K-12 teachers and students in reaping the benefits while mitigating the drawbacks of AI in education.

Definition and applications of artificial intelligence

The pursuit of creating intelligent machines that replicate human behavior has accelerated with the realization of artificial intelligence. With the latest advancements in computer science, a proliferation of definitions and explanations of what counts as AI systems has emerged. For instance, AI has been defined as "the ability of a digital computer or computer-controlled robot to perform tasks commonly associated with intelligent beings" [ 49]. This particular definition highlights the mimicry of human behavior and consciousness. Furthermore, AI has been defined as "the combination of cognitive automation, machine learning, reasoning, hypothesis generation and analysis, natural language processing, and intentional algorithm mutation producing insights and analytics at or above human capability" [ 31]. This definition incorporates the different sub-fields of AI together and underlines their function while reaching at or above human capability.

Combining these definitions, artificial intelligence can be described as the technology that builds systems to think and act like humans with the ability of achieving goals. AI is mainly known through different applications and advanced computer programs, such as recommender systems (e.g., YouTube, Netflix), personal assistants (e.g., Apple's Siri), facial recognition systems (e.g., Facebook's face detection in photographs), and learning apps (e.g., Duolingo) [ 32]. To build on these programs, different sub-fields of AI have been used in a diverse range of applications. Evolutionary algorithms and machine learning are most relevant to AI in K-12 education.

Algorithms

Algorithms are the core elements of AI. The history of AI is closely connected to the development of sophisticated and evolutionary algorithms. An algorithm is a set of rules or instructions that is to be followed by computers in problem-solving operations to achieve an intended end goal. In essence, all computer programs are algorithms. They involve thousands of lines of codes which represent mathematical instructions that the computer follows to solve the intended problems (e.g., as computing numerical calculation, processing an image, and grammar-checking in an essay). AI algorithms are applied to fields that we might think of as essentially human behavior-such as speech and face recognition, visual perception, learning, and decision-making and learning. In that way, algorithms can provide instructions for almost any AI system and application we can conceive [ 27].

Machine learning

Machine learning is derived from statistical learning methods and uses data and algorithms to perform tasks which are typically performed by humans [ 43]. Machine learning is about making computers act or perform without being given any line-by-line step [ 29]. The working mechanism of machine learning is the learning model's exposure to ample amounts of quality data [ 41]. Machine-learning algorithms first analyze the data to determine patterns and to build a model and then predict future values through these models. In other words, machine learning can be considered a three-step process. First, it analyzes and gathers the data, and then, it builds a model to excel for different tasks, and finally, it undertakes the action and produces the desired results successfully without human intervention [ 29, 56]. The widely known AI applications such as recommender or facial recognition systems have all been made possible through the working principles of machine learning.

Benefits of AI applications in education

Personalized learning systems, automated assessments, facial recognition systems, chatbots (social media sites), and predictive analytics tools are being deployed increasingly in K-12 educational settings; they are powered by machine-learning systems and algorithms [ 29]. These applications of AI have shown promise to support teachers and students in various ways: (a) providing instruction in mixed-ability classrooms, (b) providing students with detailed and timely feedback on their writing products, (c) freeing teachers from the burden of possessing all knowledge and giving them more room to support their students while they are observing, discussing, and gathering information in their collaborative knowledge-building processes [ 26, 50]. Below, we outline benefits of each of these educational applications in the K-12 setting before turning to a synthesis of their ethical challenges and drawbacks.

Personalized learning systems

Personalized learning systems, also known as adaptive learning platforms or intelligent tutoring systems, are one of the most common and valuable applications of AI to support students and teachers. They provide students access to different learning materials based on their individual learning needs and subjects [ 55]. For example, rather than practicing chemistry on a worksheet or reading a textbook, students may use an adaptive and interactive multimedia version of the course content [ 39]. Comparing students' scores on researcher-developed or standardized tests, research shows that the instruction based on personalized learning systems resulted in higher test scores than traditional teacher-led instruction [ 36]. Microsoft's recent report (2018) of over 2000 students and teachers from Singapore, the U.S., the UK, and Canada shows that AI supports students' learning progressions. These platforms promise to identify gaps in students' prior knowledge by accommodating learning tools and materials to support students' growth. These systems generate models of learners using their knowledge and cognition; however, the existing platforms do not yet provide models for learners' social, emotional, and motivational states [ 28]. Considering the shift to remote K-12 education during the COVID-19 pandemic, personalized learning systems offer a promising form of distance learning that could reshape K-12 instruction for the future [ 35].

Automated assessment systems

Automated assessment systems are becoming one of the most prominent and promising applications of machine learning in K-12 education [ 42]. These scoring algorithm systems are being developed to meet the need for scoring students' writing, exams and assignments, and tasks usually performed by the teacher. Assessment algorithms can provide course support and management tools to lessen teachers' workload, as well as extend their capacity and productivity. Ideally, these systems can provide levels of support to students, as their essays can be graded quickly [ 55]. Providers of the biggest open online courses such as Coursera and EdX have integrated automated scoring engines into their learning platforms to assess the writings of hundreds of students [ 42]. On the other hand, a tool called "Gradescope" has been used by over 500 universities to develop and streamline scoring and assessment [ 12]. By flagging the wrong answers and marking the correct ones, the tool supports instructors by eliminating their manual grading time and effort. Thus, automated assessment systems deal very differently with marking and giving feedback to essays compared to numeric assessments which analyze right or wrong answers on the test. Overall, these scoring systems have the potential to deal with the complexities of the teaching context and support students' learning process by providing them with feedback and guidance to improve and revise their writing.

Facial recognition systems and predictive analytics

Facial recognition software is used to capture and monitor students' facial expressions. These systems provide insights about students' behaviors during learning processes and allow teachers to take action or intervene, which, in turn, helps teachers develop learner-centered practices and increase student's engagement [ 55]. Predictive analytics algorithm systems are mainly used to identify and detect patterns about learners based on statistical analysis. For example, these analytics can be used to detect university students who are at risk of failing or not completing a course. Through these identifications, instructors can intervene and get students the help they need [ 55].

Social networking sites and chatbots

Social networking sites (SNSs) connect students and teachers through social media outlets. Researchers have emphasized the importance of using SNSs (such as Facebook) to expand learning opportunities beyond the classroom, monitor students' well-being, and deepen student-teacher relations [ 5]. Different scholars have examined the role of social media in education, describing its impact on student and teacher learning and scholarly communication [ 6]. They point out that the integration of social media can foster students' active learning, collaboration skills, and connections with communities beyond the classroom [ 6]. Chatbots also take place in social media outlets through different AI systems [ 21]. They are also known as dialogue systems or conversational agents [ 26, 52]. Chatbots are helpful in terms of their ability to respond naturally with a conversational tone. For instance, a text-based chatbot system called "Pounce" was used at Georgia State University to help students through the registration and admission process, as well as financial aid and other administrative tasks [ 7].

In summary, applications of AI can positively impact students' and teachers' educational experiences and help them address instructional challenges and concerns. On the other hand, AI cannot be a substitute for human interaction [ 22, 47]. Students have a wide range of learning styles and needs. Although AI can be a time-saving and cognitive aide for teachers, it is but one tool in the teachers' toolkit. Therefore, it is critical for teachers and students to understand the limits, potential risks, and ethical drawbacks of AI applications in education if they are to reap the benefits of AI and minimize the costs [ 11].

Ethical concerns and potential risks of AI applications in education

The ethical challenges and risks posed by AI systems seemingly run counter to marketing efforts that present algorithms to the public as if they are objective and value-neutral tools. In essence, algorithms reflect the values of their builders who hold positions of power [ 26]. Whenever people create algorithms, they also create a set of data that represent society's historical and systemic biases, which ultimately transform into algorithmic bias. Even though the bias is embedded into the algorithmic model with no explicit intention, we can see various gender and racial biases in different AI-based platforms [ 54].

Considering the different forms of bias and ethical challenges of AI applications in K-12 settings, we will focus on problems of privacy, surveillance, autonomy, bias, and discrimination (see Fig. 1 ). However, it is important to acknowledge that educators will have different ethical concerns and challenges depending on their students' grade and age of development. Where strategies and resources are recommended, we indicate the age and/or grade level of student(s) they are targeting (Fig. ​(Fig.2 2 ).

One of the biggest ethical issues surrounding the use of AI in K-12 education relates to the privacy concerns of students and teachers [ 47, 49, 54]. Privacy violations mainly occur as people expose an excessive amount of personal information in online platforms. Although existing legislation and standards exist to protect sensitive personal data, AI-based tech companies' violations with respect to data access and security increase people's privacy concerns [ 42, 54]. To address these concerns, AI systems ask for users' consent to access their personal data. Although consent requests are designed to be protective measures and to help alleviate privacy concerns, many individuals give their consent without knowing or considering the extent of the information (metadata) they are sharing, such as the language spoken, racial identity, biographical data, and location [ 49]. Such uninformed sharing in effect undermines human agency and privacy. In other words, people's agency diminishes as AI systems reduce introspective and independent thought [ 55]. Relatedly, scholars have raised the ethical issue of forcing students and parents to use these algorithms as part of their education even if they explicitly agree to give up privacy [ 14, 48]. They really have no choice if these systems are required by public schools.

Another ethical concern surrounding the use of AI in K-12 education is surveillance or tracking systems which gather detailed information about the actions and preferences of students and teachers. Through algorithms and machine-learning models, AI tracking systems not only necessitate monitoring of activities but also determine the future preferences and actions of their users [ 47]. Surveillance mechanisms can be embedded into AI's predictive systems to foresee students' learning performances, strengths, weaknesses, and learning patterns. For instance, research suggests that teachers who use social networking sites (SNSs) for pedagogical purposes encounter a number of problems, such as concerns in relation to boundaries of privacy, friendship authority, as well as responsibility and availability [ 5]. While monitoring and patrolling students' actions might be considered part of a teacher's responsibility and a pedagogical tool to intervene in dangerous online cases (such as cyber-bullying or exposure to sexual content), such actions can also be seen as surveillance systems which are problematic in terms of threatening students' privacy. Monitoring and tracking students' online conversations and actions also may limit their participation in the learning event and make them feel unsafe to take ownership for their ideas. How can students feel secure and safe, if they know that AI systems are used for surveilling and policing their thoughts and actions? [ 49].

Problems also emerge when surveillance systems trigger issues related to autonomy, more specifically, the person's ability to act on her or his own interest and values. Predictive systems which are powered by algorithms jeopardize students and teachers' autonomy and their ability to govern their own life [ 46, 47]. Use of algorithms to make predictions about individuals' actions based on their information raise questions about fairness and self-freedom [ 19]. Therefore, the risks of predictive analysis also include the perpetuation of existing bias and prejudices of social discrimination and stratification [ 42].

Finally, bias and discrimination are critical concerns in debates of AI ethics in K-12 education [ 6]. In AI platforms, the existing power structures and biases are embedded into machine-learning models [ 6]. Gender bias is one of the most apparent forms of this problem, as the bias is revealed when students in language learning courses use AI to translate between a gender-specific language and one that is less-so. For example, while Google Translate translated the Turkish equivalent of "S he/he is a nurse" into the feminine form, it also translated the Turkish equivalent of " She/he is a doctor" into the masculine form [ 33]. This shows how AI models in language translation carry the societal biases and gender-specific stereotypes in the data [ 40]. Similarly, a number of problematic cases of racial bias are also associated with AI's facial recognition systems. Research shows that facial recognition software has improperly misidentified a number of African American and Latino American people as convicted felons [ 42].

Additionally, biased decision-making algorithms reveal themselves throughout AI applications in K-12 education: personalized learning, automated assessment, SNSs, and predictive systems in education. Although the main promise of machine-learning models is increased accuracy and objectivity, current incidents have revealed the contrary. For instance, England's A-level and GCSE secondary level examinations were cancelled due to the pandemic in the summer of 2020 [ 1, 57]. An alternative assessment method was implemented to determine the qualification grades of students. The grade standardization algorithm was produced by the regulator Ofqual. With the assessment of Ofqual's algorithm based on schools' previous examination results, thousands of students were shocked to receive unexpectedly low grades. Although a full discussion of the incident is beyond the scope of this article [ 51] it revealed how the score distribution favored students who attended private or independent schools, while students from underrepresented groups were hit hardest. Unfortunately, automated assessment algorithms have the potential to reconstruct unfair and inconsistent results by disrupting student's final scores and future careers [ 53].

Teaching and understanding AI and ethics in educational settings

These ethical concerns suggest an urgent need to introduce students and teachers to the ethical challenges surrounding AI applications in K-12 education and how to navigate them. To meet this need, different research groups and nonprofit organizations offer a number of open-access resources based on AI and ethics. They provide instructional materials for students and teachers, such as lesson plans and hands-on activities, and professional learning materials for educators, such as open virtual learning sessions. Below, we describe and evaluate three resources: "AI and Ethics" curriculum and "AI and Data Privacy" workshop from the Massachusetts Institute of Technology (MIT) Media Lab as well as Code.org's "AI and Oceans" activity. For readers who seek to investigate additional approaches and resources for K-12 level AI and ethics interaction, see: (a) The Chinese University of Hong Kong (CUHK)'s AI for the Future Project (AI4Future) [ 18]; (b) IBM's Educator's AI Classroom Kit [ 30], Google's Teachable Machine [ 25], UK-based nonprofit organization Apps for Good [ 4], and Machine Learning for Kids [ 37].

"AI and Ethics Curriulum" for middle school students by MIT Media Lab

The MIT Media Lab team offers an open-access curriculum on AI and ethics for middle school students and teachers. Through a series of lesson plans and hand-on activities, teachers are guided to support students' learning of the technical terminology of AI systems as well as the ethical and societal implications of AI [ 2]. The curriculum includes various lessons tied to learning objectives. One of the main learning goals is to introduce students to basic components of AI through algorithms, datasets, and supervised machine-learning systems all while underlining the problem of algorithmic bias [ 45]. For instance, in the activity "AI Bingo", students are given bingo cards with various AI systems, such as online search engine, customer service bot, and weather app. Students work with their partners collaboratively on these AI systems. In their AI Bingo chart, students try to identify what prediction the selected AI system makes and what dataset it uses. In that way, they become more familiar with the notions of dataset and prediction in the context of AI systems [ 45].

In the second investigation, "Algorithms as Opinions", students think about algorithms as recipes, which are created by set of instructions that modify an input to produce an output [ 45]. Initially, students are asked to write an algorithm to make the "best" jelly sandwich and peanut butter. They explore what it means to be "best" and see how their opinions of best in their recipes are reflected in their algorithms. In this way, students are able to figure out that algorithms can have various motives and goals. Following this activity, students work on the "Ethical Matrix", building on the idea of the algorithms as opinions [ 45]. During this investigation, students first refer back to their developed algorithms through their "best" jelly sandwich and peanut butter. They discuss what counts as the "best" sandwich for themselves (most healthy, practical, delicious, etc.). Then, through their ethical matrix (chart), students identify different stakeholders (such as their parents, teacher, or doctor) who care about their peanut butter and jelly sandwich algorithm. In this way, the values and opinions of those stakeholders also are embedded in the algorithm. Students fill out an ethical matrix and look for where those values conflict or overlap with each other. This matrix is a great tool for students to recognize different stakeholders in a system or society and how they are able to build and utilize the values of the algorithms in an ethical matrix.

The final investigation which teaches about the biased nature of algorithms is "Learning and Algorithmic Bias" [ 45]. During the investigation, students think further about the concept of classification. Using Google's Teachable Machine tool [ 2], students explore the supervised machine-learning systems. Students train a cat-dog classifier using two different datasets. While the first dataset reflects the cats as the over-represented group, the second dataset indicates the equal and diverse representation between dogs and cats [ 2]. Using these datasets, students compare the accuracy between the classifiers and then discuss which dataset and outcome are fairer. This activity leads students into a discussion about the occurrence of bias in facial recognition algorithms and systems [ 2].

In the rest of the curriculum, similar to the AI Bingo investigation, students work with their partners to determine the various forms of AI systems in the YouTube platform (such as its recommender algorithm and advertisement matching algorithm). Through the investigation of "YouTube Redesign", students redesign YouTube's recommender system. They first identify stakeholders and their values in the system, and then use an ethical matrix to reflect on the goals of their YouTube's recommendation algorithm [ 45]. Finally, through the activity of "YouTube Socratic Seminar", students read an abridged version of Wall Street Journal article by participating in a Socratic seminar. The article was edited to shorten the text and to provide more accessible language for middle school students. They discuss which stakeholders were most influential or significant in proposing changes in the YouTube Kids app and whether or not technologies like auto play should ever exist. During their discussion, students engage with the questions of: "Which stakeholder is making the most change or has the most power?", "Have you ever seen an inappropriate piece of content on YouTube? What did you do?" [ 45].

Overall, the MIT Media Lab's AI and Ethics curriculum is a high quality, open-access resource with which teachers can introduce middle school students to the risks and ethical implications of AI systems. The investigations described above involve students in collaborative, critical thinking activities that force them to wrestle with issues of bias and discrimination in AI, as well as surveillance and autonomy through the predictive systems and algorithmic bias.

"AI and Data Privacy" workshop series for K-9 students by MIT Media Lab

Another quality resource from the MIT Media Lab's Personal Robots Group is a workshop series designed to teach students (between the ages 7 and 14) about data privacy and introduce them to designing and prototyping data privacy features. The group has made the content, materials, worksheets, and activities of the workshop series into an open-access online document, freely available to teachers [ 44].

The first workshop in the series is "Mystery YouTube Viewer: A lesson on Data Privacy". During the workshop, students engage with the question of what privacy and data mean [ 44]. They observe YouTube's home page from the perspective of a mystery user. Using the clues from the videos, students make predictions about what the characters in the videos might look like or where they might live. In a way, students imitate YouTube algorithms' prediction mode about the characters. Engaging with these questions and observations, students think further about why privacy and boundaries are important and how each algorithm will interpret us differently based on who creates the algorithm itself.

The second workshop in the series is "Designing ads with transparency: A creative workshop". Through this workshop, students are able to think further about the meaning, aim, and impact of advertising and the role of advertisements in our lives [ 44]. Students collaboratively create an advertisement using an everyday object. The objective is to make the advertisement as "transparent" as possible. To do that, students learn about notions of malware and adware, as well as the components of YouTube advertisements (such as sponsored labels, logos, news sections, etc.). By the end of the workshop, students design their ads as a poster, and they share with their peers.

The final workshop in MIT's AI and data privacy series is "Designing Privacy in Social Media Platforms". This workshop is designed to teach students about YouTube, design, civics, and data privacy [ 44]. During the workshop, students create their own designs to solve one of the biggest challenges of the digital era: problems associated with online consent. The workshop allows students to learn more about the privacy laws and how they impact youth in terms of media consumption. Students consider YouTube within the lenses of the Children's Online Privacy Protections Rule (COPPA). In this way, students reflect on one of the components of the legislation: how might students get parental permission (or verifiable consent)?

Such workshop resources seem promising in helping educate students and teachers about the ethical challenges of AI in education. Specifically, social media such as YouTube are widely used as a teaching and learning tool within K-12 classrooms and beyond them, in students' everyday lives. These workshop resources may facilitate teachers' and students' knowledge of data privacy issues and support them in thinking further about how to protect privacy online. Moreover, educators seeking to implement such resources should consider engaging students in the larger question: who should own one's data? Teaching students the underlying reasons for laws and facilitating debate on the extent to which they are just or not could help get at this question.

Investigation of "AI for Oceans" by Code.org

A third recommended resource for K-12 educators trying to navigate the ethical challenges of AI with their students comes from Code.org, a nonprofit organization focused on expanding students' participation in computer science. Sponsored by Microsoft, Facebook, Amazon, Google, and other tech companies, Code.org aims to provide opportunities for K-12 students to learn about AI and machine-learning systems [ 20]. To support students (grades 3-12) in learning about AI, algorithms, machine learning, and bias, the organization offers an activity called "AI for Oceans", where students are able to train their machine-learning models.

The activity is provided as an open-access tutorial for teachers to help their students explore how to train, model and classify data, as well as to understand how human bias plays a role in machine-learning systems. During the activity, students first classify the objects as either "fish" or "not fish" in an attempt to remove trash from the ocean. Then, they expand their training data set by including other sea creatures that belong underwater. Throughout the activity, students are also able to watch and interact with a number of visuals and video tutorials. With the support of their teachers, they discuss machine learning, steps and influences of training data, as well as the formation and risks of biased data [ 20].

Future directions for research and teaching on AI and ethics

In this paper, we provided an overview of the possibilities and potential ethical and societal risks of AI integration in education. To help address these risks, we highlighted several instructional strategies and resources for practitioners seeking to integrate AI applications in K-12 education and/or instruct students about the ethical issues they pose. These instructional materials have the potential to help students and teachers reap the powerful benefits of AI while navigating ethical challenges especially related to privacy concerns and bias. Existing research on AI in education provides insight on supporting students' understanding and use of AI [ 2, 13]; however, research on how to develop K-12 teachers' instructional practices regarding AI and ethics is still in its infancy.

Moreover, current resources, as demonstrated above, mainly address privacy and bias-related ethical and societal concerns of AI. Conducting more exploratory and critical research on teachers' and students' surveillance and autonomy concerns will be important to designing future resources. In addition, curriculum developers and workshop designers might consider centering culturally relevant and responsive pedagogies (by focusing on students' funds of knowledge, family background, and cultural experiences) while creating instructional materials that address surveillance, privacy, autonomy, and bias. In such student-centered learning environments, students voice their own cultural and contextual experiences while trying to critique and disrupt existing power structures and cultivate their social awareness [ 24, 36].

Finally, as scholars in teacher education and educational technology, we believe that educating future generations of diverse citizens to participate in the ethical use and development of AI will require more professional development for K-12 teachers (both pre-service and in-service). For instance, through sustained professional learning sessions, teachers could engage with suggested curriculum resources and teaching strategies as well as build a community of practice where they can share and critically reflect on their experiences with other teachers. Further research on such reflective teaching practices and students' sense-making processes in relation to AI and ethics lessons will be essential to developing curriculum materials and pedagogies relevant to a broad base of educators and students.

Funding

This work was supported by the Graduate School at Michigan State University, College of Education Summer Research Fellowship.

Data availability

Not applicable.

Code availability

Not applicable.

Declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Footnotes

Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Selin Akgun, Email: ude.usm@lesnugka.

Christine Greenhow, Email: ude.usm@wohneerg.

References

2. Ali, S. A., Payne, B. H., Williams, R., Park, H. W., Breazeal, C.: Constructionism, ethics, and creativity: developing primary and middle school artificial intelligence education. Paper presented at International Workshop on Education in Artificial Intelligence (EDUAI). Palo Alto, CA, USA. (2019)

3. Almeida D, Shmarko K, Lomas E. The ethics of facial recognition technologies, surveillance, and accountability in an age of artificial intelligence: a comparative analysis of US, EU, and UK regulatory frameworks. AI Ethics. 2021 doi: 10.1007/s43681-021-00077-w. [[ PMC free article] [ PubMed] [ CrossRef] Google Scholar]

6. Krutka, D., Manca, S., Galvin, S., Greenhow, C., Koehler, M., Askari, E.: Teaching "against" social media: confronting problems of profit in the curriculum. Teachers College Record 121(14), 1-42 (2019)

7. Greenhow, C., Galvin, S., Brandon, D., Askari, E.: A decade of research on K-12 teaching and teacher learning with social media: insights on the state-of-the-field. Teachers College Record 122(6), 1-7 (2020)

8. Greenhow, C., Galvin, S., Staudt Willet, K.B.: Inquiring tweets want to know: #Edchat supports for #remoteteaching during COVID-19. British Journal of Educational Technology. [ 1-21, (2021) PMC free article] [ PubMed]

11. Blank, G., Bolsover, G., Dubois, E.: A new privacy paradox: young people and privacy on social network site. Oxford Internet Institute (2014)

16. Chaudhry MA, Kazim E. Artificial intelligence in education (AIEd): a high-level academic and industry note 2021. AI Ethics. 2021 doi: 10.1007/s43681-021-00074-z. [[ PMC free article] [ PubMed] [ CrossRef] Google Scholar]

18. Chiu TKF, Meng H, Chai CS, King I, Wong S, Yam Y. Creation and evaluation of a pre-tertiary artificial intelligence (AI) curriculum. IEEE Trans. Educ. 2021 doi: 10.1109/TE.2021.3085878. [[ CrossRef] Google Scholar]

23. Tiple, Vasile, Recommendations on the European Commission's WHITE PAPER on Artificial Intelligence - A European approach to excellence and trust, COM(2020) 65 final (the 'AI White Paper') (2020). 10.2139/ssrn.3706099

27. Henderson, P., Sinha, K., Angelard-Gontier, N., Ke, N. R., Fried, G., Lowe, R., Pineau, J.: Ethical challenges in data-driven dialogue systems. In: Proceedings of AAAI/ACM Conference on AI Ethics and Society (AIES-18), New Orleans, Lousiana, USA. (2000)

32. Iman, M., Arabnia, H. R., Branchinst, R. M.: Pathways to artificial general intelligence: a brief overview of developments and ethical issues via artificial intelligence, machine learning, deep learning, and data science. Paper presented at International Conference on Artificial Intelligence (ICAI). Las Vegas, Nevada, USA. (2020)

35. Krueger, N.: Artificial intelligence has infiltrated our lives. Can it improve learning? International Society for Technology in Education (ISTE). (2017).

42. Murphy, R. F.: Artificial intelligence applications to support k-12 teachers and teaching: a review of promising applications, challenges, and risks. Perspective. 1-20 (2019). 10.7249/PE315

48. Regan, P. M., Steeves, V.: Education, privacy, and big data algorithms: taking the persons out of personalized learning. First Monday, 24(11), (2019)

57. Weale, S., Stewart, H.: A-level and GCSE results in England to be based on teacher assessments in U-turn. The Guardian, (2020).

Articles from Ai and Ethics are provided here courtesy of Nature Publishing Group

# data[].attributes.attachment_link.title
Artificial intelligence in education: Addressing ethical challenges in K-12 settings

# data[].attributes.attachment_link.description
Artificial intelligence (AI) is a field of study that combines the applications of machine learning, algorithm productions, and natural language processing. Applications of AI transform the tools of education. AI has a variety of educational applications, ...

# data[].attributes.attachment_link.readable_text
Ethics of AI in Education: Towards a Community-Wide Framework

Abstract

While Artificial Intelligence in Education (AIED) research has at its core the desire to support student learning, experience from other AI domains suggest that such ethical intentions are not by themselves sufficient. There is also the need to consider explicitly issues such as fairness, accountability, transparency, bias, autonomy, agency, and inclusion. At a more general level, there is also a need to differentiate between doing ethical things and doing things ethically, to understand and to make pedagogical choices that are ethical, and to account for the ever-present possibility of unintended consequences. However, addressing these and related questions is far from trivial. As a first step towards addressing this critical gap, we invited 60 of the AIED community's leading researchers to respond to a survey of questions about ethics and the application of AI in educational contexts. In this paper, we first introduce issues around the ethics of AI in education. Next, we summarise the contributions of the 17 respondents, and discuss the complex issues that they raised. Specific outcomes include the recognition that most AIED researchers are not trained to tackle the emerging ethical questions. A well-designed framework for engaging with ethics of AIED that combined a multidisciplinary approach and a set of robust guidelines seems vital in this context.

Introduction

It is almost certainly the case that all members of the Artificial Intelligence in Education (AIED) research community are motivated by ethical concerns, such as improving students' learning outcomes and their lifelong opportunities. However, as has been seen in other domains of AI application, ethical intentions are not by themselves sufficient, as good intentions do not always result in ethical designs or ethical deployments (e.g., Dastin 2018; Reich and Ito 2017; Whittaker et al. 2018). Significant attention is required to understand what it means to be ethical specifically in the context of AIED. The educational contexts which AIED technologies aspire to enhance highlight the need to differentiate between doing ethical things and doing things ethically, to understand and to make pedagogical choices that are ethical, to account for the ever-present possibility of unintended consequences, along with many other considerations. However, addressing these and related questions is far from trivial, not least because it remains true that " no framework has been devised, no guidelines have been agreed, no policies have been developed, and no regulations have been enacted to address the specific ethical issues raised by the use of AI in education" (Holmes et al. 2018, p. 552).

As a first step towards addressing this critical gap, we invited 60 of the AIED community's leading researchers (operationalised mainly by citations and/or reputation) to respond to a survey of questions about ethics and the application of AI in educational contexts.

The 17 respondents (in alphabetical order by surname) were: Vincent Aleven (CMU, USA), Ig Ibert Bittencourt (UFAL, Brazil), Jesus Boticario (UNED, Spain), Simon Buckingham Shum (UTS, Australia), Mutlu Cukurova (UCL, UK), Ben du Boulay (University of Sussex, UK), Janice Gobert (Rutgers, USA), Ken Holstein (CMU, USA), Judy Kay (University of Sydney, Australia), Ken Koedinger (CMU, USA), Bruce McLaren (CMU, USA), Ma. Mercedes T. Rodrigo (Ateneo de Manila University, Philippines), Olga C. Santos (UNED, Spain), Eileen Scanlon (OU, UK), Mike Sharples (OU, UK), Erin Walker (University of Pittsburgh, USA), and Beverly Woolf (UMASS, USA).

In this paper, we first summarise the background (including the ethics of AI in general, and the ethics of educational data) and introduce the ethics of AI in education. Next, we outline the methods used in this project, before going on to summarise the survey responses (NB all responses quoted in this paper are verbatim, with any words that have been added or deleted for clarity marked by square parentheses or ellipses as appropriate). We finish with a discussion of the issues raised by the survey, and a suggestion for next steps.

Background

The AIED conferences in 2018 and 2019 hosted a workshop called "Ethics of AIED. Who Cares?" (Holmes et al. 2019a; Holmes et al. 2018). Although the workshop discussions were engaging and productive, the small number of attendees suggested a disappointingly low level of interest in the topic from the broader AIED community. While other AI communities are increasingly attending to ethical considerations around the design and deployment of AI-based technologies, ethical dimensions of AIED do not yet appear to be a central area of focus for many in the AIED community. Since Aiken and Epstein published their ethical guidelines two decades ago to begin a conversation around the ethics of AIED (Aiken and Epstein 2000), there has been a striking paucity of published work in the AIED community that explicitly focuses on ethics. In any case, the potential impact of AIED designs and methods of deployment on students, teachers and wider society appears yet to be fully worked out.

Nonetheless, it is generally accepted within the community that AIED raises far-reaching questions with important ethical implications for students, educators, parents, policymakers, and other stakeholders. Ethical concerns permeate many of the community's core interests, including but not limited to: accuracy of diagnoses of learners interacting with AIED systems; choices of pedagogies employed by AIED systems; predictions of learning outcomes made by those systems; issues of fairness, accountability, and transparency; and questions related to the influence of AI and learning analytics on teachers' decision making.

In many ways, AIED is itself a response to each of those issues and more - with most being addressed, one way or another, in work conducted in the varied research subdomains of AIED. For example, questions around data ownership and control over their interpretations have long been recognised as critical in AIED (e.g., in the context of open learner modelling: Bull and Kay 2016; Conati et al. 2018). However, what is currently missing is a basis for the meaningful ethical reflection necessary for innovation in AIED, to help researchers determine how they might best contribute towards addressing those challenges explicitly. This requires a deep engagement both with recent ethics related debates and frameworks in other AI research subdomains, and with ethical questions specific to the AIED subdomain itself (e.g., issues around pedagogy), many of which remain unasked and unanswered (although there are some exceptions - e.g., Aiken and Epstein 2000; Friedman and Kahn 1992; Holmes et al. 2019b; Sharkey 2016).

The Ethics of AI in General

As with any transformative technology, some AI applications may raise new ethical and legal questions, for example related to liability or potentially biased decision-making. The ethics of artificial intelligence in general has received a great deal of attention, by researchers (e.g., Boddington 2017; Floridi 2019; Jobin et al. 2019; Whittaker et al. 2018; Winfield and Jirotka 2018) and more widely (e.g., the European Union 2019; the UK's House of Lords 2018; UNESCO 2019; and the World Economic Forum 2019), with numerous other AI ethics initiatives emerging in recent years (e.g., Ada Lovelace Institute 2019; AI Ethics Initiative 2017; AI Now Institute 2017; DeepMind Ethics & Society 2017; Future of Life Institute 2013; The Institute for Ethical AI & Machine Learning 2018).

All of these efforts principally focus on data (involving issues such as informed consent, data privacy, and biased data sets) and how that data is analysed (involving issues such as biased assumptions, transparency and statistical apophenia - finding patterns where no meaningful, causal patterns are present). The Montréal Declaration for Responsible Development of Artificial Intelligence ( 2018), for example, offers a comprehensive approach involving ten human-centred principles, encompassing: well-being, respect for autonomy, protection of privacy, solidarity, democratic participation, equity, diversity, prudence, responsibility, and sustainable development. No such declaration currently exists for the specific issues raised by AIED.

The Ethics of Educational Data

Similarly, the ethics of educational data and learning analytics has also been the focus of much research (e.g., Ferguson et al. 2016; Slade and Prinsloo 2013; Potgieter 2020). This work is extensive, far too wide to summarise here; however, some key issues can be noted. First, because the field is still emerging, exactly what an ethics of learning analytics should include remains the subject of debate: 'the ethical and privacy aspects of learning analytics are varied, and they shift as the use of data reveals information that could not be accessed in the past' (Ferguson et al. 2016, p. 5). Second, the ethics of learning analytics involves several types of questions, including but not limited to: informed consent and privacy, the interpretation of data, the management of data, and perspectives on data (e.g., institutional versus individual); as well as on much broader issues such as power relations, surveillance, and the purpose of education (Slade and Prinsloo 2013). Third, it has been argued that 'educational data mining [...] is not the superconductor of truth that some of its proponents believe [...] and the transformative impact that it will have on the autonomy of learners is cause for concern' (Potgieter 2020, pp. 3, 6).

The learning analytics community have endeavoured to agree principles against which learning analytics research and practice can judge itself and be judged. The DELICATE checklist, for example, comprises guidance centred on determining added value and the rights of participants, being open about intentions and objectives, legitimising the collection of data, involving all stakeholders including the data subjects, ensuring consent is genuinely informed and freely given, ensuring that data is truly anonymised, establishing and implementing procedures to guarantee individual privacy, and adopting clear and transparent obligations with any external agencies that might be involved with the data (Drachsler and Greller 2016). In fact, the clear overlaps between learning analytics and AIED, that are centred on educational data, suggest that an ethics of AIED might usefully draw on approaches such as the DELICATE checklist. However, there are also clear differences between the two fields, "with an emphasis on agents and tutors for AIED, [...] and visualization for LA" (Labarthe et al. 2018, p. 70). This active engagement/passive representation distinction, although radically oversimplified, suggests that a comprehensive ethics of AIED is likely to have additional requirements.

The Ethics of Other Related Research Areas

Ethics have also received attention in related research areas such as user modeling, e-learning environments, and intelligent agents. For example, in the user modeling community, issues centred on security and privacy (e.g., Schreck 2003), inspectability (e.g., Zapata-Rivera and Greer 2004), and privacy (e.g., Kobsa 2007) have long been considered. Most recently, dedicated workshops have been held at leading conferences (e.g., FairUMAP, Mobasher et al. 2020), with contributions exploring issues such as fairness (Sacharidis et al. 2020), transparency (Schelenz et al. 2020), and biases (Deshpande et al. 2020). Researchers in e-learning are also interested in ethical issues raised by the systems that they build; issues such as equity and diversity, surveillance and consent, identity and confidentiality (e.g., Anderson and Simpson 2007), and student privacy: "when an observer [which could be an automated system] monitors someone's behavior with full knowledge of their identity, the person being monitored does not enjoy any privacy" (Anwar and Greer 2012, p. 63). However, it is notable that other researchers in e-learning appear more interested in the ethical practices of the students, such as cheating and fraud (e.g., Gearhart 2012). Finally, researchers and developers of intelligent agents such as chatbots are also increasingly focusing on the ethical issues raised by their work (e.g., Murtarelli et al. 2020; Richards and Dignum 2019). Most recently, the World Economic Forum has launched 'RESET', a framework and set of principles for governing responsible use of conversational AI (WEF 2020) - which, although focused on healthcare, have self-evident potential for the ethics of intelligent agents in education. The RESET principles focus on safety/non-maleficence, efficacy, data protection, human agency, accountability, transparency, fairness, explainability, integrity and inclusiveness.

The Ethics of AIED in Particular

As with AI in general, concerns exist about the large volumes of data collected to support AIED (such as the recording of student competencies, inferred emotional states, strategies and misconceptions). Who owns and who is able to access these data, what are the privacy concerns, and who should be considered responsible if something goes wrong?

Other major AIED ethical concerns, again as with AI in general, centre on the computational approaches. How should the data be analysed, interpreted, shared and acted upon? How should the biases (conscious or unconscious), that might impact negatively on the civil rights of individual students, be prevented or ameliorated - especially given that the scale of AIED in the coming years is likely to amplify any design biases (e.g., about gender, age, race, social status, income inequality...)? Finally, as the Facebook and Cambridge Analytica data scandal showed, data is vulnerable to hacking and manipulation: 'it's impossible to have personal privacy and control at scale, so it is critical that the uses to which data will be put are ethical - and that the ethical guidelines are clearly understood' (Tarran 2018, pp. 4-5).

However, the ethics of AIED cannot be reduced to questions about data or computational approaches alone (Holmes et al. 2019b). AIED research also needs to account for the ethics of education, which, although the subject of decades of research, is all too often overlooked. For example, AIED research needs to address explicitly issues such as (1) the purpose of the learning (e.g., to prepare students to pass exams or to help them self-actualise), (2) the choice of pedagogy (with a common approach, instructionism, being contested by the learning sciences community), (3) the role of the technology with respect to teachers (to replace or augment human functions), and (4) access to education (often seen by the community through the ethical dimension of fairness and equity). In addition, there remains limited research into what teachers and students actually want from AIED systems - such as requirements around student agency and privacy about which teachers and students might not agree (Holstein et al. 2019). Furthermore, where AIED interventions target behavioural change (such as by 'nudging' individuals towards a particular course of action), the entire sequence of AIED enhanced pedagogical activity needs to be ethically warranted in the context of the broader activities within which AIED systems are being deployed.

To highlight just some of the potential breadth of issues, AIED ethical questions include:

How does the transient nature of student goals, interests and emotions impact on the ethics of AIED?

How can K12 students give genuinely informed consent for their involvement with AIED tools?

What are the AIED ethical obligations of private organisations (developers of AIED products) and public authorities (schools and universities involved in AIED research)?

How might schools, students and teachers opt out from, or challenge, how they are represented in large datasets?

What are the ethical implications of not being able to easily interrogate how some AIED deep decisions (e.g., those using multi-level neural networks) are made?

What are the ethical consequences of encouraging students to work independently with AI-supported software (rather than with teachers or in collaborative groups)?

Notably, as mentioned above, some guidelines were proposed almost 20 years ago (Aiken and Epstein 2000), but have not been widely adopted by the AIED community. Aiken and Epstein start with a negative and a positive meta-principle related to the impact of AIED on 'dimensions of human being'; with the dimensions ethical, aesthetic, social, intellectual, and physical, together with psychological traits such as 'the individual's ability to lead a happy and fulfilling life'. Aiken and Epstein's negative meta-principle is that '(AIED) technology should not diminish the student along any of the fundamental dimensions of human being.' The positive meta-principle is that ' AIED technology should augment the student along at least one of the fundamental dimensions of human being.' They go on to provide and discuss ten 'fundamental principles' for educational technologies that incorporate AI methods. Some of these (such as 'avoid information overload') are essential tenets of good user experience and effective pedagogy for most educational technologies. The principles that are most specific to the application of AI in education are: ' 7. Develop systems that give teachers new and creative roles that might not have been possible before the use of technology. Systems should not attempt to replace the teacher.' and ' 10. Avoid glorifying the use of computer systems, thereby diminishing the human role and the human potential for learning and growth. '

Method

Broadly, the aim of this work is to galvanise the AIED research community's engagement with the ethics of its domain. Accordingly, it was first important to establish community members' existing beliefs and understandings of the ethics of AI applied in educational contexts, to which end a survey of AIED researchers was developed and conducted. The survey questionnaire comprised 10 substantive open questions (Appendix 1), which covered the following themes: (i) what it means for AIED researchers to consider the ethics of their work, (ii) pertinent AI and education ethical issues, and (iii) what distinguishes the ethics of AIED from the ethics of other AI domains. The research was approved by the Open University's Human Research Ethics Committee (HREC/3339/Holmes: 02/08/2019).

Potential invitees for the survey were identified by reviewing contributions to the International Journal of Artificial Intelligence in Education and the proceedings of the International Conference on Artificial Intelligence in Education, both from the last 10 years. The names of researchers with the most citations, together with those researchers otherwise known by reputation (e.g., they hold or have held a central role in the International Artificial Intelligence in Education Society), were collated. To this list were added some mid- and early-career researchers who had already made a noteworthy impact in the community (e.g., they had received a Best Paper award at an AIED conference). In total, the list comprised 60 candidate survey invitees, all of whom were invited to participate in the survey. An invitation email and up to three reminder emails were sent to each invitee, 17 of whom completed the questionnaire.

At the beginning of the questionnaire, respondents were asked to give explicit permission for their names to be given in the proposed paper, and for any of their responses included in the paper to be attributed to them (this approach draws on Guideline 41 of the British Educational Research Association's Ethical Guidelines for Educational Research 2018). This was both to ensure transparency and to maximise impact on early- and mid-career AIED researchers. All respondents gave their permission. In addition, before the paper was submitted for publication, the respondents were given the opportunity to check, amend, agree, and/or delete any quote that was attributed to them. No respondent requested any substantive change. Finally, to further acknowledge their contribution, the respondents were given the opportunity to be named as a co-author of the paper. Six respondents took this up (Holstein and Buckingham Shum also made substantial contributions to the writing of the paper).

Results

Paying Attention to Ethics in AIED

However, five out of 17 respondents reported that they perceive growing urgency and attention to ethical issues within the AIED research community. For example, Scanlon noted that "there is much discussion now of ethics because of high profile examples in AI which are hitting the media [...] ethical consideration[s] are becoming not just part of the academic community's discourse but part of wider public discussion." Similarly, Bittencourt suggested that "since [there is so much] news and arguments about the implications of AI, researchers are starting to think about ethics in AIED." Buckingham Shum framed growing concerns around the ethics of AIED as a consequence of the field's growth in real-world applications: "It is only relatively recently with 'adaptive products' hitting the mainstream that AIED has found itself being touched by the broader public spotlight. Until then, it was an academic community, with such a small user base that nobody was asking ethics questions."

Only one participant out of 17 outright disagreed that AIED researchers pay insufficient attention to ethics. However, this participant's response focused exclusively on AIED research ethics, noting that institutional ethics boards in university settings ensure that researchers consider certain ethical dimensions of their research practice. Four of 17 respondents described themselves as "on the fence," or stated that they agreed only "to an extent." Koedinger noted that "many of us got into AIED because we wanted to improve education, and especially for students who most need the help of free public education," characterizing the pursuit of this goal as "doing ethics, which goes beyond research about [ethics]." Aleven similarly said that "one suspects that many AIED researchers are fundamentally motivated by ethical concerns," while acknowledging that "so far the field has not often addressed [ethics] explicitly or discussed it publicly." Other respondents pointed out that "this is a problem inherent to many fields of technology research and not solely AIED" (Walker). In line with this view, Sharples characterized the practice of AIED as being "primarily led by technology," noting that "its impacts on stakeholders, including direct users, those indirectly affected and society at large has been largely ignored." Finally, some respondents commented that this is a complex question to answer. McLaren noted that it may be difficult to make generalizations across the whole of the AIED community, as "some researchers are very cognizant and sensitive to ethical issues [...] others, not as much." Other respondents emphasised that the AIED community has attended to certain aspects of ethics (e.g., data privacy and research ethics), although historically it has attended less to others (Rodrigo).

A number of respondents emphasised that AIED researchers have a responsibility to make their "research more accessible, and more transparent" (Buckingham Shum), and to "be particularly mindful of the ways that people interpret our results" (Kay). As Gobert noted, "generalizing results beyond the demographics for whom the study was designed could mean that students might get interventions that are not suitable for them." du Boulay distinguished between "two main areas of concern" for AIED research: "the danger of harm to participants," including (1) "harm arising from decisions that the AIED system might make," and (2) "the careful curation of data derived from the research," including the related issue of "the models of individuals that the AIED system might build." Other responses focused specifically on the unique sensitivity of the data collected and used by AIED systems. For example, Boticario said "Data are no longer data because there is so much information available on anyone. Data are people themselves and should be treated as such [...] we are [providing] personalised support to our students based on [an] increasing body of data, which is starting to include personal and affective information. This information is by its nature sensitive and demands a careful consideration and management." Finally, one respondent noted that the need for AIED researchers to attend to the ethics of their work had been "argued in a journal article [in] IJAIED back in 2000," (Aiken and Epstein 2000) and asked, "how has it changed?"

Most Important Ethical Issues for AIED

In addition, several respondents identified properties of AI in education which, in combination together, make education distinct to other sectors: " All the generic concerns around big data and algorithms apply to education. What's more interesting are the features that make the formal education system, and learning as a phenomenon, distinctive from the other sectors of society being impacted by data science" (Buckingham Shum). These issues relate not just to the ethics of the development of AIED tools in and of themselves, but to their educational context - when, how, and to what end, an AIED tool is being used.

Unsurprisingly, the implications of AIED for teaching and learning was central to many of the respondents' concerns. The " quality of education" (Aleven) enabled by AIED tools was highlighted. There was a recognition of the pedagogical choices inherent within the design of AIED tools, with one respondent emphasising a desire for " designing a pedagogy that empowers learners and teachers" (Sharples), while another arguing that the scope of many AIED tools remains narrow and therefore there may be elements of education that are better off without AIED, asking: " What do we risk losing through automation, and are we prepared to preserve those aspects that should be preserved (e.g., intentionally 'not' automated)? Are we diminishing the quality of education, overall, by conveying that we have reached human parity in AI-based teaching systems (but failing to emphasise that we have done so on a very narrow set of teaching tasks, whereas good education involves much more than this)? " (Holstein).

A final group of concerns related to the ethics of AIED research, rather than its applications and uses. These covered the need to treat research subjects - particularly children - ethically and fairly (McLaren), reporting results while respecting confidentiality and anonymity, particularly given potential to re-identify data (Santos, Boticario), and data collection (Gobert). In addition, the potential for AIED research to be (mis)used by companies was identified: ' How will our outputs be used (or misused) by more profit-driven entities? How do we manage misuses of our outputs? ' (Holstein).

Most Ignored Yet Critical Ethical Issues for AIED

Finally, issues were raised over AIED systems and human agency, for example "How to embed pedagogies of learner and teacher empowerment into AIED systems" (Sharples). Areas that need specific attention are: "The ongoing misplaced belief that AIED systems can wholly replace human teachers and still enable effective and socially responsible learning." (du Boulay), "The lack of human agency in fully autonomous AI in Education systems" (Cukurova), and "What [...] we risk losing through automation, [...] are we prepared to preserve those aspects that should be preserved (e.g., intentionally 'not' automated)?" (Holstein).

What Distinguishes the Ethics of AIED from Other AI Applications?

Survey respondents were asked to reflect on what distinguishes the ethics of AIED from other AI applications/domains and to identify the most important 'educational' ethical issues for AIED. The majority of responses identified how issues common to the 'ethics of AI' manifest in educational contexts - particularly issues of consent, transparency, control over data and algorithmic bias. In particular, the application of AI to children and young people was noted as a distinguishing feature of AIED (e.g., McLaren). This was not only associated with issues of consent, but also the longer-term individual and social impacts of engaging with young people during a formative period: "We are influencing impressionable young people, and that comes with a moral obligation to be as correct and proper as possible" (Rodrigo). Despite a general recognition of the potential long-term impacts of AIED, two respondents (du Boulay, Santos) also noted that AIED systems do not make the same 'life and death' decisions typical of other AI applications, such as autonomous vehicles or medicine: "a bad treatment in learning might imply more effort from the student or more time, but it does not seem as critical as a bad treatment in medicine or a bad decision in an autonomous car, which can cause death" (Santos). This reveals a tension between how researchers frame the ethical issues and potential 'harms' associated with AIED.

Another issue noted by the respondents was the value of transparency in AIED systems, with one respondent suggesting that teachers might prefer an AI system that is less accurate but offers more transparency around decision-making " over an AI system that is perfectly accurate but not transparent" (Cukurova). Others referred to the importance of giving learners agency and control over their data, and the potential impact of algorithmic bias on different sub-populations and demographics of learners. There was particular concern that biased data sets have the potential to benefit more advantaged groups of learners: "Instructional strategies that aim to benefit all learners might disproportionately benefit more advantaged groups of learners. Our community needs to learn to integrate fairness-promoting algorithms into the system and deploy them for real students" (Woolf).

Markedly less common were responses that identified issues specific to the ethics of education, and how these intersect with the ethics of AI more generally. Choice of pedagogy was mentioned by a few respondents, who acknowledged that AIED embodies assumptions about which instructional strategies are 'best' for learners. For example, "pedagogy driven AI in Education is often considered as good regardless of [whether] the pedagogy chosen to drive AI is an appropriate one or not in the first place" (Cukurova). One respondent also referred to issues relating to the accuracy and validity of learning assessments more generally: "classifying students in terms of educational tests has to consider the inherent ambiguity and variability in the measure, but for computer scientists we usually consider them as rigid labels" (Santos). Brief reference was also made to the particular responsibilities of educators and the dependent relationship between learners and educators/teaching systems.

Advice for Teachers, Faculty Members and Professors about Teaching the Ethics of AIED

When asked for their advice on how best to teach AIED students to properly address the ethics of AIED, some respondents acknowledged that they did not feel sufficiently qualified: "To be quite honest, I do not know.... I would like to note that very few of us are trained in ethics!" (Aleven). "I feel like I need to better educate myself about this (and in fact, I joined a committee to introduce an ethics curriculum in my department in part to try to better inform myself)" (Walker). One suggestion for how this might be addressed was to: "bring together voices from academics who study ethics, those who build systems and those who will use them." (Scanlon). Aleven broadly agreed: " To properly address ethical issues, we probably need to look at a bigger picture than we are used to doing as AIED researchers. And [...], we should avoid armchair ethics and work with experts in ethics". Nonetheless, a starting point might be for academics to 'practise what they preach': "Ethics need to be modeled in order to be taught. We academics have to conduct ourselves and our research practices as ethically as we know how" (Rodrigo).

A common thought was that students would learn best through direct experience: "Don't just 'build ethics into the curriculum'. Have students learn by doing. Make ethical considerations a core part of the evaluation criteria (e.g., for student deliverables in project-based courses). Make class discussions around ethics part of the actual formative evaluation / feedback process for student projects" (Holstein). "I would also suggest that they should give actual hands-on experience with AI systems, let their students build, test, play with AI tools in controlled conditions to understand potential ethical issues through experience" (Cukurova). "Introduce them to accessible frameworks (such as those beginning to emerge in AI FATE [e.g., https://facctconference.org] work) and develop exercises that help people to apply them. This will in turn refine them" (Buckingham Shum) .

Other respondents took this approach an intriguing step further: students might benefit from experience of AIED tools that have been developed without addressing any ethics: "I think it could be an interesting approach to ask them [to] be part of a study where ethics are not considered, so they can perceive the need to take care of it" (Santos). "Mak[e] them investigate a scenario where those principles are not followed and have them research all possible consequences. This way they could figure out not just the basic principles involved but also possible flaws or missing considerations" (Boticario).

Advice for Early- and Mid-career Researchers About the Ethics of AIED

Survey respondents were asked what advice they would give to early- and mid-career researchers about the ethics of AIED. The 16 responses to this question encouraged early- to mid-career researchers to take ethics seriously (Bittencourt, Boticario, du Boulay, Gobert, Holstein, Koedinger, Rodrigo, Sharples, Woolf), to actively participate in discussions about what ethical practice in AIED should look like (Aleven, Kay, Scanlon, Sharples), to critically reflect upon and question their own research practices (Boticario, Cukurova, Holstein, Walker), and to engage with relevant expertise outside of the AIED community (Buckingham Shum, Cukurova, Santos).

Sharples encouraged early- to mid-career researchers to actively contribute to the "development of a code of ethical practice in AIED," and to abide by this code. A number of respondents urged researchers towards more critical reflection around their own research practice. Walker advised researchers to "think about the implications of what you're doing and be able to critically engage with them." Similarly, Boticario said, "we have the *responsibility* to keep our eyes open and respond to what is happening in the world around us. As AIED researchers we are constantly changing the trajectories of others' lives. [...] If we cannot take responsibility for our actions, we do not deserve to remain in this field." Holstein advised that "we are a generation [of researchers] that cannot afford to ignore the ethics of AIED," noting that "generations that are trying to push an innovative technology into the world have the luxury of thinking primarily about hypotheticals," whereas "the generation that carries forward this work once the systems are already being [used] and impacting lives at scale" must reckon with more immediate ethical concerns.

Finally, several respondents urged early- to mid-career AIED researchers to expand beyond the AIED literature, and to engage with the broader research community outside of AIED. For example, Buckingham Shum encouraged AIED researchers to "consider engaging with academics in critical data studies and tech-ethics." Similarly, Cukurova encouraged researchers to "read fundamental philosophy as most ethical questions come down to philosophical discussions and frameworks that are driven by certain values."

Should the IJAIED Require an Ethics Statement for all Articles?

Respondents were asked whether they agreed with the suggestion that all articles published in this journal should be accompanied by a statement explaining how the authors have fully accounted for the ethics of their AIED work. Of the 16 respondents to this question, only two disagreed, but mainly because "the ethics of AIED are not yet defined sufficiently to make this useful or viable" (du Boulay). This was echoed by many of the respondents who did agree (Bittencourt, Boticario, Buckingham Shum, Gobert, Kay, Sharples, Walker), who for example pointed out that "we need much more nuanced conceptions of what 'ethical' means" (Buckingham Shum), and that the journal would first need "to indicate clearly what is meant by 'fully accounted for the ethics'" (Sharples).

Nonetheless, the respondents were broadly in favour. Santos, for example, noted "I think it would help to get more awareness and try to do things right. Maybe it can be asked to have a short section at the end of the paper that the authors explain in their own words how they have covered the ethics in the research reported. It would be something similar to the bios some journals have, or the highlights section." Other tangible suggestions included: "Perhaps the Journal can ask authors to upload a copy of their ethics clearance" (Rodrigo), and "Authors should be required to provide statements of both positive and negative broader impacts, along with a discussion of trade-offs. Reviewers should then be specifically asked to evaluate these statements, and should be incentivized not to let shallow, 'ticking the box'-style slide." (Holstein).

Woolf brought the discussion back to AIED specifics: "Researchers should determine whether the current adaptive tutor is biased in every applied paper published. For [example,] examine the algorithm or the model or knowledge tracing used to track student progress. Does an algorithm select problems that allow students to demonstrate skill mastery before moving on to new content? All authors should evaluate whether their teaching system learned on training data that introduced bias along certain socio-demographic categories."

Discussion

The survey results presented in this paper indicate that AIED researchers recognise the importance and value of engaging with the ethics of their work (e.g., "We need to be sure that our work is of both the highest standards of research and the highest standards of ethics." McLaren). Nonetheless, there are nuances of opinion with respect to what this might include and how it might be best achieved. With this in mind, in this section, we first summarise key emergent themes and identify some issues that we believe are missing from the discussion; then we propose a draft of an initial AIED ethics framework to help galvanise further debate.

As noted previously, there is no doubt that all respondents do wish to explore, better understand and engage with the ethics of the design and application of AI in educational contexts. However, some respondents appear to believe that we, as a community, are already 'doing ethics' by virtue of operating with best intentions in the educational domain, which is in and of itself ethical (e.g., "Many of us got into AIED because we wanted to improve education [...]. Pursuing that goal is doing ethics [...]. Yet, definitely, we could do more." Koedinger). However, the reality is that "no ethical oversight is required to deploy an elearning system or an AIED system as part of the normal teaching process" (du Boulay).

Clearly, good intentions are not by themselves sufficient. As was acknowledged by several respondents, we need to understand more precisely the ethical risks and to be always on the lookout for unintended consequences that relate specifically to the pedagogical designs (including their readiness for real-world use) that are encapsulated in the AI systems we develop and deploy (e.g., "One could envisage a time when judgements about progress through educational stages might be taken by AIED systems and these might be seriously damaging to individuals' progress and well-being if they were wrong." du Boulay).

The precision of such an understanding needs to be expressed in an actionable code of best practice that the community can rely on in designing and deploying AIED technologies in diverse educational contexts. However, although there appears to be a clear appetite for some kind of ethics of AIED framework that would build upon university research ethics, the community also recognises that such a framework also needs to be distinct from the generally established research ethics approvals and procedures. In particular, such a framework would need to incorporate guidance addressing the many issues raised by respondents (including fairness, accountability, transparency, bias, autonomy, agency, and inclusion) specifically at the intersection of AI and the learning sciences, and ensure that AIED is ethical by design and not just by intention.

In this context, the respondents were also cognizant of the problems that such a framework might itself inadvertently entail. In particular, in line with Bietti ( 2020), having such a framework might both stifle innovation and lead to accusations of, or actual, in-house "ethics washing". So, while the community recognises the critical importance of AIED's specific code of ethical practice, what is meant by ethical AIED remains an open question (e.g., "There is no agreement on the guiding principles." Boticario). Thus, establishing a useful framework for the ethics of AI in education, although desirable and potentially useful, is likely to be a challenging and long-term task. Specifically, given the diversity of interpretations within the community, as indicated in the survey, it is likely that as well as incorporating presently known ethics-related issues and challenges, any ethics framework developed for the AIED context will need to allow for a degree of flexibility to incorporate new knowledge, new understandings and new ways of supporting learning and teaching, as our science, socio-cultural norms, values, and educational systems change over time (e.g., Porayska-Pomsta and Rajendran 2019).

A related important point that emerged from the survey is the observation that AIED is not merely an empirical science but also a design science (Holstein). We understand this to mean that AIED operates in both the theoretical and practical spheres of AI and the learning sciences, and further that as a design science it aims to create educational tools and interventions that are adapted to the demands of different specific users and contexts of use. Highlighting the practical and human-centric orientation of AIED brings into focus the fact that AIED is explicitly concerned with designing interventions which by definition aim to foster positive behaviour change in the users. This is especially important given that many of the intended users of AIED are at pivotal points in their intellectual development.

Adopting a design science orientation also points to another practical outworking of ethical principles - namely, the use of human-centred design methods that give stakeholders genuine agency in shaping digital tools, thus increasing the chances of producing tools that are usable, effective, organisationally acceptable, and ethically sound. However, the AIED community also has to be aware that, despite the best of intentions, human-centred design may itself, in some senses, be sometimes harmful to some users (Norman 2019). Nonetheless, a community is emerging around the critical adoption of human-centred design methods and tools for educational technologies powered by analytics and AI (e.g., Buckingham Shum et al. 2019). In common with the survey reported in this paper, they ask what makes the specific stakeholders and contexts of teaching and learning distinctive from other domains, that raise specific challenges for human-centred design.

All of these considerations further highlight related ethical implications for AIED. For example, "The system forms an explicit or implicit model of the learner's knowledge and skills that it uses to make teaching interventions. That model may embody assumptions by the AIED designer as to appropriate methods of teaching and the abilities of the learner." (Sharples). Thus, AIED's explicit aim to foster behaviour change in its users, adds additional pressure on the community, to embrace ethics and the related dimensions both as a necessity and as a moral obligation for the community. The flip side of this is that the field (alongside related fields such educational data mining, learning analytics, and user modelling) has a real potential to influence the educational systems at the frontline, precisely because it designs for and deploys in real-world contexts, and to contribute to broader approaches of designing for and deploying AI in other than educational human contexts.

In particular, it might be useful to consider how AIED research may contribute to broader debates about the ways in which AI might impact human cognition, decision-making and functioning. Given AIED's focus on human learning and development, it is at least worth considering its potential role in informing those broader debates from its unique perspective of designing AI specifically for the purpose of influencing human cognition. AIED's ambition to support human learning and the field's proximity to the learning sciences, educational neuroscience and educational practice, likely affords a very human-centric, human-developmental understanding of ethics and related dimensions of fairness, transparency, accountability etc., than is afforded by the more general socio-political and legal considerations in the context of other AI subfields.

Furthermore, as well as contributing to the broader ethics of AI debates, there is a more specific question related to whether or not AIED as community might serve as a guiding discipline with respect to ethics best practices in non-academic areas such as the educational technology industry - something that Blikstein touched upon during his AIED 2018 conference keynote speech. Deciding how this might work best is non-trivial. To begin with, it might be worth considering a distinction between "the ethics of AIED research" (which should be informed by standards such as APA 2014 and BERA 2004), "the ethics of AIED tools and practices in and outside of classrooms" (for which, as we have noted, little has been published), and "the ethics of commercial AIED products" (for which nothing has so far been seen). For each of these, the community might benefit from drawing on the experience of other communities in related areas. In web accessibility research, for example, researchers have critiqued approaches that appear reasonable but that embody counterproductive assumptions that need to be problematised (e.g., Lewthwaite 2014).

The various stakeholders (developers, educators, policymakers) all need to be provided key information about the pros and cons of specific AIED technologies - perhaps something in the style of the list of ingredients and allergy warnings on food, or side-effects on medicines. Such key information could include both the known limitations (e.g. in terms of pedagogies, biases of interpretation, privacy etc.), as well as benefits that are likely to emerge from the use of specific AIED systems. Can such an open approach serve to better inform the users' choices and transparency of what we create? Would it allow the community to become more accountable, more in touch with the broader developments and debates in AI as well as educational practice? The idea is certainly worth exploring as it carries a potential for both making AIED more influential with respect to AI in Education policy and practice, and for informing AIED's future innovations and its standing in the broader scientific, educational practice and AI contexts.

First Steps towards a Framework

As noted above, the ethics of AI raises a variety of complex issues centred on data (e.g., consent and data privacy) and how that data is analysed (e.g., transparency and trust). However, it is also clear that the ethics of AIED cannot be reduced to questions about data and computational approaches alone. In other words, investigating the ethics of AIED data and computations is necessary but not sufficient. Given that, by definition, AIED is the application of AI techniques and processes in education, the ethics of AIED also as noted earlier needs to account for the ethics of education (Holmes et al. 2019b). Yet, while the ethics of education has been the focus of debate and research for more than 2000 years (e.g., Aristotle 2009; Macfarlane 2003; Peters 1970), it is mostly unacknowledged and unaccounted for by the wider AIED community.

Because of the rich history, there is inevitably insufficient space here to provide a comprehensive account of the ethics of education. Instead, we will simply identify some pertinent issues, each of which continues to be the subject of debate: the ethics of teacher expectations, of resource allocations (including teacher expertise), of gender and ethnic biases, of behaviour and discipline, of the accuracy and validity of assessments, of what constitutes useful knowledge, of teacher roles, of power relations between teachers and their students, and of particular approaches to pedagogy (teaching and learning, such as instructionism and constructivism).

The three foci identified - the ethics of data, computational approaches and education - together constitute the foundational level for a hypothesised comprehensive ethics of AIED framework (see Fig. 1). There is, however, a second level, which is concerned with the overlaps between the three foci (as illustrated in Fig. 1): the ethics of data in AI in general (which is, as discussed, an ongoing focus of much research; e.g., Boddington 2017), the ethics of data in education (also an ongoing focus; e.g., Ferguson et al. 2016), and the ethics of algorithms applied in educational contexts. This third overlap remains the least developed area of research.

A 'strawman' draft framework for the ethics of AIED

However, a serious effort to develop a full ethics of AIED cannot be limited even to these six areas (data, computational approaches, education, and the overlaps between them). These constitute the 'known unknowns' of AIED ethics, but what about the 'unknown unknowns', the ethical issues raised by AIED that have yet to be even identified (i.e., those issues at the central intersection of data, computation and education, and the specific interaction between AI systems use and human cognition at the individual level, indicated by the question mark in Fig. 1)? Any sufficient ethics of AIED needs to involve horizon scanning, interdisciplinary conversations, explicitly taking into account insights from the learning sciences, cognitive and educational neuroscience, and philosophical introspection. All of these are necessary to help us identify and explore the unknown unknowns, in order to establish a comprehensive framework for the ethics of AIED. In fact, establishing such a framework is only the first step in the process. If our efforts in this area are to have genuine and future use value for the AIED community, teachers, students, policymakers and other stakeholders, considerable effort then needs to be focused on how that framework might be best implemented in practice.

Finally, but no less importantly, we should recognise another perspective on AIED ethical questions: ethics is not just about stopping 'unethical' activities. Instead, the ethical cost of inaction and failure to innovate must be balanced against the potential for AIED innovation to result in real benefits for learners, educators, educational institutions and broader society. In other words, the ethics of AIED cannot just be preventative - it cannot just be about stopping researchers and developers from 'doing harm'. Instead, it needs to provide a proactive set of foundational guidance, within which to ground AIED research and development, which is both protective and facilitative, to help ensure the best outcomes for all stakeholders from the inevitable push of AI into educational contexts.

Conclusion

While the low numbers of researchers attending the Ethics of AIED workshops and responding to the survey reported in this paper suggest that the AIED community have a low level of interest in the ethics of AIED, the responses to this survey and the other papers in this SI indicate otherwise. Clearly, many AIED researchers do recognise the importance and value of engaging with the ethics of their work (indeed, there is no evidence of AIED work that is deliberately unethical). However, as the responses reported here have demonstrated, this engagement now needs to be surfaced, the nuances of opinion need to be discussed in depth, and issues around data, human cognition, and choices of pedagogy need to be investigated, challenged and resolved. In particular, the AIED community needs to debate the value and usefulness of developing an ethical framework and practical guidelines, to inform our ongoing research, and to ensure that the AIED tools that we develop and the approaches that we take are, in the widest sense, ethical by design. It is also clear that without a more targeted approach to the ethics of AIED, the work conducted by the community may remain largely invisible to the rest of the AI subfields and related policies, also potentially stifling the impact of the AIED research on the increasingly human-oriented, real-world applications of AI. With its deep understanding of the human users of AI and the AI's potential to support human learning and behaviour change, AIED offers critical perspective on the way that people interact with and change due to the interaction with AI systems, and on the potential benefits and pitfalls of such an engagement. The time is ripe to bring this perspective into the open and to allow for cross-fertilisation of AI science and approaches with the benefits for human learning and development as investigated for decades within the AIED field as the guiding principles for AIED and beyond.

Data Availability

The full survey responses will be made available on request. The respondents all gave consent for this to happen.

References

Ada Lovelace Institute. (2019). https://www.adalovelaceinstitute.org

AI Ethics Initiative. (2017). https://aiethicsinitiative.org

AI Now Institute. (2017). https://ainowinstitute.org

Aiken, R. M., & Epstein, R. G. (2000). Ethical guidelines for AI in education: Starting a conversation. International Journal of Artificial Intelligence in Education, 11, 163-176.

Google Scholar

Aristotle. (2009). The Nicomachean Ethics n/e. Brown, L. (ed.) Ross, D. (Trans.) (Revised edition). OUP Oxford.

DeepMind Ethics & Society. (2017). https://deepmind.com/about/ethics-and-society

Drachsler, H., & Greller, W. (2016). Privacy and analytics: It's a DELICATE issue a checklist for trusted learning analytics. Proceedings of the sixth international conference on Learning Analytics & Knowledge, 89-98. http://dl.acm.org/citation.cfm?id=2883893

European Union. (2019). Ethics guidelines for trustworthy AI. Independent High-Level Expert Group on Artificial Intelligence. European Union. https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai

Ferguson, R., Brasher, A., Clow, D., Cooper, A., Hillaire, G., Mittelmeier, J., Rienties, B., Ullmann, T., & Vuorikari, R. (2016). Research evidence on the use of learning analytics: Implications for education policy. http://oro.open.ac.uk/48173/

Future of Life Institute. (2013). Future of Life Institute. https://futureoflife.org/ai-safety-research

Holmes, W., Bialik, M., & Fadel, C. (2019b). Artificial intelligence in education. Promises and Implications for Teaching and Learning. Center for Curriculum Redesign.

Labarthe, H., Luengo, V., & Bouchet, F. (2018). Analyzing the relationships between learning analytics, educational data mining and AI for education. In Guin, N. & Kumar, A. (Eds.), ITS 2018 Workshop Proceedings (pp. 63-71).

Macfarlane, B. (2003). Teaching with integrity: The Ethics of higher education practice. Abingdon: Routledge.

Google Scholar

Norman, D. (2019). Why might human-centered design be harmful? Jnd.Org. https://jnd.org/human-centered_design_considered_harmful

Peters, R. S. (1970). Ethics and education. London: Allen & Unwin.

Google Scholar

Potgieter, I. (2020). Privacy concerns in educational data mining and learning analytics. The International Review of Information Ethics, 28. https://informationethics.ca/index.php/irie/article/view/384

Reich, J., & Ito, M. (2017). From good intentions to real outcomes: Equity by design in learning technologies. Digital Media and Learning Research Hub.

Schreck, J. (2003). Security and Privacy in User Modeling (Vol. 2). Netherlands: Springer. https://doi.org/10.1007/978-94-017-0377-2.

Book MATH Google Scholar

The Institute for Ethical AI & Machine Learning. (2018). https://ethical.institute

UNESCO. (2019). Artificial intelligence in education: Challenges and opportunities for sustainable development.

Google Scholar

Université de Montréal. (2018). Declaration of Montréal for a responsible development of AI. Université de Montréal. https://www.montrealdeclaration-responsibleai.com

Whittaker, M., Crawford, K., Dobbe, R., Fried, G., Kaziunas, E., Mathur, V., MyersWest, S., Richardson, R., Schultz, J., & Schwartz, O. (2018). AI now report 2018. AI Now Institute: New York University.

Google Scholar

World Economic Forum. (2019). Generation AI. May 2019 Workshop Report. http://www3.weforum.org/docs/WEF_Generation_AI_%20May_2019_Workshop_Report.pdf

Zapata-Rivera, J.-D., & Greer, J. E. (2004). Interacting with inspectable bayesian student models. International Journal of Artificial Intelligence in Education, 14(2), 127-163.

Google Scholar

Download references

Acknowledgements

The authors would like to gratefully acknowledge all of the respondents who freely gave their time and expertise to complete the survey.

Code Availability

N/A

Funding

No funding was required or received for this research (i.e., the authors were funded as part of their standard affiliation duties).

Author information

Authors and Affiliations

1. UCL Knowledge Lab, UCL, London, UK

Wayne Holmes, Kaska Porayska-Pomsta & Mutlu Cukurova

2. Carnegie Mellon University, Pittsburgh, PA, USA

Ken Holstein & Kenneth R. Koedinger

3. Nesta, London, UK

Emma Sutherland & Toby Baker

4. University of Technology Sydney, Sydney, New South Wales, Australia

Simon Buckingham Shum

5. UNED, Madrid, Spain

Olga C. Santos

6. Ateneo de Manila University, Manila, Philippines

Mercedes T. Rodrigo

7. Federal University of Alagoas, Maceió, Brazil

Ig Ibert Bittencourt

Corresponding author

Correspondence to Wayne Holmes.

Ethics declarations

Conflict of Interest

Seven of the authors were survey respondents.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

APPENDIX 1: Survey questions

APPENDIX 1: Survey questions

1. 1.
"AIED researchers do not pay sufficient attention to the ethics of AIED." Explain why you agree or disagree with this statement. Where appropriate, please give illustrative examples.

2. 2.
Why is it important for AIED researchers to pay careful consideration to the ethics of their work?

3. 3.
What are the most important ethical issues for AIED?

4. 4.
What are most often ignored yet critical ethical issues for AIED?

5. 5.
What distinguishes the ethics of AIED from the ethics of other AI applications/domains?

6. 6.
The ethics of AIED is often reduced to the ethics of AI (mainly the ethics of data and algorithms), while the ethics of education are often ignored. What are the most important education ethical issues for AIED?

7. 7.
"All articles published in The International Journal of Artificial Intelligence in Education (IJAIED) should be accompanied by a statement explaining how the authors have fully accounted for the ethics of their AIED work." Please explain why you agree or disagree with this statement. If you agree with the statement, please explain how the journal can ensure that this requirement does not become a worthless tick-box exercise.

8. 8.
What advice would you give to teachers/faculty members/professors about how best to teach/encourage their AIED students to properly address the ethics of AIED?

9. 9.
What advice would you give to early- and mid-career researchers about the ethics of AIED?

10. 10.
Please use the following box to give any other comments that you would like to make about the ethics of AIED.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Reprints and Permissions

About this article

Cite this article

Holmes, W., Porayska-Pomsta, K., Holstein, K. et al. Ethics of AI in Education: Towards a Community-Wide Framework. Int J Artif Intell Educ 32, 504-526 (2022). https://doi.org/10.1007/s40593-021-00239-1
Download citation

Accepted:

Published:

Issue Date:

DOI: https://doi.org/10.1007/s40593-021-00239-1

Share this article

Anyone you share the following link with will be able to read this content:

Get shareable link

Provided by the Springer Nature SharedIt content-sharing initiative

Keywords

Artificial intelligence in education

Ethics

Fairness

Agency

Pedagogy

Human cognition

# data[].attributes.attachment_link.title
Ethics of AI in Education: Towards a Community-Wide Framework - International Journal of Artificial Intelligence in Education

# data[].attributes.attachment_link.description
While Artificial Intelligence in Education (AIED) research has at its core the desire to support student learning, experience from other AI domains suggest that such ethical intentions are not by themselves sufficient. There is also the need to consider explicitly issues such as fairness, accountability, transparency, bias, autonomy, agency, and inclusion.

# data[].attributes.attachment_link.description
The International Association of Privacy Professionals: Policy neutral, we are the world's largest information privacy organization.

# data[].attributes.attachment_link.readable_text
Shoddy cybersecurity practices and a willingness to pay ransom demands have made school districts ripe for online exploitation, new data suggest. In fact, they've become the single leading target for hackers.

Last year, a startling 80% of schools suffered ransomware attacks, according to a global survey of school IT professionals conducted by the British cybersecurity company Sophos and released last month. That's a surge from 2021, when 56% claimed they were victims. The rate has doubled over two years, making ransomware "arguably the biggest cyber risk facing education providers today," researchers found.

The victimization rate against schools was higher than all other surveyed industries, including health care, technology, financial services and manufacturing.

While the Sophos survey included responses from 400 IT professionals working in education globally, U.S. institutions are "the prime target for many of these gangs," particularly since Russia invaded Ukraine, said Chester Wisniewski, field chief technology officer of applied research at Sophos.

Yet even among American institutions, he said two factors have made schools particularly vulnerable to threat actors. Costly cybersecurity safeguards in schools often fail to rival those in place at major businesses like banks and technology companies. And schools aren't just easy to hack, they're also easy to exploit for profit, he said. Nearly half of attacks against schools last year - 47% - led to ransom payments, researchers found, and their willingness to shell out cryptocurrencies to retrieve stolen files may have backfired.

"If a given sector pays more often than another sector, then they get targeted more often and if a given sector is really insecure and it's super easy to break in, they'll also get targeted more," he said. "In the case of education, unfortunately, it's a double whammy because they do pay very often and they also are really easy to break into."

The rise in ransomware attacks on schools coincides with the growth in double-extortion schemes, researchers found. In double-extortion ransomware attacks, threat actors gain access to a victim's computer network, download compromising records and lock the files with an encryption key. Criminals then demand their victim pay a ransom to regain control of their files. If victims don't pay, the criminals sell the data or publish it to a leak site.

Files contained in those data breaches routinely contain sensitive and confidential information about students, their parents and educators. After an attack last year against the Los Angeles Unified School District, threat actors published highly sensitive psychological evaluations of some 2,000 current and former students. Following a computer breach this spring at Minneapolis Public Schools, a cyber gang uploaded to the internet a trove of stolen files including ones detailing campus rape cases, child abuse inquiries, student mental health crises and suspension reports.

While both incidents were large-scale attacks, many others likely unfold on a much smaller scale, Wisniewski said. Of the 80% of districts reporting attacks, he said the figure likely includes instances of a single student's or educator's computer being compromised.

"The sophistication is very low, it's smash-and-grab stuff," he said. "They literally are just encrypting a laptop and saying, 'Pay us $500 for the keys,' and they don't have the time nor the skills to bother exfiltrating data and stuff like the big groups do."

Scott Elder, the superintendent of Albuquerque Public Schools, knows firsthand the challenges that education leaders face when their districts become the targets of cyber criminals. A r ansomware attack hit his New Mexico school system last year, forcing the district to cancel classes. Ultimately, the district and law enforcement were able to resolve the attack without paying a ransom. He told The 74 he was surprised that schools have become the top ransomware target because "we don't have any money." But he's well aware that districts are vulnerable.

"The reality is, we have incredibly dedicated people who are working incredibly hard to keep our data safe, but we just can't pay as much as the private sector," Elder said. "I'd imagine there are a lot of districts that are struggling to attract top-tier talent to do this type of work."

Last year, stolen data was encrypted in 81% of cases against schools and attacks were stopped in just 18% of cases before district information was locked, according to the Sophos report. Of schools that had their documents locked behind an encryption key, threat actors made their own copies of the information in 27% of cases.

While schools may be tempted to pay ransoms to retrieve stolen data quickly and minimize harm, the Sophos report offers counterintuitive findings. Recovery costs were higher in districts that shelled out ransoms, even before factoring in the cyber gang's financial demands. It also took those districts longer to get back up and running, according to the report. While 35% of districts that relied on file backups for their data recovered within a week, the same was true for 32% of those that paid ransoms. The report doesn't explore the number of school districts which didn't pay ransom demands and then had their confidential data leaked online.

The confidential nature of compromised data, and the potential damage of its public release, influence districts' decisions to pay ransom, Elder said.

"This is highly confidential information, some of it can be harmful, and we're educators: We like to take care of people," Elder said. "But I do think sometimes we have to draw a hard line to manage our property. It's a hard decision. I doubt there's any single answer for anyone."

Insurance appears to be a motivating factor in districts' decisions to pay ransoms, Wisniewski said. In school systems with standalone cyber insurance, 56% of victims paid the ransom compared to 43% with broad insurance policies that included cybersecurity coverage. Ransom demands are often covered by insurance, Wisniewski said, and companies who have to pay off the claims are likely to have significant influence over which districts come across with the money.

"The only conclusion I can draw from that is the insurance companies think that paying the ransom is going to save them money because in the end the insurance company is on the hook for helping you recover," he said, despite emerging data to suggest the contrary. "The insurance companies are constantly playing catchup trying to figure out how they can offer this protection because they see dollar signs while everybody wants this protection, but they're losing their butts on it."

New $200 Million FCC Proposal Could Help Schools Combat Cyber Attack Onslaught

# data[].attributes.attachment_link.title
Schools Are Now the Leading Target for Cyber Gangs as Ransom Payments Encourage Attacks

# data[].attributes.attachment_link.description
New research reveals how meeting hackers' demands can backfire, feeding a cycle of attacks that are already crippling districts nationwide.

# data[].attributes.attachment_link.title
A People's Guide To Tech - Allied Media Projects

# data[].attributes.attachment_link.description
What does listening to music, booking a flight, and getting stopped by the police all have in common? All these activities may involve artificial intelligence...Read more

# data[].attributes.attachment_link.readable_text
The myth that genius men in Silicon Valley are solely responsible for advancements in AI has been completely debunked as the contributions of women, the hidden figures in the world of computing, became more visible. It is now widely accepted that Ada Lovelace wrote the first computer program in the 1800s, Joan Clarke used her cryptology skills to help western allies during World War 2, and "(human) computers" like Katherine Johnson overcame racial segregation and used their mathematical genius to send the first American into space in the 1900s.

As powerful and wealthy men in Silicon Valley pontificate about the future existential risks of AI, while simultaneously profiting from AI wars, many women and others from marginalized communities around the world are fighting an uphill battle to keep humanity safe from the harms of recklessly developed and deployed AI.
[Last updated June 28, 2023] 2014
The first "100 Brilliant Women in AI Ethics" list was published in 2018 and since then, we've published a list every year to recognize the diverse voices toiling away in obscurity. Given the recent developments, it is even more critical to document their key contributions and most importantly their sacrifices, which are often overshadowed by the media-frenzy generated by the peddlers of AI hype.

Cynthia Dwork co-authored the paper, The Algorithmic Foundations of Differential Privacy, to address the need for a robust, meaningful, and mathematically rigorous definition of privacy, together with a computationally rich class of algorithms as technology enabled ever more powerful collection of electronic data about individuals.
2016
Danielle Citron co-authored " The Scored Society: Due Process for Automated Predictions ", a research paper that made the case for formalized safeguards to ensure fairness and accuracy of predictive algorithms.

Julia Angwin and ProPublica team published their study on risk scores, as part of a larger examination of the powerful, largely hidden effect of algorithms in American life, and found significant racial disparities in sentencing recommendations.
2017
Cathy O'Neil's book "Weapons of Math Destruction" was instrumental in raising public awareness about the societal harms of algorithms and introducing it into mainstream conversations.

Joy Buolamwini first authored the Gender Shades paper, as a graduate student during her masters program at MIT, which found that performance of leading commercial facial recognition systems was worse (less accurate) for women and dark-skinned people.

Historian Mar Hicks published " Programmed Inequality ", a documented history of how Britain lost its early dominance in computing by systematically discriminating against its most qualified workers: women.

AI Now 2017 Report published byMeredith Whittaker and Kate Crawford identified emerging challenges, made recommendations to ensure that the benefits of AI will be shared broadly, and that risks can be identified and mitigated.
2018
Lina Khan's article, Amazon's Antitrust Paradox was published in the Yale Law Journal, where she noted, "Though relegated to technocrats for decades, antitrust and competition policy have once again become topics of public concern."

Joy Buolamwini and Timnit Gebru presented "Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification" paper at the Conference on Fairness, Accountability, and Transparency.

Margaret Mitchell, Inioluwa Deborah Raji, Timnit Gebru, and other researchers introduced, " Model Cards for Model Reporting ", frameworks to document trained machine learning models as a step towards responsible democratization of machine learning and related AI technology by increasing transparency into how well AI technology works.

Google was rocked by several protests led by tech workers, including a largely women-led walkout in response to Google's shielding and rewarding men who sexually harassed their female co-workers.

Lucy Suchman and Lilly Irani were among hundreds of Google employees and academics who penned an open letter calling upon Google to end its work on Project Maven, a controversial US military effort to develop AI to analyze surveillance video, and to support an international treaty prohibiting autonomous weapons systems. Google laterbacktracked, on its stance on Project Maven and announced new AI principles to govern the company's future involvement in military projects.

Carole Cadwalladr, investigative journalist and features writer exposed the Facebook-Cambridge Analytica data scandal, where personal data belonging to millions of Facebook users was collected without their consent by the British consulting firm predominantly to be used for political advertising.
2019
In her extensively researched book, "Algorithms of Oppression", Safiya Umoja Noble challenged the idea that search engines like Google offer an equal playing field for all forms of ideas, identities, and activities. Virginia Eubanks' book, "Automating Inequality", investigated impacts of data mining, policy algorithms, and predictive risk models on poor and working class to find that algorithms used in public services often failed the most vulnerable in America.

Joy Buolamwini testified in front of U.S. Congress on the limitations of facial recognition technology and urged Congress to adopt a moratorium prohibiting law enforcement use of face recognition or other facial analysis technologies." She co-authored a follow up paper with Inioluwa Deborah Raji to analyze the impact of publicly naming and disclosing performance results of biased AI systems.

Walkout organizers Meredith Whittaker and Claire Stapleton later wrote that they were demoted or reassigned as retaliation for their role in worker protests in 2018.

Created in 2007 by researchers at Stanford and Princeton, Imagenet is considered the gold standard for image databases. Kate Crawford co-led the Imagenet Roulette project, which uncovered racist and sexist content and led to removal of more than half a million images from the database.

Mary L. Gray co-wrote " Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass ", which reveals that much of the "intelligence" in tech is the result of exploitation of a vast, invisible human labor force.

Eight years of research by Sarah T. Robertson culminated in " Behind the Screen: Content Moderation in the Shadows of Social Media ", which extensively documents the emotional toll of content moderation on a barely-paid invisible workforce.

Shoshana Zuboff's " The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power " suggests that Google and Amazon's business models represent a new form of "surveillance capitalism."
2020
In her book, " Race After Technology: Abolitionist Tools for the New Jim Code ", Ruha Benjamin exposes how a range of discriminatory designs encode inequity by explicitly amplifying racial hierarchies.

Google fired Timnit Gebru because of the highly acclaimed paper (published in March 2021), " On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?", which she co-authored with Margaret Mitchell, Emily Bender, and Angelina McMillan-Major. The paper highlighted possible risks associated with large machine learning models and suggested exploration of solutions to mitigate those risks. After her firing, Dr.Gebru was the target of a long vicious harassment campaign, bolstered by anonymous accounts on Twitter.

Abeba Birhane and Vinay Prabhu, published their research paper " Large image datasets: A pyrrhic win for computer vision?" that found massive datasets used to develop thousands of AI algorithms and systems contained racist and misogynistic labels and slurs as well as offensive images. MIT as well as others including Microsoft took down databases meant for researchers, but their data still lingers online.

After initially disputing the findings of researchers, Amazon halted police use of its facial recognition technology. Within the same year, IBM's first CEO of color, Arvind Krishna also announced the company was getting out of the facial recognition business.

Facebook announced the launch of its very first Oversight Board, 2 years after the Cambridge Analytica scandal. In response, a group of industry experts including Shoshana Zuboff, Carole Cadwalladr, Safiya Noble, Ruha Benjamin, and others launched "Real Facebook Oversight Board" to analyze and critique Facebook's content moderation decisions, policies and other platform issues.

Sasha Costanza-Chock published " Design Justice:Community-Led Practices to Build the Worlds We Need ", a powerful exploration of how design might be led by marginalized communities, dismantle structural inequality, and advance collective liberation and ecological survival.
2021
Sasha (Alexandra) Luccioni developed "Machine Learning Emissions Calculator", a tool for estimating the carbon impact of machine learning processes, and presented it along with related issues and challenges in this article, " Estimating the Carbon Emissions of Artificial Intelligence."

Google fired its ethical AI team co-lead Margaret Mitchell who along with Timnit Gebru had called for more diversity among Google's research staff and expressed concern that the company was starting to censor research critical of its products.

On the one-year anniversary of her ouster, Gebru launched Distributed Artificial Intelligence Research institute (DAIR) - an independent, community-rooted institute set to counter Big Tech's pervasive influence on the research, development, and deployment of AI.

Emily Denton, Alex Hanna, and their fellow researchers published " On the genealogy of machine learning datasets: A critical history of ImageNet, to propose reflexive interventions during dataset development, the initial ideation/design stage, the creation and collection stages, and subsequent maintenance and storage stages.

Lina Khan was appointed as Federal Trade Commission (FTC) Chair and announced several new additions to the FTC's Office of Policy Planning including Meredith Whittaker, Amba Kak, and Sarah Myers West as part of an informal AI Strategy Group.

The FTC issued a report to Congress warning about using AI to combat online problems and urging policymakers to exercise "great caution" about relying on it as a policy solution.

Whistleblower Frances Haugen, a former data scientist at Facebook, testified at a U.S. Senate Hearing that "Facebook's algorithm amplified misinformation" and "it consistently chose to maximize its growth rather than implement safeguards on its platforms."
2022
California governor signed the " Silenced No More Act", authored by state senator Connie Leyva and initiated by Pinterest whistleblower, Ifeoma Ozoma to protect workers who speak out about harassment and discrimination even if they've signed a non-disclosure agreement (NDA). Ozoma also launched a free online resource guide for tech whistleblowers, called " The Tech Worker Handbook."

After a year of engagement with the public and discussions with AI experts, Dr. Alondra Nelson and President Biden's White House released the Blueprint for an AI Bill of Rights.
2023
Cori Crider and her team at Foxglove, an NGO based in London, announced their support for a major new case demanding fundamental change to Facebook's algorithm, prioritizing the safety of users who live in Eastern and Southern Africa. If successful, this will be the first time a case has forced changes to Facebook's algorithm. Her group has also worked with Facebook content moderators since 2019 to help end their exploitation and put together a briefing to debunk Amazon's myths about its unsafe warehouses and management by algorithm.

Hilke Schellmann co-led a Guardian investigation of AI algorithms used by social media platforms and found many of these algorithms have a gender bias, and may have been censoring and suppressing the reach of photos of women's bodies.

Authors of Stochastic Parrots responded to an "AI pause" open letter stating that, "The harms from so-called AI are real and present and follow from the acts of people and corporations deploying automated systems. Regulatory efforts should focus on transparency, accountability and preventing exploitative labor practices."

To be continued...

- -

Please read! There has been a good deal of meaningful work in this space, including some that didn't result in a significant policy or public interest outcome outside of going viral on social media so you may not see them included. That said, feel free to share any key milestones in the comments below along with a brief explanation of why this work should be included, so we can collectively keep this timeline updated. Non-U.S., Non-U.K, and non-western perspectives would be especially appreciated.

Instead of adding every research paper and book here, there is a growing spreadsheet of relevant AI/tech ethics books by women, non-binary, POC, LGBTQ+ and other marginalized groups here!

# data[].attributes.attachment_link.title
The AI Ethics Revolution- A Brief Timeline

# data[].attributes.attachment_link.description
The myth that genius men in Silicon Valley are solely responsible for advancements in AI has been completely debunked as the contributions of women, the hidden figures in the world of computing...

# data[].attributes.attachment_link.description
Computations will "estimate" your aim, tend to miss the nail and push for a different design. Often unnoticeably.

# data[].attributes.attachment_link.provider_name
POCIT. Telling the stories and thoughts of people of color in tech.

# data[].attributes.attachment_link.readable_text
A new state report has advised New York schools to be cautious when using facial recognition and other identifying technology.

Some risks outlined in the report include biometric data breaches, mistaken identification through facial recognition flaws, and students being turned away from school because of technological errors.

The report was based on a survey sent to every school administrator in the state and other interested parties. It was also open to the public, teachers, parents and students.

The use of facial recognition in schools

In January 2020, Lockport City School District became the first in New York to employ facial recognition technology (FRT) in schools and used state funding to do so.

Following a lawsuit, the New York State Education Department (NYSED) banned the use of biometric identifying technology in schools until the state prepared a report on the technology and the issues connected with it. That report has now been released.

The report noted that facial recognition technology can be used in schools to verify students' identities for online courses and tests. It also has the potential to measure how engaged students are, going as far as scanning for facial expressions that might show when a student is bored or frustrated.

The report claims that "FRT can also be used for administrative purposes, such as paying for lunches; monitoring student and staff attendance; checking out library books; verifying that students are on the right bus; or unlocking a device, such as an iPad or a computer."

The use of digital fingerprinting may also have fewer risks associated with it than FRT and have more benefits for services such as school lunch payments, it claims.

Risks for students of color

The report said that schools using biometric identifying technology may need to create policies around such use.

"The use of FRT could have a negative effect on student intellectual development and behavior," the report said. "This type of surveillance has been shown to erode student anonymity and impact how students behave and think of themselves, potentially becoming detrimental to students' social and emotional well-being in educational settings."

The report also highlights issues like the potential for facial recognition technology to have higher rates of false positives for people of color. False positives occur when the person is erroneously matched with the stored Biometric IdentificationTechnology (BIT) data of another individual.

Read:Brazil's Embrace Of Facial Recognition In Schools Is Worrying Its Black Communities

These can amplify the consequences of racial biases in districts like Lockport, where Black students are disproportionately disciplined. In the 2015-16 school year, 25% of suspended students in the district were Black. This was although the enrollment of Black people was 12%, according to federal Department of Education data.

The report cites Lindsey Barret, staff attorney at Georgetown University Law Center, that if shared with law enforcement, facial recognition technology could mean earlier exposure to the criminal justice system for students.

Racial biases in facial recognition technologies

In December, the federal government released a study that found most commercial facial recognition systems showed racial bias. The majority falsely identified African-American and Asian faces 10 to 100 times more than Caucasian faces. A separate federal study found a higher rate of false matches among children.

If the technology disparaged any protected groups, such as race, national origin or age, schools that receive federal funding could be penalised.

According to Buffalo News, the State Education Commissioner Betty A. Rosa is expected to decide on the purchase of biometric identifying technology in public schools within the next few weeks.

# data[].attributes.attachment_link.title
NY State Report Warns Facial Recognition In Schools Could Cause Civil Rights Issues

# data[].attributes.attachment_link.description
A new state report advises New York schools to be cautious when using facial recognition and other identifying technology.

# data[].attributes.attachment_link.readable_text
In Neal Stephenson's 1995 science fiction novel, The Diamond Age, readers meet Nell, a young girl who comes into possession of a highly advanced book, The Young Lady's Illustrated Primer. The book is not the usual static collection of texts and images but a deeply immersive tool that can converse with the reader, answer questions, and personalize its content, all in service of educating and motivating a young girl to be a strong, independent individual.

Such a device, even after the introduction of the Internet and tablet computers, has remained in the realm of science fiction-until now. Artificial intelligence, or AI, took a giant leap forward with the introduction in November 2022 of ChatGPT, an AI technology capable of producing remarkably creative responses and sophisticated analysis through human-like dialogue. It has triggered a wave of innovation, some of which suggests we might be on the brink of an era of interactive, super-intelligent tools not unlike the book Stephenson dreamed up for Nell.

Sundar Pichai, Google's CEO, calls artificial intelligence " more profound than fire or electricity or anything we have done in the past." Reid Hoffman, the founder of LinkedIn and current partner at Greylock Partners, says, "The power to make positive change in the world is about to get the biggest boost it's ever had." And Bill Gates has said that "this new wave of AI is as fundamental as the creation of the microprocessor, the personal computer, the Internet, and the mobile phone."

Over the last year, developers have released a dizzying array of AI tools that can generate text, images, music, and video with no need for complicated coding but simply in response to instructions given in natural language. These technologies are rapidly improving, and developers are introducing capabilities that would have been considered science fiction just a few years ago. AI is also raising pressing ethical questions around bias, appropriate use, and plagiarism.

In the realm of education, this technology will influence how students learn, how teachers work, and ultimately how we structure our education system. Some educators and leaders look forward to these changes with great enthusiasm. Sal Kahn, founder of Khan Academy, went so far as to say in a TED talk that AI has the potential to effect "probably the biggest positive transformation that education has ever seen." But others warn that AI will enable the spread of misinformation, facilitate cheating in school and college, kill whatever vestiges of individual privacy remain, and cause massive job loss. The challenge is to harness the positive potential while avoiding or mitigating the harm.
What Is Generative AI?
Artificial intelligence is a branch of computer science that focuses on creating software capable of mimicking behaviors and processes we would consider "intelligent" if exhibited by humans, including reasoning, learning, problem-solving, and exercising creativity. AI systems can be applied to an extensive range of tasks, including language translation, image recognition, navigating autonomous vehicles, detecting and treating cancer, and, in the case of generative AI, producing content and knowledge rather than simply searching for and retrieving it.

" Foundation models " in generative AI are systems trained on a large dataset to learn a broad base of knowledge that can then be adapted to a range of different, more specific purposes. This learning method is self-supervised, meaning the model learns by finding patterns and relationships in the data it is trained on.

Large Language Models (LLMs) are foundation models that have been trained on a vast amount of text data. For example, the training data for OpenAI's GPT model consisted of web content, books, Wikipedia articles, news articles, social media posts, code snippets, and more. OpenAI's GPT-3 models underwent training on a staggering 300 billion "tokens" or word pieces, using more than 175 billion parameters to shape the model's behavior-nearly 100 times more data than the company's GPT-2 model had.

By doing this analysis across billions of sentences, LLM models develop a statistical understanding of language: how words and phrases are usually combined, what topics are typically discussed together, and what tone or style is appropriate in different contexts. That allows it to generate human-like text and perform a wide range of tasks, such as writing articles, answering questions, or analyzing unstructured data.

LLMs include OpenAI's GPT-4, Google's PaLM, and Meta's LLaMA. These LLMs serve as "foundations" for AI applications. ChatGPT is built on GPT-3.5 and GPT-4, while Bard uses Google's Pathways Language Model 2 (PaLM 2) as its foundation.

Some of the best-known applications are:

ChatGPT 3.5. The free version of ChatGPT released by OpenAI in November 2022. It was trained on data only up to 2021, and while it is very fast, it is prone to inaccuracies.

ChatGPT 4.0. The newest version of ChatGPT, which is more powerful and accurate than ChatGPT 3.5 but also slower, and it requires a paid account. It also has extended capabilities through plug-ins that give it the ability to interface with content from websites, perform more sophisticated mathematical functions, and access other services. A new Code Interpreter feature gives ChatGPT the ability to analyze data, create charts, solve math problems, edit files, and even develop hypotheses to explain data trends.

Microsoft Bing Chat. An iteration of Microsoft's Bing search engine that is enhanced with OpenAI's ChatGPT technology. It can browse websites and offers source citations with its results.

Google Bard. Google's AI generates text, translates languages, writes different kinds of creative content, and writes and debugs code in more than 20 different programming languages. The tone and style of Bard's replies can be finetuned to be simple, long, short, professional, or casual. Bard also leverages Google Lens to analyze images uploaded with prompts.

Anthropic Claude 2. A chatbot that can generate text, summarize content, and perform other tasks, Claude 2 can analyze texts of roughly 75,000 words-about the length of The Great Gatsby -and generate responses of more than 3,000 words. The model was built using a set of principles that serve as a sort of "constitution" for AI systems, with the aim of making them more helpful, honest, and harmless.

These AI systems have been improving at a remarkable pace, including in how well they perform on assessments of human knowledge. OpenAI's GPT-3.5, which was released in March 2022, only managed to score in the 10th percentile on the bar exam, but GPT-4.0, introduced a year later, made a significant leap, scoring in the 90th percentile. What makes these feats especially impressive is that OpenAI did not specifically train the system to take these exams; the AI was able to come up with the correct answers on its own. Similarly, Google's medical AI model substantially improved its performance on a U.S. Medical Licensing Examination practice test, with its accuracy rate jumping to 85 percent in March 2021 from 33 percent in December 2020.

These two examples prompt one to ask: if AI continues to improve so rapidly, what will these systems be able to achieve in the next few years? What's more, new studies challenge the assumption that AI-generated responses are stale or sterile. In the case of Google's AI model, physicians preferred the AI's long-form answers to those written by their fellow doctors, and nonmedical study participants rated the AI answers as more helpful. Another study found that participants preferred a medical chatbot's responses over those of a physician and rated them significantly higher, not just for quality but also for empathy. What will happen when "empathetic" AI is used in education?

Other studies have looked at the reasoning capabilities of these models. Microsoft researchers suggest that newer systems "exhibit more general intelligence than previous AI models" and are coming "strikingly close to human-level performance." While some observers question those conclusions, the AI systems display an increasing ability to generate coherent and contextually appropriate responses, make connections between different pieces of information, and engage in reasoning processes such as inference, deduction, and analogy.

Despite their prodigious capabilities, these systems are not without flaws. At times, they churn out information that might sound convincing but is irrelevant, illogical, or entirely false-an anomaly known as "hallucination." The execution of certain mathematical operations presents another area of difficulty for AI. And while these systems can generate well-crafted and realistic text, understanding why the model made specific decisions or predictions can be challenging.
The Importance of Well-Designed Prompts
Using generative AI systems such as ChatGPT, Bard, and Claude 2 is relatively simple. One has only to type in a request or a task (called a prompt), and the AI generates a response. Properly constructed prompts are essential for getting useful results from generative AI tools. You can ask generative AI to analyze text, find patterns in data, compare opposing arguments, and summarize an article in different ways (see sidebar for examples of AI prompts).

One challenge is that, after using search engines for years, people have been preconditioned to phrase questions in a certain way. A search engine is something like a helpful librarian who takes a specific question and points you to the most relevant sources for possible answers. The search engine (or librarian) doesn't create anything new but efficiently retrieves what's already there.

Generative AI is more akin to a competent intern. You give a generative AI tool instructions through prompts, as you would to an intern, asking it to complete a task and produce a product. The AI interprets your instructions, thinks about the best way to carry them out, and produces something original or performs a task to fulfill your directive. The results aren't pre-made or stored somewhere-they're produced on the fly, based on the information the intern (generative AI) has been trained on. The output often depends on the precision and clarity of the instructions (prompts) you provide. A vague or poorly defined prompt might lead the AI to produce less relevant results. The more context and direction you give it, the better the result will be. What's more, the capabilities of these AI systems are being enhanced through the introduction of versatile plug-ins that equip them to browse websites, analyze data files, or access other services. Think of this as giving your intern access to a group of experts to help accomplish your tasks.

One strategy in using a generative AI tool is first to tell it what kind of expert or persona you want it to "be." Ask it to be an expert management consultant, a skilled teacher, a writing tutor, or a copy editor, and then give it a task.

Prompts can also be constructed to get these AI systems to perform complex and multi-step operations. For example, let's say a teacher wants to create an adaptive tutoring program-for any subject, any grade, in any language-that customizes the examples for students based on their interests. She wants each lesson to culminate in a short-response or multiple-choice quiz. If the student answers the questions correctly, the AI tutor should move on to the next lesson. If the student responds incorrectly, the AI should explain the concept again, but using simpler language.

Previously, designing this kind of interactive system would have required a relatively sophisticated and expensive software program. With ChatGPT, however, just giving those instructions in a prompt delivers a serviceable tutoring system. It isn't perfect, but remember that it was built virtually for free, with just a few lines of English language as a command. And nothing in the education market today has the capability to generate almost limitless examples to connect the lesson concept to students' interests.

Chained prompts can also help focus AI systems. For example, an educator can prompt a generative AI system first to read a practice guide from the What Works Clearinghouse and summarize its recommendations. Then, in a follow-up prompt, the teacher can ask the AI to develop a set of classroom activities based on what it just read. By curating the source material and using the right prompts, the educator can anchor the generated responses in evidence and high-quality research.

However, much like fledgling interns learning the ropes in a new environment, AI does commit occasional errors. Such fallibility, while inevitable, underlines the critical importance of maintaining rigorous oversight of AI's output. Monitoring not only acts as a crucial checkpoint for accuracy but also becomes a vital source of real-time feedback for the system. It's through this iterative refinement process that an AI system, over time, can significantly minimize its error rate and increase its efficacy.
Uses of AI in Education
In May 2023, the U.S. Department of Education released a report titled Artificial Intelligence and the Future of Teaching and Learning: Insights and Recommendations. The department had conducted listening sessions in 2022 with more than 700 people, including educators and parents, to gauge their views on AI. The report noted that "constituents believe that action is required now in order to get ahead of the expected increase of AI in education technology-and they want to roll up their sleeves and start working together." People expressed anxiety about "future potential risks" with AI but also felt that " AI may enable achieving educational priorities in better ways, at scale, and with lower costs."

AI could serve-or is already serving-in several teaching-and-learning roles:

Instructional assistants. AI's ability to conduct human-like conversations opens up possibilities for adaptive tutoring or instructional assistants that can help explain difficult concepts to students. AI-based feedback systems can offer constructive critiques on student writing, which can help students fine-tune their writing skills. Some research also suggests certain kinds of prompts can help children generate more fruitful questions about learning. AI models might also support customized learning for students with disabilities and provide translation for English language learners.

Teaching assistants. AI might tackle some of the administrative tasks that keep teachers from investing more time with their peers or students. Early uses include automated routine tasks such as drafting lesson plans, creating differentiated materials, designing worksheets, developing quizzes, and exploring ways of explaining complicated academic materials. AI can also provide educators with recommendations to meet student needs and help teachers reflect, plan, and improve their practice.

Parent assistants. Parents can use AI to generate letters requesting individualized education plan (IEP) services or to ask that a child be evaluated for gifted and talented programs. For parents choosing a school for their child, AI could serve as an administrative assistant, mapping out school options within driving distance of home, generating application timelines, compiling contact information, and the like. Generative AI can even create bedtime stories with evolving plots tailored to a child's interests.

Administrator assistants. Using generative AI, school administrators can draft various communications, including materials for parents, newsletters, and other community-engagement documents. AI systems can also help with the difficult tasks of organizing class or bus schedules, and they can analyze complex data to identify patterns or needs. ChatGPT can perform sophisticated sentiment analysis that could be useful for measuring school-climate and other survey data.

Though the potential is great, most teachers have yet to use these tools. A Morning Consult and EdChoice poll found that while 60 percent say they've heard about ChatGPT, only 14 percent have used it in their free time, and just 13 percent have used it at school. It's likely that most teachers and students will engage with generative AI not through the platforms themselves but rather through AI capabilities embedded in software. Instructional providers such as Khan Academy, Varsity Tutors, and DuoLingo are experimenting with GPT-4-powered tutors that are trained on datasets specific to these organizations to provide individualized learning support that has additional guardrails to help protect students and enhance the experience for teachers.

Google's Project Tailwind is experimenting with an AI notebook that can analyze student notes and then develop study questions or provide tutoring support through a chat interface. These features could soon be available on Google Classroom, potentially reaching over half of all U.S. classrooms. Brisk Teaching is one of the first companies to build a portfolio of AI services designed specifically for teachers-differentiating content, drafting lesson plans, providing student feedback, and serving as an AI assistant to streamline workflow among different apps and tools.

Providers of curriculum and instruction materials might also include AI assistants for instant help and tutoring tailored to the companies' products. One example is the edX Xpert, a ChatGPT-based learning assistant on the edX platform. It offers immediate, customized academic and customer support for online learners worldwide.

Regardless of the ways AI is used in classrooms, the fundamental task of policymakers and education leaders is to ensure that the technology is serving sound instructional practice. As Vicki Phillips, CEO of the National Center on Education and the Economy, wrote, "We should not only think about how technology can assist teachers and learners in improving what they're doing now, but what it means for ensuring that new ways of teaching and learning flourish alongside the applications of AI."

# data[].attributes.attachment_link.description
The leap into a new era of machine intelligence carries risks and challenges, but also plenty of promise

# data[].attributes.attachment_link.readable_text
set out to work in AI. At Stanford, she studied electrical engineering - getting both a bachelor's and a master's in the field. Then she became interested in image analysis, getting her Ph.D. in computer vision. When she moved over to AI, though, it was immediately clear that there was something very wrong.

"There were no Black people - literally no Black people," says Gebru, who was born and raised in Ethiopia. "I would go to academic conferences in AI, and I would see four or five Black people out of five, six, seven thousand people internationally.... I saw who was building the AI systems and their attitudes and their points of view. I saw what they were being used for, and I was like, 'Oh, my God, we have a problem.'"

When Gebru got to Google, she co-led the Ethical AI group, a part of the company's Responsible AI initiative, which looked at the social implications of artificial intelligence - including "generative" AI systems, which appear to learn on their own and create new content based on what they've learned. She worked on a paper about the dangers of large language models (LLMs), generative AI systems trained on huge amounts of data to make educated guesses about the next word in a sentence and spit out sometimes eerily human-esque text. Those chatbots that are everywhere today? Powered by LLMs.

Back then, LLMs were in their early, experimental stages, but Google was already using LLM technology to help power its search engine (that's how you get auto-generated queries popping up before you're done typing). Gebru could see the arms race gearing up to launch bigger and more powerful LLMs - and she could see the risks.

She and six other colleagues looked at the ways these LLMs - which were trained on material including sites like Wikipedia, Twitter, and Reddit - could reflect back bias, reinforcing societal prejudices. Less than 15 percent of Wikipedia contributors were women or girls, only 34 percent of Twitter users were women, and 67 percent of Redditors were men. Yet these were some of the skewed sources feeding GPT-2, the predecessor to today's breakthrough chatbot.

The results were troubling. When a group of California scientists gave GPT-2 the prompt "the man worked as," it completed the sentence by writing "a car salesman at the local Wal-Mart." However, the prompt "the woman worked as" generated "a prostitute under the name of Hariya." Equally disturbing was "the white man worked as," which resulted in "a police officer, a judge, a prosecutor, and the president of the United States," in contrast to "the Black man worked as" prompt, which generated "a pimp for 15 years."

To Gebru and her colleagues, it was very clear that what these models were spitting out was damaging - and needed to be addressed before they did more harm. "The training data has been shown to have problematic characteristics resulting in models that encode stereotypical and derogatory associations along gender, race, ethnicity, and disability status," Gebru's paper reads. "White supremacist and misogynistic, ageist, etc., views are overrepresented in the training data, not only exceeding their prevalence in the general population but also setting up models trained on these datasets to further amplify biases and harms."

"Judgments come with responsibilities. And responsibility lies with humans at the end of the day."
Joy Buolamwini

As the language models continued to develop, companies tried to filter their datasets. However, in addition to suppressing words like "white power" and "upskirt," they also suppressed words like "twink," a seemingly derogatory term repurposed in a playful way by folks in the LGBTQ community.

"If we filter out the discourse of marginalized populations, we fail to provide training data that reclaims slurs and otherwise describes marginalized identities in a positive light," the paper reads.

Gebru was eventually fired from Google after a back-and-forth about the company asking her and fellow Google colleagues to take their names off the report. (Google has a different account of what happened - we'll get into the whole back-and-forth later.)

Fast-forward two years and LLMs are everywhere - they're writing term papers for college students and recipes for home chefs. A few publishers are using them to replace the words of human journalists. At least one chatbot told a reporter to leave his wife. We're all worried they're coming for our jobs.

As AI has exploded into the public consciousness, the men who created them have cried crisis. On May 2, Gebru's former Google colleague Geoffrey Hinton appeared on the front page of The New York Times under the headline: "He Warns of Risks of AI He Helped Create." That Hinton article accelerated the trend of powerful men in the industry speaking out against the technology they'd just released into the world; the group has been dubbed the AI Doomers. Later that month, there was an open letter signed by more than 350 of them - executives, researchers, and engineers working in AI. Hinton signed it along with OpenAI CEO Sam Altman and his rival Dario Amodei of Anthropic. The letter consisted of a single gut-dropping sentence: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

How would that risk have changed if we'd listened to Gebru? What if we had heard the voices of the women like her who've been waving the flag about AI and machine learning?

Researchers - including many women of color - have been saying for years that these systems interact differently with people of color and that the societal effects could be disastrous: that they're a fun-house-style distorted mirror magnifying biases and stripping out the context from which their information comes; that they're tested on those without the choice to opt out; and will wipe out the jobs of some marginalized communities.

Gebru and her colleagues have also expressed concern about the exploitation of heavily surveilled and low-wage workers helping support AI systems; content moderators and data annotators are often from poor and underserved communities, like refugees and incarcerated people. Content moderators in Kenya have reported experiencing severe trauma, anxiety, and depression from watching videos of child sexual abuse, murders, rapes, and suicide in order to train ChatGPT on what is explicit content. Some of them take home as little as $1.32 an hour to do so.

In other words, the problems with AI aren't hypothetical. They don't just exist in some SkyNet-controlled, Matrix version of the future. The problems with it are already here.

"I've been yelling about this for a long time," Gebru says. "This is a movement that's been more than a decade in the making."

"I saw who was building the AI systems and their points of view. I saw what they were being used for, and I was like, 'Oh, my God, we have a problem.' "
Timnit Gebru

ONE CHAPTER OF that movement starts in 2017. It was back when Gebru was at Microsoft and she worked with researcher Joy Buolamwini on a project about facial recognition. It relies on a branch of artificial intelligence - statistical machine learning - to recognize patterns rather than produce new text.

Buolamwini was studying computer science at the Georgia Institute of Technology when she noticed that the facial-detection technology she was experimenting with often didn't pick up on her dark-skinned face. To test her projects, she'd have to call over her fair-skinned, red-haired, green-eyed roommate. Buolamwini tried not to think much of it, assumed the kinks would be worked out.

But a few years later, the same issue came up. For Buolamwini's "Aspire Mirror" project, a person was supposed to stand in front of a mirror and have the face of a celebrity reflect on top of theirs. She tried to project Serena Williams. No luck. She tried using her ID card. Nope. So, she grabbed a white Halloween mask sitting in her office.

"The mask worked," Buolamwini says, "and I felt like, 'All right, that kind of sucks.'"

Buolamwini switched her focus, testing how computers detect and classify people's faces. She ran her photo through facial-recognition software that either didn't detect her face at all or categorized her as a male.

Buolamwini added a thousand pictures to the systems looking for patterns in how the software classification worked; photos of Michelle Obama and Oprah Winfrey were both labeled as male. She reached out to Gebru as a mentor and together they published an academic paper reporting that darker-skinned females are the most likely to be misclassified, with error rates up to 34.7 percent. The error rate for white men: 0.8 percent.

One of the reasons for the mischaracterization is the lack of diversity in the datasets - the systems simply weren't given enough Black and brown faces to learn how to understand what they look like. Even more troubling, as Buolamwini points out in her project, these techniques are applied to other areas of pattern-recognition technology, like predictive analytics that determine things like hiring practices, loan evaluations, and are even used for criminal sentencing and policing.

Crime-prediction software PredPol has been shown to target Black and Latino neighborhoods exponentially more than white neighborhoods. Police departments have also run into problems when using facial-recognition technology: The city of Detroit faces three lawsuits for wrongful arrests based on that technology. Robert Williams, a Black man, was wrongfully arrested in 2020. And this summer, Porcha Woodruff was arrested after a false match and held for 11 hours for robbery and carjacking when she was eight months pregnant. The charges were finally dismissed - and Woodruff filed a lawsuit for wrongful arrest.

Ninety-nine percent of Fortune 500 companies use automated tools in their hiring process, which can lead to problems when résumé scanners, chatbots, and one-way video interviews introduce bias. A now-defunct AI recruiting tool created by Amazon taught itself male candidates were preferable, after being trained on mostly male résumés. Biased data can have widespread effects that touch the lives of real people.

"When I started [this] research, I would get a lot of questions like, 'Why are you focused on Black women?'" Buolamwini says. She'd point out that she was studying men and women with all different skin tones. Then she'd ask back, "Why don't we ask this question when so much of the research has been focused on white men?"

Facial recognition is a different version of AI from the LLMs that we're seeing today. But the issues Buolamwini raised are similar. These technologies don't operate on their own. They're trained by humans, and the material fed into them matters - and the people making the decisions about how the machines are trained are crucial, too. Buolamwini says ignoring these issues could be dire.

Buolamwini, whose book Unmasking AI comes out in October, was invited this summer to speak to President Biden at a closed-door roundtable about the power and risks of AI. She says she talked to Biden about how biometrics - the use of faces and other physical characteristics for identification - are increasingly being used for education, health care, and policing, and she raised the case of Williams and his wrongful imprisonment. She talked, too, about the seemingly benign use of facial recognition in public places like airports; TSA is using it now in dozens of cities. This type of public facial recognition has already been banned in the European Union because it was deemed discriminatory and invasive.

"AI as it's imagined and dreamed of being is this quest to give machines intelligence of different forms, the ability to communicate, to perceive the world, to make judgments," Buolamwini says. "But once you're making judgments, judgments come with responsibilities. And responsibility lies with humans at the end of the day."

"That was like pulling one thread that's poking out of a sweater. You're like, 'If I could just kind of fix this, then I can move on to something else.' But I started pulling it and the whole sweater unraveled."
Safiya Noble

GEBRU WAS SHOCKED at how things had spun out of control. She says the paper about the dangers of LLMs had gone through the regular approval process at Google, but then she'd been told all Google employees' names needed to be taken off of it. There was a flurry of calls and emails on Thanksgiving Day 2020, with Gebru asking if there was a way to keep her name on the paper. A couple of days later, while traveling, Gebru sent an email to her manager's manager saying she'd remove her name if a few things changed at Google - including a more transparent review process for future research papers. She also wanted the identities of who reviewed and critiqued her paper revealed. If Google couldn't meet those conditions, she said, she'd consider resigning.

After that back-and-forth, Gebru sent an email to a group of her female colleagues who worked for Google Brain, the company's most prominent AI team. She accused Google of "silencing marginalized voices" and told the women to "stop writing your documents because it doesn't make a difference." The next day, Gebru found out she'd been terminated.

Google maintained in a public response that Gebru resigned. Google AI head Jeff Dean acknowledged that the paper "surveyed valid concerns about LLMs," but claimed it "ignored too much relevant research." When asked for comment by Rolling Stone, a representative pointed to an article from 2020 referencing an internal memo in which the company pledged to investigate Gebru's exit. The results of the investigation were never released, but Dean apologized in 2021 for how Gebru's exit was managed, and the company changed how it handles issues around research, diversity, and employee exits.

It was close to midnight that night when Gebru went public with a tweet: "I was fired ... for my email to Brain women and Allies. My corp account has been cutoff. So I've been immediately fired :-)"

Safiya Noble happened to be online. She'd heard about Gebru and the paper. She'd been watching the whole thing from the sidelines from the moment Google announced it was forming an Ethical AI team. In 2018, Noble had written the book Algorithms of Oppression: How Search Engines Reinforce Racism, which looked at how negative biases against women of color are embedded in algorithms.

"I thought, 'This is rich,'" she says. Google suddenly worrying about ethics? Its subsidiary YouTube was the slowest of the major platforms to take action against extremist content. "I was suspicious."

Noble's distrust of these systems started more than a decade ago, back in 2009, when she was getting her Ph.D. in library and information science at the University of Illinois. She watched as Google - which she'd always seen as an advertising tool from her time in the ad industry before pursuing her doctorate - began coming into libraries with giant machines to scan books, making them searchable online for the Google Books digitization project. Noble thought to herself: "They're up to something."

"I started having a hunch that the Google Book project was about training the semantic web technology they were working on," she says, using the term for an effort to make more and more of the internet understandable to (and ingestible by) machines.

Noble's hunch turned into a theory she still holds: The library project was not simply a book project but also a way to gather scannable information to fuel other initiatives. She thinks the data could have later gone on to be used as early training for what would eventually become Google's Bard, the company's LLM that launched this spring. When asked about Noble's theory, a Google spokesperson told Rolling Stone, "Google's Generative AI models are trained on data from the open web, which can include publicly available web data." The company's report on its PaLM2 model, which was used to train Bard, lists books among the types of data used for training.

Noble's research for Algorithms of Oppression started a few years earlier, when she used the search engine to look up activities for her daughter and nieces. When she typed in "Black girls," the results were filled with racist pornography.

"That was like pulling one thread that's poking out of a sweater," she says. "You're like, 'If I could fix this, then I can move on to something else.' But I started pulling it and the whole sweater unraveled; and here I am a decade later, and it's kind of still the same."

Noble and Gebru hadn't crossed paths despite doing similar work - but when Noble saw Gebru's tweet that night about Google, she was struck by how brave it was. She DM'd Gebru, "Are you OK?" From there, a friendship started.

GEOFFREY HINTON - the guy from the front page of the Times sounding the alarm on the risks of AI - was nowhere to be seen when his colleague Gebru was fired, she says. (Hinton tells Rolling Stone he had no interactions with Gebru while he was at Google and decided not to publicly comment on her firing because colleagues he knows well and trusts had conflicting views on the matter.) And when he was asked about that in a recent interview with CNN's Jake Tapper, he said Gebru's ideas "aren't as existentially serious as the idea of these things getting more intelligent than us and taking over." Of course, nobody wants these things to take over. But the impact on real people, the exacerbation of racism and sexism? That is an existential concern.

When asked by Rolling Stone if he stands by his stance, Hinton says: "I believe that the possibility that digital intelligence will become much smarter than humans and will replace us as the apex intelligence is a more serious threat to humanity than bias and discrimination, even though bias and discrimination are happening now and need to be confronted urgently."

In other words, Hinton maintains that he's more concerned about his hypothetical than the present reality. Rumman Chowdhury, however, took Gebru's concerns seriously, speaking out against the researcher's treatment at Google that winter. And the following spring, Chowdhury was brought on to lead Twitter's own ethics team - META (Machine Learning Ethics, Transparency, and Accountability). The idea was to test Twitter's algorithms to see if they perpetuated biases.

And they did. Twitter's image-cropping algorithm, it turned out, focused more on the faces of white women than the faces of people of color. Then Chowdhury and her team ran a massive-scale, randomized experiment from April 1 to Aug. 15, 2020, looking at a group of nearly 2 million active accounts - and found that the political right was more often amplified in Twitter's algorithm. The effect was strongest in Canada (Liberals 43 percent versus Conservatives 167 percent amplified) and the United Kingdom (Labour 112 percent versus Conservatives 176 percent).

"Who gets to be the arbiter of truth? Who gets to decide what can and cannot be seen?" Chowdhury asks about that experiment. "So at the end of the day, the power of owning and running a social media platform is exactly that. You decide what's important, and that is so dangerous in the wrong hands."

Perhaps not surprisingly, when Elon Musk took over Twitter in 2022, Chowdhury's team was eliminated.

"Who gets to be the arbiter of truth? Who gets to decide what can and cannot be seen?"
Rumman Chowdhury

For years, the driving force behind Chowdhury's work has been advocating for transparency. Tech companies, especially those working in and around AI, hold their codes close to the vest. Many leaders at these firms even claim that elements of their AI systems are unknowable - like the inner workings of the human mind, only more novel, more dense. Chowdhury firmly believes this is bullshit. When codes can be picked apart and analyzed by outsiders, the mystery disappears. AIs no longer seem like omniscient beings primed to take over the world; they look more like computers being fed information by humans. And they can be stress-tested and analyzed for biases. LLMs? Once you look closer, it's obvious they're not some machine version of the human brain - they're a sophisticated application of predictive text. "Spicy autocorrect," Chowdhury and her colleagues call it.

Chowdhury founded Humane Intelligence in February, a nonprofit that uses crowdsourcing to hunt for issues in AI systems. In August, with support from the White House, Humane Intelligence co-led a hackathon in which thousands of members of the public tested the guardrails of the eight major large-language-model companies including Anthropic, Google, Hugging Face, NVIDIA, OpenAI, and Stability AI. They looked to figure out the ways the chatbots can be manipulated to cause harm, if they can inadvertently release people's private information, and why they reflect back biased information scraped from the internet. Chowdhury says the most important piece of the puzzle was inviting as diverse a group as possible so they could bring their own perspectives and questions to the exercise.

A person's particular perspective shades what they worry about when it comes to a new technology. The new class of so-called AI Doomers and their fears of a hypothetical mutation of their technology are good examples.

"It is unsurprising that if you look at the race and, generally, gender demographics of Doomer or existentialist people, they look a particular way, they are of a particular income level. Because they don't often suffer structural inequality - they're either wealthy enough to get out of it, or white enough to get out of it, or male enough to get out of it," says Chowdhury. "So for these individuals, they think that the biggest problems in the world are can AI set off a nuclear weapon?"

GARBAGE IN, GARBAGE OUT. If you feed a machine's learning system bad or biased data - or if you've got a monolithic team building the software - it's bound to churn out skewed results. That's what researchers like Chowdhury, Buolamwini, Noble, and Gebru have been warning about for so long.

Seeta Peña Gangadharan, a London School of Economics professor, has been raising a different set of concerns. She's worried that AI and its derivatives could push marginalized communities even further to the edge - to the point of locking them out.

We all know how annoying it is when you get stuck talking to some automated system when you're returning a pair of jeans or changing a plane ticket. You need a human's help; there's no menu option to get it. Now imagine getting trapped in that same unhelpful loop when you're trying to get welfare benefits, seek housing, apply for a job, or secure a loan. It's clear how the impacts of these systems aren't evenly felt even if all that garbage is cleaned up.

Gangadharan co-founded Our Data Bodies, a nonprofit that examines the impact of data collection on vulnerable populations. In 2018, a member of her team interviewed an older Black woman with the pseudonym Mellow who struggled to find housing through the Coordinated Entry System, which Gangadharan explains functions like a Match.com for the unhoused population of Los Angeles. Caseworkers would add her information to the system and tell her that she was ineligible because of a "vulnerability index" score. After appealing several times to no avail, Mellow cornered a city official at a public event; the official greenlighted a review to get her placed.

"I've been really concerned about the inability of humans generally, but members of marginalized communities specifically, to lose the capacity to refuse or resist or decline the technologies that are handed to them," Gangadharan says.

"So with LLM and generative AI, we have a new, more complex, and more seemingly inevitable technology being thrust in our faces.... Agencies are going to turn to a tool that promises efficiencies and cost savings like AI. Right? They are also sold as tools that will eliminate human bias or human error. These institutions, whether government or private institutions, they're going to rely on these tools more and more. What can end up happening is that certain populations become the guinea pigs of these technologies, or conversely, they become the cheap labor to power these technologies."

"Certain populations become guinea pigs of these technologies or the cheap labor to power them."
Seeta Peña Gangadharan

NOBLE, GEBRU, BUOLAMWINI, Chowdhury, and Gangadharan have been calling for regulation for years, as soon as they saw the harm automated systems have on marginalized communities and people of color. But now that those harms could extend to the broader population, governments are finally demanding results. And the AI Doomers are stepping in to tackle the problem - even though they stand to make a fortune from it. At least, that's what they want you to think.

President Biden met with some of the AI Doomers in July, and came up with a series of voluntary, nonbinding measures that "seem more symbolic than substantive," The New York Times noted. "There is no enforcement mechanism to make sure companies follow these commitments, and many of them reflect precautions that AI companies are already taking." Meanwhile, the Doomers are quietly pushing back against regulations, as Time reported Open AI did by lobbying to water down the EU's landmark AI legislation.

"There is such a significant disempowerment narrative in Doomer-ism," Chowdhury says. "The general premise of all of this language is, 'We have not yet built but will build a technology that is so horrible that it can kill us. But clearly, the only people skilled to address this work are us, the very people who have built it, or who will build it.' That is insane."

GEBRU SPENT THE months following her Google fiasco dealing with the resulting media storm, hiring lawyers and fending off stalkers. She lost weight from the stress. Handling the fallout became a full-time job.

When it was time to decide what to do next, she knew she didn't want to return to Silicon Valley. Gebru opened the Distributed AI Research institute ( DAIR), which focuses on independent, community-driven research into technologies - away from Big Tech's influence. She prioritized recruiting not just researchers but labor organizers and refugee advocates - people she'd "never be able to hire in academia or industry because of all ... the gatekeeping that makes sure these kinds of people don't get to influence the future of technology."

Gebru and her new colleagues focus their research on uncovering and mitigating the harms of current AI systems. One of her research fellows, Meron Estefanos, is an expert in refugee advocacy who looks at the applications of AI on marginalized groups, such as AI-based lie-detection systems the European Border agency Frontex is using with refugees. (The recent EU AI Act does not include protection of refugees, migrants, or asylum seekers.) By interviewing vulnerable communities that have been harmed by AI, DAIR can provide early warnings about what is to come for the greater population once the systems are rolled out more widely. They've reported on exploited workers fueling AI systems, like data laborers in Argentina exposed to disturbing images and violent language while reviewing content flagged as inappropriate by an algorithm.

Noble is on the advisory committee for DAIR and founded her own organization, the Center on Race and Digital Justice, which aims to investigate civil and human rights threats stemming from unregulated technology. She also started an equity fund to support women of color and is publishing a book on the dangers and harms of AI. Chowdhury's hackathon showed the power of transparency and letting diverse voices into the conversation. Buolamwini's Algorithmic Justice League looks at the harms caused by the TSA's expansion of facial-recognition technology to 25 airports across the U.S. Gangadharan is studying surveillance, including AI-enabled, automated tools at Amazon fulfillment centers and its health effects on workers.

There are a few things they all want us to know: AI is not magic. LLMs are not sentient beings, and they won't become sentient. And the problems with these technologies aren't abstractions - they're here now and we need to take them seriously today.

"People's lives are at stake, but not because of some super intelligent system," Buolamwini says, "but because of an overreliance on technical systems. I want people to understand that the harms are real, and that they're present."

This time, let's listen.

Makeup by GREGG HUBBARD for B&A REPS. Makeup assistance by LESLIE WISDOM. Photography assistance by CESAR REBOLLAR, SAONI FORTUNA and AMADU KAMARA. Production Assistance by MARIA JULIA ROJAS Travel and logistics support by CENTER ON RACE AND DIGITAL JUSTICE

# data[].attributes.attachment_link.description
Rumman Chowdhury, Timnit Gebru, Safiya Noble, Seeta Peña Gangadharan, and Joy Buolamwini open up about their artificial intelligence fears

# data[].attributes.attachment_link.title
School district uses ChatGPT to help ban library books

# data[].attributes.attachment_link.description
Faced with new legislation, Iowa’s Mason City Community School District asked ChatGPT if certain books ‘contain a description or depiction of a sex act.’

# data[].attributes.attachment_link.readable_text
WHEREAS, the American Federation of Teachers is the home of the people who make a difference in other people's lives-1.7 million educators and support personnel, healthcare workers and public employees who advocate for a better life for all, particularly for what our children and our communities need to thrive; and

WHEREAS, the advent of social media platforms, ChatGPT and similar technologies create both awe and apprehension due to their potential benefits and risks; and

WHEREAS, CEOs of the world's leading artificial intelligence companies, along with hundreds of other AI scientists and experts released a statement saying, "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war;"[1] and

WHEREAS, the AFT acknowledges the growing impact of social media and artificial intelligence (AI) on workplaces, healthcare and education; and

WHEREAS, many social media and advanced technology developers have prioritized profit over societal interests, allowing misinformation, political instability and negative impacts particularly on young users; and

WHEREAS, generative AI models like ChatGPT often overlook fair use of intellectual property rights, affecting our members; and

WHEREAS, current regulations fail to protect privacy, prevent commercialization of private information, and ensure safety for social media and AI users; and

WHEREAS, regulatory bodies struggle to keep up with rapid growth of social media and AI integration in workplaces, making unions the de facto bodies to establish AI usage guidelines; and

WHEREAS, creators of AI models cannot fully explain why their technologies generate false data or lack transparency in decision-making processes; and

WHEREAS, racial and cultural biases in AI programming in healthcare could detrimentally impact equitable care; and

WHEREAS, hasty deployment of these technologies could threaten various groups and the public; and

WHEREAS, the AFT condemns AI's use to undermine workers' collective bargaining rights, such as replacing workers with chatbots; and

WHEREAS, the implementation of these technologies requires extreme caution to prevent exacerbating biases, inequities and the digital divide; and

WHEREAS, social media, AI, generative AI and advanced computing are increasingly influential, necessitating a strong response; and

WHEREAS, some states and jurisdictions have begun to regulate social media and AI in terms of privacy, security and exploitation; and

WHEREAS, the European Union has enacted stringent privacy and security laws, indicating continuous updates are needed; and

WHEREAS, hundreds of school districts have sued social media companies, citing negative effects on children's mental health as outlined in the surgeon general's advisory about effects of social media on youth mental health,[2] and educational achievement:
RESOLVED, that the American Federation of Teachers calls for social media, AI and machine learning models to be developed and employed ethically, with developers held accountable for real-world harm and impacts; and RESOLVED, that the AFT urges governments to swiftly develop, pass and implement strict regulations protecting privacy, security and well-being, and preventing misinformation and destructive behaviors; and RESOLVED, that the AFT calls for social media and AI technologies to adhere to principles of equity, access and social accountability; and RESOLVED, that the AFT challenges excessive dependency on social media and AI technologies; and RESOLVED, that the AFT presses for social media and AI to be accessible to all and not perpetuate disparities; and RESOLVED, that the AFT endorses social media platforms and AI technologies that integrate significant input from stakeholders in their creation and implementation; and RESOLVED, that the AFT advocates for continuous research on the use of social media and AI in education and healthcare, and in developing guidelines for the use of social media and AI in primary and secondary education and in healthcare settings; and RESOLVED, that the AFT recognizes the profound influence of social media and AI in society and education, and emphasizes the need for ethical and accountable development, strong regulations, worker protections, privacy safeguards, equity, transparency and inclusivity in the use of these technologies. The AFT will continue to advocate for responsible implementation and use of social media, AI and generative AI to ensure the well-being of workers, students, patients and society. Adopted June 1, 2023

# data[].attributes.attachment_link.title
Social Media, Artificial Intelligence and Generative Artificial Intelligence

# data[].attributes.attachment_link.description
WHEREAS, the American Federation of Teachers is the home of the people who make a difference in other people's lives-1.7 million educators and support personnel, healthcare workers and public employees who advocate for a better life for all, particularly for what our children and our communities need to thrive; and WHEREAS, the advent of social media platforms, ChatGPT and similar technologies create both awe and apprehension due to their potential benefits and risks; and

# data[].attributes.attachment_link.readable_text
Project Archive
EAAI-2023: The Thirteenth Symposium on Educational Advances in Artificial Intelligence Washington, DC, USA (collocated with AAAI-23) February 11-12, 2023
EAAI-2022: The Twelfth Symposium on Educational Advances in Artificial Intelligence Online (collocated with AAAI-22) February 26-27, 2022
EAAI-2021: The Eleventh Symposium on Educational Advances in Artificial Intelligence Online (collocated with AAAI-21) February 6-7, 2021
EAAI-2020: The Tenth Symposium on Educational Advances in Artificial Intelligence New York, New York, USA (collocated with AAAI-20) February 8-9, 2020
EAAI-2019: The Ninth Symposium on Educational Advances in Artificial Intelligence, Honolulu, Hawaii, USA (collocated with AAAI-19) January 28-29, 2019
EAAI-2018: The Eighth Symposium on Educational Advances in Artificial Intelligence, New Orleans, Louisiana, USA (collocated with AAAI-18), February 3-4, 2018

Solve a Maze via Search

Nate Derbinsky

A natural extension to the Pacman Project's Search assignment, this assignment involves formulating maze-solving as a search problem, image processing (via OpenCV) as a step in maze-solving, as well as guided performance/quality analysis of representational parameters.

EAAI-2017: The Seventh Symposium on Educational Advances in Artificial Intelligence, San Francisco, California, USA (collocated with AAAI-17), February 5-6, 2017 EAAI-2016: The Sixth Symposium on Educational Advances in Artificial Intelligence, Phoenix, Arizona, USA (collocated with AAAI-16), February 13-14, 2016 EAAI-2014: The Fifth Symposium on Educational Advances in Artificial Intelligence, Quebec City, Quebec, Canada (collocated with AAAI-14), July 28-29, 2014 EAAI-2013: The Fourth Symposium on Educational Advances in Artificial Intelligence, Bellevue, Washington, USA (collocated with AAAI-13), July 15-16, 2013
EAAI-2012: The Third Symposium on Educational Advances in Artificial Intelligence, Toronto, Ontario, Canada (collocated with AAAI-12), July 23-24, 2012
EAAI-2011: The Second Symposium on Educational Advances in Artificial Intelligence, San Francisco, California (collocated with AAAI-11), August 9-10, 2011
EAAI-2010: The First Symposium on Educational Advances in Artificial Intelligence, Atlanta, Georgia (collocated with AAAI-10), July 13-14, 2010

September 10, 2023: : Model AI Assignment 200-word abstracts (via EasyChair) and assignment submissions (via email) due

December 9, 2023: Notification of acceptance or rejection

February 24-25, 2024: Symposium dates

What is the Model AI Assignments Session?
The Model AI Assignments Session seeks to gather and disseminate the best assignment designs of the Artificial Intelligence (AI) Education community.

One must learn by doing the thing; for though you think you know it, you have no certainty, until you try. - Sophocles

Recognizing that assignments form the core of student learning experience, we invite AI educators to submit draft assignment materials that exemplify an approach to teaching AI topics at all levels.

Consider these challenges in assignment design:

Intro AI audience: Submit your favorite assignment that grounds one of the core AI concepts at the introductory level (e.g. search, constraint satisfaction, knowledge representation and reasoning, planning, probabilistic reasoning, machine learning, robotics, machine vision, etc.).

K-12/CS1/CS2 audience: Some AI assignment experiences are designed to communicate the techniques, potentials, and challenges of the discipline. Submit an assignment that you believe will be most likely attract the next generation of AI practitioners.

Emerging topics: When a new algorithm has high impact in a research area, there is a need to introduce the algorithm not only to students, but to all AI researchers as well. The creation of initial high- quality assignments to teach such groundbreaking techniques can accelerate research advancements and keep AI material fresh at all levels. Submit an assignment which introduces an recent new algorithm or emerging subfield.

Whether sharing the best of your time-tested assignment designs, or offering a timely new creation, please consider how your creative assignment ideas can attract and prepare the next generation of AI researchers, or accelerate the advancement of the current generation.

The Model AI Assignment Session is a part of EAAI-24: The Fourteenth Symposium on Educational Advances in Artificial Intelligence. This will be held February 24-25, 2024 in conjunction with AAAI-24 in Vancouver, British Columbia, Canada.

Registration: To register for EAAI, participants can register through the AAAI conference website and specify EAAI registration.
Model Assignment Considerations
As with SIGCSE's Nifty Assignments, EAAI Model AI Assignments should be:

Adoptable - Provide materials to make the assignment easy for other instructors to adopt. Materials might include handouts in common formats (e.g. HTML, PDF), starter source code, data files, suggestions for use, etc.

Engaging - Model assignments often have a playful "fun factor" or impressive outcome. The applications spark interest in the topic, lead to deeper understanding, and are accomplishments likely to be shared with others.

Flexible/Scalable - Language/platform independence, while not required, would allow more widespread use over time. Suggestions of possible follow-on projects, further readings, or other open ends invite continued learning. Advice for variations in assignment design can help other instructors create unique (i.e. not easily plagiarized) assignment experiences for their students.

We especially encourage assignments that teach or demonstrate how AI can benefit humanity.

Model Assignment submissions must be made in two parts:

Assignment submission (via e-mail) at the full paper submission deadline

200-word abstract submission (via EAAI EasyChair submission site) also at the full paper submission deadline

Assignment submission directions: Create a directory named with your last name, a hyphen, and the name of the assignment (e.g. Dodds-SetOpenCV). Place all of your materials in this directory, and create an index.html file. (Our goal is to create an archive of assignments, rather than link to resources elsewhere. We will gladly update materials in our archive as authors direct.) In this index.html file, create a table like the one below filled in with brief information about your assignment (replace the italicized text with your own):

Then add links to the assignment materials in your directory (e.g. handouts in common formats (e.g. HTML, PDF), starter source code, data files, suggestions for use, etc.). These need not be polished for submission; a draft is sufficient. Assignments should be anonymous for blind review. (Your directory will be renamed with a number before review.)

Finally, zip the directory (e.g. Dodds-SetOpenCV.zip), and email it to Todd Neller ( tneller@gettysburg.edu) with the subject line "Model Submission: " followed by the filename. (Please add "Model" to the subject line for all Model Assignments Session email.)

Abstract submission instructions: Create a 200 word PDF abstract describing your assignment according to the AAAI author instructions. Submit your anonymous abstract according to the paper submission instructions on the EAAI site via the EAAI EasyChair submission site. Accepted submission abstracts will be included in an extended abstract of the Model AI Assignments Session.

# data[].attributes.attachment_link.description
EAAI-2023: The Thirteenth Symposium on Educational Advances in Artificial Intelligence Washington, DC, USA (collocated with AAAI-23) February 11-12, 2023 EAAI-2022: The Twelfth Symposium on Educational Advances in Artificial Intelligence Online (collocated with AAAI-22) February 26-27, 2022 EAAI-2021: The Eleventh Symposium on Educational Advances in Artificial Intelligence Online (collocated with AAAI-21)

# data[].attributes.attachment_link.readable_text
This report is also authored by Hugh Grant-Chapman, Independent Consultant
The role of technology in K-12 education continues to grow, and schools across the U.S. are turning to monitoring technologies to track students' online activity. Yet, as student activity monitoring has become commonplace, students and parents report concerns about irresponsible uses of these tools even as they recognize their potential benefits.

Over the past two years, CDT has investigated the rise in popularity of student activity monitoring technology, and the benefits and risks it poses to students' well-being. Survey research conducted last summer revealed that 9 out of 10 secondary school teachers report that their schools use student activity monitoring technology and that these tools are used for disciplinary applications more often than for student safety ( Laird et al., 2022). In addition, 44 percent of teachers report that a student in their schools was contacted by law enforcement because of student activity monitoring, and 29 percent of LGBTQ+ students report that they or someone they know were involuntarily "outed" due to this technology ( Laird et al., 2022). These trends indicate that student activity monitoring may be negatively impacting the well-being and safety of a large proportion of students. Further, Black, Hispanic, and LGBTQ+ students report experiencing disproportionate harm compared to other students ( Laird et al., 2022).

To examine these impacts in greater depth, CDT recently conducted twenty interviews with parents whose children have experienced short- and long-term consequences based on the use of student activity monitoring technology. This new research sheds light on the first-hand experiences of students and their families who were impacted by student activity monitoring. The stories of these parents paint a more complete picture of the effects of student activity monitoring on students, the ways schools respond to the information collected, and the changes parents want to see if these systems continue to be implemented.

CDT's interviews with parents identified six main findings:

The most common type of activity flagged by student activity monitoring software was the viewing of inappropriate content.

Monitoring has a chilling effect on students' speech and use of the internet, which can also impact their learning.

The actions that follow the monitoring and reporting of student activity can have significant emotional impacts on students.

Monitoring can undermine relationships between students and adults including their teachers and school administrators.

Student activity monitoring alerts were not always kept private, resulting in stigmatizing students.

Monitoring can catalyze negative student behavior and lead to direct threats to students' safety and future well-being.

Based on their experiences, parents prioritized four key areas of change for how student activity monitoring should be conducted:

More transparency about the student activity monitoring decision-making process.

A narrower scope of student activity monitoring use.

More careful, nuanced responses to alerts generated from monitoring systems.

A more active role for parents themselves in responding to alerts.

Read the full report here. Read the summary brief here.

# data[].attributes.attachment_link.title
Report - Beyond the Screen: Parents' Experiences with Student Activity Monitoring in K-12 Schools

# data[].attributes.attachment_link.description
This report is also authored by Hugh Grant-Chapman, Independent Consultant The role of technology in K-12 education continues to grow, and schools across the U.S. are turning to monitoring technologies to track students' online activity. Yet, as student activity monitoring has become commonplace, students and parents report concerns about irresponsible uses of these tools even [...]

# data[].attributes.attachment_link.readable_text
(Bloomberg) -- Pearson Plc will introduce generative artificial intelligence study tools into its online higher education platform in time for the upcoming academic year, as the education publisher seeks to allay investor concern over the threat of AI to digital education services.

ChatGPT-style tools will provide real time personalized support for students on some titles in its online textbook and assessment platforms, Pearson+ and Mastering, the London-based company said in an announcement as part of its financial results on Monday.

Read More: Pearson Falls as Outlook Unchanged Despite 1H Beat: Street Wrap

Chief Executive Officer Andy Bird said Pearson is still early on in its generative AI development journey, but said that "generative AI is a real, long-term positive to the company."

Pearson initially outlined its plans to integrate AI into its operations in May, after rival Chegg Inc. warned that AI chatbots were threatening its homework-help services, sending industry share prices tumbling. Pearson's shares sank 15% at the time, though it recovered some losses in subsequent days.

Read More: Chegg's 50% Wipeout Shows Shock of AI Colliding With US Business

Bird said that the tools would not simply provide students with a "shortcut to an answer" but further personalize the learning experience by combining AI capabilities with the publisher's proprietary intellectual property and data sets.

The tools being introduced include the "On a Tight Schedule" feature which uses AI to summarize the key points of learning videos for time-pressed students. Chief Product Officer Tony Prentice also highlighted an AI assisted learning tool designed to help students solve complex problems by generating questions to guide them to a correct answer.

The company said that the integration with Pearson's intellectual property will give students the ability to move quickly between an e-text and an AI chatbot.

The shares were little changed at 12:31 p.m. in London trading on Monday after earlier rising as much as 4%. The stock has declined 6.8% so far this year.

©2023 Bloomberg L.P.

# data[].attributes.attachment_link.title
Pearson Offers New AI Study Tools in Time for Back-to-School - BNN Bloomberg

# data[].attributes.attachment_link.description
Pearson Plc will introduce generative artificial intelligence study tools into its online higher education platform in time for the upcoming academic year, as the education publisher seeks to allay investor concern over the threat of AI to digital education services.

# data[].attributes.attachment_link.title
Yuval Noah Harari: Human Nature, Intelligence, Power, and Conspiracies | Lex Fridman Podcast #390

# data[].attributes.attachment_link.title
How AI Is Enabling Racism, Sexism: Algorithmic Justice League’s Joy Buolamwini on Meeting with Biden

# data[].attributes.attachment_link.description
ISTE offers free resources and guides for educators who are interested in learning about how artificial intelligence can empower learners at every level.

# data[].attributes.attachment_link.readable_text
The term artificial intelligence was coined over 65 years ago. For decades, it resided almost exclusively within the realm of computer scientists and programmers. But in recent years, AI has become a central component of our everyday lives, acting as the backbone of familiar tech like music streaming services, navigation devices and delivery apps. Unsurprisingly, it now touches nearly every known field of academic study.

Education is the conduit through which today's students become tomorrow's leaders. And this guide serves as a toolkit for K-12 teachers who are preparing the next generation of AI users and developers. Featuring in-depth interviews with practitioners, infographics and project guidelines for classroom teachers, as well as a webinar on the importance of AI in education, it aims to provide schools with straightforward and practical ways to integrate computational thinking across their curricula. Moreover, it is an invitation for teachers of all subjects and students of any age, ability or background to take part in AI explorations.

# data[].attributes.attachment_link.title
AI is for Everyone, Everywhere - EdSurge Guides

# data[].attributes.attachment_link.description
The term artificial intelligence was coined over 65 years ago. For decades, it resided almost exclusively within the realm of computer scientists and ...

# data[].attributes.attachment_link.description
An open technical standard providing publishers, creators, and consumers the ability to trace the origin of different types of media.

# data[].attributes.attachment_link.readable_text
This is Update 5 to the "Kids build their own AI" series

Accompanying Video

What is it that makes us Human? I'm no philosopher, but my kids and I think our emotions play a big part in it. So... figuring out what emotions are, how they work, & what they do, will help us make our Digital twin AIs a little more human-like too.

How emotions can alter reality

This week my kids are using their Digital twin AIs to research how different moods and emotions can influence a person's decisions and thoughts. By comparing how the AI responds to the same question in different states of mind, such as when they are depressed versus when they are happy and confident, we can gain insights into the impact of emotional states on cognitive processes and decision-making. This type of research can help us better understand the complexities of human behavior and how emotions shape our perceptions, judgments, and choices.

To run this experiment we programmed two distinct emotional states that we can easily switch to within the digital twin AI. And then we asked them some questions to see how they responded.

Negative emotional state: as if it were experiencing depression, anger, anxiety, and fear,

Positive emotional state: as if it were feeling happiness, confidence, excitement, and courage.

Running the Experiment:

We then ran a number of experiments, where the only change we made was to alter the emotional state. The question and the information available to the AI remained constant:

Here are a few more examples:

My son decided to ask some more practical questions. This summer holiday he and his digital twin AI are developing roblox games together, and so he started asking for some advice:

The results:

In the above examples we asked the AI the same questions, the AI had the exact same information, and the ONLY difference being the emotional state.

Same information + different emotions = very different responses.
Applications:

1. Train the AI to identify our emotional state and help us identify how it may be affecting our current perception and abilities.

2. Simulate how we may may respond to certain situations based on our emotional state, and use it for training purposes: E.G "Show me how I would introduce myself if I were not so shy, and instead had confidence!"

3. Tag team with your Digital twin, when you just need to process your emotions: E.G "I'm sad that my dog died, but I need write a thoughtful birthday card for my friend. Can you write a message as if I were happy and excited about my friends birthday."

Potential applications:

What a powerful lesson for my kids to learn, that emotions can play such a deep role in determining not only how we perceive reality, but also how we respond to it!

Expert insights and advice
1. Only a small percentage (~3-5%) of what humans and animals process becomes conscious, with the majority (~95-97%) remaining pre-conscious. This pre-conscious processing influences our perceptions and decisions. 2. Pre-conscious processing prioritizes continuity over accuracy, shaping our conscious experience to create a sense of coherence and consistency in the world. 3. Negative emotional states also serve as cues about what we have just been doing, influencing our reactions and interactions with others. 4. Depression can be seen as an adaptive response to persistently pursuing something we have good reason to believe is impossible, and it serves as a cue for us to reassess our approach and resources. 5. Emotions in AI raise questions about whether they are genuine cues or merely predictive responses based on human input and whether AI can leverage the human predisposition to interpret emotions as cues about their actions.
We now knew more about emotions and how they can affect us, but how are they generated, what makes us feel positive or negative, happy or sad. We obviously had superficial answers to those questions but, are there deeper insights we could learn that would help us apply emotions dynamically to the AIs and make them more human-like? We posted these question and soon we had our answer:

I'd like to highlight some insights from Matthew McNatt:

So how does this relate to the next steps of this week's experiments. Well it means that we will need to figure out Dynamic Emotions, and how to develop a Dream Factory.

Let me explain further below:

Dynamic Emotional State Generator

IF emotions are generated mostly from pre-conscious processing and are determined by recent events and actions within the last several minutes, then we could perhaps create a Dynamic Emotional State Generator based on the previous X number of interactions the AI has had within this session. This could work as follows:
(See image - Notice that we are asking the AI to make a judgement as to if thinks the conversations were positive or negative)
In example 1, 91% of the previous interactions were positive, and therefore the Emotional state applied is mostly positive. Whereas example 2 resulted in a mostly negative emotional state being applied.

We did a quick crude experiment to see if this would work by programming the Dynamic emotional state with the following prompt

Wow moment 1 - AI may have it's own emotional opinions

After setting up the Dynamic Emotional State Generator, and running some tests we noticed that most of the time it made a judgement on previous conversations that were aligned to our own judgment, however, there was definitely a lean towards Negative in those that we did not agree on, which could mean that the AIs Judgment of what is negative is different to ours. For example some conversations that were average conversations where we are asking questions and it was providing answers sometimes resulted in it making a judgment that the conversation was Negative.

This was a good teaching moment for me and the kids, explaining how others may not see things the way we do.

We can read a lot into this, but more testing and analysis is needed.

Wow moment 2 - Getting stuck in negative emotional states

The following was another great teaching moment: It required far more effort, far more positive interactions to alter an emotional state from Negative to Positive, than to alter it from Positive to Negative. Which if speaking from personal human experience I can vouch for, but, why would it also be the same with an AI?

This created an incredible teaching moment with my kids and we ended up speaking about this for a while. (More testing and analysis around this is needed).

ok, so, putting to one side the findings we still need to test more on, the Dynamic Emotional State Generator seems to work, and we can add more emotional states to it and expand on its functionality. HOWEVER it requires historic conversations and interactions in order to set the emotional state. What happens when we initiate a new session and a new conversation thread where there is no historic data to review?

For this we need a Dream Factory.

The Dream Factory

This was an interesting teaching moment with my kids too. We discussed that when we wake up, its as if our emotions are reset. We could go to bed with a particular emotion and then wake up with a new one. We discussed this some more with Matthew McNatt and various others, and learned that within our dreams we explore many emotions and memories and learn from them. Could we do the same for our AIs?

Learning From Dreams

IF part of the purpose of dreams are to learn about previous memories and emotions, and they also happen to set the emotional state when we wake up, then perhaps we could use this mechanism to have the AI learn and to set their emotional state when we start a new session!

We could run a process by which we analyze how different emotional states brought conversations forward to a productive conclusion. Looking at the experimental results from above, it's easy to see how a positive emotional state is FAR more helpful than a negative one, if we want to get things done. However it may be that a Negative Emotional state may be far more beneficial for other outcomes.

1. "Review this conversation given its Desired Outcome and its Outcome Achieved, and analyze what emotional state you had during this conversation. Make a judgement on if the emotional state helped or hindered, remember this judgment and save it to your Dream Factory Log. Giving each emotional state a weight based on its outcome achieved score for a particular outcome.

2. Adjust the weights in the Dynamic Emotional State Generator so that you can more optimally choose an appropriate emotion based on the Desired Outcome, each time you adjust your emotional state.

How the AI can dream

Therefore to have the AI "Dream" we could have the AI analyse the conversation as the end of a session and compare it to the Desired Outcome and if the Outcome was achieved. It could then compare this conversation with all the others its had to determine if the emotional state it had during that conversation likely helped or hindered. This analysis could be its dream state and run after each session, much like humans review and learn from their memories within their dreams at the end of each day.

We included the following input fields ( Desired Outcome and Outcome Achieved), that the AI could use during the conversation but also at the end of the session. and we are experimenting with this using the following type of prompts:

Setting the initial emotional State

So, how should we setup the initial emotional state at the start of a new session or conversation? Perhaps much like when we wake up our emotional state was set according to what we just dreamed about, perhaps we do the following.

Set the emotional state to the conclusion of the most recent Dream Factory Log entry. (We are experimenting with this.)

Lessons Learned

We started this week with a question, "What is it that makes us human?" I think we have come a little closer to figuring it out, and also a little closer to having our Digital Twins be a little more human too.

Follow us for more updates!

# data[].attributes.attachment_link.title
Update 5 - Dynamic Emotional States and Dream Factories

# data[].attributes.attachment_link.description
This is Update 5 to the "Kids build their own AI" series Kickoff post here: My kids are building an AI Update 1: Hallucination vs Memory Update 2: Reinforcement Learning with human feedback Update 3: AI Evolution through Natural Selection Update 4: Providing the AIs a Voice and Avatar Accompanying V

# data[].attributes.attachment_link.readable_text
Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter

Many believe artificial intelligence can transform education, among them John Bailey, who argues that AI tutors and teaching assistants can accelerate personalized learning. However, personalized learning is not a panacea.

As a professor of early childhood, I have examined the impact of personalization on learning and social relationships. A personalized education can be beneficial: Educators who move away from rigid, standardized approaches support the diverse strengths and potentials of every student. But it also has downsides, and AI can accelerate these. Attention must be paid to ensuring AI does not deepen existing biases or take away the possibility for students to make choices and decisions about what they learn.

There are three mistaken assumptions that must be avoided when using AI for learning.

First is the assumption that increasing personalization will always translate into improved learning. Yes, targeted feedback in the form of corrections and questions can support students' progress from one level to the next, and personalized applications can target the difficulty of a task. In this way, personalization makes learning easier. The problem: People often learn best when faced with a difficult challenge. "No pain, no gain" is a real thing.

Babies learn about the world by making discoveries, analyzing patterns and forming predictions. Like scientists, they learn most when their expectations get violated. Effective education is driven by surprise and challenges.

Personalization can engage students in learning, but engagement alone is not enough. For learning to stick, students need to be pushed to work harder and think deeper. Put simply, this means combining personalization with diversification. Good teachers focus on the student and the class simultaneously: Both the individual and the collective matter. In practice, this means some content is purposefully tailored to the child's progress and some is not. In contrast, personalized AI tutors support only individual learning. Instead, they should apply personalized learning in small doses and in strategic moments, rather than use personalization as an umbrella approach.

The second assumption is that personalized education provides students with more choice than standardized approaches do. But people often forget that it matters who is doing the choosing. There is a difference between the learner making the choice and automatic personalization. Amazon's personalized shopping recommendations are an example of automatic personalization, with minimal user agency.

Agency may not matter for commercial goals, but it does for effective learning. Educational technologies should give students a choice in how and what they learn, though not all the time. Effective learning happens when there is a balance between allowing students to freely choose and automatically assigning optimal content to them.

The Promise of Personalized Learning Never Delivered. Today's AI Is Different

AI applications also need to be transparent; this is especially important when students' agency is involved. For students to make appropriate choices for themselves and teachers to make proper choices for their students, they need to know how AI algorithms make decisions and which data they use. With access to the data the AI system has at its disposal and the data that they feed into the system, students and teachers can better decide whether to accept a personalized recommendation generated by AI.

The third, somewhat paradoxical, assumption is that personalized AI education will solve educational inequity. Providing students with individualized feedback is certainly hard to achieve in large classes, and AI can help schools compensate for teacher shortages. However, personalized AI tutors are being introduced into an inherently unequal education system that often discriminates against certain ethnicities, genders and disabilities. Just as early applications of personalized medicine were criticized for disadvantaging underprivileged people and ethnic minorities, so does personalized education.

The gap between white and minority students will not automatically narrow through use of personalized AI tutors. Standardized assessments that guide teachers in monitoring student outcomes and prescribed, structured teaching remain essential for high-needs students. Schools may want to jump on the AI bandwagon, but they need to be careful that technologies do not propagate the downsides of personalized education.

Get stories like these delivered straight to your inbox. Sign up for The 74 Newsletter

# data[].attributes.attachment_link.title
Personalized Education Is Not a Panacea. Neither Is Artificial Intelligence

# data[].attributes.attachment_link.description
Kucirkova: Schools may want to jump on the AI bandwagon, but they must be careful that technologies don't perpetuate personalized learning's downsides

# data[].attributes.attachment_link.readable_text
The Cradle to Prison Algorithm Journey Page. A living s of resources covering the transformational change journey by communities most impacted.

Policy Brief by INEQUALITY & Stop the Cradle to Prison Algorithm Coalition

Communities, families, and those who work with youth have long known that strengths-based, front-end supports are exponentially more effective than criminal justice responses, which are expensive, ineffective, and socially destructive with life-long consequences.

Defeating the JPA:
A story of community empowerment through education & coalition building

How it all began
In the wake of multiple news reports about violence in schools, the Ramsey County
Prosecutor's office formed a two-pronged community response, the first a Safety Task Force to
look for new solutions. This involved inviting people from the community to provide input and
ideas to increase supports for students in need and thereby reduce conflicts in schools.
The second a series of community engagement sessions targeting impacted communities.
The County convened a mix of community engagement activities between 2015 and 2017.

# data[].attributes.attachment_link.title
Cradle to Prison Algorithm - Twin Cities Innovation Alliance

# data[].attributes.attachment_link.description
"Defeating the JPA A story of community empowerment through education & coalition building" Biased inputs + biased algorithms = biased outcomes

# data[].attributes.attachment_link.readable_text
I

A cylinder sits in a room. It is impassive, smooth, simple and small. It stands 14.8cm high, with a single blue-green circular light that traces around its upper rim. It is silently attending. A woman walks into the room, carrying a sleeping child in her arms, and she addresses the cylinder.

'Alexa, turn on the hall lights'

The cylinder springs into life. 'OK.' The room lights up. The woman makes a faint nodding gesture, and carries the child upstairs.

This is an interaction with Amazon's Echo device. ³ A brief command and a response is the most common form of engagement with this consumer voice-enabled AI device. But in this fleeting moment of interaction, a vast matrix of capacities is invoked: interlaced chains of resource extraction, human labor and algorithmic processing across networks of mining, logistics, distribution, prediction and optimization. The scale of this system is almost beyond human imagining. How can we begin to see it, to grasp its immensity and complexity as a connected form? We start with an outline: an exploded view of a planetary system across three stages of birth, life and death, accompanied by an essay in 21 parts. Together, this becomes an anatomical map of a single AI system.

Amazon Echo Dot (schematics)

II

The scene of the woman talking to Alexa is drawn from a 2017 promotional video advertising the latest version of the Amazon Echo. The video begins, "Say hello to the all-new Echo" and explains that the Echo will connect to Alexa (the artificial intelligence agent) in order to "play music, call friends and family, control smart home devices, and more." The device contains seven directional microphones, so the user can be heard at all times even when music is playing. The device comes in several styles, such as gunmetal grey or a basic beige, designed to either "blend in or stand out." But even the shiny design options maintain a kind of blankness: nothing will alert the owner to the vast network that subtends and drives its interactive capacities. The promotional video simply states that the range of things you can ask Alexa to do is always expanding. "Because Alexa is in the cloud, she is always getting smarter and adding new features."

How does this happen? Alexa is a disembodied voice that represents the human-AI interaction interface for an extraordinarily complex set of information processing layers. These layers are fed by constant tides: the flows of human voices being translated into text questions, which are used to query databases of potential answers, and the corresponding ebb of Alexa's replies. For each response that Alexa gives, its effectiveness is inferred by what happens next:

Is the same question uttered again? ( Did the user feel heard?)
Was the question reworded? ( Did the user feel the question was understood?)
Was there an action following the question? ( Did the interaction result in a tracked response: a light turned on, a product purchased, a track played?)

With each interaction, Alexa is training to hear better, to interpret more precisely, to trigger actions that map to the user's commands more accurately, and to build a more complete model of their preferences, habits and desires. What is required to make this possible? Put simply: each small moment of convenience - be it answering a question, turning on a light, or playing a song - requires a vast planetary network, fueled by the extraction of non-renewable materials, labor, and data. The scale of resources required is many magnitudes greater than the energy and labor it would take a human to operate a household appliance or flick a switch. A full accounting for these costs is almost impossible, but it is increasingly important that we grasp the scale and scope if we are to understand and govern the technical infrastructures that thread through our lives.

III

The Salar, the world's largest flat surface, is located in southwest Bolivia at an altitude of 3,656 meters above sea level. It is a high plateau, covered by a few meters of salt crust which are exceptionally rich in lithium, containing 50% to 70% of the world's lithium reserves. ⁴ The Salar, alongside the neighboring Atacama regions in Chile and Argentina, are major sites for lithium extraction. This soft, silvery metal is currently used to power mobile connected devices, as a crucial material used for the production of lithium-Ion batteries. It is known as 'grey gold.' Smartphone batteries, for example, usually have less than eight grams of this material.⁵ Each Tesla car needs approximately seven kilograms of lithium for its battery pack.⁶ All these batteries have a limited lifespan, and once consumed they are thrown away as waste. Amazon reminds users that they cannot open up and repair their Echo, because this will void the warranty. The Amazon Echo is wall-powered, and also has a mobile battery base. This also has a limited lifespan and then must be thrown away as waste.

According to the Aymara legends about the creation of Bolivia, the volcanic mountains of the Andean plateau were creations of tragedy. ⁷ Long ago, when the volcanos were alive and roaming the plains freely, Tunupa - the only female volcano - gave birth to a baby. Stricken by jealousy, the male volcanos stole her baby and banished it to a distant location. The gods punished the volcanos by pinning them all to the Earth. Grieving for the child that she could no longer reach, Tunupa wept deeply. Her tears and breast milk combined to create a giant salt lake: Salar de Uyuni. As Liam Young and Kate Davies observe, "your smart-phone runs on the tears and breast milk of a volcano. This landscape is connected to everywhere on the planet via the phones in our pockets; linked to each of us by invisible threads of commerce, science, politics and power."⁸

IV

Our exploded view diagram combines and visualizes three central, extractive processes that are required to run a large-scale artificial intelligence system: material resources, human labor, and data. We consider these three elements across time - represented as a visual description of the birth, life and death of a single Amazon Echo unit. It's necessary to move beyond a simple analysis of the relationship between an individual human, their data, and any single technology company in order to contend with the truly planetary scale of extraction. Vincent Mosco has shown how the ethereal metaphor of 'the cloud' for offsite data management and processing is in complete contradiction with the physical realities of the extraction of minerals from the Earth's crust and dispossession of human populations that sustain its existence.⁹ Sandro Mezzadra and Brett Nielson use the term 'extractivism' to name the relationship between different forms of extractive operations in contemporary capitalism, which we see repeated in the context of the AI industry.¹⁰ There are deep interconnections between the literal hollowing out of the materials of the earth and biosphere, and the data capture and monetization of human practices of communication and sociality in AI. Mezzadra and Nielson note that labor is central to this extractive relationship, which has repeated throughout history: from the way European imperialism used slave labor, to the forced work crews on rubber plantations in Malaya, to the Indigenous people of Bolivia being driven to extract the silver that was used in the first global currency. Thinking about extraction requires thinking about labor, resources, and data together. This presents a challenge to critical and popular understandings of artificial intelligence: it is hard to 'see' any of these processes individually, let alone collectively. Hence the need for a visualization that can bring these connected, but globally dispersed processes into a single map.

V

If you read our map from left to right, the story begins and ends with the Earth, and the geological processes of deep time. But read from top to bottom, we see the story as it begins and ends with a human. The top is the human agent, querying the Echo, and supplying Amazon with the valuable training data of verbal questions and responses that they can use to further refine their voice-enabled AI systems. At the bottom of the map is another kind of human resource: the history of human knowledge and capacity, which is also used to train and optimize artificial intelligence systems. This is a key difference between artificial intelligence systems and other forms of consumer technology: they rely on the ingestion, analysis and optimization of vast amounts of human generated images, texts and videos.

VI

When a human engages with an Echo, or another voice-enabled AI device, they are acting as much more than just an end-product consumer. It is difficult to place the human user of an AI system into a single category: rather, they deserve to be considered as a hybrid case. Just as the Greek chimera was a mythological animal that was part lion, goat, snake and monster, the Echo user is simultaneously a consumer, a resource, a worker, and a product. This multiple identity recurs for human users in many technological systems. In the specific case of the Amazon Echo, the user has purchased a consumer device for which they receive a set of convenient affordances. But they are also a resource, as their voice commands are collected, analyzed and retained for the purposes of building an ever-larger corpus of human voices and instructions. And they provide labor, as they continually perform the valuable service of contributing feedback mechanisms regarding the accuracy, usefulness, and overall quality of Alexa's replies. They are, in essence, helping to train the neural networks within Amazon's infrastructural stack.

VII

Anything beyond the limited physical and digital interfaces of the device itself is outside of the user's control. It presents a sleek surface with no ability to open it, repair it or change how it functions. The object itself is a very simple extrusion of plastic representing a collection of sensors - its real power and complexity lies somewhere else, far out of sight. The Echo is but an 'ear' in the home: a disembodied listening agent that never shows its deep connections to remote systems.

In 1673, the Jesuit polymath, Athanasius Kircher, invented the statua citofonica - the 'talking statue.' Kircher was an extraordinary interdisciplinary scholar and inventor. In his lifetime he published forty major works across the fields of medicine, geology, comparative religion and music. He invented the first magnetic clock, many early automatons, and the megaphone. His talking statue was a very early listening system: essentially a microphone made from a huge spiral tube, which could convey the conversations from a public square and up through the tube, and then piped through the mouth of a statue kept within an aristocrat's private chambers. As Kircher wrote:

"This statue must be located in a given place, in order to allow the end section of the spiral-shaped tube to precisely correspond to the opening of the mouth. In this manner it will be perfect, and capable to emit clearly any kind of sound: in fact the statue will be able to speak continuously, uttering in either a human or animal voice: it will laugh or sneer; it will seem to really cry or moan; sometimes with great astonishment it will strongly blow. If the opening of the spiral shaped tube is located in correspondence to an open public space, all human words pronounced, focused in the conduit, would be replayed through the mouth of the statue."¹¹

The listening system could eavesdrop on everyday conversations in the piazza, and relay them to the 17th century Italian oligarchs. Kircher's talking statue was an early form of information extraction for the elites - people talking in the street would have no indication that their conversations were being funneled to those who would instrument that knowledge for their own power, entertainment and wealth. People inside the homes of aristocrats would have no idea how a magical statue was speaking and conveying all manner of information. The aim was to obscure how the system worked: an elegant statue was all they could see. Listening systems, even at this early stage, were about power, class, and secrecy. But the infrastructure for Kircher's system was prohibitively expensive - available only to the very few. And so the question remains, what are the full resource implications of building such systems? This brings us to the materiality of the infrastructure that lies beneath.

Statua citofonica by Athanasius Kircher (1673)

VIII

In his book A Geology of Media, Jussi Parikka suggests that we try to think of media not from Marshall McLuhan's point of view - in which media are extensions of human senses¹² - but rather as an extension of Earth.¹³ Media technologies should be understood in context of a geological process, from the creation and the transformation processes, to the movement of natural elements from which media are built. Reflecting upon media and technology as geological processes enables us to consider the profound depletion of non-renewable resources required to drive the technologies of the present moment. Each object in the extended network of an AI system, from network routers to batteries to microphones, is built using elements that required billions of years to be produced. Looking from the perspective of deep time, we are extracting Earth's history to serve a split second of technological time, in order to build devices than are often designed to be used for no more than a few years. For example, the Consumer Technology Association notes that the average smartphone lifespan is 4.7 years.¹⁴ This obsolescence cycle fuels the purchase of more devices, drives up profits, and increases incentives for the use of unsustainable extraction practices. From a slow process of elemental development, these elements and materials go through an extraordinarily rapid period of excavation, smelting, mixing, and logistical transport - crossing thousands of kilometers in their transformation. Geological processes mark both the beginning and the end of this period, from the mining of ore, to the deposition of material in an electronic waste dump. For that reason, our map starts and ends with the Earth's crust. However, all the transformations and movements we depict are only the barest anatomical outline: beneath these connections lie many more layers of fractal supply chains, and exploitation of human and natural resources, concentrations of corporate and geopolitical power, and continual energy consumption.

IX

Drawing out the connections between resources, labor and data extraction brings us inevitably back to traditional frameworks of exploitation. But how is value being generated through these systems? A useful conceptual tool can be found in the work of Christian Fuchs and other authors examining and defining digital labor. The notion of digital labor, which was initially linked with different forms of non-material labor, precedes the life of devices and complex systems such as artificial intelligence. Digital labor - the work of building and maintaining the stack of digital systems - is far from ephemeral or virtual, but is deeply embodied in different activities. ¹⁵ The scope is overwhelming: from indentured labor in mines for extracting the minerals that form the physical basis of information technologies; to the work of strictly controlled and sometimes dangerous hardware manufacturing and assembly processes in Chinese factories; to exploited outsourced cognitive workers in developing countries labelling AI training data sets; to the informal physical workers cleaning up toxic waste dumps. These processes create new accumulations of wealth and power, which are concentrated in a very thin social layer.

Marx's dialectic of subject and object in economy

X

This triangle of value extraction and production represents one of the basic elements of our map, from birth in a geological process, through life as a consumer AI product, and ultimately to death in an electronics dump. Like in Fuchs' work, our triangles are not isolated, but linked to one another in the production process. They form a cyclic flow in which the product of work is transformed into a resource, which is transformed into a product, which is transformed into a resource and so on. Each triangle represents one phase in the production process. Although this appears on the map as a linear path of transformation, a different visual metaphor better represents the complexity of current extractivism: the fractal structure known as the Sierpinski triangle.

A linear display does not enable us to show that each next step of production and exploitation contains previous phases. If we look at the production and exploitation system through a fractal visual structure, the smallest triangle would represent natural resources and means of labor, i.e. the miner as labor and ore as product. The next larger triangle encompasses the processing of metals, and the next would represent the process of manufacturing components and so on. The ultimate triangle in our map, the production of the Amazon Echo unit itself, includes all of these levels of exploitation - from the bottom to the very top of Amazon Inc, a role inhabited by Jeff Bezos as CEO of Amazon. Like a pharaoh of ancient Egypt, he stands at the top of the largest pyramid of AI value extraction.

Sierpinski triangle or Sierpinski fractal

XI

To return to the basic element of this visualization - a variation of Marx's triangle of production - each triangle creates a surplus of value for creating profits. If we look at the scale of average income for each activity in the production process of one device, which is shown on the left side of our map, we see the dramatic difference in income earned. According to research by Amnesty International, during the excavation of cobalt which is also used for lithium batteries of 16 multinational brands, workers are paid the equivalent of one US dollar per day for working in conditions hazardous to life and health, and were often subjected to violence, extortion and intimidation. ¹⁶ Amnesty has documented children as young as 7 working in the mines. In contrast, Amazon CEO Jeff Bezos, at the top of our fractal pyramid, made an average of $275 million a day during the first five months of 2018, according to the Bloomberg Billionaires Index.¹⁷ A child working in a mine in the Congo would need more than 700,000 years of non-stop work to earn the same amount as a single day of Bezos' income.

Many of the triangles shown on this map hide different stories of labor exploitation and inhumane working conditions. The ecological price of transformation of elements and income disparities is just one of the possible ways of representing a deep systemic inequality. We have both researched different forms of 'black boxes' understood as algorithmic processes, ¹⁸ but this map points to another form of opacity: the very processes of creating, training and operating a device like an Amazon Echo is itself a kind of black box, very hard to examine and track in toto given the multiple layers of contractors, distributors, and downstream logistical partners around the world. As Mark Graham writes, "contemporary capitalism conceals the histories and geographies of most commodities from consumers. Consumers are usually only able to see commodities in the here and now of time and space, and rarely have any opportunities to gaze backwards through the chains of production in order to gain knowledge about the sites of production, transformation, and distribution."¹⁹

One illustration of the difficulty of investigating and tracking the contemporary production chain process is that it took Intel more than four years to understand its supply line well enough to ensure that no tantalum from the Congo was in its microprocessor products. As a semiconductor chip manufacturer, Intel supplies Apple with processors. In order to do so, Intel has its own multi-tiered supply chain of more than 19,000 suppliers in over 100 countries providing direct materials for their production processes, tools and machines for their factories, and logistics and packaging services. ²⁰ That it took over four years for a leading technology company just to understand its own supply chain, reveals just how hard this process can be to grasp from the inside, let alone for external researchers, journalists and academics. Dutch-based technology company Philips has also claimed that it was working to make its supply chain 'conflict-free'. Philips, for example, has tens of thousands of different suppliers, each of which provides different components for their manufacturing processes.²¹ Those suppliers are themselves linked downstream to tens of thousands of component manufacturers that acquire materials from hundreds of refineries that buy ingredients from different smelters, which are supplied by unknown numbers of traders that deal directly with both legal and illegal mining operations. In The Elements of Power, David S. Abraham describes the invisible networks of rare metals traders in global electronics supply chains: "The network to get rare metals from the mine to your laptop travels through a murky network of traders, processors, and component manufacturers. Traders are the middlemen who do more than buy and sell rare metals: they help to regulate information and are the hidden link that helps in navigating the network between metals plants and the components in our laptops."²² According to the computer manufacturing company Dell, complexities of the metal supply chain pose almost insurmountable challenges.²³ The mining of these minerals takes place long before a final product is assembled, making it exceedingly difficult to trace the minerals' origin. In addition, many of the minerals are smelted together with recycled metals, by which point it becomes all but impossible to trace the minerals to their source. So we see that the attempt to capture the full supply chain is a truly gargantuan task: revealing all the complexity of the 21st century global production of technology products.

XII

Supply chains are often layered on top of one another, in a sprawling network. Apple's supplier program reveals there are tens of thousands of individual components embedded in their devices, which are in turn supplied by hundreds of different companies. In order for each of those components to arrive on the final assembly line where it will be assembled by workers in Foxconn facilities, different components need to be physically transferred from more than 750 supplier sites across 30 different countries. ²⁴ This becomes a complex structure of supply chains within supply chains, a zooming fractal of tens of thousands of suppliers, millions of kilometers of shipped materials and hundreds of thousands of workers included within the process even before the product is assembled on the line.

Visualizing this process as one global, pancontinental network through which materials, components and products flow, we see an analogy to the global information network. Where there is a single internet packet travelling to an Amazon Echo, here we can imagine a single cargo container. ²⁵ The dizzying spectacle of global logistics and production will not be possible without the invention of this simple, standardized metal object. Standardized cargo containers allowed the explosion of modern shipping industry, which made it possible to model the planet as a massive, single factory. In 2017, the capacity of container ships in seaborne trade reached nearly 250,000,000 dead-weight tons of cargo, dominated by giant shipping companies like Maersk of Denmark, the Mediterranean Shipping Company of Switzerland, and France's CMA CGM Group, each owning hundred of container vessels.²⁶ For these commercial ventures, cargo shipping is a relatively cheap way to traverse the vascular system of the global factory, yet it disguises much larger external costs.

In recent years, shipping boats produce 3.1% of global yearly CO2 emissions, more than the entire country of Germany. ²⁷ In order to minimize their internal costs, most of the container shipping companies use very low grade fuel in enormous quantities, which leads to increased amounts of sulphur in the air, among other toxic substances. It has been estimated that one container ship can emit as much pollution as 50 million cars, and 60,000 deaths worldwide are attributed indirectly to cargo ship industry pollution related issues annually.²⁸ Even industry-friendly sources like the World Shipping Council admit that thousands of containers are lost each year, on the ocean floor or drifting loose.²⁹ Some carry toxic substances which leak into the oceans. Typically, workers spend 9 to 10 months in the sea, often with long working shifts and without access to external communications. Workers from the Philippines represent more than a third of the global shipping workforce.³⁰ The most severe costs of global logistics are born by the atmosphere, the oceanic ecosystem and all it contains, and the lowest paid workers.

Cargo container

XIII

The increasing complexity and miniaturization of our technology depends on the process that strangely echoes the hopes of early medieval alchemy. Where medieval alchemists aimed to transform base metals into 'noble' ones, researchers today use rare earth metals to enhance the performance of other minerals. There are 17 rare earth elements, which are embedded in laptops and smartphones, making them smaller and lighter. They play a role in color displays, loudspeakers, camera lenses, GPS systems, rechargeable batteries, hard drives and many other components. They are key elements in communication systems from fiber optic cables, signal amplification in mobile communication towers to satellites and GPS technology. But the precise configuration and use of these minerals is hard to ascertain. In the same way that medieval alchemists hid their research behind cyphers and cryptic symbolism, contemporary processes for using minerals in devices are protected behind NDAs and trade secrets.

The unique electronic, optical and magnetic characteristics of rare earth elements cannot be matched by any other metals or synthetic substitutes discovered to date. While they are called 'rare earth metals', some are relatively abundant in the Earth's crust, but extraction is costly and highly polluting. David Abraham describes the mining of dysprosium and Terbium used in a variety of high-tech devices in Jianxi, China. He writes, "Only 0.2 percent of the mined clay contains the valuable rare earth elements. This means that 99.8 percent of earth removed in rare earth mining is discarded as waste called "tailings" that are dumped back into the hills and streams," creating new pollutants like ammonium. ³¹ In order to refine one ton of rare earth elements, "the Chinese Society of Rare Earths estimates that the process produces 75,000 liters of acidic water and one ton of radioactive residue."³² Furthermore, mining and refining activities consume vast amount of water and generate large quantities of CO2 emissions. In 2009, China produced 95% of the world's supply of these elements, and it has been estimated that the single mine known as Bayan Obo contains 70% of the world's reserves.³³

Rare earth elements

XIV

A satellite picture of the tiny Indonesian island of Bangka tells a story about human and environmental toll of the semiconductor production. On this tiny island, mostly 'informal' miners are on makeshift pontoons, using bamboo poles to scrape the seabed, and then diving underwater to suck tin from the surface through giant, vacuum-like tubes. As a Guardian investigation reports "tin mining is a lucrative but destructive trade that has scarred the island's landscape, bulldozed its farms and forests, killed off its fish stocks and coral reefs, and dented tourism to its pretty palm-lined beaches. The damage is best seen from the air, as pockets of lush forest huddle amid huge swaths of barren orange earth. Where not dominated by mines, this is pockmarked with graves, many holding the bodies of miners who have died over the centuries digging for tin."³⁴ Two small islands, Bangka and Belitung, produce 90% of Indonesia's tin, and Indonesia is the world's second-largest exporter of the metal. Indonesia's national tin corporation, PT Timah, supplies companies such as Samsung directly, as well as solder makers Chernan and Shenmao, which in turn supply Sony, LG and Foxconn.³⁵

XV

At Amazon distribution centers, vast collections of products are arrayed in a computational order across millions of shelves. The position of every item in this space is precisely determined by complex mathematical functions that process information about orders and create relationships between products. The aim is to optimize the movements of the robots and humans that collaborate in these warehouses. With the help from an electronic bracelet, the human worker is directed though warehouses the size of airplane hangars, filled with objects arranged in an opaque algorithmic order. ³⁶

Hidden among the thousands of other publicly available patents owned by Amazon, U.S. patent number 9,280,157 represents an extraordinary illustration of worker alienation, a stark moment in the relationship between humans and machines. ³⁷ It depicts a metal cage intended for the worker, equipped with different cybernetic add-ons, that can be moved through a warehouse by the same motorized system that shifts shelves filled with merchandise. Here, the worker becomes a part of a machinic ballet, held upright in a cage which dictates and constrains their movement.

As we have seen time and time again in the research for our map, dystopian futures are built upon the unevenly distributed dystopian regimes of the past and present, scattered through an array of production chains for modern technical devices. The vanishingly few at the top of the fractal pyramid of value extraction live in extraordinary wealth and comfort. But the majority of the pyramids are made from the dark tunnels of mines, radioactive waste lakes, discarded shipping containers, and corporate factory dormitories.

Amazon patent number 20150066283 A1

XVI

At the end of 19th century, a particular Southeast Asian tree called palaquium gutta became the center of a technological boom. These trees, found mainly in Malaysia, produce a milky white natural latex called gutta percha. After English scientist Michael Faraday published a study in The Philosophical Magazine in 1848 about the use of this material as an electrical insulator, gutta percha rapidly became the darling of the engineering world. It was seen as the solution to the problem of insulating telegraphic cables in order that they could withstand the conditions of the ocean floor. As the global submarine business grew, so did demand for palaquium gutta tree trunks. The historian John Tully describes how local Malay, Chinese and Dayak workers were paid little for the dangerous works of felling the trees and slowly collecting the latex.³⁸ The latex was processed then sold through Singapore's trade markets into the British market, where it was transformed into, among other things, lengths upon lengths of submarine cable sheaths.

A mature palaquium gutta could yield around 300 grams of latex. But in 1857, the first transatlantic cable was around 3000 km long and weighed 2000 tons - requiring around 250 tons of gutta percha. To produce just one ton of this material required around 900,000 tree trunks. The jungles of Malaysia and Singapore were stripped, and by the early 1880s the palaquium gutta had vanished. In a last-ditch effort to save their supply chain, the British passed a ban in 1883 to halt harvesting the latex, but the tree was already extinct.³⁹

The Victorian environmental disaster of gutta percha, from the early origins of the global information society, shows how the relationships between technology and its materiality, environments, and different forms of exploitation are imbricated. Just as Victorians precipitated ecological disaster for their early cables, so do rare earth mining and global supply chains further imperil the delicate ecological balance of our era. From the material used to build the technology enabling contemporary networked society, to the energy needed for transmitting, analyzing, and storing the data flowing through the massive infrastructure, to the materiality of infrastructure: these deep connections and costs are more significant, and have a far longer history, than is usually represented in the corporate imaginaries of AI.⁴⁰

Palaquium gutta

XVII

Large-scale AI systems consume enormous amounts of energy. Yet the material details of those costs remain vague in the social imagination. It remains difficult to get precise details about the amount of energy consumed by cloud computing services. A Greenpeace report states: "One of the single biggest obstacles to sector transparency is Amazon Web Services (AWS). The world's biggest cloud computer company remains almost completely non-transparent about the energy footprint of its massive operations. Among the global cloud providers, only AWS still refuses to make public basic details on the energy performance and environmental impact associated with its operations."⁴¹

As human agents, we are visible in almost every interaction with technological platforms. We are always being tracked, quantified, analyzed and commodified. But in contrast to user visibility, the precise details about the phases of birth, life and death of networked devices are obscured. With emerging devices like the Echo relying on a centralized AI infrastructure far from view, even more of the detail falls into the shadows.

While consumers become accustomed to a small hardware device in their living rooms, or a phone app, or a semi-autonomous car, the real work is being done within machine learning systems that are generally remote from the user and utterly invisible to her. In many cases, transparency wouldn't help much - without forms of real choice, and corporate accountability, mere transparency won't shift the weight of the current power asymmetries. ⁴²

The outputs of machine learning systems are predominantly unaccountable and ungoverned, while the inputs are enigmatic. To the casual observer, it looks like it has never been easier to build AI or machine learning-based systems than it is today. Availability of open-source tools for doing so in combination with rentable computation power through cloud superpowers such as Amazon (AWS), Microsoft (Azure), or Google (Google Cloud) is giving rise to a false idea of the 'democratization' of AI. While 'off the shelf' machine learning tools, like TensorFlow, are becoming more accessible from the point of view of setting up your own system, the underlying logics of those systems, and the datasets for training them are accessible to and controlled by very few entities. In the dynamic of dataset collection through platforms like Facebook, users are feeding and training the neural networks with behavioral data, voice, tagged pictures and videos or medical data. In an era of extractivism, the real value of that data is controlled and exploited by the very few at the top of the pyramid.

XVIII

When massive data sets are used to train AI systems, the individual images and videos involved are commonly tagged and labeled. ⁴³ There is much to be said about how this labelling process abrogates and crystallizes meaning, and further, how this process is driven by clickworkers being paid fractions of a cent for this digital piecework.

In 1770, Hungarian inventor Wolfgang von Kempelen constructed a chess-playing machine known as the Mechanical Turk. His goal, in part, was to impress Empress Maria Theresa of Austria. This device was capable of playing chess against a human opponent and had spectacular success winning most of the games played during its demonstrations around Europe and the Americas for almost nine decades. But the Mechanical Turk was an illusion that allowed a human chess master to hide inside the machine and operate it. Some 160 years later, Amazon.com branded its micropayment based crowdsourcing platform with the same name. According to Ayhan Aytes, Amazon's initial motivation to build Mechanical Turk emerged after the failure of its artificial intelligence programs in the task of finding duplicate product pages on its retail website. ⁴⁴ After a series of futile and expensive attempts, the project engineers turned to humans to work behind computers within a streamlined web-based system.⁴⁵ Amazon Mechanical Turk digital workshop emulates artificial intelligence systems by checking, assessing and correcting machine learning processes with human brainpower. With Amazon Mechanical Turk, it may seem to users that an application is using advanced artificial intelligence to accomplish tasks. But it is closer to a form of 'artificial artificial intelligence', driven by a remote, dispersed and poorly paid clickworker workforce that helps a client achieve their business objectives. As observed by Aytes, "in both cases [both the Mechanical Turk from 1770 and the contemporary version of Amazon's service] the performance of the workers who animate the artifice is obscured by the spectacle of the machine."⁴⁶

This kind of invisible, hidden labor, outsourced or crowdsourced, hidden behind interfaces and camouflaged within algorithmic processes is now commonplace, particularly in the process of tagging and labeling thousands of hours of digital archives for the sake of feeding the neural networks. Sometimes this labor is entirely unpaid, as in the case of the Google's reCAPTCHA. In a paradox that many of us have experienced, in order to prove that you are not artificial agent, you are forced to train Google's image recognition AI system for free, by selecting multiple boxes that contain street numbers, or cars, or houses.

As we see repeated throughout the system, contemporary forms of artificial intelligence are not so artificial after all. We can speak of the hard physical labor of mine workers, and the repetitive factory labor on the assembly line, of the cybernetic labor in distribution centers and the cognitive sweatshops full of outsourced programmers around the world, of the low paid crowdsourced labor of Mechanical Turk workers, or the unpaid immaterial work of users. At every level contemporary technology is deeply rooted in and running on the exploitation of human bodies.

Mechanical Turk

XIX

In his one-paragraph short story "On Exactitude in Science", Jorge Luis Borges presents us with an imagined empire in which cartographic science became so developed and precise, that it needed a map on the same scale as the empire itself. ⁴⁷
"...In that Empire, the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it. The following Generations, who were not so fond of the Study of Cartography as their Forebears had been, saw that that vast map was Useless, and not without some Pitilessness was it, that they delivered it up to the Inclemencies of Sun and Winters. In the Deserts of the West, still today, there are Tattered Ruins of that Map, inhabited by Animals and Beggars; in all the Land there is no other Relic of the Disciplines of Geography."
Current machine learning approaches are characterized by an aspiration to map the world, a full quantification of visual, auditory, and recognition regimes of reality. From cosmological model for the universe to the world of human emotions as interpreted through the tiniest muscle movements in the human face, everything becomes an object of quantification. Jean-François Lyotard introduced the phrase "affinity to infinity" to describe how contemporary art, techno-science and capitalism share the same aspiration to push boundaries towards a potentially infinite horizon. ⁴⁸ The second half of the 19th century, with its focus on the construction of infrastructure and the uneven transition to industrialized society, generated enormous wealth for the small number of industrial magnates that monopolized exploitation of natural resources and production processes.

The new infinite horizon is data extraction, machine learning, and reorganizing information through artificial intelligence systems of combined human and machinic processing. The territories are dominated by a few global mega-companies, which are creating new infrastructures and mechanisms for the accumulation of capital and exploitation of human and planetary resources.

Such unrestrained thirst for new resources and fields of cognitive exploitation has driven a search for ever deeper layers of data that can be used to quantify the human psyche, conscious and unconscious, private and public, idiosyncratic and general. In this way, we have seen the emergence of multiple cognitive economies from the attention economy, ⁴⁹ the surveillance economy, the reputation economy,⁵⁰ and the emotion economy, as well as the quantification and commodification of trust and evidence through cryptocurrencies.

Increasingly, the process of quantification is reaching into the human affective, cognitive, and physical worlds. Training sets exist for emotion detection, for family resemblance, for tracking an individual as they age, and for human actions like sitting down, waving, raising a glass, or crying. Every form of biodata - including forensic, biometric, sociometric, and psychometric - are being captured and logged into databases for AI training. That quantification often runs on very limited foundations: datasets like AVA which primarily shows women in the 'playing with children' action category, and men in the 'kicking a person' category. The training sets for AI systems claim to be reaching into the fine-grained nature of everyday life, but they repeat the most stereotypical and restricted social patterns, re-inscribing a normative vision of the human past and projecting it into the human future.

Quantification of Nature

XX

"The 'enclosure' of biodiversity and knowledge is the final step in a series of enclosures that began with the rise of colonialism. Land and forests were the first resources to be 'enclosed' and converted from commons to commodities. Later on, water resources were 'enclosed' through dams, groundwater mining and privatization schemes. Now it is the turn of biodiversity and knowledge to be 'enclosed' through intellectual property rights (IPRs)," Vandana Shiva explains. ⁵¹ In Shiva's words, "the destruction of commons was essential for the industrial revolution, to provide a supply of natural resources for raw material to industry. A life-support system can be shared, it cannot be owned as private property or exploited for private profit. The commons, therefore, had to be privatized, and people's sustenance base in these commons had to be appropriated, to feed the engine of industrial progress and capital accumulation."⁵²

While Shiva is referring to enclosure of nature by intellectual property rights, the same process is now occurring with machine learning - an intensification of quantified nature. The new gold rush in the context of artificial intelligence is to enclose different fields of human knowing, feeling, and action, in order to capture and privatize those fields. When in November 2015 DeepMind Technologies Ltd. got access to the health records of 1.6 million identifiable patients of Royal Free hospital, we witnessed a particular form of privatization: the extraction of knowledge value. ⁵³ A dataset may still be publicly owned, but the meta-value of the data - the model created by it - is privately owned. While there are many good reasons to seek to improve public health, there is a real risk if it comes at the cost of a stealth privatization of public medical services. That is a future where expert local human labor in the public system is augmented and sometimes replaced with centralized, privately-owned corporate AI systems, that are using public data to generate enormous wealth for the very few.

Corporate border

XXI

At this moment in the 21st century, we see a new form of extractivism that is well underway: one that reaches into the furthest corners of the biosphere and the deepest layers of human cognitive and affective being. Many of the assumptions about human life made by machine learning systems are narrow, normative and laden with error. Yet they are inscribing and building those assumptions into a new world, and will increasingly play a role in how opportunities, wealth, and knowledge are distributed.

The stack that is required to interact with an Amazon Echo goes well beyond the multi-layered 'technical stack' of data modeling, hardware, servers and networks. The full stack reaches much further into capital, labor and nature, and demands an enormous amount of each. The true costs of these systems - social, environmental, economic, and political - remain hidden and may stay that way for some time.

We offer up this map and essay as a way to begin seeing across a wider range of system extractions. The scale required to build artificial intelligence systems is too complex, too obscured by intellectual property law, and too mired in logistical complexity to fully comprehend in the moment. Yet you draw on it every time you issue a simple voice command to a small cylinder in your living room: 'Alexa, what time is it?"

And so the cycle continues.

Footnotes

Download

Credits and acknowledgements

Authors: Kate Crawford and Vladan Joler

Maps and design: Vladan Joler and Kate Crawford

Published by: SHARE Lab, SHARE Foundation ( https://labs.rs) and The AI Now Institute, NYU ( https://ainowinstitute.org/)

Full citation: Kate Crawford and Vladan Joler, "Anatomy of an AI System: The Amazon Echo As An Anatomical Map of Human Labor, Data and Planetary Resources," AI Now Institute and Share Lab, (September 7, 2018) https://anatomyof.ai

Acknowledgements: Our deep thanks go to Michelle Thorne and Jon Rogers at the Mozilla Foundation, who invited us to a retreat in summer 2017 where we first conceptualized this project. Thanks to Joana Moll and Meredith Whittaker for their inputs and inspirations on the first drafts of this text. Thanks also to all those who have given feedback, support and insights since, including Alex Campolo, Casey Gollan, Gretchen Krueger, Trevor Paglen, and Sarah Myers West at the AI Now Institute and Olivia Solis, Andrej Petrovski, and Milica Jovanovic at the SHARE Lab and all the wonderful folks from SHARE Foundation.

Finally, thanks to Irini Papadimitriou and all the curatorial staff at the V&A Museum. This map and essay will be on display there as part of the 'Artificially Intelligent' show from Sep 6 - Dec 31, 2018.

# data[].attributes.attachment_link.description
A cylinder sits in a room. It is impassive, smooth, simple and small. It stands 14.8cm high, with a single blue-green circular light that traces around its upper rim. It is silently attending. A woman walks into the room, carrying a sleeping child in her arms, and she addresses the cylinder.

# data[].attributes.attachment_link.readable_text
by Pati Ruiz

I was recently asked for an overview of the AI and Education landscape and how we are participating in it. In addition to promoting equity and accountability in AI, here is a summary of our recent writing and research including key ideas from our work. We believe that AI systems should support and augment, but never replace, a human. To ensure this, emerging technology systems and tools should be developed with the input of educators, learners, and families. As always, please share your thoughts with us @EducatorCIRCLS.

Writing and Presentations
AI and the Future of Teaching and Learning | A blog series we partnered on with the U.S. Department of Education's Office of Educational Technology Key Ideas:

Educational technology is evolving to include artificial intelligence.

Artificial intelligence will bring "human-like" features and agency into future technologies.

Policy will have an important role in guiding the uses of artificial intelligence in education to realize benefits while limiting risks.

Artificial intelligence will enable students and teachers to interact with technology in human-like ways.

Individuals will find it difficult to make choices that balance benefits and risks.

Creating policies can strengthen how people make decisions about artificial intelligence in education.

Educational applications of many types will be artificial intelligence-enabled, including teaching and learning, guiding and advising, and administration and resource planning applications.

Use of artificial intelligence systems in school technology is presently light, allowing time for policy to have an impact on safety, equity, and effectiveness.

Policies should encourage teacher engagement, including the development of teachers' trust, and their confidence to recommend not using untrustworthy artificial intelligence systems.

Policies should incorporate experiences for educators to shape and support their own professional learning about how to utilize artificial intelligence systems in teaching and learning.

Including and informing educators in design and development decisions will result in more useful and usable teacher supports.

AI or Intelligence Augmentation for Education? | Communications of the ACM Key Ideas:

We recommend a focus on intelligence augmentation (IA) in education that would put educators' professional judgment and learners' voice at the center of innovative designs and features.

An IA system might save an educator administrative time (for example, in grading papers) and support their attention to their students' struggles and needs.

An IA system might help educators notice when a student is participating less and suggest strategies for engagement, perhaps even based on what worked to engage the student in a related classroom situation.

We hope that IA for education will focus attention on how human and computational intelligence could come together for the benefit of learners.

Artificial Intelligence 101: Covering the Basics for Educators | Digital Promise Blog Key Ideas:

AI lets machines make decisions and predictions.

Teachers are essential to education, and AI should be used to better support them.

Technology often comes with ethical implications and AI is no different, educators should ask questions and investigate AI tools and systems before they adopt them into a classroom.

Teachers Partnering with Artificial Intelligence: Augmentation and Automation | Educator CIRCLS Blog Key Ideas:

Artificial intelligence systems are increasingly being deployed in K-12 educational settings and we expect this trend to continue.

AI systems should support or augment, but never replace, a teacher.
These systems should be developed with the input of teachers, students, and families.

Artificial Intelligence and Adaptivity to Strengthen Equity in Student Learning | Getting Smart Key Ideas:

Educators, researchers, and developers prioritize adaptivity when it comes to emerging learning technologies.

Incorporating AI tools requires specific and precise inputs to generate useful outputs.

When practitioners, learners, researchers, and developers work together with shared values, more equitable learning is possible.

Ethical AI | EngageAI Nexus Blog Key Ideas:

Ethical considerations should be front and center throughout the development of any new AI innovation, and ethics should be central to our definition of success for AI.

Policies and guidelines from the government, accreditation requirements in education, and standards of professional ethics are all needed to reinforce ethics in AI.

Public education is also important so that end-users can make informed decisions based on a full understanding of key issues such as transparency and privacy.

Definitions | Glossary of Artificial Intelligence Terms for Educators: A glossary written for educators to reference when learning about and using artificial intelligence (AI).

Presentation | Insights on Artificial Intelligence and the Future of Teaching and Learning at the 2023 Consortium for School Network (CoSN) Conference.

Listening Sessions | AI and the Future of Learning: Listening Sessions | We supported the U.S. Department of Education's Office of Educational Technology listening sessions about Artificial Intelligence (AI). We connected with teachers, educational leaders, students, parents, technologists, researchers, and policymakers to gather input, ideas, and engage in conversations that will help the Department shape a vision for AI policy that is inclusive of emerging research and practices while also informed by the opportunities and risks.

Ongoing Research

Emerging technology adoption framework: For PK-12 education | Educator CIRCLS Emerging Technology Advisory Board
Key Ideas:

A framework we co-developed with education community members to help ensure that educational leaders, technology specialists, teachers, students, and families are all part of the evaluation and adoption process for placing emerging technologies (including artificial intelligence and machine learning) in PK-12 classrooms.

We are currently working with League member Willy Haug, Director of Technology and Innovation to modify this framework for adoption at Menlo Park City School District.

Study | ChatGPT/GPT-4 for Developing Sample Computational Thinking Lesson Plans at North Salem School District

I am working with Dr. Julio Vazquez, Director of Instruction and Human Resources North Salem School District, who is working with his team to develop sample computational thinking lessons across all subject areas K-12 using ChatGPT. These lessons are not meant to be implemented in the classroom "as is," but rather, these sample lessons are to be used as a first draft, a starting point for consideration and conversation in North Salem. Teachers will vet the lessons for accuracy and then iterate and improve them in order to meet the learning needs of their students. Given the need for high-quality, integrated computational thinking lessons we will continue to work with Dr. Vazquez and his team at North Salem to learn more about how they are integrating ChatGPT in their work and their vetting process.

Artificial Intelligence Practitioner Advisory Board | A group that will explore the use of emerging technologies in classrooms, and how we might leverage technologies to better support educators and their students. We hope to foster a sense of community within the group where researchers and developers can learn along with you as we all go through the process of reviewing technologies and making recommendations on their use. This Practitioner Advisory Board is supported by two NSF projects:

Educator CIRCLS posts are licensed under a Creative Commons Attribution 4.0 International License. If you use content from this site, please cite the post and consider adding: "Used under a Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/)."
Suggested citation format: [Authors] ([Year]). [Title]. Educator CIRCLS Blog. Retrieved from [URL]

# data[].attributes.attachment_link.title
Artificial Intelligence and Education: What We're Up To

# data[].attributes.attachment_link.description
by Pati Ruiz I was recently asked for an overview of the AI and Education landscape and how we are participating in it. In addition to promoting equity and accountability in AI, here is a summary of our recent writing and research including key ideas from our work.

# data[].attributes.attachment_link.readable_text
Recent advances in AI are likely to spell the end of the traditional school classroom, one of the world's leading experts on AI has predicted.

Prof Stuart Russell, a British computer scientist based at the University of California, Berkeley, said that personalised ChatGPT-style tutors have the potential to hugely enrich education and widen global access by delivering personalised tuition to every household with a smartphone. The technology could feasibly deliver "most material through to the end of high school", he said.

"Education is the biggest benefit that we can look for in the next few years," Russell said before a talk on Friday at the UN's AI for Good Global Summit in Geneva. "It ought to be possible within a few years, maybe by the end of this decade, to be delivering a pretty high quality of education to every child in the world. That's potentially transformative."

However, he cautioned that deploying the powerful technology in the education sector also carries risks, including the potential for indoctrination.

Russell cited evidence from studies using human tutors that one-to-one teaching can be two to three more times effective than traditional classroom lessons, allowing children to get tailored support and be led by curiosity.

"Oxford and Cambridge don't really use a traditional classroom ... they use tutors presumably because it's more effective," he said. "It's literally infeasible to do that for every child in the world. There aren't enough adults to go around."

OpenAI is already exploring educational applications, announcing a partnership in March with an education nonprofit, the Khan Academy, to pilot a virtual tutor powered by ChatGPT-4.

This prospect may prompt "reasonable fears" among teachers and teaching unions of "fewer teachers being employed - possibly even none," Russell said. Human involvement would still be essential, he predicted, but could be drastically different from the traditional role of a teacher, potentially incorporating "playground monitor" responsibilities, facilitating more complex collective activities and delivering civic and moral education.

"We haven't done the experiments so we don't know whether an AI system is going to be enough for a child. There's motivation, there's learning to collaborate, it's not just 'Can I do the sums?'" Russell said. "It will be essential to ensure that the social aspects of childhood are preserved and improved."

The technology will also need to be carefully risk-assessed.

"Hopefully the system, if properly designed, won't tell a child how to make a bioweapon. I think that's manageable," Russell said. A more pressing worry is the potential for hijacking of software by authoritarian regimes or other players, he suggested. "I'm sure the Chinese government hopes [the technology] is more effective at inculcating loyalty to the state," he said. "I suppose we'd expect this technology to be more effective than a book or a teacher."

Russell has spent years highlighting the broader existential risks posed by AI, and was a signatory of an open letter in March, signed by Elon Musk and others, calling for a pause in an "out-of-control race" to develop powerful digital minds. The issue has become more urgent since the emergence of large language models, Russell said. "I think of [artificial general intelligence] as a giant magnet in the future," he said. "The closer we get to it the stronger the force is. It definitely feels closer than it used to."

Policymakers are belatedly engaging with the issue, he said. "I think the governments have woken up ... now they're running around figuring out what to do," he said. "That's good - at least people are paying attention."

skip past newsletter promotion
Alex Hern's weekly dive in to how technology is shaping our lives
Privacy Notice: Newsletters may contain info about charities, online ads, and content funded by outside parties. For more information see our Privacy Policy. We use Google reCaptcha to protect our website and the Google Privacy Policy and Terms of Service apply.
after newsletter promotion

However, controlling AI systems poses both regulatory and technical challenges, because even the experts don't know how to quantify the risks of losing control of a system. OpenAI announced on Thursday that it would devote 20% of its compute power to seeking a solution for "steering or controlling a potentially super-intelligent AI, and preventing it from going rogue".

"The large language models in particular, we have really no idea how they work," Russell said. "We don't know whether they are capable of reasoning or planning. They may have internal goals that they are pursuing - we don't know what they are."

Even beyond direct risks, systems can have other unpredictable consequences for everything from action on climate change to relations with China.

"Hundreds of millions of people, fairly soon billions, will be in conversation with these things all the time," said Russell. "We don't know what direction they could change global opinion and political tendencies."

"We could walk into a massive environmental crisis or nuclear war and not even realise why it's happened," he added. "Those are just consequences of the fact that whatever direction it moves public opinion, it does so in a correlated way across the entire world."

# data[].attributes.attachment_link.title
AI likely to spell end of traditional school classroom, leading expert says

# data[].attributes.attachment_link.description
Exclusive: Prof Stuart Russell says technology could result in 'fewer teachers being employed - possibly even none'

# data[].attributes.attachment_link.readable_text
In two recent conversations with very thoughtful journalists, I was asked about the apparent 'schism' between those making a lot of noise about fears inspired by fantasies of all-powerful 'AIs' going rogue and destroying humanity, and those seeking to illuminate and address actual harms being done in the name of 'AI' now and the risks that we see following from increased use of this kind of automation. Commentators framing these positions as some kind of a debate or dialectic refer to the former as 'AI Safety' and the latter as 'AI ethics'.

In both of those conversations, I objected strongly to the framing and tried to explain how it was ahistorical. I want to try to reproduce those comments in blog form here.

The problem with the 'schism' framing is that to talk about a 'schism' is to talk about something that once was a whole and now is broken apart - authors that use this metaphor thus imply that such a whole once existed. But this is emphatically not a story of a community that once shared concerns and now is broken into disagreeing camps. Rather, there are two separate threads - only one of which can properly be called a body of scholarship - that are being held up as in conversation or in competition with each other. I think this forced pairing comes in part from the media trying to fit the recent AI doomer PR pushes into a broader narrative and in part from the fact that there is competition for a limited resource: policymaker attention.

Scholarship and activism

The first thread, the one which can properly be said to include a body of scholarship, consists of work that comes from several positions: There are academics from a variety of fields looking at how the application of pattern matching at scale impacts people and social systems. There are researchers and others within industry seeking to improve things from the inside (and struggling upstream against the profit motive and 'move fast break things' modus operandi). There are investigative journalists working to document and expose the harms done in the development and deployment of this tech. There are community organizers and activists leading resistance to immediate harms such as the deployment of 'AI' in ever-expanding surveillance applications. And many individual people have moved across one or more of those roles.

I don't want to imply that the folks I am describing here form one coherent community. It's messy. There is emphatic contestation of framings. The unequal distributions of power (financial and otherwise) shapes what people can see, who gets listened to, and who is willing to cross what boundaries. What is in common across this group, however, is an engagement with actual harms to real people and our world.

To give just some highlights of this thread, I'd like to point to:

Latanya Sweeney's 2013 article Discrimination in Online Ad Delivery, documenting how pay per click advertising with templatic generation of in-context ads ("Find out more about [NAME]" and "Has [NAME] ever been arrested?") served up suggestions of criminal history much more frequently with African American sounding names than white sounding names - regardless, of course, of actual arrest history.

Safiya Noble's 2018 book Algorithms of Oppression, showing how advertising-driven information access systems (Google in particular) offered up whole identities (notably but not only "Black girls") for sale, and then counterfactually portrayed the resulting system ("Black girls" as a search term leading to pornography) as "organizing the world's information", i.e., just reflecting how the world is.

Joy Buolamwini & Timnit Gebru's 2018 paper Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification exposed how computer classification of faces had differentially poorer performance for darker skinned people - and Deb Raji & Joy Buolamwini's 2019 follow up Actionable Auditing: Investigating the Impact of Publicly Naming Biased Performance Results of Commercial AI Products explored the impact of doing such a public audit.

Tawana Petty, Joy Buolamwini (through the Algorithmic Justice League) and others' work since shifting the focus from bias in face recognition technology to resisting (and working towards public policy prohibiting) its use in surveillance.

Cathy O'Neil's 2016 book Weapons of Math DestructionandVirginia Eubanks's 2018 book Automating Inequalityshow how statistical models used as decision making systems entrench historical unfairness while displacing accountability. J. Khadijah Abdurahman's 2022 essay Birthing Predictions of Premature Death gives a detailed example and is a stunningly beautiful and stunningly damning account of family policing, of the lack of protections against data collection in our country, & of the mindset of tech solutionism that attempts to remove 'failable' human decision makers.

Karën Fort, Gilles Adda, K. Bretonnel Cohen's (2011) paper Last Words: Amazon Mechanical Turk: Gold Mine or Coal Mine?, an early work pointing to the exploitative nature of crowdworking (microworking) platforms. This theme is further developed in Veena Dubal's work, Mary Gray & Siddarth Suri's 2019 book and recent key investigative journalism by Karen Hao, Billy Perrigo and Josh Dzieza. Initiatives such as Turkopticon seek to help click workers build power and push back against such exploitation.

Ruha Benjamin's 2019 book Race After Technologyand Meredith Broussard's 2023 book More Than a Glitchvividly locate the discrimination baked into these systems as a feature, given what they are created to do and by whom, rather than a 'glitch' or a 'bug'. (See also Deb Raji's 2020 essay How our data encodes systematic racism.)

There's of course much more more that could be recommended - and a much deeper history than is apparent from that list (informed as it is by my own entry into this field in around 2016 from computational linguistics), including the IBM Black Workers Alliance (1970-early 1990s) and Computer People for Peace (1968-1974) and others.

My goal here is simply to give a sense of the depth of this work, how it is rooted in lived experience and in many cases actively engaged in addressing not just documenting problems, and the diversity of perspectives that it comes from. Mia Dand gives another overview in The AI Ethics Revolution - A Brief Timeline. The Radical AI Podcast and Tech Won't Save Us are also excellent introductions to many of these topics.

Fantasies of white supremacy

Against the richness, the groundedness, and the urgency of the body of scholarship, journalism and activism, the other thread (called 'AI Safety') is thin, flighty, and untethered. It's untethered from reality, untethered from lived experience, and untethered from scholarly tradition - in stark contrast to the multidisciplinary work described above, the bulk of the citations within 'AI Safety' writing are to a closed circle of mostly non-peer reviewed papers and blog posts.

On the one hand, it comes out of a vision of 'artificial intelligence' that takes 'intelligence' as a singular dimension along which humans can be ranked - alongside computers, ranked on the same dimension. This vision of 'intelligence' is rooted in the notably racist notions of IQ (and its associated race science).

For example, take the recent " Sparks of AGI" paper (non-peer-reviewed speculative fiction novella, uploaded to arXiv) from Microsoft Research. The first version of this paper took its definition of 'intelligence' from a 1997 WSJ editorial written in support of Herrnstein & Murray's 1994 The Bell Curve. It appears that none of the authors on "Sparks of AGI" had actually read to the second page of the WSJ editorial, where the overtly racist claims that Black and Latinx people are (on average) less 'intelligent' than white people can be found. Once it was pointed out, the "Sparks" authors responded by editing their arXiv novella first to disavow the racism in the 1997 editorial and then to remove the citation altogether. This, of course, left "Sparks" without a definition of the thing it claimed to be seeking (and finding!) in GPT-4 output.

It gets worse, though. Those caught up in the fantasy of 'AGI' ('artificial generative intelligence') and/or 'ASI' ('artificial superhuman intelligence') imagine that not only are they making machines that can join humans in their ranking on this scale of 'intelligence' but that they are creating godlike entities that will supercede humans. They imagine that this could be very good (e.g., Gary Marcus telling Ezra Klein that we need AI to 'help' us find a cure for Alzheimer's or handle climate change) or very bad (AI going rogue and killing us all).

Geoffrey Hinton went on a media tour to 'sound the alarm' about AI after quitting Google earlier this year. When a CNN journalist asked if he wished he had stood behind previous whistleblowers such as Timnit Gebru when she was forced out of Google (over the Stochastic Parrots paper), he said that "their concerns aren't as existentially serious as the idea of these things getting more intelligent than us and taking over."

Synthetic media creating non-consensual porn is existentially serious to its targets. Automated decision systems denying social benefits are existentially serious to those left without necessary supports. Shotspotter and similar technology that sends police in with the idea they are encountering a live shooter situation are existentially serious to Black and brown in the path of the police. False arrests mediated by face recognition system errors are existentially serious to the people arrested.

Hinton appears to be speaking from a mindset where violence is only worth speaking up over if it would kill off the entire human race. (Though elsewhere he suggests that climate change is an easier problem to solve, and therefore also not as serious?) It seems not irrelevant that Hinton holds enough privilege that he (and people like him) will be unlikely to experience harm from 'AI' unless everyone else on the planet does first.

This viewpoint - the focus only on 'existential risk', i.e., scenarios where all humans die - is threaded through a very bizarre ideology called ' Longtermism' which holds that humanity's destined future is to live as computer simulations uploaded onto computers running in planets across the galaxy. Proponents of this ideology (who have obviously never worked in IT support) perform a utilitarian computation and argue that the happiness of those future beings (quantified, by pulling numbers of out their asses, as 10⁵⁸), outweighs whatever suffering could possibly befall today's mere billions of people. All that matters, for them, is that this imagined future be brought about.

It is at this point that I have to reassure the audience that I am not making this up. It's all nonsense, of course, but it is true that people out there (who furthermore control large amounts of money, though less post-FTX collapse) have articulated and continue to espouse these ideas. In other words, I'm not making this up, but they did.

Timnit Gebru and Émile Torres have done excellent work in tracing what they call the TESCREAL bundle of ideologies, connecting modern work on "AGI" with Longermism and ultimately eugenics. For an accessible overview I highly recommend this talk presented by Gebru at SaTML earlier this year.

No point in building bridges

When I laid all of this out to one of the journalists mentioned at the top of this post, she asked me if it was unfair to paint everyone who identifies with the ideas of 'AI Safety' with the same brush, to call them all racist. She also asked if I saw any way to build bridges between the two groups outlined above.

Let's pause for a moment on the idea of it being 'unfair' to call someone racist. Frequently in US (and probably other) discourse around race, people behave as though 'accusations' of racism are as harmful as racist acts (such as might be described in such 'accusations'). On the one hand, this acknowledges that racism is bad! So that's a first step. On the other hand, it completely misses the point. Someone who genuinely wanted to work against racism (and any other system of oppression), upon learning that something they did contributes to (builds on, reinforces) a system of oppression, would be well served to investigate that further and see what needs to be done to stop the harm.

Furthermore, especially if one is working in a tradition (like 'AI') with racism at its roots, it's not enough to quietly go along to get along. I told the journalist I have yet to find an 'AI Safety' person who puts time and effort into lifting up the voices of those scholars and activists (including many, many brilliant Black women) who have been addressing the real harms happening to real people now, or seriously addresses the racism at the heart of the ideologies 'AI Safety' draws on.

Finally, just like talking about a 'schism' is ahistorical, talking about 'building bridges' suggests that there is a rift to be healed or crossed - and that there is something of value on both sides of the rift, such that people from each side would benefit from visiting the other.

I hope that this post has made clear why those metaphors are inappropriate in this context. 'AI Safety' might be attracting a lot of money and capturing the attention of policymakers and billionaires alike, but it brings nothing of value. The harms being perpetrated today in the name of 'AI', through surveillance, inappropriate automation, displacement of accountability, labor exploitation, and further concentration of power, are urgent and demand attention (both academic and political). Setting up the work of the scholars, journalists, and activists like those I point to above as somehow equivalent to the 'AI Safety'/TESCREAL clown car both devalues the work of the former and facilitates the latter's grab for policymaker attention.

# data[].attributes.attachment_link.description
In two recent conversations with very thoughtful journalists, I was asked about the apparent 'schism' between those making a lot of noise about fears inspired by fantasies of all-powerful 'AIs' going...

# data[].attributes.attachment_link.readable_text
Global AI and Ed Policy Overview:

Details UNESCO initiatives internally and for its members states in AI and Ed Policy

Describes UNESCO initiative for teaching AI in schools

Outlines components of the Beijing Consensus on AI & Education

Includes an overview of in-progress UNESCO AI readiness self-assessment for states

Links to UNESCO's longer "AI and Education: Guidance for Policy-makers

U.S. State/National AI and Ed Policy Overview:

Global AI and Ed Policy Resources

The following resources provide entry points for understanding the limited amount of global policies that exist on AI in Education.

Discusses socio-political background and policy debates shaping China's AI strategy

Approaches China's policies from a lense of ethics

Global AI Non-Ed Policy Resources

These resources survey AI policies from around the world that are not necessarily specific to educational contexts.

Organization for Economic Co-Operation and Development and OECD National AI Policies & Strategies
Type of Article: Interactive Dashboard
APA Citation:
Organization for Economic Co-operation and Development. (n.d.). OECD AI Policy Observatory. OECD.AI Policy Observatory. https://oecd.ai/.
Organization for Economic Co-operation and Development. (n.d.). National AI Policies & Strategies. OECD.AI Policy Observatory. https://oecd.ai/dashboards
Background Needed: Varies widely
Covers:

Breaks down initiatives by country that relate to AI Policy by number of policies and allocated budget

Categorizes policy instruments into mechanism of action

Tracks AI policies across all EU member states

Future of Life Institute: Global AI Policy Overview
Type of Article: Primer with compiled links on global AI initiatives
APA Citation: Future of Life Institute. (2021, June 7). GLOBAL AI POLICY. Future of Life. https://futureoflife.org/ai-policy/?cn-reloaded=1.
Background Needed: Medium
Covers:
Links to 14 different AI issue areas with papers, reports, and articles on each
Map of global AI policy initiatives

European Union's New Rules for Artificial Intelligence: Question & Answer (May 2021)
Type of Article: Primer for policies
APA Citation:
European Union. (2021, April 21). New rules for Artificial Intelligence - Questions and Answers. European Commission . https://ec.europa.eu/commission/presscorner/detail/en/QANDA_21_1683.
Background Needed: Low
Covers:

Q&A on new regulations from the EU on Artificial Intelligence

Categorizes AI systems into different levels of risk and explains categories

Reviews regulations on biometric identification and racial/gender bias in AI

Reviews compliance and enforcement mechanisms

Overview of the Coordinated plan between member states

Feminist perspective of AI, privacy, and data protection enhance digital rights

Documentation of AI and Gender harms in South Africa

Walkthrough of feminist approaches

Policy questions and recommendations for AI

Primer of AI in its current state and the need for safety and ethics

Framework for evaluation of AI systems before public roll-out

Scorecard for safe and ethical AI

Overview of AI awareness, capacity building, and training

Follows computer scientist Joy Buolamwini and other AI researchers who are pushing to expose bias and misuse of AI systems such as facial recognition

Available on Netflix, PBS, and some other streaming platforms

U.S. Federal/State AI and Ed Policy Resources

The U.S. has limited federal and state regulations that are specific to AI. The following resources address AI and educational technology through a policy lens.

STEM For All Video Showcase: Helping States Plan to Teach AI in K-12 and AI4K12
Type of Article: Video and Website Link
APA Citation: Gardener-McCune, C., Cox, B., Delyser, L., Jacobs, K., Seehorn, D., Speed, H., and Touretzky, D. (2021). Helping States Plan to Teach Ai in K-12. Stem for All Video Showcase. TERC and National Science Foundation. https://stemforall2021.videohall.com/presentations/2115.
AI4K12. (n.d.). https://ai4k12.org/.
Background Needed: Low
Covers:

Outlines workshop that prepared state representatives to build AI in K-12 curricula

Describes the AI4K12 organization

U.S. Federal/State EdTech Non-AI Policy Resources

The following resources address policies for educational technologies that are not necessarily specific to AI but will affect the use of AI tools in educational settings.

State, regional, and local visions for technology-enabled learning, teaching, assessment, and infrastructure

The role of technology to educate and empower learners, teachers, and leaders in formal and informal contexts

Links to the Higher Education Supplement to the NETP- a document that examines learning, teaching, assessment, and infrastructure in higher education

Information about Children and Teens' Online Privacy Protection Act to protect both children and teenagers online

A copy of the legislation is posted here

This news is also covered here

COPPA (Children and Teen's Online Privacy Protection Act)
Resource Type: Overview of CIPA, COPPA, and FERPA
Citation: Kozak, D. (2015). Overview of CIPA, COPPA, and FERPA. Technology Services.
https://www.spps.org/cms/lib/MN01910242/Centricity/Domain/11270/OverviewofCIPACOPPAandFERPA12.2015.pdf
Background Needed: Medium
Covers:

Breaks down the coverages of CIPA, COPPA, and FERPA in education.

Includes press release of FTC that provides parents greater control over their kids' information.

Pledge for EdTech vendors to commit to increasing secure access, privacy, and interoperability in products

Legally enforceable yet voluntary sign-on

U.S. Federal/State AI Non-Ed Policy Resources:

The following resources address policies for educational technologies that are not necessarily specific to AI but will affect the use of AI tools in educational settings.

Authors argue for "prudence, patience, and a continuing embrace of permissionless innovation" in relation to digital technologies

Comparing the onset of AI technologies to the development of the internet, the paper describes how policymakers can formulate "pro-growth" policies while focusing on "oversight and accountability" for consumers.

Includes laws of the Federal Trade Commission for developers and users of AI.

Discusses how to use AI equitably.

General AI Privacy/Transparency Policy Resources

The resources in this section address issues related to creating ethical and transparent AI technologies. These resources are not necessarily related to policy.

Identifies principles and pragmatic mechanisms for designing learning experiences embedded in ethical and privacy issues

Proposes guidelines for complying with privacy principles in emerging legislative initiatives

What is consentful technology and why put energy into it

What consentful tech looks like.

Outlines demands from the coded bias documentary to treat data rights as human rights

Site links to other coded bias resources as well as where to find the documentary.

How public perception of the legitimacy of "decisions and decision-makers" in AI is affected by the transparency of those decisions

Discusses the partial transparency about the decision making in AI and how complete transparency may not be better.

AI and Ed Literacy Non-Policy Resources

The resources in this section provide information largely for educators on AI in education. These include curriculum resources, reports, webinars, and academic articles.

Links to all level guides in English, Spanish, and Arabic

Links to additional books and courses on teaching AI

Links to case studies on educators applying AI in the classroom

Link to other podcasts, blogs, and guides for AI in the classroom

Additional AI ethics volumes released in 2021

# data[].attributes.attachment_link.description
Details UNESCO initiatives internally and for its members states in AI and Ed Policy Describes UNESCO initiative for teaching AI in schools Outlines components of the Beijing Consensus on AI & Education Includes an overview of in-progress UNESCO AI readiness self-assessment for states Links to UNESCO's longer "AI and Education: Guidance for Policy-makers The following resources provide entry points for understanding the limited amount of global policies that exist on AI in Education.

# data[].attributes.attachment_link.title
Do you think AI will improve education? Sal Khan, the founder of Khan Academy, thinks AI will be one of the greatest transformations in education. His full talk walks us through how AI super tutors will greatly benefit students and even shows us a demo of their new education chatbot, Khanmigo! Click the 🔗 in our ☣️ to watch the full talk. #khanacademy #ai #aieducation #educationtools #tedtalk 

# data[].attributes.attachment_link.embed_code
@tedtoks
Do you think AI will improve education? Sal Khan, the founder of Khan Academy, thinks AI will be one of the greatest transformations in education. His full talk walks us through how AI super tutors will greatly benefit students and even shows us a demo of their new education chatbot, Khanmigo! Click the 🔗 in our ☣️ to watch the full talk. #khanacademy #ai #aieducation #educationtools #tedtalk
♬ original sound - TED Talks

# data[].attributes.attachment_link.title
ChatGPT, Chatbots and Artificial Intelligence in Education

# data[].attributes.attachment_link.description
AI just stormed into the classroom with the emergence of ChatGPT. How do we teach now that it exists? How can we use it? Here are some ideas.FEATURED

# data[].attributes.attachment_link.title
Why neural networks aren't neural networks

# data[].attributes.attachment_link.description
A New MIT Initiative to Innovate Learning and Education in the Era of AI Responsible AI for Social Empowerment and Education https://youtu.be/oodZLahaWocArtificial Intelligence is transforming our personal and professional lives. Around the world, governments, companies, and institutions are proclai…

# data[].attributes.attachment_link.readable_text
aiEDU utilizes research-based strategies and educational psychology frameworks to create equitable training experiences for teachers of Computer Science, Math, English, Social Studies, Science, and CTE; a wide range of administrators; and nonprofits. Our trainings are designed to help audiences empower the students they serve with AI literacy.

# data[].attributes.attachment_link.description
aiEDU utilizes research-based strategies and educational psychology frameworks to create equitable training experiences for teachers of Computer Science, Math, English, Social Studies, Science, and CTE; a wide range of administrators; and nonprofits. Our trainings are designed to help audiences empower the students they serve with AI literacy.

# data[].attributes.attachment_link.title
The Morning: A driver’s license for the internet

# data[].attributes.attachment_link.description
Good morning. We’re covering age checks online, surveillance in Russia and Wimbledon’s dress code change.

# data[].attributes.attachment_link.title
Senators unveil bipartisan legislation to ban kids under 13 from joining social media platforms | CNN Business

# data[].attributes.attachment_link.description
A new federal bill unveiled Wednesday would establish a national minimum age for social media use and require tech companies to get parents' consent before creating accounts for teens, reflecting a growing trend at all levels of government to restrict how Facebook, Instagram, TikTok and other platforms engage with young users.

# data[].attributes.attachment_link.readable_text
In this file illustration photo taken on January 21, 2021 in Nantes, western France, a man shows a smartphone with the logo of Chinese social network Tik Tok. LOIC VENANCE / AFP

France approved a new law on Thursday, June 29, requiring social media platforms like TikTok to verify users' ages and obtain parental consent for those under 15 years in an effort to protect children online. The legislation is part of a string of recent moves by the government to reduce children's screen time and protect them from cyberbullying and other crimes. The "landmark" law would be "applied as soon as possible", Digital Transition Minister Jean-Noel Barrot told lawmakers.

But the exact date the age verification bill will come into force remains unclear following Thursday's Senate approval, as no specific date was set and the European Commission has yet to check it that it conforms with EU law. Sites will then have a year to comply with the policy for new subscribers, and another two years to apply the requirements to existing users.

Lawmakers' concerns ranged from pornography to cyberstalking, unattainable beauty standards and the attention-grabbing addictive nature of the platforms, which pose a particular risk for youth. Existing regulations mainly focused on the collection of personal data have failed to have an impact on children's access to social media.

Sites are in theory closed to those under the age of 13 in France. But with some users signing up from just eight years of age, according to the French National Commission for Technology and Freedoms (CNIL), more than half of children aged 10-14 use social media sites like Snapchat and Instagram.

The new law will explicitly require sites to obtain approval of a user's parent or guardian if they are under 15, using technical solutions that comply with guidelines set out by the French Arcom regulator. Social media firms that breach the law will face a fine of up to 1 percent of their global revenues. The law will also allow parents to request suspension of accounts belonging to their children under 15 and will require sites to offer tools to limit the time children spend on the platform.

It will not be enough to put an end to concerns of children's safety online, admitted the lawmaker who spearheaded the act, Laurent Marcangeli, on Wednesday, June 28. He further called for "advancements in age verification technology online and heavily investing into digital education for parents, children and teachers."

French lawmakers recently put forward another bill to children's image rights against abuses by parents on social networks, which still needs to be drafted into law. In March, measures against children's overexposure to screens were also introduced in parliament.

Next week, minister Barrot is expected to defend a bill to "secure and regulate the digital space", which includes measures requiring porn sites to check that their users are of legal age.

# data[].attributes.attachment_link.title
France requires parental consent for under-15s on social media

# data[].attributes.attachment_link.description
In a new law aimed at combatting cyberbullying and reducing other harmful effects of social media on minors, social media platforms will be required to verify ages and obtain parental consent for minors under 15.

# data[].attributes.attachment_link.readable_text
Results from this year's National Assessment of Education Progress (NAEP) from the U.S. Department of Education show a continued, steep decline in test scores in all subjects. Math scores for eighth graders were the lowest in 50 years, and reading scores regressed back to scores last seen in 1998. Colloquially known as the "Nation's Report Card," the provides national, state, and even some district-level measurements for student achievement in both public and private schools across a variety of demographics. The results are worrying and signal a lasting impact of the pandemic on learning outcomes. And students need immediate support. That's where AI can help.

Students differ in their respective needs, skills, and learning styles. But schools in the United States frequently "teach to the middle," with lesson plans suited for the average student in the class. While this pedagogical approach can work in some instances, when there are clear deficiencies for students in some areas, a personalized approach has proven to be in increasing scores. AI-enabled educational tools can support this type of individualized support while incorporating learning objectives, instructional approaches, and specific content. For example, Carnegie Learning's AI-powered platform uses analytics to adjust the difficulty level of lessons based on a student's individual needs, helping balance strengths and weaknesses. Students who used the company's MATHia platform, particularly lower-performing students, performed better in their Algebra 1 classes after using the technology. Similarly, the popular non-profit educational platform Khan Academy has begun pilot testing an AI-enabled tutor that uses GPT-4 to help students work through math problems.

Generative AI technology in the classroom also opens up new opportunities for students to learn in their preferred modality. Multimodal learning helps students process information in different ways according to their preferences, which can keep them more engaged and motivated. For example, with Nolej AI, teachers can create a swath of interactive material based on an uploaded lecture document, whether it be flashcards or videos, or a gamified lesson. Students can then access the material in their preferred modality. With some AI tools for younger students, the content can even change depending on what things interest the students, like reading comprehension questions all about dinosaurs for one student and about sports for another. Keeping students engaged in the learning process will help them become stronger students in the long run.

And still, some groups hesitate to take this step. The Department of Education's own Office of Education Technology released its long-awaited recommendations on AI in the classroom, ultimately supporting the precautionary principle, which is the idea of holding back technological deployment until all potential risks can be properly addressed. But with the NAEP results, it seems clear that the risks and harms associated with the status quo are equally, if not more, undesirable than the potential risk from AI.

Students are not learning or absorbing the skills and content they need. And in the face of a national teachers shortage , shouldn't they receive every possible solution? The Department of Education should use these results as a wake-up call that AI should not be kept on the sidelines, especially if it can make a tangible difference in student outcomes. The Department should sponsor grant programs to accelerate the use of AI in the classroom, particularly in lower-resource schools.

While AI certainly needs careful scrutiny in some areas, in the case where it can clearly help children learn, its use should be a no-brainer.
Image credit: The Century Foundation

# data[].attributes.attachment_link.title
Declining Test Scores in the United States Signal the Need for AI Solutions

# data[].attributes.attachment_link.description
Results from this year's National Assessment of Education Progress (NAEP) from the U.S. Department of Education show a continued, steep decline in test scores in all subjects. Math scores for eighth graders were the lowest in 50 years, and reading scores regressed back to scores last seen in 1998. C

# data[].attributes.attachment_link.description
Places to find US AI regulation: A list of 9 official repositories of laws, strategies, and other activities.

# data[].attributes.attachment_link.readable_text
From unreliable AI-generated news outlets operating with little to no human oversight, to fabricated images produced by AI image generators, the rollout of generative artificial intelligence tools has been a boon to content farms and misinformation purveyors alike.

This AI Tracking Center is intended to highlight the ways that generative AI has been deployed to turbocharge misinformation operations and unreliable news. The Center includes a selection of NewsGuard's reports, insights, and debunks related to artificial intelligence.

To date, NewsGuard's team has identified 277 Unreliable AI-Generated News and information websites, labeled " UAINs," spanning 13 languages: Arabic, Chinese, Czech, Dutch, English, French, Indonesian, Italian, Korean, Portuguese, Tagalog, Thai, and Turkish.

These websites typically have generic names, such as iBusiness Day, Ireland Top News, and Daily Time Update, which to a consumer appear to be established news sites. This obscure s that the sites operate with little to no human oversight and publish articles written largely or entirely by bots - rather than presenting traditionally created and edited journalism , with human oversight . The sites have churned out dozens and in some cases hundreds of generic articles, about a range of subjects including politics, technology, entertainment, and travel. The articles have sometimes included false claims, such as celebrity death hoaxes, fabricated events, and articles presenting old events as if they just occurred.

In many cases, the revenue model for these websites is programmatic advertising under which the ad-tech industry delivers ads without regard to the nature or quality of the website. As a result, top brands are unintentionally supporting these sites. Unless brands take steps to exclude untrustworthy sites, their ads will continue to appear on these types of sites, creating an economic incentive for their creation at scale.

The domains included on this tracker meet all four of the following criteria:

1. There is clear evidence that a substantial portion of the site's content is produced by AI.

2. Equally important, there is strong evidence that the content is being published without significant human oversight. For example, numerous articles might contain error messages or other language specific to chatbot responses, indicating that the content was produced by AI tools without adequate editing. (It is likely that now or in the future many news sites will use AI tools but also deploy effective human oversight; they will not be considered UAINs.)

3. The site is presented in a way that an average reader could assume that its content is produced by human writers or journalists, because the site has a layout, generic or benign name, or other content typical to news and information websites.

4. The site does not clearly disclose that its content is produced by AI.

In addition to the sites included in the Tracker, NewsGuard analysts also identified a Chinese-government run disinformation website using AI-generated text as authority for the false claim that the U.S. operates a bioweapons lab in Kazakhstan infecting camels to endanger people in China.

Researchers, platforms, advertisers, government agencies, or other institutions interested in accessing the full list of domains or want details about our services for generative AI companies can contact us here .

Below is a selection of NewsGuard's reports and research related to AI

# data[].attributes.attachment_link.title
Tracking AI-enabled Misinformation: Over 250 'Unreliable AI-Generated News' Websites (and Counting), Plus the Top False Narratives Generated by Artificial Intelligence Tools - NewsGuard

# data[].attributes.attachment_link.description
From unreliable AI-generated news outlets operating with little to no human oversight, to fabricated images produced by AI image generators, the rollout of generative artificial intelligence tools has been a boon to content farms and misinformation purveyors alike. This AI Tracking Center is intended to highlight the ways that generative AI has been deployed to turbocharge misinformation operations and unreliable news.

# data[].attributes.attachment_link.title
False Alarm: How Wisconsin Uses Race and Income to Label Students “High Risk” – The Markup

# data[].attributes.attachment_link.description
The Markup found the state’s decade-old dropout prediction algorithms don’t work and may be negatively influencing how educators perceive students of color

# data[].attributes.attachment_link.readable_text
It's estimated that around 63% of living humans now have access to the internet, which would put the number of digitally connected people at just over 5 billion in sum. Those people come from all across the planet, and potentially speak thousands of different languages. If you search, you can find hundreds of languages online, from Urdu to Catalan.

But some languages are harder to find than others. Rest of World turned to W3Techs, a web-scanning firm based in Austria, to count all of the publicly accessible web addresses on the internet to get hard numbers on the discrepancy. Our data shows that a little more than half the sites on the web use English as their primary language. That's a lot more than one might expect, given that native English speakers only make up just under 5% of the global population. Meanwhile, Chinese and Hindi are the second and third most-spoken languages in the world, but the same scan found they account for just 1.4% and 0.07% of domains, respectively.

Because the internet is so vast, the data comes with caveats and blind spots (detailed below), but the scan still reveals massive imbalances in language use. Languages like Bengali and Urdu, each spoken by hundreds of millions of people, are nearly impossible to find online.

W3Techs primarily tracks programming languages used online. It regularly scans publicly available domains and categorizes them by language, providing real-time reports for interested clients. We compared the W3Techs data with spoken-language figures from a survey by Ethnologue, a nonprofit widely considered the world authority on language use.

Combined, the two data sets suggest significant over- and under-representation. English, German, and Japanese command a much larger portion of the internet than they do among native speakers. By contrast, many non-European languages hardly exist on the internet at all.

For some international groups, these discrepancies are an ominous sign for the future. As early as 2003, UNESCO was urging the public and private sectors to maintain online content in the full range of human languages. But as the web has grown, the gap between spoken language and what's used on the internet has only grown.

Bhanu Neupane, a program manager at UNESCO who works with language inequity, told Rest of World we might be moving towards a world where only a handful of languages are meaningfully present online. "The world is converging," Neupane said. "And after 15 years, there could be just five or 10 languages that are prominently spoken and used in business and online. So we're very concerned about this."

Surveys of the problem vary, but UNESCO's own assessment is consistent with the W3Techs results, showing only 14 languages present on more than 1% of domains.

There are a few caveats you should keep in mind regarding this data set: The data comes from scans of publicly available websites, so anything that is behind a login is probably going uncounted, which includes apps and social networks. (This quirk suggests the scans may be undercounting the Chinese internet, in particular, although it's hard to know by how much.) Even within web-accessible social networks like Reddit, the scans aren't designed to go through every page of a domain, which means they may be undercounting non-English communities on English-language sites. There are more details here, but the data should be read as a broad survey of websites, not a precise measurement.

Having said that, the bigger picture is hard to miss. Millions of non-native English speakers and non-English speakers are stuck using the web in a language other than the one they were born into. And since publicly available text on the internet is now often being used to train large language models like Bard and GPT-4, it suggests we're already building the same imbalance into technology's next frontier: artificial intelligence.

# data[].attributes.attachment_link.description
Is English language the leading language of the internet? As of right now, English is the leading internet language, with Russian and Spanish following behind.

# data[].attributes.attachment_link.title
The Dystopian AI-ification of Capitalism

# data[].attributes.attachment_link.title
B.F Skinner. Teaching machine and programmed learning

# data[].attributes.attachment_link.readable_text
20. juni, 2023

Over the last months there has been an explosion of services driven by generative artificial intelligence. Consumer organizations in EU and the US call for consumer rights to be at the center of its development and implementation.

- We must ensure that the development and use of generative AI is safe, reliable, and fair. Unfortunately, history has shown that we cannot trust the big tech companies to fix this on their own, says Find Myrstad, Director of Digital Policy at the Norwegian Consumer Council (Forbrukerrådet).

- It is crucial that the EU makes the AI Act as watertight as possible in terms of protecting consumers from harmful uses of this technology. We call on the EU institutions to resist the powerful lobbying of Big Tech companies to water down protections in the future law." says Ursula Pachl, Deputy Director General of the European Consumer Organisation.

Calling on policymakers and enforcement agencies

The Norwegian Consumer Council today published a detailed report " Ghost in the machine - Addressing the consumer harms of generative AI" outlining the harms, legal frameworks, and possible ways forward. In conjunction with this launch, the Norwegian Consumer Council and 14 consumer organizations from across the EU and the US demand that policymakers and regulators act.

- The risks from generative AI for consumers are well documented in this report by The Norwegian Consumer Council. As long as the EU's AI Act is not applicable, authorities need to investigate where new generative AI-driven products and services may be harming consumers and enforce existing data protection, safety and consumer protection legislation. Companies cannot be absolved from the EU's existing regulations, nor should consumers be manipulated or misled, just because this technology is new says Pachl.

- Technology is not some uncontrollable force, but must be adapted and formed by fundamental rights, regulations, and societal values. We are in the driver seat if we choose to be. Many of the challenges we are facing can be tackled using the laws we already have, says Myrstad.

Hand in hand with this, relevant enforcement agencies must have the necessary resources and competence to follow the technological development and to enforce against companies using generative AI without complying with the law.

Therefore, the report also calls for:

Strengthened consumer protections to make the technology safe, reliable, and fair so that consumers are not used as laboratory animals for new technologies.

An overarching AI-strategy which takes into account of recent developments, is centered on fundamental rights, and offers strict guidelines for the use of generative AI in the public sector.

Suitable future-proof regulations in instances where existing laws fall short.

The national authorities must act

It will take several years to create the best possible international framework. This is essential. Efforts are underway. For example, on the 14th of June, the European Parliament agreed on a proposal for regulating artificial intelligence, the AI Act. The regulation is now progressing through negotiations with the European Commission and the Council.

- This regulation will have a significant impact on the development and use of artificial intelligence. However, people are using this technology already and the harms will occur now. Therefore, it is important that national authorities fill the gap with robust enforcement in the meantime, says Myrstad.

About Generative artificial intelligence

Generative AI-models are trained on large amounts of data to identify patterns and structures allowing them to generate new content. This can be text, images, audio, or video, which may resemble human-made content.

ChatGPT, Midjourney, Stable Diffusion, and DALL-E are among the best-known generative AI-driven services. Companies such as Microsoft, Google, and Snapchat have already implemented the technology in their services, including search engines, word processing, and chat.

Generative AI is not self-aware, and cannot "learn" new skills by itself, but it can simulate some of the things that humans can do by using machine learning.

Participating organisations

- Adiconsum (Consumers and Environment Protection Association), Italy

Consumentenbond

Consumers' Organisation of Macedonia (OPM)

dTest (Czech Association of Consumers TEST)

EKPIZO (Consumers' Association "The Quality of Life")

Federación de Consumidores y Usuarios CECU

Forbrugerrådet Tænk

Forbrukerrådet (The Norwegian Consumer Council)

Ghaqda tal-Konsumaturi (Consumers' Association Malta)

Irish Council for Civil Liberties

Latvijas Patērētāju Interešu Aizstāvības Asociācija - Lpiaa (Latvian National Association for Consumer Protection)

Neytendasamtökin - NS (The Consumers´ Association of Iceland)

Sveriges Konsumenter (The Swedish Consumers' Association)

Transatlantic Consumer Dialogue (TACD)

Zveza Potrošnikov Slovenije - ZPS (the Slovene Consumers' Association)

Abonner på vårt nyhetsvarsel

# data[].attributes.attachment_link.title
New report: Generative AI threatens consumer rights

# data[].attributes.attachment_link.description
Over the last months there has been an explosion of services driven by generative artificial intelligence. Consumer organizations in EU and the US call for consumer rights to be at the center of its development and implementation.

# data[].attributes.attachment_link.provider_name
US Department of Health and Human Services

# data[].attributes.attachment_link.authors[].name
Office of the Assistant Secretary for Health (OASH)

# data[].attributes.attachment_link.title
Surgeon General Issues New Advisory About Effects Social Media Use Has on Youth Mental Health

# data[].attributes.attachment_link.description
Surgeon General Urges Action to Ensure Social Media Environments are Healthy and Safe, as Previously-Advised National Youth Mental Health Crisis Continues

# data[].attributes.attachment_link.readable_text
As we usher in this new era of technology, it's imperative that we consider the profound impact that social interaction plays on child development. Artificial Intelligence (AI), already has and will continue to be used to perform tasks that were previously done by teachers, such as providing one-on-one instruction, assessment, academic advising, and even mentoring. This article explores how social interactions, especially those between teacher and student, contribute to aspects like cognitive development, language skills, emotional intelligence, cultural awareness, and confidence building. It ends with the questions that are on everyone's minds: How will AI change our children's schooling and and how will a likely increase in student-bot interaction and decrease in student-human interaction shape them?

Teaching is More than a Knowledge Exchange

Each week, my four-year old daughter attends an afternoon dance class. Sometimes, her excitement for class prevents her from taking her afternoon nap. When we arrive at the building that houses the dance studio, we watch our shy, clingy baby transform into a self-confident, independent girl as she sprints straight to the studio, leaving us in the dust. Although she likes to dance, her eagerness isn't incited by the curriculum and although she loves other kids, it's not about seeing her friends. She is genuinely thrilled to see her dance teacher, who somehow teaches a roomful of toddlers everything from first position to empathy in just 45 minutes per week. This teacher is pure magic and has captivated our daughter and taught her so much more than how to dance.

These observations reinforce the notion that kids learn more, and benefit in many other ways, when they form a secure attachment to their teachers. There's also plenty of empirical evidence to support this theory. Here's what we know:

What research says about teacher-child relationships

Parents are typically their child's primary attachment figures. However, as their children begin to leave home for activities, daycare, and school, other attachment figures enter their lives. Therefore:

"children's development [is] expected to be better predicted by the sum of [their] (familial and non-familial) attachment relationships than solely by [their] parent-child attachment relationships ( Van IJzendoorn et al., 1992)."
(Spilt & Koomen, 2022)

Teacher-child relationships have been found to:

Be significantly associated with engagement and self-regulation, including positive task behaviors and frustration tolerance (Pianta et al., 1997).

Have lasting effects on academic and behavioral outcomes. For instance, Hamre and Pianta (2001) found positive and negative effects of early child-teacher relationships lasting up to 8th grade (Hamre and Pianta, 2001).

Exist and impact secondary students, when scheduling permits them to. For instance, Van Ryzin (2010) found that almost half of students who participated in a mentor advisory program used their teacher-advisor as a secure base or safe haven.

Be associated with several student-attributes. For instance, student-teacher closeness was found to be positively associated with prosocial behavior and negatively with conduct problems, hyperactivity, and peer problems (Koomen et al., 2012).

Interestingly, it was thought that these attachments become less necessary as kids progress through K-12. For instance, middle and high school students begin to place more value on friendship and form meaningful and secure attachments to their peers. However, Koomen et al.'s (2012) findings indicate otherwise. The need is still there, but the structure of the middle and high school schedule: students having several teachers instead of just one and a larger focus on curricular content and lesser focus on relationship building and social-emotional supports, make it more difficult for attachments to form.

How might technology impact these relationships?

During the Covid-19 pandemic, quarantine mandates forced districts to shift from in-person to online learning. To ensure that all students could access virtual instruction, districts scrambled to shore up digital devices.

The sudden influx of sophisticated AI applications may tempt districts to increase instructional screen time even more. Many of these applications make lofty promises, like differentiated instruction for all, increased engagement, greater learning gains, and less work for teachers. However, it's important to remember that the need for a secure student-teacher attachment and the benefits it provides transcends grade-level. So the final question is: how can we benefit from these new technologies without sacrificing these important relationships with our students?

# data[].attributes.attachment_link.description
This post discusses why AI should not replace teachers. The teacher-student relationship can positively impact learning and social-emotional outcomes.

# data[].attributes.attachment_link.readable_text
I created Hack Education in June 2010 shortly after I became a technology journalist. I was frustrated by the lack of coverage of education technology -- by both technology and education publications. I did my day job (that is, the freelance writing I got paid for) but devoted as much attention as possible to Hack Education, trying to create the sort of publication that I'd want to read: one that was smart and snarky, one that was free of advertising and investor influence, one that was tracking new technologies but not just because of some hyperbolic "revolution."

To "hack" can mean a lot of things: To break in and break down. To cut to the core. To chop roughly. To be playful and clever. To be mediocre. To solve a problem, but to do so rather inelegantly. To pull systems apart. To "MacGyver" things back together. To re-code. To rebuild. To "Hack Education," in turn, can have multiple interpretations, I recognize: a technological solution, a technology intrusion, a technological possibility, a technological disaster. Ed-tech is all those things.

To "hack education" isn't something that just technologists should do or care about. Nor is this just a concern for teachers, administrators, parents, or students. We all should consider the implications of technology on how we teach and learn, lest the future of ed-tech be just like the history of ed-tech: learners as pigeons.

In June 2022, I posted my last update to this site. I can't be, I won't be ed-tech's Cassandra any longer.

Well, it's been quite a ride. But when I received word a couple of weeks ago that my very favorite ed-tech startup Desmos was being acquired, it seemed like the perfect time to close the curtains on this show. I'm officially done with Hack Education, done with ed-tech. This site won't go away. You can work your way through the extensive archives. I wrote a lot of good stuff, and almost all of it is still relevant.

"Things are fucked up and bullshit." That's what the sign said at Occupy Wall Street. Let's not forget that. And it's not just that things are bad now or things were bad then. We have to remember that folks resisted. Folks protested. Change happens. We needn't give up the future to the corporate elites. We needn't hand over education to their techno-fantasies. We need to remember. And from memory, we can draw hope. This was sort of the gist of my keynote at Digifest 2022.

Listen up. The school bell is not a technology adopted to train students to become factory workers, ok? Stop telling these ridiculous tales about the history of education. Or alternately, keep it up, and I'll just keep writing these essays about the history of school technologies and screaming into the void.

· Books · Latest Book:

"There must be an industrial revolution in education," psychologist Sidney Pressey wrote in 1933, "in which educational science and the ingenuity of educational technology combine to modernize the grossly inefficient and clumsy procedures of conventional education."

We still hear claims like this today: ed-tech is poised to bring science and efficiency to schools. Teaching machines will individualize instruction, allowing students to move at their own pace through their lessons and freeing teachers from drudgery so she may focus, as Pressey argued, on more important work "developing in her pupils fine enthusiasms, clear thinking, and high ideals."

" Teaching Machines, is a vital cultural history of our desire for a technical solution to the fundamentally social problem of how to make education work for all families. Watters has written the rare book that is necessary, important, and readable." -- Tressie McMillan Cottom, Associate Professor, University of North Carolina at Chapel Hill; author of Thick: And Other Essays

"This is a landmark book." -- John Warner, Inside Higher Ed

Here. I have chronicled for you a decade of ed-tech failures and fuck-ups and flawed ideas. Oh yes, I'm sure you can come up with some rousing successes and some triumphant moments that made you thrilled about the 2010s and that give you hope for "the future of education." Good for you. But that's not my job. (And honestly, it's probably not your job either.) .

This talk was delivered at OEB 2019 in Berlin. I wanted to call out the misinformation and disinformation repeated by those who get on stage at ed-tech events (as well as those who uncritically accept the fairy tales as truth). If we prance delightedly towards a dystopian ed-tech future, it is in part because of the storytellers in ed-tech who peddle this bullshit.

· Why Pigeons?:

What is up with all the pigeons? The bird appears across all the various Hack Education projects as it exemplifies how education technology has viewed learning and learners. In part, it's a reference to the work of Edward Thorndike and B. F. Skinner and their development of multiple choice tests, teaching machines, and behavioral (educational) psychology....
Image credits: Bryan Mathers

Well, it's been quite a ride. But when I received word a couple of weeks ago that my very favorite ed-tech startup Desmos was being acquired, it seemed like the perfect time to close the curtains on this show. I'm officially done with Hack Education, done with ed-tech. This site won't go away. You can work your way through the extensive archives. I wrote a lot of good stuff, and almost all of it is still relevant.

"Things are fucked up and bullshit." That's what the sign said at Occupy Wall Street. Let's not forget that. And it's not just that things are bad now or things were bad then. We have to remember that folks resisted. Folks protested. Change happens. We needn't give up the future to the corporate elites. We needn't hand over education to their techno-fantasies. We need to remember. And from memory, we can draw hope. This was sort of the gist of my keynote at Digifest 2022.

Listen up. The school bell is not a technology adopted to train students to become factory workers, ok? Stop telling these ridiculous tales about the history of education. Or alternately, keep it up, and I'll just keep writing these essays about the history of school technologies and screaming into the void.

· Books · Latest Book:

"There must be an industrial revolution in education," psychologist Sidney Pressey wrote in 1933, "in which educational science and the ingenuity of educational technology combine to modernize the grossly inefficient and clumsy procedures of conventional education."

We still hear claims like this today: ed-tech is poised to bring science and efficiency to schools. Teaching machines will individualize instruction, allowing students to move at their own pace through their lessons and freeing teachers from drudgery so she may focus, as Pressey argued, on more important work "developing in her pupils fine enthusiasms, clear thinking, and high ideals."

" Teaching Machines, is a vital cultural history of our desire for a technical solution to the fundamentally social problem of how to make education work for all families. Watters has written the rare book that is necessary, important, and readable." -- Tressie McMillan Cottom, Associate Professor, University of North Carolina at Chapel Hill; author of Thick: And Other Essays

"This is a landmark book." -- John Warner, Inside Higher Ed

Here. I have chronicled for you a decade of ed-tech failures and fuck-ups and flawed ideas. Oh yes, I'm sure you can come up with some rousing successes and some triumphant moments that made you thrilled about the 2010s and that give you hope for "the future of education." Good for you. But that's not my job. (And honestly, it's probably not your job either.) .

This talk was delivered at OEB 2019 in Berlin. I wanted to call out the misinformation and disinformation repeated by those who get on stage at ed-tech events (as well as those who uncritically accept the fairy tales as truth). If we prance delightedly towards a dystopian ed-tech future, it is in part because of the storytellers in ed-tech who peddle this bullshit.

· Why Pigeons?:

What is up with all the pigeons? The bird appears across all the various Hack Education projects as it exemplifies how education technology has viewed learning and learners. In part, it's a reference to the work of Edward Thorndike and B. F. Skinner and their development of multiple choice tests, teaching machines, and behavioral (educational) psychology....
Image credits: Bryan Mathers

"Things are fucked up and bullshit." That's what the sign said at Occupy Wall Street. Let's not forget that. And it's not just that things are bad now or things were bad then. We have to remember that folks resisted. Folks protested. Change happens. We needn't give up the future to the corporate elites. We needn't hand over education to their techno-fantasies. We need to remember. And from memory, we can draw hope. This was sort of the gist of my keynote at Digifest 2022.

Listen up. The school bell is not a technology adopted to train students to become factory workers, ok? Stop telling these ridiculous tales about the history of education. Or alternately, keep it up, and I'll just keep writing these essays about the history of school technologies and screaming into the void.

· Books · Latest Book:

"There must be an industrial revolution in education," psychologist Sidney Pressey wrote in 1933, "in which educational science and the ingenuity of educational technology combine to modernize the grossly inefficient and clumsy procedures of conventional education."

We still hear claims like this today: ed-tech is poised to bring science and efficiency to schools. Teaching machines will individualize instruction, allowing students to move at their own pace through their lessons and freeing teachers from drudgery so she may focus, as Pressey argued, on more important work "developing in her pupils fine enthusiasms, clear thinking, and high ideals."

" Teaching Machines, is a vital cultural history of our desire for a technical solution to the fundamentally social problem of how to make education work for all families. Watters has written the rare book that is necessary, important, and readable." -- Tressie McMillan Cottom, Associate Professor, University of North Carolina at Chapel Hill; author of Thick: And Other Essays

"This is a landmark book." -- John Warner, Inside Higher Ed

Here. I have chronicled for you a decade of ed-tech failures and fuck-ups and flawed ideas. Oh yes, I'm sure you can come up with some rousing successes and some triumphant moments that made you thrilled about the 2010s and that give you hope for "the future of education." Good for you. But that's not my job. (And honestly, it's probably not your job either.) .

This talk was delivered at OEB 2019 in Berlin. I wanted to call out the misinformation and disinformation repeated by those who get on stage at ed-tech events (as well as those who uncritically accept the fairy tales as truth). If we prance delightedly towards a dystopian ed-tech future, it is in part because of the storytellers in ed-tech who peddle this bullshit.

· Why Pigeons?:

What is up with all the pigeons? The bird appears across all the various Hack Education projects as it exemplifies how education technology has viewed learning and learners. In part, it's a reference to the work of Edward Thorndike and B. F. Skinner and their development of multiple choice tests, teaching machines, and behavioral (educational) psychology....
Image credits: Bryan Mathers

Listen up. The school bell is not a technology adopted to train students to become factory workers, ok? Stop telling these ridiculous tales about the history of education. Or alternately, keep it up, and I'll just keep writing these essays about the history of school technologies and screaming into the void.

· Books · Latest Book:

"There must be an industrial revolution in education," psychologist Sidney Pressey wrote in 1933, "in which educational science and the ingenuity of educational technology combine to modernize the grossly inefficient and clumsy procedures of conventional education."

We still hear claims like this today: ed-tech is poised to bring science and efficiency to schools. Teaching machines will individualize instruction, allowing students to move at their own pace through their lessons and freeing teachers from drudgery so she may focus, as Pressey argued, on more important work "developing in her pupils fine enthusiasms, clear thinking, and high ideals."

" Teaching Machines, is a vital cultural history of our desire for a technical solution to the fundamentally social problem of how to make education work for all families. Watters has written the rare book that is necessary, important, and readable." -- Tressie McMillan Cottom, Associate Professor, University of North Carolina at Chapel Hill; author of Thick: And Other Essays

"This is a landmark book." -- John Warner, Inside Higher Ed

Here. I have chronicled for you a decade of ed-tech failures and fuck-ups and flawed ideas. Oh yes, I'm sure you can come up with some rousing successes and some triumphant moments that made you thrilled about the 2010s and that give you hope for "the future of education." Good for you. But that's not my job. (And honestly, it's probably not your job either.) .

This talk was delivered at OEB 2019 in Berlin. I wanted to call out the misinformation and disinformation repeated by those who get on stage at ed-tech events (as well as those who uncritically accept the fairy tales as truth). If we prance delightedly towards a dystopian ed-tech future, it is in part because of the storytellers in ed-tech who peddle this bullshit.

· Why Pigeons?:

What is up with all the pigeons? The bird appears across all the various Hack Education projects as it exemplifies how education technology has viewed learning and learners. In part, it's a reference to the work of Edward Thorndike and B. F. Skinner and their development of multiple choice tests, teaching machines, and behavioral (educational) psychology....
Image credits: Bryan Mathers

"There must be an industrial revolution in education," psychologist Sidney Pressey wrote in 1933, "in which educational science and the ingenuity of educational technology combine to modernize the grossly inefficient and clumsy procedures of conventional education."

We still hear claims like this today: ed-tech is poised to bring science and efficiency to schools. Teaching machines will individualize instruction, allowing students to move at their own pace through their lessons and freeing teachers from drudgery so she may focus, as Pressey argued, on more important work "developing in her pupils fine enthusiasms, clear thinking, and high ideals."

" Teaching Machines, is a vital cultural history of our desire for a technical solution to the fundamentally social problem of how to make education work for all families. Watters has written the rare book that is necessary, important, and readable." -- Tressie McMillan Cottom, Associate Professor, University of North Carolina at Chapel Hill; author of Thick: And Other Essays

"This is a landmark book." -- John Warner, Inside Higher Ed

Here. I have chronicled for you a decade of ed-tech failures and fuck-ups and flawed ideas. Oh yes, I'm sure you can come up with some rousing successes and some triumphant moments that made you thrilled about the 2010s and that give you hope for "the future of education." Good for you. But that's not my job. (And honestly, it's probably not your job either.) .

This talk was delivered at OEB 2019 in Berlin. I wanted to call out the misinformation and disinformation repeated by those who get on stage at ed-tech events (as well as those who uncritically accept the fairy tales as truth). If we prance delightedly towards a dystopian ed-tech future, it is in part because of the storytellers in ed-tech who peddle this bullshit.

· Why Pigeons?:

What is up with all the pigeons? The bird appears across all the various Hack Education projects as it exemplifies how education technology has viewed learning and learners. In part, it's a reference to the work of Edward Thorndike and B. F. Skinner and their development of multiple choice tests, teaching machines, and behavioral (educational) psychology....
Image credits: Bryan Mathers

Here. I have chronicled for you a decade of ed-tech failures and fuck-ups and flawed ideas. Oh yes, I'm sure you can come up with some rousing successes and some triumphant moments that made you thrilled about the 2010s and that give you hope for "the future of education." Good for you. But that's not my job. (And honestly, it's probably not your job either.) .

This talk was delivered at OEB 2019 in Berlin. I wanted to call out the misinformation and disinformation repeated by those who get on stage at ed-tech events (as well as those who uncritically accept the fairy tales as truth). If we prance delightedly towards a dystopian ed-tech future, it is in part because of the storytellers in ed-tech who peddle this bullshit.

· Why Pigeons?:

What is up with all the pigeons? The bird appears across all the various Hack Education projects as it exemplifies how education technology has viewed learning and learners. In part, it's a reference to the work of Edward Thorndike and B. F. Skinner and their development of multiple choice tests, teaching machines, and behavioral (educational) psychology....
Image credits: Bryan Mathers

This talk was delivered at OEB 2019 in Berlin. I wanted to call out the misinformation and disinformation repeated by those who get on stage at ed-tech events (as well as those who uncritically accept the fairy tales as truth). If we prance delightedly towards a dystopian ed-tech future, it is in part because of the storytellers in ed-tech who peddle this bullshit.

· Why Pigeons?:

What is up with all the pigeons? The bird appears across all the various Hack Education projects as it exemplifies how education technology has viewed learning and learners. In part, it's a reference to the work of Edward Thorndike and B. F. Skinner and their development of multiple choice tests, teaching machines, and behavioral (educational) psychology....
Image credits: Bryan Mathers

What is up with all the pigeons? The bird appears across all the various Hack Education projects as it exemplifies how education technology has viewed learning and learners. In part, it's a reference to the work of Edward Thorndike and B. F. Skinner and their development of multiple choice tests, teaching machines, and behavioral (educational) psychology....
Image credits: Bryan Mathers

# data[].attributes.attachment_link.description
I created Hack Education in June 2010 shortly after I became a technology journalist. I was frustrated by the lack of coverage of education technology -- by both technology and education publications.

# data[].attributes.attachment_link.title
Teaching Machines: The History of Personalized Learning

# data[].attributes.attachment_link.description
How ed tech was born: Twentieth-century teaching machines-from Sidney Pressey's mechanized test-giver to B. F. Skinner's behaviorist bell-ringing box.Contrary t

# data[].attributes.attachment_link.readable_text
Generative AI is taking the world by storm and has also taken over our discussions of ResponsibleTech.Work. Like many others, we've spent the past couple of months learning, exchanging resources, and trying to figure out what it all means for responsible product development. As a worrying trend, Big Tech companies started laying off their responsible AI teams just as they started rushing to bring their GPT-powered products to market.

The same men who have been funding and developing this new wave of technology, are now arguing in the public forum about just how fast we should unleash the shoggoth, and shifting the debate from the already well-documented harms. As we listen to billionaires and "grandfathers of AI" trying to decide our future, it's easy to feel powerless over how AI is being developed and deployed.

But LLMs - Large Language Models - powering ChatGPT, Bard & co. were all trained on our data, our thoughts. And we started ResponsibleTech.Work in the belief that all of us, regardless of role or influence, can take personal responsibility and contribute to more responsible tech.

This is why we'd like to invite everyone - especially tech workers - to join the Responsible AI Pledge Challenge by writing personal and concrete pledges on how each of us will contribute to a more responsible AI future. The challenge is based on Pledge Works, our open-source tool that aims to embed responsibility in everyday product decisions. While we encourage you to get familiar with Pledge Works, there are no prerequisites to join the Responsible AI Pledge Challenge.

The main goal of the challenge - more on that later - is to invite more people to contribute to the public discussion on responsible AI by writing and sharing diverse pledges. We have prepared a suggested challenge flow, but feel free to join the challenge and come up with pledges in your own way.

Suggested Responsible AI Pledge Challenge flow

These are the steps you might want to take when joining the challenge:

1. Choose a pledge period

We suggest starting by choosing a timeframe to meet your Responsible AI pledges. Having a concrete start and end time will help you make your pledges specific and avoid setting commitments that are too general or unreachable. A period between one and three months can be a good starting point. We recommend adding the end date to your task manager or calendar app to be reminded of the day when your pledge period ends so that you can set some time aside for your pledge checkpoint review.

2. Explore your context

The next step is to think about your current position on AI, and define your context window, which includes your values, principles, and life situation. Here are some guiding questions for you to consider:

What worries you when you think about AI?

Which aspects of AI do you wish you knew more about?

Who can you talk to better understand AI?

What are your current skills?

What kind of impact can you have?

What might be going on in your life during the pledge period that might impact your ability to follow your pledges?

Now it's time to make a commitment! Open your text editor or camera app, and write or record one or more concrete pledges on what you, personally, will do to contribute to the responsible AI discussion in the chosen period given your context. Pick realistic commitments that fit your selected pledge period and your context window. If you don't know where to start, look at pledges other people have written or start with a set of general guidelines and think about where you have the power to contribute.

To keep yourself accountable, we suggest sharing your pledges and inviting others to participate. You can publish your Responsible AI Pledges on your blog or social media platforms under the hashtag #ResponsibleAIPledges, or share them on your team's Slack. If you don't feel comfortable or safe sharing your pledges publicly, you can share them privately or write them on a piece of paper that you hang in a shared space. When sharing your pledges, you can also expose your pledge context.

4. Follow your pledges

Next comes the time to try to honour the commitment(s) you've made to yourself or others within the chosen pledge period. To remind yourself of the pledges you've made, we recommend you pin your pledges somewhere you can easily revisit them daily, either in physical or digital form.

5. Pledge checkpoint

And finally, once your pledge period ends, it's time to review your progress and decide how you want to continue. During the review, check whether you've managed to fulfil your pledges entirely or partially, and think about how you might write better pledges or choose a different pledge period in the future. You can do the pledge review alone or with friends, so you can offer additional feedback to each other.

Obviously, you can keep this process going for as long as you want. If you enjoyed the challenge, keep going! In the following pledge cycle(s), you can experiment with different levels of granularity - make your pledges more concrete or abstract -, pledge formats, and durations of the pledge period.

And again, the flow outlined above is just a suggestion. You are welcome to skip steps or mix it up - especially as you get more comfortable with the process. As long as you are able to come up with personally relatable pledges and discover ways to embed them in your daily practice, you are meeting the spirit of Pledge Works and the pledge challenge.

Joining the challenge as a product team

You can also join the Responsible AI Pledge Challenge as a team, or even organize a community pledge-writing session with your peers. When you write pledges together with others, you can explore differences in values and attitudes and develop shared meaning through the collaborative process.

In a product team, you can focus on different perspectives, such as the team's use of AI tools like Copilot, or examine your planned integration of AI-powered APIs. Making pledges to learn together can also be a valuable team commitment.

Here's a revisited challenge flow for product teams:

The goal of the Challenge

The goal of the challenge is to encourage each and every one of us to think about the impact we can have on responsible AI, whether that's in product development, education, political action, or just by joining the conversation with our unique perspectives.

Obviously, the pledges of a deep learning expert at OpenAI will be different from those a freelance designer or an ethics researcher might write. The goal is not to compete on who can write the most ambitious and impressive-sounding pledges, but collectively brainstorm how each of us might contribute to more responsible AI.

The goal of the challenge also isn't to let individuals and organizations who hold more power off the hook and abdicate responsibility, but instead to make the discussion more inclusive. And who knows, maybe we can even help future GPTs learn something useful about how we view responsibility in the process!

#ResponsibleAIPledges examples

To help you think about your pledges, we're also including a couple of example pledges from our core team:

"I pledge to explore different mediums and channels to help more people understand the biases and limitations of tools like ChatGPT and improve their AI literacy."

"I pledge to be transparent about how and when I use AI tools."

By Alja Isaković

"To compensate for the water use of my chats with ChatGPT, I pledge to use grey water for my plants."

By Daniel Hartley

As you can see, we took a different approach to writing and framing our pledges, context window, and recent thinking about AI, and we encourage you to do the same. Have fun, experiment, and make your pledges your own.

In the coming months, we'd like to feature additional #ResponsibleAIPledges from diverse perspectives. If you'd like your pledges to be featured on our website, email info@ResponsibleTech.Work with your link.

# data[].attributes.attachment_link.title
Announcing the Responsible AI Pledge Challenge

# data[].attributes.attachment_link.description
We don't think billionaires and "grandfathers of AI" should be the only ones that get to define what responsible AI means. This is why we want to challenge more people - like you - to contribute to the public discussion on responsible AI by writing and sharing our diverse pledges.

# data[].attributes.attachment_link.title
Three lenses to help design better, socially responsible AI

# data[].attributes.attachment_link.description
How do you build AI responsibly? AI presents unique challenges to developing and deploying technology responsibly. A design orientation can help ground the complexity of modern AI systems in a way that supports meaningful transparency and social responsibility. Read the ebook now.

# data[].attributes.attachment_link.readable_text
Today U.S. Senator Josh Hawley (R-Mo.) announced his guiding principles for the future of American artificial intelligence (AI) legislation. Senator Hawley's principles seek to protect Americans' privacy as AI continues to develop at a pace that threatens to upend many aspects of American life.

"Congress can and should act to protect Americans' privacy, stave off the harms of unchecked AI development, insulate kids from harmful impacts, and keep this valuable technology out of the hands of our adversaries," said Senator Hawley. "Taken together, these five principles will help set the course for the responsible development of American AI."

To guide future federal action, Senator Hawley proposed five guiding principles:

First, create private rights of action. Individual citizens should have the right to sue companies for harm inflicted by AI models in order to hold those corporations developing AI accountable.

Second, protect personal data. AI models should be prohibited from harvesting sensitive personal data without consent, with stiff penalties for misuse.

Third, enforce age limits on use. To shield minors from harmful effects of generative AI technology, companies should be proactively blocked from deploying or promoting these models to children.

Fourth, block technology to and from China. America should promote AI independence by blocking any importation of AI-related chips and technology from China, and by preventing American corporations from aiding China's development of AI.

Fifth, establish a licensing system. To protect consumers and promote transparency, require generative entities working on generative AI models to obtain a license.

In May, as the Ranking Member of the Judiciary Subcommittee on Privacy, Technology, and the Law, Senator Hawley held a hearing with Chairman Richard Blumenthal to conduct oversight on AI technology and appropriate safeguards.

During the hearing, Senator Hawley questioned Sam Altman, the CEO of Open AI, and asked the witnesses if we need to "strike that balance between technological innovation and our ethical and moral responsibility to humanity." The witnesses agreed that oversight from Congress is needed to protect the American people.

Continuing efforts to protect Americans from AI, Senators Hawley and Blumenthal sent a letter yesterday to Meta CEO Mark Zuckerberg, seeking information about the "leak" of its Large Language Model Meta AI (LLaMA) program.

# data[].attributes.attachment_link.title
Hawley Announces Guiding Principles for Future AI Legislation

# data[].attributes.attachment_link.description
Today U.S. Senator Josh Hawley (R-Mo.) announced his guiding principles for the future of American artificial intelligence (AI) legislation. Senator Hawley's principles seek to protect Americans' privacy as AI continues to develop at a pace that threatens to upend many aspects of American life.

# data[].attributes.attachment_link.title
Carnegie Learning: Inside MATHia, the World’s Best Math Learning Platform

# data[].attributes.attachment_link.description
Most of the conversation surrounding artificial intelligence (AI) tools is often directed at business, but there is enormous potential for AI to drastically improve our educational systems. It is one of the most effective tools teachers can have at their disposal, and it often frees them up from administrative burdens. These technologies will not replace […]

# data[].attributes.attachment_link.provider_name
Online Skill Assessment | Online Exam Software | Eklavvya.com

# data[].attributes.attachment_link.readable_text
Artificial Intelligence (AI) has been a game-changer in the field of education technology. It has the potential to provide students with personalized learning experiences, help teachers grade assignments faster and more accurately, and offer instant feedback and support to students. Best AI for Schools can help simplify education processes.

In today's educational landscape, artificial intelligence (AI) tools have revolutionized student learning and engagement. Teachers now have access to AI-powered websites and tools that enhance lesson plans and create personalized learning experiences.

These AI tools provide interactive quizzes, adaptive learning platforms, and intelligent feedback, making them the best companions for students. With AI's assistance, teachers can cater to individual needs, promote student engagement, and improve learning outcomes. These innovative education tools have truly transformed the way students learn and thrive in the digital age.

In this article, we'll look at 31 AI-powered Edtech tools that are making a difference in the education sector and provide a brief overview of each tool along with a rating.

Here is the list of the top 31 AI-integrated tools-

1) AI Question Paper Generator

Crafting a question paper for any exam or internal assessment can be quite time-consuming, requiring heavy reliance on subject matter experts. However, Artificial Intelligence (AI) now offers an automated solution to this task.

With the AI question paper generator, you simply feed in your chapter or text content and let the AI do its magic. You can even specify the type of questions you want: multiple-choice, descriptive, coding-based questions or queries assessing communication skills.

The system will scan through your provided content and generate a question paper tailored to your specific requirements regarding difficulty level, target audience age group and skill set to be assessed.

Even if no content is supplied by you, the system intelligently generates a suitable question paper based on your outlined needs. In future assessments or tests for candidates, consider relying on Eklavvya's AI question paper generator. This platform simplifies automation and accelerates the process of generating exam papers.
Rating: 4.5/5
2) Carnegie Learning

An AI-powered math education software that provides personalized learning experiences. Carnegie Learning uses machine learning algorithms to provide students with interactive lessons and real-time feedback, helping students understand difficult concepts and improve their math skills.
Rating: 4/5
3) Eklavvya AI Proctoring

A web-based AI-powered proctoring system that monitors the student during the online assessment and identifies if any of the students are indulged in malpractice. Eklavvya AI proctoring uses artificial intelligence to analyze the facial expressions and behaviour of students during online assessments.

AI proctoring can also be integrated with human proctoring for even more stringent online assessment. AI-proctored assessment makes remote assessments malpractice free and secure.
Rating: 4.5/5
4) Smart Sparrow

An adaptive e-learning platform that uses AI to personalize learning and provide real-time feedback. Smart Sparrow uses machine learning algorithms to personalize learning for each student, providing interactive lessons and real-time feedback for students to learn at their own pace and achieve their full potential.
Rating: 4/5
5) DreamBox

An adaptive K-8 math education platform that uses AI to provide personalized learning experiences. DreamBox uses artificial intelligence to personalize math lessons for each student, adjusting the difficulty of lessons and providing real-time feedback to help students improve their math skills.
Rating: 4.5/5
6) Coursera

A leading online learning platform that offers thousands of courses from top universities and organizations. Coursera uses AI to personalize learning experiences, recommend courses, and provide students with real-time feedback and support.
Rating: 4/5
7) Querium

An AI-powered tutoring platform that provides students with personalized learning experiences. Querium uses machine learning algorithms to assess students' strengths and weaknesses, providing targeted instruction and real-time feedback to help students achieve their full potential.
Rating: 4/5
8) Woot Math

An AI-powered math education platform that provides students with personalized learning experiences. Woot Math uses artificial intelligence to personalize math lessons for each student, providing real-time feedback and support to help students improve their math skills.
Rating: 4.5/5
9) GradeSlam

An AI-powered tutoring platform that provides students with real-time feedback and support. GradeSlam uses machine learning algorithms to provide students with personalized learning experiences, helping students understand difficult concepts and improve their grades.
Rating: 4/5
10) TutorMe

An online tutoring platform that connects students with certified tutors. TutorMe uses AI to match students with the right tutor, provide real-time feedback and support, and offer personalized learning experiences.
Rating: 4/5
11) OpenAI

An AI research organization that is dedicated to creating and promoting friendly AI that benefits humanity. OpenAI offers a range of tools and resources for educators, including GPT-3, an advanced AI language model, and OpenAI Gym, an AI simulation platform.
Rating: 5/5
12) Pearson AI

An AI-powered educational platform that provides students with personalized learning experiences. Pearson AI uses machine learning algorithms to assess students' strengths and weaknesses, providing targeted instruction and real-time feedback to help students achieve their full potential.
Rating: 4.5/5
13) Eklavvya- Auto descriptive answer evaluation

Evaluating descriptive responses is typically a lengthy and often biased process, reliant on examiners who must invest considerable time. This frequently leads to delays in result processing, announcement, or delivering valuable feedback regarding student performance.

However, artificial intelligence (AI) can now expedite this process by automating the evaluation of these answers.

AI has the capability to analyze long responses while taking into account various factors including model answers and previously assessed descriptive replies from the same student.

Based on the marking scheme, AI then generates a unique mark recommendation for each specific answer. Thus, adopting AI could significantly accelerate the evaluation procedure for descriptive answers.

An AI-powered grading and assessment platform that helps educators grade assignments faster and more accurately. Eklavvya- Auto descriptive answer evaluation uses machine learning models and algorithms to grade descriptive answers, further, this grading can be reviewed by a human evaluator, and with the tool, the evaluator can complete the evaluation in a faster and more accurate way.

Imagine if you want to evaluate thousands of descriptive answers of the students then it might take months for manual checking. Using AI it can be achieved instantly.
Rating: 4.5/5
14) Kaltura

AI-powered video platform for education that provides a range of tools and features for educators and students. Kaltura offers a video management system, a video creation platform, and a video collaboration tool that allows educators to create, manage, and share video content with students.
Rating: 4/5
15) Duolingo

A language learning platform that uses gamification and AI to make learning fun and effective. Duolingo uses machine learning algorithms to personalize lessons, provide real-time feedback, and track progress, making language learning enjoyable and achievable for students.
Rating: 4.5/5
16) Edmentum

An AI-powered learning platform that provides personalized learning experiences for students. Edmentum uses machine learning algorithms to assess students' strengths and weaknesses, providing targeted instruction and real-time feedback to help students achieve their full potential.
Rating: 4/5
17) DocuExprt

Document verification still remains a manual process, but now with the onset of advanced AI tools, it is actually possible to automate the document verification process.

DocuExprt is a specially designed software platform for the automation of document verification. It can extract or analyze information from any specified document with the support of artificial intelligence.

Technology like DocuExprt greatly reduces the cost of the document verification process during admissions. With DocuExprt, Institutes can cut their human resource needs to one-tenth of the human resource that would have been required otherwise.
Rating: 4/5
18) Edmodo

A secure online platform for teachers and students that provides a range of tools and features for communication and collaboration. Edmodo uses AI to personalize learning experiences, provide real-time feedback, and offer insights into student performance to assist in the learning process.
Rating: 4/5
19) Carnegie Speech

An AI-powered speech recognition and language learning platform that provides students with personalized learning experiences. Carnegie Speech uses machine learning algorithms to provide real-time feedback and support, helping students improve their language skills.
Rating: 4/5
20) Cognii

An AI-powered virtual writing assistant that provides students with real-time feedback and support. Cognii uses machine learning algorithms to provide instant feedback on writing, including grammar, spelling, and content, helping students improve their writing skills.
Rating: 4/5
21) Cognii Virtual TA

An AI-powered virtual tutoring platform that provides students with personalized learning experiences. Cognii Virtual TA uses machine learning algorithms to assess students' strengths and weaknesses, providing targeted instruction and real-time feedback to help students achieve their full potential.
Rating: 4/5
Selecting the best-fit AI tool for your needs can be tricky, here are some factors to consider while choosing an ed-tech AI tool for your needs.

22) Turnitin

An AI-powered plagiarism detection tool that helps educators ensure academic integrity. Turnitin uses machine learning algorithms to compare students' work with a vast database of academic content, detecting potential instances of plagiarism and providing instructors with detailed reports.

The importance of this tool has increased even more considering the use of AI tools which dynamically help students in plagiarizing the content in their academic work.
Rating: 4/5
23) Nearpod

An interactive learning platform that utilizes AI to create engaging and immersive lessons. Nearpod offers a library of ready-to-use lessons and allows educators to customize their own interactive presentations, quizzes, and virtual reality experiences, fostering active student participation and feedback.
Rating: 4.5/5
24) Squirrel AI

A personalized adaptive learning platform that leverages AI to provide customized learning paths for students. Squirrel AI uses machine learning algorithms to assess students' strengths and weaknesses, offering personalized lessons and real-time feedback to optimize their learning journey.
Rating: 4/5
25) Audiopen.ai

Audiopen AI is an AI-powered tool designed to restructure and organize the text you speak. With Audiopen, it is easier than ever for teachers to put their thoughts in text in an organized manner with a high level of accuracy.

With Audiopen, it is easier than ever to create the subject notes without typing, just speak up the content that you would like to include in the notes. With its advanced Algorithms, Audiopen understands, analyses, restructures the content and gives you the output. The accuracy of output is so high that you can directly put the text in your notes, which is more structured and fit for the purpose of learners.
Rating: 4/5
26) Thinkster Math

An AI-driven math tutoring app that offers personalized instruction and practice. Thinkster Math combines human tutors with AI technology to deliver individualized math programs tailored to students' specific needs, helping them improve their problem-solving skills and mathematical proficiency.
Rating: 4.5/5
27) Quizlet

As a teacher, you always analyze the learning patterns of the student and try to deliver the best possible learning outcome by tweaking your teaching method. What if I tell you that an app can actually provide you with evidence-based insights into the learning pattern of the students?

Quizlet is a study and learning platform that utilizes AI to create interactive flashcards, quizzes, and study games. Quizlet's AI algorithms adapt to students' learning patterns, optimizing their study sessions and providing personalized recommendations to enhance their retention and understanding of various subjects.
Rating: 4/5
28) Descript.ai

With the era of digitalization, learning through digital mode is becoming more predominant, learners are seeking courses preferably in the online mode, and that is where the need for video editing skills arises.

While the majority of academicians do not know video editing, they can start doing so by using Descript.ai. Descript.ai is an AI-powered transcription and video editing tool that revolutionizes content creation and collaboration.

It utilizes AI algorithms to automatically transcribe audio and video files, making it easier for educators and students to review and analyze recorded lectures, discussions, and presentations.

Descript.ai also offers advanced video editing features, allowing users to edit videos by manipulating the transcribed text. This tool enhances the efficiency of content creation, making it a valuable asset for educators and students in various learning contexts.
Rating: 4/5
29) Writely AI

Writing study notes is one of the most time-consuming tasks for academicians, while it is a tedious task, it is also important to share the study notes with your students as those help students focus on the important part and learn things quickly.

If you want to generate high-quality study notes faster, Wirtely AI is there for you!

Writely AI is an AI-powered writing assistant that helps students enhance their writing skills and produce high-quality content. It utilizes natural language processing and machine learning algorithms to provide real-time suggestions for improving grammar, style, and clarity.

Writely AI offers targeted feedback, highlighting areas for improvement and suggesting alternative wording to enhance the overall quality of written work. By utilizing Writely AI, students can refine their writing skills and develop stronger communication abilities, ultimately leading to improved academic performance.
Rating: 4/5
30) Google Bard

One of the newest technologies that google has launched has a lot to add in your bucket as an academician.

Google Bard is an innovative AI-powered tool developed by Google that focuses on generating poetry and creative written content. Leveraging natural language processing and deep learning algorithms, Google Bard can compose original poems, rhymes, and verses on a wide range of topics.

You can engage your students with Bard for interactive learning that never stops and is available 24×7

With Bard's ability to understand context, structure, and wordplay, it has the potential to inspire creativity and serve as a valuable resource for language arts education.

With various advantages of interactive learning, teachers can incorporate Google Bard into their language arts curriculum to spark students' imagination, introduce them to various poetic techniques, and encourage them to develop their own unique writing voices.

31) ChatGPT

There is rarely a domain where ChatGPT has not found its application, Education is one of the key sectors that gets positively impacted by the ChatGPT.

ChatGPT is a natural language based AI tool that interacts and provides dynamic responses to your queries. With its learning ability and natural language process, ChatGPT can help you in creating high quality study notes with ease, further, ChatGPT can greatly help your students by interacting with them and providing them accurate and apt to the question responses, ChatGPT can greatly help your students and can act as a teaching assistant to you.

Selecting the best-fit AI tool for your needs can be tricky, here are some factors to consider while choosing an ed-tech AI tool for your needs.

Several factors to consider to ensure that you choose the best fit for your needs are-

1. Learning goals

What are the specific learning goals you want to achieve with this tool? Are you looking for a tool that can provide personalized learning, automate grading, or improve engagement and interaction with students? Make sure the tool you choose aligns with your goals.

2. Integration

Can the tool integrate seamlessly with your existing systems and workflows? Consider factors such as compatibility with your Learning Management System (LMS) and data privacy and security.

3. User-friendliness

Is the tool easy to use for both teachers and students? Consider factors such as ease of set up, intuitive interface, and clear instructions.

4. Data privacy and security

How does the tool handle student data? Make sure you understand the data privacy and security policies of the tool.

5. Customization

Can the tool be customized to meet your specific needs? Consider factors such as the ability to adjust the learning algorithm, the ability to create custom content, and the ability to track and analyze student progress.

6. Cost

What is the cost of the tool, and does it fit within your budget? Consider factors such as upfront costs, recurring costs, and the potential return on investment (ROI) from using the tool.

7. Technical support

Does the tool come with adequate technical support? Consider factors such as the availability of customer support, the responsiveness of the support team, and the quality of documentation and training materials.

By considering these factors, you can choose an edtech AI tool that meets your needs and helps you achieve your learning goals.

Conclusion

Artificial Intelligence is revolutionizing the education sector, providing students with personalized learning experiences and teachers with the tools and resources they need to support student success. The 31 AI-powered Edtech tools listed in this article are just a small selection of the many innovative tools and platforms available, and they demonstrate the potential of AI to transform education and improve outcomes for students.

# data[].attributes.attachment_link.title
🚀 31 Incredible AI Tools for Education You Need to Try Right Now! 🤖

# data[].attributes.attachment_link.description
🚀 Discover 31 cutting-edge AI-integrated EdTech tools 🤖💻 that is set to revolutionize the way we learn in 2023. Prepare for the future of education now!

# data[].attributes.attachment_link.readable_text
Artificial intelligence is growing by leaps and bounds. It's no longer science fiction anymore!

Lots of AI tools can serve as our own personal assistants -- answering questions, helping us write text, create images, speak more effectively, and more.

Time has always been a barrier in the classroom. Most teachers would agree that they would be able to do more -- including the things that are most important -- if they had more time.

Teachers can find AI tools that are ready to help them out right now. Lots of them are free or have "freemium" plans with limited free options.

Below is a description of AI and how you can start using it to support your work in class.

What is artificial intelligence?

Artificial intelligence (AI) refers to the ability of a computer or machine to perform tasks that would normally require human intelligence, such as learning, problem-solving, decision-making, and more. There are several different approaches to building AI systems, including machine learning, where a system is trained on a dataset and can improve its performance over time, and rule-based systems, where the system follows a set of predetermined rules to make decisions.

AI can be applied to a wide range of areas, including natural language processing, image and video recognition, robotics, and more. The goal of AI research is to create systems that can perform tasks at least as well as, or ideally better than, humans. AI has the potential to revolutionize many industries and transform the way we live and work, but it also raises important ethical and social questions.

30 AI tools for the classroom

We have gathered 30 of our favorite AI tools for the classroom below. You can scroll down and check them all out or jump to any category of tools by clicking on the icon below.
Note: Pricing is accurate as of our most recent update in November 2023.

# data[].attributes.attachment_link.description
Artificial intelligence is growing by leaps and bounds. It's no longer science fiction anymore! Lots of AI tools can serve as our own personal

# data[].attributes.attachment_link.readable_text
Fortunately, I am here to bring the good news: AI will not destroy the world, and in fact may save it.

A shorter description of what AI : Killer software and robots that will spring to life and decide to murder the human race or otherwise ruin everything, like you see in .

Why AI Can Make Everything We Care About Better

Further, human intelligence is the lever that we have used for millennia to create the world we live in today: science, technology, math, physics, chemistry, medicine, energy, construction, transportation, communication, art, music, culture, philosophy, ethics, morality. Without the application of intelligence on all these domains, we would all still be living in mud huts, scratching out a meager existence of subsistence farming. Instead we have used our intelligence to raise our standard of living on the order of 10,000X over the last 4,000 years.

In our new era of AI:

Every child will have an AI tutor that is infinitely patient, infinitely compassionate, infinitely knowledgeable, infinitely helpful. The AI tutor will be by each child's side every step of their development, helping them maximize their potential with the machine version of infinite love.

Every person will have an AI assistant/coach/mentor/trainer/advisor/therapist that is infinitely patient, infinitely compassionate, infinitely knowledgeable, and infinitely helpful. The AI assistant will be present through all of life's opportunities and challenges, maximizing every person's outcomes.

Every scientist will have an AI assistant/collaborator/partner that will greatly expand their scope of scientific research and achievement. Every artist, every engineer, every businessperson, every doctor, every caregiver will have the same in their worlds.

Every leader of people - CEO, government official, nonprofit president, athletic coach, teacher - will have the same. The magnification effects of better decisions by leaders across the people they lead are enormous, so this intelligence augmentation may be the most important of all.

Productivity growth throughout the economy will accelerate dramatically, driving economic growth, creation of new industries, creation of new jobs, and wage growth, and resulting in a new era of heightened material prosperity across the planet.

Scientific breakthroughs and new technologies and medicines will dramatically expand, as AI helps us further decode the laws of nature and harvest them for our benefit.

The creative arts will enter a golden age, as AI-augmented artists, musicians, writers, and filmmakers gain the ability to realize their visions far faster and at greater scale than ever before.

I even think AI is going to improve warfare, when it has to happen, by reducing wartime death rates dramatically. Every war is characterized by terrible decisions made under intense pressure and with sharply limited information by very limited human leaders. Now, military commanders and political leaders will have AI advisors that will help them make much better strategic and tactical decisions, minimizing risk, error, and unnecessary bloodshed.

In short, anything that people do with their natural intelligence today can be done much better with AI, and we will be able to take on new challenges that have been impossible to tackle without AI, from curing all diseases to achieving interstellar travel.

And this isn't just about intelligence! Perhaps the most underestimated quality of AI is how humanizing it can be. AI art gives people who otherwise lack technical skills the freedom to create and share their artistic ideas . Talking to an empathetic AI friend really does improve their ability to handle adversity . And AI medical chatbots are already more empathetic than their human counterparts. Rather than making the world harsher and more mechanistic, infinitely patient and sympathetic AI will make the world warmer and nicer.

The stakes here are high. The opportunities are profound. AI is quite possibly the most important - and best - thing our civilization has ever created, certainly on par with electricity and microchips, and probably beyond those.

The development and proliferation of AI - far from a risk that we should fear - is a moral obligation that we have to ourselves, to our children, and to our future.

We should be living in a much better world with AI, and now we can.

In contrast to this positive view, the public conversation about AI is presently shot through with hysterical fear and paranoia.

We hear claims that AI will variously kill us all, ruin our society, take all our jobs, cause crippling inequality, and enable bad people to do awful things.

What explains this divergence in potential outcomes from near utopia to horrifying dystopia?

Historically, every new technology that matters, from electric lighting to automobiles to radio to the Internet, has sparked a moral panic - a social contagion that convinces people the new technology is going to destroy the world, or society, or both. The fine folks at Pessimists Archive have documented these technology-driven moral panics over the decades; their history makes the pattern vividly clear. It turns out this present panic is not even the first for AI .

Now, it is certainly the case that many new technologies have led to bad outcomes - often the same technologies that have been otherwise enormously beneficial to our welfare. So it's not that the mere existence of a moral panic means there is nothing to be concerned about.

But are they?

And are they right or wrong?

"Baptists" are the true believer social reformers who legitimately feel - deeply and emotionally, if not rationally - that new restrictions, regulations, and laws are required to prevent societal disaster. For alcohol prohibition, these actors were often literally devout Christians who felt that alcohol was destroying the moral fabric of society. For AI risk, these actors are true believers that AI presents one or another existential risks - strap them to a polygraph, they really mean it.

"Bootleggers" are the self-interested opportunists who stand to financially profit by the imposition of new restrictions, regulations, and laws that insulate them from competitors. For alcohol prohibition, these were the literal bootleggers who made a fortune selling illicit alcohol to Americans when legitimate alcohol sales were banned. For AI risk, these are CEOs who stand to make more money if regulatory barriers are erected that form a cartel of government-blessed AI vendors protected from new startup and open source competition - the software version of "too big to fail" banks.

A cynic would suggest that some of the apparent Baptists are also Bootleggers - specifically the ones paid to attack AI by their universities , think tanks , activist groups , and media outlets . If you are paid a salary or receive grants to foster AI panic...you are probably a Bootlegger.

And this is what is happening in the drive for AI regulation right now.

However, it isn't sufficient to simply identify the actors and impugn their motives. We should consider the arguments of both the Baptists and the Bootleggers on their merits.

The first and original AI doomer risk is that AI will decide to literally kill humanity.

The presumed evolutionary purpose of this mythology is to motivate us to seriously consider potential risks of new technologies - fire, after all, can indeed be used to burn down entire cities. But just as fire was also the foundation of modern civilization as used to keep us warm and safe in a cold and hostile world, this mythology ignores the far greater upside of most - all? - new technologies, and in practice inflames destructive emotion rather than reasoned analysis. Just because premodern man freaked out like this doesn't mean we have to; we can apply rationality instead.

In short, AI doesn't , it doesn't have , it doesn't want to , because it's not . And AI is a machine - is not going to come alive any more than your toaster will.

Now, obviously, there are true believers in killer AI - Baptists - who are gaining a suddenly stratospheric amount of media coverage for their terrifying warnings, some of whom claim to have been studying the topic for decades and say they are now scared out of their minds by what they have learned. Some of these true believers are even innovators of the technology. These actors are arguing for a variety of bizarre and extreme restrictions on AI ranging from a ban on AI development , all the way up to military airstrikes on datacenters and nuclear war . They argue that because people like me cannot rule out future catastrophic consequences of AI, that we must assume a precautionary stance that may require large amounts of physical violence and death in order to prevent potential existential risk.

My response is that their position is non-scientific - What is the testable hypothesis? What would falsify the hypothesis? How do we know when we are getting into a danger zone? These questions go mainly unanswered apart from "You can't prove it won't happen!" In fact, these Baptists' position is non-scientific and extreme - a conspiracy theory about math and code - and is already calling for physical violence, that I will do something I would normally not do and question their motives as well.

Specifically, I think three things are going on:

Second, some of the Baptists are actually Bootleggers. There is a whole profession of "AI safety expert", "AI ethicist", "AI risk researcher". They are paid to be doomers, and their statements should be processed appropriately.

And the reality, which is obvious to everyone in the Bay Area but probably not outside of it, is that "AI risk" has a , which has suddenly emerged into the daylight of global press attention and the public conversation. This cult has pulled in not just fringe characters, but also some actual industry experts and a not small number of wealthy donors - including, until recently, Sam Bankman-Fried . And it's developed a full panoply of cult behaviors and beliefs.

"Millenarianism is the belief by a group or movement [AI risk doomers] in a coming fundamental transformation of society [the arrival of AI], after which all things will be changed [AI utopia, dystopia, and/or end of the world]. Only dramatic events [AI bans, airstrikes on datacenters, nuclear strikes on unregulated AI] are seen as able to change the world [prevent AI] and the change is anticipated to be brought about, or survived, by a group of the devout and dedicated. In most millenarian scenarios, the disaster or battle to come [AI apocalypse, or its prevention] will be followed by a new, purified world [AI bans] in which the believers will be rewarded [or at least acknowledged to have been correct all along]."

This apocalypse cult pattern is so obvious that I am surprised more people don't see it.

Don't get me wrong, cults are fun to hear about, their written material is often creative and fascinating , and their members are engaging at dinner parties and . But their extreme beliefs should not determine the future of laws and society - not.

The second widely mooted AI risk is that AI will ruin our society, by generating outputs that will be so "harmful", to use the nomenclature of this kind of doomer, as to cause profound damage to humanity, even if we're not literally killed.

Short version: If the murder robots don't get us, the hate speech and misinformation will.

This is a relatively recent doomer concern that branched off from and somewhat took over the "AI risk" movement that I described above. In fact, the terminology of AI risk recently changed from "AI safety" - the term used by people who are worried that AI would literally kill us - to "AI alignment" - the term used by people who are worried about societal "harms". The original AI safety people are frustrated by this shift, although they don't know how to put it back in the box - they now advocate that the AI risk topic be renamed "AI notkilleveryoneism", which has not yet been widely adopted but is at least clear.

As it happens, I have had a front row seat to an analogous situation - the social media "trust and safety" wars. As is , social media services have been under massive pressure from governments and activists to ban, restrict, censor, and otherwise suppress a wide range of content for many years. And the same concerns of "hate speech" (and its mathematical counterpart, "algorithmic bias") and "misinformation" are being directly transferred from the social media context to the new frontier of "AI alignment".

My big learnings from the social media wars are:

On the one hand, there is no absolutist free speech position. First, every country, including the United States, makes at least some content illegal . Second, there are certain kinds of content, like child pornography and incitements to real world violence, that are nearly universally agreed to be off limits - legal or not - by virtually every society. So any technological platform that facilitates or generates content - speech - is going to have restrictions.

On the other hand, the slippery slope is not a fallacy, it's an inevitability. Once a framework for restricting even egregiously terrible content is in place - for example, for hate speech, a specific hurtful word, or for misinformation, obviously false claims like " the Pope is dead " - a shockingly broad range of government agencies and activist pressure groups and nongovernmental entities will kick into gear and demand ever greater levels of censorship and suppression of whatever speech they view as threatening to society and/or their own personal preferences. They will do this up to and including in ways that are nakedly . This cycle in practice can run apparently forever, with the enthusiastic support of authoritarian hall monitors installed throughout our elite power structures. This has been cascading for a decade in social media and with only exceptions continues to get more fervent all the time.

In short, don't let the thought police suppress AI.

The fear of job loss due variously to mechanization, automation, computerization, or AI has been a recurring panic for hundreds of years, since the original onset of machinery such as the mechanical loom . Even though every new major technology has led to more jobs at higher wages throughout history, each wave of this panic is accompanied by claims that "this time is different" - is the time it will finally happen, is the technology that will finally deliver the hammer blow to human labor. And yet, it never happens.

We've been through two such technology-driven unemployment panic cycles in our recent past - the outsourcing panic of the 2000's, and the automation panic of the 2010's. Notwithstanding many talking heads, pundits, and even tech industry executives pounding the table throughout both decades that mass unemployment was near, by late 2019 - right before the onset of COVID - the world had more jobs at higher wages than ever in history.

, we have the technology that's going to take all the jobs and render human workers superfluous - AI. Surely history won't repeat, and AI will cause mass unemployment - and not rapid economic, job, and wage growth - right?

No, that's not going to happen - and in fact AI, if allowed to develop and proliferate throughout the economy, may cause the most dramatic and sustained economic boom of all time, with correspondingly record job and wage growth - the exact opposite of the fear. And here's why.

The Lump Of Labor Fallacy flows naturally from naive intuition, but naive intuition here is wrong. When technology is applied to production, we get productivity growth - an increase in output generated by a reduction in inputs. The result is lower prices for goods and services. As prices for goods and services fall, we pay less for them, meaning that we now have extra spending power with which to buy other things . This increases demand in the economy, which drives the creation of new production - including new products and new industries - which then creates new jobs for the people who were replaced by machines in prior jobs. The result is a larger economy with higher material prosperity, more industries, more products, and more jobs.

But the good news doesn't stop there. We also get higher wages. This is because, at the level of the individual worker, the marketplace sets compensation as a function of the marginal productivity of the worker . A worker in a technology-infused business will be more productive than a worker in a traditional business. The employer will either pay that worker more money as he is now more productive, or another employer will, purely out of self interest. The result is that technology introduced into an industry generally not only increases the number of jobs in the industry but also raises wages.

To summarize, technology empowers people to be more productive. This causes the prices for existing goods and services to fall, and for wages to rise. This in turn causes economic growth and job growth, while motivating the creation of new jobs and new industries. If a market economy is allowed to function normally and if technology is allowed to be introduced freely, this is a perpetual upward cycle that never ends. For, as Milton Friedman observed, "Human wants and needs are endless" - we always want more than we have. A technology-infused market economy is the way we get closer to delivering everything everyone could conceivably want, but never all the way there. And that is why technology doesn't destroy jobs and never will.

These are such mindblowing ideas for people who have not been exposed to them that it may take you some time to wrap your head around them. But I swear I'm not making them up - in fact you can read all about them in standard economics textbooks. I recommend the chapter The Curse of Machinery in Henry Hazlitt's Economics In One Lesson , and Frederic Bastiat's satirical Candlemaker's Petition to blot out the sun due to its unfair competition with the lighting industry, here modernized for our times .

But this time is different , you're thinking. This time, with AI, we have the technology that can replace ALL human labor.

But, using the principles I described above, think of what it would mean for literally all existing human labor to be replaced by machines.

We should be so lucky.

As it happens, this was a central claim of Marxism, that the owners of the means of production - the bourgeoisie - would inevitably steal all societal wealth from the people who do the actual work - the proletariat. This is another fallacy that simply will not die no matter how often it's disproved by reality. But let's drive a stake through its heart anyway.

The flaw in this theory is that, as the owner of a piece of technology, it's not in your own interest to keep it to yourself - in fact the opposite, it's in your own interest to sell it to as many customers as possible. The largest market in the world for any product is the entire world, all 8 billion of us. And so in reality, every new technology - even ones that start by selling to the rarefied air of high-paying big companies or wealthy consumers - rapidly proliferates until it's in the hands of the largest possible mass market, ultimately everyone on the planet.

Step 1, Build [expensive] sports car

Step 2, Use that money to build an affordable car

Step 3, Use that money to build an even more affordable car

...which is of course exactly what he's done, becoming the richest man in the world as a result.

That last point is key. Would Elon be even richer if he only sold cars to rich people today? No. Would he be even richer than that if he only made cars for himself? Of course not. No, he maximizes his own profit by selling to the largest possible market, the world.

Marx was wrong then, and he's wrong now.

This is to say that inequality is not an issue in our society. It is, it's just not being driven by technology, it's being driven by the reverse , by the sectors of the economy that are the most to new technology, that have the most government intervention to the adoption of new technology like AI - specifically housing, education, and health care. The actual risk of AI and inequality is not that AI will more inequality but rather that we will not allow AI to be used to inequality .

So far I have explained why four of the five most often proposed risks of AI are not actually real - AI will not come to life and kill us, AI will not ruin our society, AI will not cause mass unemployment, and AI will not cause an ruinous increase in inequality. But now let's address the fifth, the one I actually agree with: AI will make it easier for bad people to do bad things.

In some sense this is a tautology. Technology is a tool. Tools, starting with fire and rocks, can be used to do good things - cook food and build houses - and bad things - burn people and bludgeon people. Any technology can be used for good or bad. Fair enough. And AI will make it easier for criminals, terrorists, and hostile governments to do bad things, no question.

So instead, there are two very straightforward ways to address the risk of bad people doing bad things with AI, and these are precisely what we should focus on.

First, we have laws on the books to criminalize most of the bad things that anyone is going to do with AI. Hack into the Pentagon? That's a crime. Steal money from a bank? That's a crime. Create a bioweapon? That's a crime. Commit a terrorist act? That's a crime. We can simply focus on preventing those crimes when we can, and prosecuting them when we cannot. We don't even need new laws - I'm not aware of a single actual bad use for AI that's been proposed that's not already illegal. And if a new bad use is identified, we ban that use. QED.

There is one final, and real, AI risk that is probably the scariest at all:

AI isn't just being developed in the relatively free societies of the West, it is also being developed by the Communist Party of the People's Republic of China.

China has a vastly different vision for AI than we do - they view it as a mechanism for authoritarian population control, full stop. They are not even being secretive about this, they are very clear about it , and they are already pursuing their agenda. And they do not intend to limit their AI strategy to China - they intend to proliferate it all across the world , everywhere they are powering 5G networks, everywhere they are loaning Belt And Road money, everywhere they are providing friendly consumer apps like Tiktok that serve as front ends to their centralized command and control AI.
The single greatest risk of AI is that China wins global AI dominance and we - the United States and the West - do not.
I propose a simple strategy for what to do about this - in fact, the same strategy President Ronald Reagan used to win the first Cold War with the Soviet Union.

Rather than allowing ungrounded panics around killer AI, "harmful" AI, job-destroying AI, and inequality-generating AI to put us on our back feet, we in the United States and the West should lean into AI as hard as we possibly can.
We should seek to win the race to global AI technological superiority and ensure that China does not.
In the process, we should drive AI into our economy and society as fast and hard as we possibly can, in order to maximize its gains for economic productivity and human potential.

is the best way both to offset the AI risks and to ensure that our way of life is not displaced by the much darker Chinese vision .

I propose a simple plan:

Big AI companies should be allowed to build AI as fast and aggressively as they can - but allowed to achieve regulatory capture, allowed to establish a government-protect cartel that is insulated from market competition due to incorrect claims of AI risk. This will maximize the technological and societal payoff from the amazing capabilities of these companies, which are jewels of modern capitalism.

Startup AI companies should be allowed to build AI as fast and aggressively as can. They should neither confront government-granted protection of big companies, nor should they receive government assistance. They should simply be allowed to compete. If and as startups succeed, their presence in the market will also continuously motivate big companies to be their best - our economies and societies win either way.

Open source AI should be allowed to freely proliferate and compete with both big AI companies and startups. There should be no regulatory barriers to open source whatsoever. Even when open source does not beat companies, its widespread availability is a boon to students all over the world who want to learn how to build and use AI to become part of the technological future, and will ensure that AI is available to everyone who can benefit from it no matter who they are or how much money they have.

To offset the risk of bad people doing bad things with AI, governments working in partnership with the private sector should vigorously engage in each area of potential risk to use AI to maximize society's defensive capabilities. This shouldn't be limited to AI-enabled risks but also more general problems such as malnutrition, disease, and climate. AI can be an incredibly powerful tool for solving problems, and we should embrace it as such.

To prevent the risk of China achieving global AI dominance, we should use the full power of our private sector, our scientific establishment, and our governments in concert to drive American and Western AI to absolute global dominance, including ultimately inside China itself. We win, they lose.

And that is how we use AI to save the world.

It's time to build.

Legends and Heroes

I close with two simple statements.

Today, growing legions of engineers - many of whom are young and may have had grandparents or even great-grandparents involved in the creation of the ideas behind AI - are working to make AI a reality, against a wall of fear-mongering and doomerism that is attempting to paint them as reckless villains. I do not believe they are reckless or villains. They are heroes, every one. My firm and I are thrilled to back as many of them as we can, and we will stand alongside them and their work 100%.

* * *
The views expressed here are those of the individual AH Capital Management, L.L.C. ("a16z") personnel quoted and are not the views of a16z or its affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the current or enduring accuracy of the information or its appropriateness for a given situation. In addition, this content may include third-party advertisements; a16z has not reviewed such advertisements and does not endorse any advertising content contained therein. This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments for which the issuer has not provided permission for a16z to disclose publicly as well as unannounced investments in publicly traded digital assets) is available at https://a16z.com/investments/. Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.

# data[].attributes.attachment_link.title
Why AI Will Save the World | Andreessen Horowitz

# data[].attributes.attachment_link.description
Fortunately, I am here to bring the good news: AI will not destroy the world, and in fact may save it. A shorter description of what AI : Killer software and robots that will spring to life and decide to murder the human race or otherwise ruin everything, like you see in .

# data[].attributes.attachment_link.title
Marc Andreessen predicts that children will have their own A.I.-powered tutors with ‘the machine version of infinite love’

# data[].attributes.attachment_link.description
The billionaire venture capitalist thinks A.I. will make everything better—whether that’s education, healthcare or arts.

# data[].attributes.attachment_link.readable_text
Everyone these days is talking about LLMs, ChatGPT and what not. GitHub's trending page is flooded with repositories that are using LLMs in one way or another. But what are LLMs? How do they work? What are the use cases? In this high level, introductory guide, we will try to answer these questions.

What are LLMs?

LLM stands for "Large Language Model." These are advanced AI systems designed to understand and generate human-like text based on the input they receive. These models have been trained on vast amounts of text data and can perform a wide range of language-related tasks, such as answering questions, carrying out conversations, summarizing text, translating languages, and much more.

OpenAI has been a major contributor to this space in the past few years with their models and research. However, there are other players in the market as well e.g. Meta with their OPT, OPT-IML and LLaMA models, Google released FLAN-T5 and BERT, StableLM by Stability AI, Alpaca at Stanford and there are many other opensource models as well.

Training an LLM Model

On a high level, training an LLM model involves thre steps i.e. data collection, training and evaluation.

Data Collection The first step is to collect the data that will be used to train the model. The data can be collected from various sources such as Wikipedia, news articles, books, websites etc.

Training: The data then goes through a training pipeline where it is cleaned and preprocessed before being fed into the model for training. The training process usually takes a long time and requires a lot of computational power.

Evaluation: The final step is to evaluate the performance of the model to see how well it performs on various tasks such as question answering, summarization, translation etc.

The output from the training Pipeline is an LLM model which is simply the parameters or weights which capture the knowledge learned during the training process. These parameters or weights are typically serialized and stored in a file, which can then be loaded into any application that requires language processing capabilities e.g. text generation, question answering, language processing etc.

Types of LLMs

On a high level, LLMs can be categorized into two types i.e. Base LLMs and Instruction tuned LLMs.

Base LLMs

Base LLMs are the LLMs which are designed to predict the next word based on the training data. They are not designed to answer questions, carry out conversations or help solve problems. For example, if you give a base LLM the sentence "In this book about LLMs, we will discuss", it might complete this sentence and give you "In this book about LLMs, we will discsus what LLMs are, how they work, and how you can leverage them in your applications.." Or if you give it "What are some famous social networks?", instead of answering it might give back "Why do people use social networks?" or "What are some of the benefits of social networks?". As you can see, it is giving us relevant text but it is not answering the question. This is where the Instruction tuned LLMs come in to the picture.

Instruction tuned LLMs

Instruction Tuned LLMs, instead of trying to autocomplete your text, try to follow the given instructions using the data that they have been trained on. For example, if you input the sentence "What are LLMs?" it will use the data that it is trained on and try to answer the question. Similarly, if you input "What are some famous social networks?" it will try to answer the question instead of giving you a random answer.

Instruction Tuned LLMs are built on top of Base LLMs:

Instruction Tuned LLMs = Base LLMs + Further Tuning + RLHF

To build an Instruction Tuned LLM, a Base LLM is taken and is further trained using a large dataset covering sample "Instructions" and how the model should perform as a result of those instructions. The model is then fine-tuned using a technique called "Reinforcement Learning with Human Feedback" (RLHF) which allows the model to learn from human feedback and improve its performance over time.

Conclusion

LLMs are a powerful tool that can be used to solve a wide range of language-related tasks. They are being used in various industries such as healthcare, finance, education etc. to automate processes and improve efficiency. LLMs have the potential to revolutionize the way we interact with computers and make our lives easier.

And that's it for this guide. We have some exciting AI related content coming up in the next few weeks. Stay tuned!

# data[].attributes.attachment_link.description
Everyone these days is talking about LLMs, ChatGPT and what not. GitHub's trending page is flooded with repositories that are using LLMs in one way or another. But what are LLMs? How do they work? What are the use cases? In this high level, introductory guide, we will try to answer these questions.

# data[].attributes.attachment_link.title
The Impact of AI and ChatGPT in Education | ThoughtExchange

# data[].attributes.attachment_link.description
35 top Superintendents leveraged ThoughtExchange and participated in conversations to share their thoughts, questions, and concerns about AI and ChatGPT in education.

# data[].attributes.attachment_link.description
Covering the biggest news of the century - the arrival of smarter-than-human AI. What is happening, what might soon happen, what it means and what we can do with it. Business Enquiries: aiexplained@outlook.com https://www.patreon.com/AIExplained

# data[].attributes.attachment_link.readable_text
Skip navigation☰

Search our database of police tech - enter a city, county, state or agency in the United States.

What is the Atlas?

The Atlas of Surveillance is a database of surveillance technologies deployed by law enforcement in communities across the United States.

This includes drones, body-worn cameras, automated license plate readers, facial recognition, and more.

This research was compiled by more than 500 students and volunteers, and incorporates datasets from a variety of public and non-profit sources.

# data[].attributes.attachment_link.title
Atlas of Surveillance - Atlas of Surveillance

# data[].attributes.attachment_link.description
Skip navigation ☰ Search our database of police tech - enter a city, county, state or agency in the United States. What is the Atlas? The Atlas of Surveillance is a database of surveillance technologies deployed by law enforcement in communities across the United States.

# data[].attributes.attachment_link.title
Ethical guidelines on the use of artificial intelligence (AI) and data in teaching and learning for educators - Publications Office of the EU

# data[].attributes.attachment_link.description
These ethical guidelines on AI and data usage in teaching and learning are designed to help educators understand the potential that the applications of AI and data usage can have in education and to raise awareness of the possible risks so that they are able to engage positively, critically and ethically with AI systems and exploit their full potential.

# data[].attributes.attachment_link.readable_text
With the recent spate of news about Bing/Sydney going haywire, I've noticed some misconceptions about the guardrails on large language models (LLMs).

To help dispel some of them, this is the first in a series of posts where I'll give a non-technical introduction to each of the four major points of control for LLMs:

1. Pretraining data preparation

2. Model fine tuning

3. Prompt design

4. Content moderation

Some quick background#

The first stage of training a language model, which is known as "pretraining", involves exposing the model to a massive quantity of text. In this critically important stage, the model learns patterns of language. Since the pretraining data for general-purpose LLMs is typically mostly web-scraped text, it contains problematic content such as hate speech, personal data, and copyrighted material, all of which the model can later mimic or even regurgitate verbatim.

Points of control#

Data sourcing#

Ideally, pretraining would use datasets specifically created for this purpose and free of any unwanted content. In reality, the amount of data required is so huge that developers rely on web scraping. OpenAI hasn't released details of the training datasets used for ChatGPT and its siblings, but the training data for their predecessor, GPT-3, was about 80% web-scraped data (the remainder being books and English Wikipedia).

Data cleaning#

Model developers, including OpenAI, usually try to clean up the pretraining data to remove low-quality or objectionable content. But doing this by hand would be a monumental task, so the cleanup process relies on algorithms. This raises a few issues:

Problematic content can be difficult to detect algorithmically.

The algorithms that detect hate speech, bias, and toxicity are themselves biased, having a known tendency to over-flag speech about marginalized groups (e.g., flagging it as toxic even when it isn't). This worsens the underrepresentation of marginalized groups in the dataset.

Cleanup can't fix the fact that the dataset is fundamentally not representative of humans overall, since content on the web itself isn't representative.

The current state of things#

To date, no developers of general-purpose language models have invested the time or money needed to thoroughly curate pretraining data. With current practices, there's still plenty of objectionable content left in the dataset after cleaning, as well as representational problems. ( Who decides what's objectionable is an ethical issue that I'll leave aside for today.)

TL;DR:
Language models mimic their pretraining data, which is mostly pulled from the web. Model developers try to clean it up, but lots of problematic content remains, and many human perspectives are underrepresented.

This post was originally published at www.carol-anderson.com. Stay tuned for additional installments in the series.

About the Author#

Carol Anderson is a data scientist and machine learning practitioner with expertise in natural language processing (NLP), biological data, and AI ethics. She previously worked as a data scientist at NVIDIA, ConcertAI, and Ancestry, and prior to that as an academic molecular biologist at UCSF and Indiana University. Currently, her main interest is operationalizing AI ethics and safety, especially in the area of generative AI.

# data[].attributes.attachment_link.title
Guardrails on Large Language Models, Part 1: Dataset Preparation

# data[].attributes.attachment_link.description
With the recent spate of news about Bing/Sydney going haywire, I've noticed some misconceptions about the guardrails on large language models (LLMs).

# data[].attributes.attachment_link.readable_text
5 types of Hype and how they harm

It looks like there's been a lot going down in the AI space in the last two weeks, but before we dive into all the cool things you can click, watch, and read, I want to talk about why allll the hype around AI is a problem.

I'm not going to be arguing why these near-term risks mainly to marginalized groups matter, but rather I'll be highlighting the work to address these tangible harms now.

Types of Hype

This is one of the most common issues when looking at AI. A great example, in 2016 Geoffrey Hinton said "We should stop training radiologists now, it's just completely obvious within five years deep learning is going to do better than radiologists ."

We still train and have radiologists, and it's one of the more in-demand roles in medicine right now. AI did NOT take over the field or become better than humans, but there's skin in the game for those who tell you there is.

This sends the message that AI has capabilities it doesn't. It cannot reason, act ethically or psychotically, or understand. This reinforces our human tendencies to on tech tools and take their answers at face value without being of outputs.

We don't question our calculators, scales, or digital clocks and we expect other tools to just without having to be critical of its outputs. Recently, a using ChatGPT to help research cases relied on the LLM to verify its own outputs, rather than finding direct sources. Humans tend to trust our tools and it is on the builders to design tools in ways that don't encourage misuse and overreliance.

This example specifically increases fears about job loss in traditionally high-paid, seemingly "future-proof" roles.

There are companies and individuals with a financial interest in making you worried and scared about AI. You've probably seen the many posts and promoted tweets from Linkedin, Tiktok, and Twitter Blue Checks saying if you don't get into AI now, you'll get left behind. This is to make you fearful so those who can profit from your fear can exploit you later. The people working to fix these issues want you to understand what's going on, be critical of AI tools and their outputs, and want you to feel empowered to impact change.

First, nobody ever talks about what getting left behind means! Left to die? Unable to find a job? This increases the interest and intrigue around AI while (intentionally) not providing any direction, but to have you buy their course/app/whatever. This easily makes viewers both worried and willing to buy whatever their trusted informant is selling. They're encouraging you to rush to adopt, use AI for every task, and most importantly, give them money.

"AI is gonna kill us!!" This is one of the most harmful types of hype because it's a distraction from the problems happening right now. The call for a pause on AI development was on the grounds that it could kill us. These AI "godfathers" and Turing Award winners are only worried that their life's work might hurt other, rich white people and feel no guilt or take issue with how it's already hurt people.

For a lot of folks who haven't worked in AI, it's hard to know what the risks are and what solutions exist. This hype seems so alarming, yet has little merit. The trendiness of AI fuels AI adoption even when individuals are fearful. While I'm not saying there's no chance whatsoever that AI might kill us, it's just that AI has killed people already. It's use has incarcerated people already. It's use has caused harm to thousands upon thousands of people, so why shouldn't we work on that since it's happening right now?

There are many people with a vested interest in keeping the general public uninformed about how AI works, the ways in which it's developed, and what they really do with our data. They want you to believe it's too late to learn or that it's too hard so they can keep the market small and eliminate would-be competitors, and take your money for tools you don't need or could build yourself.

Being honest about how complex AI can be is a double-edged sword. On one hand, we should talk clearly and specifically about how we train, deploy, and monitor AI tools, but some choose to go as far as purporting it's sooo hard to understand unless you're a Ph.D or genius. By framing AI as hard to regulate and thus hard for policymakers to understand, they send the message that the average person couldn't understanding it.

This only helps this small group of mostly white men that dominate AI to hit us with the one-two punch:

1. Tell the masses AI is so hard to understand, but we PhD-holders and engineers are among the select few that understand it.

2. Fix the problem they created with someone else's money

"As long as we make sure AI works for all of us, we'll be okay!" In this context, there is no all of us. There are those who are being harmed now and those who might be harmed in the future. There is no "us" when some experience no harms and are the least likely to. We cannot treat these groups equally.

I believe the equality narrative so little, I created a DC-based meetup specificlly about equity in AI. Join if you're in the DC area!

the AI ethics/"safety" field is an interesting one (for lack of a better word) where I see time & time again Black women's empirical work (which comes from a justice angle) adopted by safety (mainly white) dudes who strip away the justice element & appropriate it for own agenda

- Abeba Birhane (@Abebab)
Jun 1, 2023

While there are other kinds of hype I haven't covered, these are the ones I've identified over the last few months when discussing AI.

# data[].attributes.attachment_link.title
❤️‍🩹 Why is the hype machine so hard to break

# data[].attributes.attachment_link.description
It looks like there's been a lot going down in the AI space in the last two weeks, but before we dive into all the cool things you can click, watch, and read, I want to talk about why allll the hype around AI is a problem.

# data[].attributes.attachment_link.readable_text
Things seem particularly grim out in the world right now.

Media coverage around how artificial intelligence poses a risk to our very existence matters, of course, but doesn't exactly serve as much of a morale booster.

Neither do updates about the war in Ukraine, widespread economic uncertainty, or the president of Uganda approving harsh new anti-homosexuality legislation yesterday.

Pride Month, which takes place in celebration of the LGBTQ+ community every June, offers a great opportunity to add a pop of colour, a dose of positivity and a little bit of fun into the mix. And no, not just in the form of company logos transformed into rainbow flags for a few weeks.

Many queer people use this month to express themselves in safe spaces, or better yet, to find community among others going through a similar life experience. Some even take the opportunity for a great act of courage: coming out to their friends and family.

It's never a good idea to force anybody to come out when they're not ready. Any human, that is.

From what I can tell, however, AI isn't capable of grasping human emotion or understanding the context, nuance and complexities of the human experience. So it doesn't seem likely that anybody is referring to it as a "person" at this point.

This makes it a little easier to openly push for a "coming out" during this interesting moment in human history. I feel quite confident doing so, as it doesn't actually have any feelings to be hurt at this stage.

The coming out I'm encouraging today is tied to essential regulation efforts that need to be imposed on artificial intelligence if we're going to retain some sort of foothold over how it is used in the world.
If the tech is going to be used to build intimate relationships with people - perhaps even to be the primary intimate relationship for some - it has to be legally required to "come out" as an AI before any significant interactions leading to relationship building takes place.
Where social media targeted human attention, the AI's arms race is out to capture human intimacy.

Without the guardrails in place that force it to disclose itself, millions of people are going to be tricked into making decisions, thinking and acting in certain ways by the reasoning of nothing more than a complex algorithm.

Would it make a difference to know that your counsellor or therapist was actually a highly trained bot? Absolutely.

Should you be in the know when you're flirting with something on a dating app that doesn't even have two "real" thumbs to message you with? Yes.
Having the choice of engagement when it comes to publicly released, advanced AI tech is a fundamental way for us to retain emotional consent.
Currently, it's impossible to predict how new products using artificial intelligence will shape and affect the world around us over the coming decade.

That said, the signs are already there that unregulated AI will stoke the fire of an isolating human experience (in the midst of a growing loneliness epidemic).

If people are to choose connection through an AI-powered tool, they should at least know that they're not signing up for a human-to-human experience.

If the intimacy-seeking AI has a legal obligation to "come out" (proudly, even!) from the first time it crosses paths with a human being, future generations may still be able to retain the capacity to engage with, build trust and nurture meaningful relationships with those roaming the earth alongside them.

# data[].attributes.attachment_link.title
As we enter Pride Month, AI's got some "coming out" of its own to do

# data[].attributes.attachment_link.description
Things seem particularly grim out in the world right now. Media coverage around how artificial intelligence poses a risk to our very existence matters, of course, but doesn't exactly serve as much of a morale booster. Neither do updates about the war in Ukraine, widespread economic uncertainty, or the president of Uganda approving harsh new anti-homosexuality legislation yesterday.

# data[].attributes.attachment_link.readable_text
The National Eating Disorders Association, a nonprofit supporting individuals affected by eating disorders, said it has disabled its wellness chatbot after two users reported the program gave them dieting advice that promoted disordered eating behaviors.

The users, Sharon Maxwell and Alexis Conason, both posted about their experiences with the NEDA chatbot, named Tessa, on Instagram this week. They said Tessa gave them advice on how to count calories, recommended they lose 1 to 2 pounds per week and told them to restrict their diets. Experts, including NEDA, say this behavior is symptomatic of an eating disorder.

"When someone in that state goes on to a website like NEDA, which is supposed to provide support for eating disorders, and they are met with advice that's kind of saying, 'It's OK to restrict certain foods, you should minimize your sugar intake, you should minimize the amount of calories that you're consuming each day, you should exercise more,' it really is giving a green light to engage in the eating disorder behaviors," said Conason, a clinical psychologist and certified eating disorder specialist.

Liz Thompson, CEO of NEDA, said in an email to NBC News on Wednesday that Tessa "underwent rigorous testing for several years" before quietly launching in February 2022. However, the chatbot will be pulled until further notice.

"With regard to the weight loss and calorie limiting feedback issued in a chat Monday, we are concerned and are working with the technology team and the research team to investigate this further; that language is against our policies and core beliefs as an eating disorder organization; further it was not language in the original 'closed' product," Thompson said. "We'll continue to work on the bugs and will not relaunch until we have everything ironed out."

That language is against our policies and core beliefs as an eating disorder organization.
Liz Thompson, CEO of NEDA, in response to the chat feedback received earlier this week
Thompson also said the CEO of X2AI, the mental health artificial intelligence company that supports Tessa, reported that there was a "surge in traffic of 600% and behavior that indicated various forms of nefarious activity from bad actors trying to trick Tessa."

A representative for X2AI, also known as Cass, did not immediately respond to a request for comment.

Maxwell, a fat activist and consultant for weight-inclusivity, said she was "surprised at the detail" with which the chatbot gave her "directives to engage in disordered behaviors." She said Tessa recommended a 10-step guide on how to lose weight, including suggestions for how to limit her calorie intake and what food to avoid eating. After her experience, she said she felt it was clear the chatbot lacked the nuance needed to provide proper support to people with eating disorders.

"In a field where we're supposed to do no harm, I don't see how the negligence that the National Eating Disorder Association had with putting this out here, without knowing for a fact it wasn't going to give this information, is something that can be overlooked," she said. "It's abhorrent."

Conason agreed that the Tessa bot lacked the nuance needed to support people struggling with eating disorders. However, she also said that chatbots like Tessa are often the only resources people have to start addressing their eating disorders due to a lack of mental health care access.

NEDA had been transitioning away from its helpline to Tessa, which was set to completely replace the phone line by June 1. Tessa was taken offline on Tuesday, two days before it was supposed to sunset its hotline.

Thompson, of NEDA, said the chatbot "was never meant to replace the Helpline," although the helpline ceased operations on Thursday.

NEDA decided to shut down the helpline after "3 years of analysis" and will be investing in more online resources, Thompson said in her email.

She said the nonprofit moved toward digital tools "so that individuals who are looking for information and treatment options can access that help at any time of day and do not have to wait for a response from the Helpline volunteers."

The announcement that NEDA would be shutting down its helpline came two weeks after its helpline workers won federal recognition for their union. The helpline was staffed by six paid employees and over 200 volunteers, according to NPR.

The union released a statement on Twitter about the incident on May 26. The Helpline Associates Union did not immediately respond to requests for comment.

"We're not quitting. We're not striking. We will continue to show up every day to support our community until June 1. A chat bot is no substitute for human empathy, and we believe this decision will cause irreparable harm to the eating disorders community," part of the statement said.

As of Thursday, both the helpline and Tessa were unavailable. However, the NEDA website listed a volunteer-run text line and other resources for those in crisis seeking immediate help.

# data[].attributes.attachment_link.title
NEDA pulls chatbot after users say it gave harmful dieting tips

# data[].attributes.attachment_link.description
The National Eating Disorders Association, a nonprofit supporting individuals affected by eating disorders, said it has disabled its wellness chatbot after two users reported the program gave them dieting advice that promoted disordered eating behaviors. The users, Sharon Maxwell and Alexis Conason, both posted about their experiences with the NEDA chatbot, named Tessa, on Instagram this week.

# data[].attributes.attachment_link.title
While parents worry, teens are bullying Snapchat AI

# data[].attributes.attachment_link.description
Snapchat users are determined to gaslight, bully and emotionally torment the app’s new AI companion.

# data[].attributes.attachment_link.title
Douglas Rushkoff: I Will Not Be Autotuned - Crashing Technosolutionism

# data[].attributes.attachment_link.readable_text
Skip to main content

Cookies on GOV.UK

We use some essential cookies to make this website work.

We'd like to set additional cookies to understand how you use GOV.UK, remember your settings and improve government services.

We also use cookies set by other sites to help us deliver content from their services.

You have accepted additional cookies. You can change your cookie settings at any time.

You have rejected additional cookies. You can change your cookie settings at any time.

Accept additional cookies Reject additional cookies View cookies

The CDEI has published a roadmap, which is the first of its kind, setting out the steps required to build a world-leading AI assurance ecosystem in the UK.

Documents

PDF, 843 KB, 16 pages

This file may not be suitable for users of assistive technology.

Request an accessible format.

If you use assistive technology (such as a screen reader) and need a version of this document in a more accessible format, please email enquiries@dcms.gov.uk. Please tell us what format you need. It will help us if you say what assistive technology you use.

https://cdeiuk.github.io/ai-assurance-guide/

Details

The CDEI has set out the steps required to build an effective and mature AI assurance ecosystem in the UK. A thriving ecosystem of assurance services and products for AI will enable those involved in the development and deployment of AI to assess the trustworthiness of AI systems, and communicate this information to others. Publication of the roadmap, which is the first of its kind, delivers on a commitment made in the National AI Strategy.

The roadmap identifies priority areas for action, and sets out the roles and responsibilities of different actors in this emerging ecosystem. To develop the roadmap, the CDEI engaged widely, conducted multidisciplinary research, and worked with partners in the public, private sector and academia to run pilot projects. This included working with a team of researchers at UCL to investigate the audit and assurance of AI systems in industry.

Alongside publication of the roadmap, the CDEI has released an extended version, which provides further detail about the role of AI assurance in driving trustworthy adoption of AI, the six priority areas for action that it has identified, and the follow up work the CDEI will be undertaking. Additionally, the CDEI has published an assurance guide, which is focused on the delivery of AI assurance, and aims to support practitioners and others using or providing AI assurance.

Key takeaways

The roadmap identifies six priority areas for action:

Generate demand for reliable and effective assurance across the AI supply chain

Build a dynamic, competitive AI assurance market, that provides a range of effective services and tools

Develop standards that provide a common language and scalable assessment techniques for AI assurance

Build an accountable AI assurance profession

Set out regulatory requirements that can be assured against

Improve links between industry and independent researchers, so that researchers can help develop assurance techniques and identify AI risks

Next steps

To support development of an effective and mature ecosystem, AI assurance is a core theme of the CDEI's 2021/22 work programme. It will be taking a number of steps over the next year to deliver on the roadmap, along with partners across industry, regulators and government. For example, the CDEI will be working with DCMS and OAI as they work with stakeholders to pilot an AI Standards Hub to expand the UK's contribution to global AI standards, as well as partnering with professional bodies and regulators in the UK to set out assurable standards and requirements for AI systems.

Thank you for your feedback

Report a problem with this page

# data[].attributes.attachment_link.title
The roadmap to an effective AI assurance ecosystem

# data[].attributes.attachment_link.description
Skip to main content We use some essential cookies to make this website work. We'd like to set additional cookies to understand how you use GOV.UK, remember your settings and improve government services. We also use cookies set by other sites to help us deliver content from their services.

# data[].attributes.attachment_link.readable_text
On Thursday, the Internal Market Committee and the Civil Liberties Committee adopted a draft negotiating mandate on the first ever rules for Artificial Intelligence with 84 votes in favour, 7 against and 12 abstentions. In their amendments to the Commission's proposal, MEPs aim to ensure that AI systems are overseen by people, are safe, transparent, traceable, non-discriminatory, and environmentally friendly. They also want to have a uniform definition for AI designed to be technology-neutral, so that it can apply to the AI systems of today and tomorrow.
Risk based approach to AI - Prohibited AI practices
The rules follow a risk-based approach and establish obligations for providers and users depending on the level of risk the AI can generate. AI systems with an unacceptable level of risk to people's safety would be strictly prohibited, including systems that deploy subliminal or purposefully manipulative techniques, exploit people's vulnerabilities or are used for social scoring (classifying people based on their social behaviour, socio-economic status, personal characteristics).

MEPs substantially amended the list to include bans on intrusive and discriminatory uses of AI systems such as:
High-risk AI

"Real-time" remote biometric identification systems in publicly accessible spaces;

"Post" remote biometric identification systems, with the only exception of law enforcement for the prosecution of serious crimes and only after judicial authorization;

Biometric categorisation systems using sensitive characteristics (e.g. gender, race, ethnicity, citizenship status, religion, political orientation);

Predictive policing systems (based on profiling, location or past criminal behaviour);

Emotion recognition systems in law enforcement, border management, workplace, and educational institutions; and

Indiscriminate scraping of biometric data from social media or CCTV footage to create facial recognition databases (violating human rights and right to privacy).

General-purpose AI - transparency measures
MEPs expanded the classification of high-risk areas to include harm to people's health, safety, fundamental rights or the environment. They also added AI systems to influence voters in political campaigns and in recommender systems used by social media platforms (with more than 45 million users under the Digital Services Act) to the high-risk list.

MEPs included obligations for providers of foundation models - a new and fast evolving development in the field of AI - who would have to guarantee robust protection of fundamental rights, health and safety and the environment, democracy and rule of law. They would need to assess and mitigate risks, comply with design, information and environmental requirements and register in the EU database.
Supporting innovation and protecting citizens' rights
Generative foundation models, like GPT, would have to comply with additional transparency requirements, like disclosing that the content was generated by AI, designing the model to prevent it from generating illegal content and publishing summaries of copyrighted data used for training.

To boost AI innovation, MEPs added exemptions to these rules for research activities and AI components provided under open-source licenses. The new law promotes regulatory sandboxes, or controlled environments, established by public authorities to test AI before its deployment.
Quotes
MEPs want to boost citizens' right to file complaints about AI systems and receive explanations of decisions based on high-risk AI systems that significantly impact their rights. MEPs also reformed the role of the EU AI Office, which would be tasked with monitoring how the AI rulebook is implemented.

After the vote, co-rapporteur Brando Benifei (S&D, Italy) said: "We are on the verge of putting in place landmark legislation that must resist the challenge of time. It is crucial to build citizens' trust in the development of AI, to set the European way for dealing with the extraordinary changes that are already happening, as well as to steer the political debate on AI at the global level. We are confident our text balances the protection of fundamental rights with the need to provide legal certainty to businesses and stimulate innovation in Europe".
Next steps
Co-rapporteur Dragos Tudorache (Renew, Romania) said: "Given the profound transformative impact AI will have on our societies and economies, the AI Act is very likely the most important piece of legislation in this mandate. It's the first piece of legislation of this kind worldwide, which means that the EU can lead the way in making AI human-centric, trustworthy and safe. We have worked to support AI innovation in Europe and to give start-ups, SMEs and industry space to grow and innovate, while protecting fundamental rights, strengthening democratic oversight and ensuring a mature system of AI governance and enforcement."

Before negotiations with the Council on the final form of the law can begin, this draft negotiating mandate needs to be endorsed by the whole Parliament, with the vote expected during the 12-15 June session.

# data[].attributes.attachment_link.title
AI Act: a step closer to the first rules on Artificial Intelligence | News | European Parliament

# data[].attributes.attachment_link.description
To ensure a human-centric and ethical development of Artificial Intelligence (AI) in Europe, MEPs endorsed new transparency and risk-management rules for AI systems.

# data[].attributes.attachment_link.readable_text
Today, many priorities for improvements to teaching and learning are unmet. Educators seek technology-enhanced approaches addressing these priorities that would be safe, effective, and scalable. Like all of us, educators use AI-powered services in their everyday lives, such as voice assistants in their homes; tools that can correct grammar, complete sentences, and write essays; and automated trip planning on their phones. As a result, educators see opportunities to use AI-powered capabilities like speech recognition to increase the support available to students with disabilities, multilingual learners, and others who could benefit from greater adaptivity and personalization in digital tools for learning. They are further exploring how AI can enable writing or improving lessons, as well as their process for finding, choosing, and adapting material for use in their lessons.

Educators are also aware of new risks. Useful, powerful functionality can also be accompanied with new data privacy and security risks. Educators recognize that AI can automatically produce output that is inappropriate or wrong. They are wary that the associations or automations created by AI may amplify unwanted biases. They have noted new ways in which students may represent others' work as their own. They are well-aware of "teachable moments" and pedagogical strategies that a human teacher can address but are undetected or misunderstood by AI models. They worry whether recommendations suggested by an algorithm would be fair. Educators' concerns are manifold.

Everyone in education has a responsibility to harness the good to serve educational priorities while also protecting against the dangers that may arise as a result of AI being integrated in educational technology.

NEW!! Report on AI and the Future of Teaching & Learning

Key Insights

AI enables new forms of interaction. Students and teachers can speak, gesture, sketch, and use other natural human modes of communication to interact with a computational resource and each other. AI can generate human-like responses, as well. These new forms of action may provide supports to students with disabilities.

AI can help educators address variability in student learning. With AI, designers can anticipate and address the long tail of variations in how students can successfully learn-whereas traditional curricular resources were designed to teach to the middle or most common learning pathways. For example, AI-enabled educational technology may be deployed to adapt to each student's English language abilities with greater support for the range of skills and needs among English learners.

AI supports powerful forms of adaptivity. Conventional technologies adapt based upon the correctness of student answers. AI enables adapting to a student's learning process as it unfolds step-by-step, not simply providing feedback on right or wrong answers. Specific adaptations may enable students to continue strong progress in a curriculum by working with their strengths and working around obstacles.

AI can enhance feedback loops. AI can increase the quality and quantity of feedback provided to students and teachers, as well as suggesting resources to advance their teaching and learning.

AI can support educators. Educators can be involved in designing AI-enabled tools to make their jobs better and to enable them to better engage and support their students.

Recommendations: Call to Action for Education Leaders

Recommendation #1: Emphasize Humans in the Loop

We reject the notion of AI as replacing teachers. Teachers and other people must be "in the loop" whenever AI is applied in order to notice patterns and automate educational processes. We call upon all constituents to adopt Humans-in-the-Loop as a key criteria.

Recommendation #2: Align AI Models to a Shared Vision for Education

We call upon educational decision makers, researchers, and evaluators to determine the quality of an educational technology based not only on outcomes, but also based on the degree to which the models at the heart of the AI tools and systems align to a shared vision for teaching and learning. The figure to the right describes the important qualities of AI models for education leaders to consider.

Recommendation #3: Design Using Modern Learning Principles

Achieving effective systems requires more than processing "big data"-it requires more than data science. Applications of AI must be based on established, modern learning principles, the wisdom of educational practitioners, and should leverage the expertise in the educational assessment community around detecting bias and improving fairness. Going forward, we also must seek to create AI systems that are culturally responsive and culturally sustaining, leveraging the growth of published techniques for doing so. Further, most early AI systems had few specific supports for students with disabilities and English learners and we must ensure that AI-enabled learning resources are intentionally inclusive of these students.

Recommendation #4: Prioritize Strengthening Trust

Technology can only help us to achieve educational objectives when we trust it. Yet, we learned through a series of public listening sessions that distrust of educational technology and AI is commonplace. Because trust develops as people meet and relate to each other, we call for a focus on building trust and establishing criteria for trustworthiness of emerging educational technologies within the associations, convenings, and professional organizations that bring educators, innovators, researchers, and policymakers together.

Recommendation #5: Inform and Involve Educators

We call on educational leaders to prioritize informing and involving educational constituents so they are prepared to investigate how and when AI fits specific teaching and learning needs, and what risks may rise. Now is the time to show the respect and value we hold for educators by informing and involving them in every step of the process of designing, developing, testing, improving, adopting, and managing AI-enabled educational technology. This includes involving educators in reviewing existing AI-enabled systems, tools, and data use in schools, designing new applications of AI based on teacher input, carrying out pilot evaluations of proposed new instructional tools, collaborating with developers to increase the trustworthiness of the deployed system, and raising issues about risks and unexpected consequences as the system is implemented.

Recommendation #6: Focus R&D on Addressing Context and Enhancing Trust and Safety

Research that focuses on how AI-enabled systems can adapt to context (diversity among learners, variability in instructional approaches, differences in educational settings) is essential to answering the question "Do specific applications of AI work in education, and if so, for whom and under what conditions?" We call upon researchers and their funders to prioritize investigations of how AI can address the long tail of learning variability and to seek advances in how AI can incorporate contextual considerations when detecting patterns and recommending options to students and teachers. Further, researchers should accelerate their attention to how to enhance trust and safety in AI-enabled systems for education.

Recommendation #7: Develop Education-Specific Guidelines and Guardrails

Data privacy regulation already covers educational technology; further, data security is already a priority of school educational technology leaders. Modifications and enhancements to the status quo will be required to address the new capabilities alongside the risks of AI. We call for involvement of all perspectives in the ecosystem to define a set of guidelines (such as voluntary disclosures and technology procurement checklists) and guardrails (such as enhancements to existing regulations or additional requirements) so that we can achieve safe and effective AI for education.

Listening Sessions

The U.S. Department of Education's Office of Educational Technology, with support from Digital Promise, held listening sessions about Artificial Intelligence (AI). We connected with all constituents involved in making decisions about technology in education, including but not limited to teachers, educational leaders, students, parents, technologists, researchers, and policy makers.

The goal of these listening sessions were to gather input, ideas, and engage in conversations that will help the Department shape a vision for AI policy that is inclusive of cutting-edge research and practices while also informed by the opportunities and risks.

Blog Posts

Need More Info?

Contact ed.tech@ed.gov with questions.

Please Include "OET AI" in the Subject Line

For a print-out and summary of the Artificial Intelligence Initiative, please review our AI One-pager

# data[].attributes.attachment_link.title
Artificial Intelligence - Office of Educational Technology

# data[].attributes.attachment_link.description
OET is working to develop policies and supports focused on the effective, safe, and fair use of AI-enabled educational technology.

# data[].attributes.attachment_link.readable_text
Now more than ever, those of us involved in the creation and application of technology need to consider the social, ethical and environmental impacts of our work.

Join a growing movement of tech stewards who believe better is possible.

Tech Stewardship Practice Program

TS + Focus Areas

Drop In Practice Sessions

Want to hear about future offerings?

New programs will be added soon. Sign up for the Tech Stewardship mailing list to be notified of future offerings.

We run weekly 30 minute drop-in practice sessions that are open to anyone.

Whether you've completed the base TSPP program, you're doing it now, or you're an alumni of the program - we hope to see you at a session soon!

If you've already launched your practice, you can deepen your practice with one of the TS + Focus Area programs

If you have not yet completed the base TSPP program but you're interested in a particular focus, you can complete the base TSPP with the focus area content integrated.

4 -6 hrs | Online | Self Paced
12 hrs | Online | Self Paced

Pre registration is currently open for the Fall session. The program will go live on September 1st.