# data[].attributes.attachment_link.title
DoAI Lesson 6 Bias In Artificial Intelligence

# data[].attributes.attachment_link.title
AI Chatbots Tend Toward Flattery. Why That’s Bad for Students

# data[].attributes.attachment_link.description
Flattering technology can make people less willing to admit they are wrong.

# data[].attributes.attachment_link.title
AITA? AI won’t tell you, and it’s affecting behavior and relationships

# data[].attributes.attachment_link.description
A new study of AI sycophancy shows how asking agreeable chatbots for advice can change your behavior

# data[].attributes.attachment_link.readable_text
Yesterday we had the pleasure of working with leaders and teachers from L'Anse Creuse School District outside of Detroit for one of our Train-the-Trainer Institutes. We had a great time digging into all things GenAI!

Our 1-day institute focuses on two key PD sessions: Introduction to Generative AI for Educators and Prompting 101. We work to upskill the new trainers on foundational concepts of GenAI, before equipping them with strategies to turnkey this work in their school.

In our Prompting 101 session we focus on strategies for getting the best out of popular and powerful free GenAI tools like ChatGPT, Claude, and Gemini.

What's great is there are many different prompt frameworks out there for educators to use - including our 5S Framework: Set the scene (priming), be Specific, Simplify language, Structure output, and Share feedback.

We also break down a good prompting in the following four steps:

1. Clarity is Key

Explicitly state what you would like the model to do. The more specific your prompt, the more accurate and tailored the AI's response will be. General prompts will result in general responses

2. Pick the Right Prompting Technique

You may be able to get what you need from one well-structured prompt (one-shot prompting), but there are other techniques too. You can provide examples in your prompt to guide the AI's responses (few-shot prompting), or cut down your requests into steps (chain-of-thought prompting).

3. Provide Context

The chatbot is called a "context window" for a reason! Give AI as much necessary background information as possible. This will help it prepare a response that fits your needs.

4. Format Matters

A well-structured prompt guides the AI in understanding the exact nature of your request. Use clear and direct language, and structure your prompt logically.

So what does that look like in practice for a one-shot prompt? An OK prompt for educators might look like this:

"Create a lesson plan about multiplying fractions for 5th graders"

A better prompt would look like:

"Act as an expert mathematician and a teacher skilled in designing engaging learning experiences for upper elementary students. Design a lesson plan about multiplying fractions for 5th grade students."

And an even more effective prompt would be:

"You are an expert mathematician and teacher skilled in Universal Design for Learning. Design an accessible lesson plan about multiplying fractions for 5th grade students interested in soccer. The lesson should include a hands-on activity and frequent opportunities for collaboration. Format your response in a table."

We take this approach every time we create on of our more than 100 customizable prompts in our Prompt Library.

# data[].attributes.attachment_link.title
Effective Prompting for Educators - AI for Education

# data[].attributes.attachment_link.description
In our Prompting 101 session we focus on strategies for getting the best out of popular and powerful free GenAI tools like ChatGPT, Claude, and Gemini.What's great is there are many different prompt frameworks out there for educators to use - including our 5S Framework: Set the scene (priming), be S

# data[].attributes.attachment_link.readable_text
Deepfakes are videos, photos, or audio recordings that use artificial intelligence to make it look or sound like someone is saying or doing something that is not real.
Deepfake Techniques

Face Swapping: Replacing a person's face with another in a video or photo using AI.

Voice Cloning: Using AI to mimic someone's voice.

Lip Syncing: Making a person's lips appear to match different audio using AI.

The Dangers of Deepfakes
While deepfakes can be used for harmless or even educational purposes, they can also be used unethically in the following ways:

Identity theft: Deepfakes can impersonate individuals in videos or audio recordings, potentially allowing criminals to commit fraud under someone else's name.

Threats/blackmail: Deepfakes can create convincing footage or audio that puts the subject in a bad light, which can then be used to threaten or blackmail them.

Cyberbullying: Deepfakes can be used to create embarrassing or hurtful videos or audio recordings of someone.

Propaganda/fake news: Deepfakes can be used to create fake videos or audio recordings that trick people and sway their opinions on important topics.

Election manipulation: Deepfakes can be used to make candidates look bad or confuse voters by creating fake speeches or interviews.

How to Identify Deepfakes

Detecting deepfakes involves looking for inconsistencies and anomalies in videos. photos, or audio recordings that may not be immediately apparent. It's important to note that deepfake technology is quickly improving, so it may soon be impossible to tell a fake from something real.

Look closely at people's faces in the video. If their skin looks too smooth, their eyes don't blink normally, or their facial expressions seem off, it might be a deepfake.

If the voice doesn't sound quite right, like it's too robotic or the tone doesn't match how the person usually speaks, it could be a sign of a deepfake.

Pay attention to how the lips move. If they're not syncing up with the words perfectly, or the movement looks strange, the video might be manipulated.

Consider the video's overall appearance. Sometimes, the area around the person or behind them might look blurry or glitchy. This can be a sign that the video has been altered.

Or if the shadows on the person's face or in the background don't match up with the lighting in the rest of the video, it could be a clue that it's a deepfake.

A variety of Large tech companies such as Google, Meta, and OpenAI are joining together to form the Coalition for Content Provenance and Authenticity (C2PA) in order to develop methods of labeling AI-generated content to clearly identify when and how it was generated or altered. 1. In January 2024, thousands of AI-generated robocalls impersonating President Joe Biden went out to Democratic voters discouraging them from voting in the New Hampshire primary. How might this type of deepfake use impact the integrity of elections? How can we combat this? This question encourages a discussion on the potential manipulation of voter perceptions and behavior through false representations of political candidates. 2. One 2023 study found that 98% of the deepfake videos online consisted of nonconsensual pornography. The targets of 99% of this content were women, including pop star Taylor Swift. Why do you think this type of content is so widespread, and what are the social and ethical implications? This question prompts a critical discussion about the pervasive use of deepfakes to create nonconsensual pornography, focusing on the disproportionate impact on women. 3. In summer of 2023, a group of New Jersey teenagers used AI to create deepfake pornographic images of an underaged female classmate, which were circulated over group text. What psychological harms could this kind of incident cause for the victim in terms of trust, safety, and mental health? Should legal action be taken against the students involved? This question raises serious issues about the exploitation of minors using deepfake technology, as well as implications for online privacy and consent. It further prompts students to consider appropriate legal and disciplinary responses. 4. In April 2024, a high school principal in Maryland fell victim to a deepfake attack when a disgruntled athletic director faked a recording of the principal making racist remarks. What are the consequences of using deepfakes to impersonate authority figures? Consider implications for the victim, the attacker, and the system as a whole. This question encourages exploration of the ethical, social, and legal repercussions of such digital impersonations at the system level, and how they can undermine trust within communities. 5. What do you think has been the impact of deepfakes online? How has it changed how you interact with the internet? This question invites students to analyze and discuss the various effects that deepfakes have had on the online environment. Considerations include influencing public perception, personal privacy, and trust in digital media. 6. Do you think deepfakes should be banned? Why or why not? This question prompts students to consider the pros and cons of the availability of deepfake technology, weighing the implications for freedom of expression, technological innovation, and personal security. deepfake detectors are available online, but as these operate based on a model that calculates probabilities, which 8. With advancements in AI technology, deepfakes are likely to become even more sophisticated. How should society prepare for these changes? Consider the roles of individuals, educators, policymakers, and technology companies in shaping a response. This question facilitates a discussion on the necessary preparations for future advancements in AI and deepfake technology. will never be 100% reliable.
What Can You Do?

Question everything: AI-generated content has already flooded the internet, so be cautious. Always investigate if a suspicious image, video, or audio clip is real.

Discussion Questions

Use the following scenario-based discussion questions to build background knowledge on the impact of deepfakes on society to inform students' responses to the more general questions on the following page. Links to the news sources for each scenario can be found in the Sources and Further Reading section of this document.

7. Can you think of any positive use cases for deepfake content? Under what circumstances can deepfakes be created and shared ethically?

T his question encourages students to identify and discuss potential positive applications of deepfake technology. Discuss scenarios such as education, entertainment, and philanthropy, and consider the benefits, risks, and circumstances that would justify deepfake use.

Examples to Use in the Classroom

The following links are just a few of the thousands of examples of deepfakes that exist.

# data[].attributes.attachment_link.title
Uncovering Deepfakes: Classroom Guide - AI for Education

# data[].attributes.attachment_link.description
Since 2019, the presence of Deepfakes - hyperrealistic AI-generated video, photo, and audio forgeries - has surged by 550%. This blurring of reality requires a new approach to vetting if online content is real or fake.

# data[].attributes.attachment_link.title
AI’s Impact on the Environment — AI for Education

# data[].attributes.attachment_link.description
We have a variety of prompts to help you lesson plan and do administrative tasks with GenAI chatbots like ChatGPT, Claude, Bard, and Perplexity.

# data[].attributes.attachment_link.readable_text
Not an educator? Learn about Google AI Essentials - applicable across roles and industries.

As a teacher, we know your time is valuable and student needs are broad. With Generative AI for Educators, you'll learn how to use generative AI tools to help you save time on everyday tasks, personalize instruction, enhance lessons and activities in creative ways, and more.

100%

online, self-paced learning

2 hours

to complete the course

Learn more about the course

Generative AI for Educators

Developed by AI experts at Google in collaboration with MIT RAISE, this course will help you bring AI into your practice. You'll also gain a foundational understanding of AI - you'll learn what it is, the opportunities and limitations of this technology, and how to use it responsibly.

You'll learn how to use generative AI tools to:

Save time on everyday tasks like drafting emails and other correspondence

Personalize instruction for different learning styles and abilities

Enhance lessons and activities in creative ways

You'll learn how to use generative AI tools to:

Save time on everyday tasks like drafting emails and other correspondence

Personalize instruction for different learning styles and abilities

Enhance lessons and activities in creative ways

Save time and enhance student learning with AI skills

When you complete the online course, you'll gain essential AI skills you can apply to your workflow immediately. By using AI as a helpful collaboration tool, you can work smarter, not harder.

Earn a certificate which you can present to your district for professional development (PD) credit, depending on district and state requirements.

Get hands-on experience using generative AI tools and apply your new skills right away.

Google and MIT's Generative AI for Educators course marks a transformative moment for Oklahoma's education. Having navigated the educational system as a dyslexic student and parent, I recognize the critical need for personalized learning. This initiative equips educators to unlock each student's potential, tailoring learning experiences to their unique minds. It's a step towards a future where education mirrors the individuality of a fingerprint, empowering students to shape tomorrow with their ingenuity.

Nellie Tayloe Sanders

Oklahoma Secretary of Education

We are excited to be a part of this partnership with Google to equip our middle and high school teachers with transformative Generative AI professional learning courses. This responds to the need for continuous innovation in our school District. By empowering educators with the latest advancements in AI, we're not only uplifting their professional practices, but also giving them evolving tools to enhance the ever-changing landscape of teaching and learning.

Jose L. Dotres, Superintendent of Schools

Miami-Dade County Public Schools

The Google Generative AI for Educators opportunity for teachers will help put our educators on the front lines of generative AI opportunities for our 27,000 students. In a rapidly evolving world, our teachers cannot afford to fall behind in accessing powerful generative tools that will help them develop new approaches to teaching and learning.

Michael Matsuda, Superintendent of Schools

Anaheim Union High School District

Frequently asked questions

Generative AI for Educators was designed by AI experts at Google in collaboration with MIT RAISE (Responsible AI for Social Empowerment and Education).

Check out Experience AI. Created by Google DeepMind and The Raspberry Pi Foundation, this no cost program provides ready-to-use resources to introduce AI technology in middle school classrooms, including lesson plans, slide decks, worksheets, and videos on AI and machine learning.

# data[].attributes.attachment_link.keywords[]
using artificial intelligence in education

# data[].attributes.attachment_link.title
Generative AI for Educators - Grow with Google

# data[].attributes.attachment_link.description
Learn AI skills with Google Generative AI for Educators. Save time, personalize instruction, and enhance lessons, no previous experience required.

# data[].attributes.attachment_link.readable_text
Not an educator? Learn about Google AI Essentials - applicable across roles and industries.

As a teacher, we know your time is valuable and student needs are broad. With Generative AI for Educators, you'll learn how to use generative AI tools to help you save time on everyday tasks, personalize instruction, enhance lessons and activities in creative ways, and more.

100%

online, self-paced learning

2 hours

to complete the course

Learn more about the course

Generative AI for Educators

Developed by AI experts at Google in collaboration with MIT RAISE, this course will help you bring AI into your practice. You'll also gain a foundational understanding of AI - you'll learn what it is, the opportunities and limitations of this technology, and how to use it responsibly.

You'll learn how to use generative AI tools to:

Save time on everyday tasks like drafting emails and other correspondence

Personalize instruction for different learning styles and abilities

Enhance lessons and activities in creative ways

You'll learn how to use generative AI tools to:

Save time on everyday tasks like drafting emails and other correspondence

Personalize instruction for different learning styles and abilities

Enhance lessons and activities in creative ways

Save time and enhance student learning with AI skills

When you complete the online course, you'll gain essential AI skills you can apply to your workflow immediately. By using AI as a helpful collaboration tool, you can work smarter, not harder.

Earn a certificate which you can present to your district for professional development (PD) credit, depending on district and state requirements.

Get hands-on experience using generative AI tools and apply your new skills right away.

Google and MIT's Generative AI for Educators course marks a transformative moment for Oklahoma's education. Having navigated the educational system as a dyslexic student and parent, I recognize the critical need for personalized learning. This initiative equips educators to unlock each student's potential, tailoring learning experiences to their unique minds. It's a step towards a future where education mirrors the individuality of a fingerprint, empowering students to shape tomorrow with their ingenuity.

Nellie Tayloe Sanders

Oklahoma Secretary of Education

We are excited to be a part of this partnership with Google to equip our middle and high school teachers with transformative Generative AI professional learning courses. This responds to the need for continuous innovation in our school District. By empowering educators with the latest advancements in AI, we're not only uplifting their professional practices, but also giving them evolving tools to enhance the ever-changing landscape of teaching and learning.

Jose L. Dotres, Superintendent of Schools

Miami-Dade County Public Schools

The Google Generative AI for Educators opportunity for teachers will help put our educators on the front lines of generative AI opportunities for our 27,000 students. In a rapidly evolving world, our teachers cannot afford to fall behind in accessing powerful generative tools that will help them develop new approaches to teaching and learning.

Michael Matsuda, Superintendent of Schools

Anaheim Union High School District

Frequently asked questions

Generative AI for Educators was designed by AI experts at Google in collaboration with MIT RAISE (Responsible AI for Social Empowerment and Education).

Check out Experience AI. Created by Google DeepMind and The Raspberry Pi Foundation, this no cost program provides ready-to-use resources to introduce AI technology in middle school classrooms, including lesson plans, slide decks, worksheets, and videos on AI and machine learning.

# data[].attributes.attachment_link.keywords[]
using artificial intelligence in education

# data[].attributes.attachment_link.title
Generative AI for Educators - Grow with Google

# data[].attributes.attachment_link.description
Learn AI skills with Google Generative AI for Educators. Save time, personalize instruction, and enhance lessons, no previous experience required.

# data[].attributes.attachment_link.description
Please make sure your browser supports JavaScript and cookies and that you are not blocking them from loading. For more information you can review our Terms of Service and Cookie Policy.

# data[].attributes.attachment_link.description
Please make sure your browser supports JavaScript and cookies and that you are not blocking them from loading. For more information you can review our Terms of Service and Cookie Policy.

# data[].attributes.attachment_link.description
We use cookies on this site to enhance your user experience. For more information on how we use cookies, read our privacy notice.

# data[].attributes.attachment_link.title
AI Literacy Framework for Primary & Secondary Education

# data[].attributes.attachment_link.description
Empower learners for the age of AI with the AILit Framework—a joint EC & OECD initiative, supported by Code.org and international experts.

# data[].attributes.attachment_link.description
Train a computer to recognize your own images, sounds, & poses. A fast, easy way to create machine learning models for your sites, apps, and more - no expertise or coding required.

# data[].attributes.attachment_link.title
3 types of bias in AI | Machine learning

# data[].attributes.attachment_link.title
3 types of bias in AI | Machine learning

# data[].attributes.attachment_link.title
Twin Pics - Daily AI Image Generation Game

# data[].attributes.attachment_link.description
Use AI to generate a matching pic. Get closer to score higher.

# data[].attributes.attachment_link.description
Fall is going to be very different this year. Educators need to be ready.

# data[].attributes.attachment_link.title
Talking to Teachers About AI in Schools - Leon Furze.pdf

# data[].attributes.attachment_link.title
Talking to Teachers About AI in Schools - Leon Furze.pdf

# data[].attributes.attachment_link.readable_text
As a former secondary English teacher, senior examination assessor, and lecturer for initial teacher education, I understand the allure of using Generative AI (GenAI) for grading student work. We're all familiar with the workload of assessment and reporting. The idea of a tool that could save time and streamline the grading process is undeniably appealing. It's no surprise, then, that the market is flooded with AI-powered grading solutions, all promising to make our lives easier.

However, as I've explored in previous posts on the capabilities and limitations of GenAI, I firmly believe that this technology is fundamentally unsuited for high-stakes student assessment. At its core, GenAI generates probabilistic outputs based on patterns in training data, lacking true understanding and the ability to make qualitative judgments. This leads to inconsistency and bias in grading, raising serious concerns about fairness and reliability.

The use of AI in grading also raises a host of ethical and equity issues. As I wrote in " Generative AI doesn't 'democratize creativity' ", the notion that AI levels the playing field is often an illusion. In reality, relying on AI for grading may exacerbate existing inequities and privilege certain groups of students over others.

In this post, I'll go deeper into the reasons why I believe GenAI should not be used for grading, drawing on recent experiments and real-world examples. I'll also explore the potential risks and unintended consequences of AI-powered assessment. By the end, I hope to convince you that, despite the temptation, GenAI is a dead-end when it comes to evaluating student work.

The fundamental limitations of LLMs

At the heart of the problem with using GenAI for grading is the fundamental way these systems work. GenAI produces outputs based on probabilistic patterns in its training data, without any real understanding, reasoning, or ability to make qualitative judgments. Essentially, it's making what appear to be educated guesses, but are in effect just statistical patterns.

This means that the grades GenAI assigns can vary significantly based on seemingly minor differences in prompt language or details in the student work. In a quick experiment, I fed the same Year 9 persuasive writing piece into ChatGPT multiple times, changing only the student name. The grades ranged from 78 to 95 out of 100 - a massive discrepancy based on a single variable.

Such inconsistency completely undermines the fairness and reliability of the grading process. It creates a false sense of objectivity and accuracy, when in reality, the grades are no more meaningful than a roll of the dice.

But it gets worse. GenAI models are trained on vast datasets scraped from the internet, which can encode all sorts of societal biases and discrimination. The models can make inferences about student attributes like race, gender, or background based on their writing, potentially disadvantaging certain groups.

And here's the rub: this kind of bias is a lot harder to detect and address compared to human grader bias. We've got strategies like anonymisation and moderation to mitigate human bias, imperfect as they are. But with AI, simply removing the student's name doesn't cut it. The bias is baked into the model at a much deeper level, based on the training data and the patterns it's picked up.

So not only are the grades inconsistent and unreliable, they're also likely to be biased in ways we can't easily control for. It's a recipe for disaster when it comes to fair, equitable assessment.

Improving the prompt only improves the appearance of accuracy

One of the biggest criticisms in the (lengthy) comment thread alongside the original post was that my prompt was overly simplistic. Why didn't I use a more elaborate prompt, guiding the LLMs response with criteria, an assignment sheet, or maybe even exemplar marked student work?

It's a fair question. And yes, more detailed prompts can help anchor the AI's responses and make them somewhat more consistent. But let's be clear - it's a superficial fix. The AI still doesn't have any deep understanding of the work it's grading. It's just getting better at pattern-matching and churning out responses that fit the rubric.

Others argued that human graders are biased and inconsistent too. And that's of course true to an extent. But here's the thing: we know about human bias, and we've got strategies to mitigate it, like blind marking and moderation. We invest in professional development for markers. When I assessed the senior English certificate examination (VCE English), we marked blind, double marked, and had a qualified team of expert assessors with years of training to moderate results. And crucially, human graders have the capacity for contextual understanding and empathy that AI just doesn't.

The argument that "AI is biased but so are humans," is a false equivalence: apples and oranges. While it's true that both AI and humans can exhibit biases, the nature and impact of those biases are not comparable. Human biases stem from individual experiences, cultural backgrounds, and societal influences, whereas AI biases are typically the result of biases present in the training data or introduced by the designers. Additionally, human educators can actively work to recognise and mitigate their biases, while AI systems lack this self-awareness and agency.

Bias is an incredibly complex problem in AI systems, and there might be even more subtle issues than we're aware of. It seems as though the more sophisticated models become, the better they are at inferring details about authors based on even minor details in texts. For example, the output of an AI response to an assessment item can be influenced dramatically by changing only one word, as in these experiments by Melissa Warr and Punya Mishra where they changed one word ("classical" for "rap" music) and it impacted the outcome.

These findings are supported in the article Dialect prejudice predicts AI decisions about people's character, employability, and criminality by Valentin Hofmann, Pratyusha Ria Kalluri, Dan Jurafsky, Sharese King who demonstrate that "dialect prejudice has the potential for harmful consequences by asking language models to make hypothetical decisions about people, based only on how they speak." This could, of course, impact students through their writing if AI is used to grade written assessments.

I often hear the argument that AI is improving at a breakneck pace, and these issues will be ironed out "any day now". It's the "this is the worst AI we'll ever use" argument favoured by people like Professor Ethan Mollick. But while the tech is certainly developing fast, the fundamental problems aren't going away.

Bigger language models and more training data alone won't magically give AI genuine understanding or eliminate bias. We can't take the hype and marketing at face value. As educators, we need to critically evaluate the evidence and implications, not just jump on the bandwagon.

The other equity issues of GenAI grading

One of my biggest concerns about using AI for grading is the potential for it to worsen existing equity gaps in education: access to AI tools is far from equal.

Imagine a student from a low-income family, living in a remote area with patchy internet. They're relying on free, basic AI tools because that's all they can access. Contrast that with a student from a well-off background, who can afford subscriptions to advanced models like GPT-4 and has top-notch devices at their disposal.

From the outset, the field is far from level. The student with access to better AI tools can generate higher quality work, iterate and refine it, and potentially game the system when it comes to things like detection tools. Meanwhile, the disadvantaged student is stuck with clunky, less sophisticated outputs.

Now, layer AI grading on top of that. If the grading AI is swayed by the polish and complexity of the work - which, as we've seen, it often is - then the student with better AI tools has an unfair advantage. They're likely to score higher grades, through no fault or merit of their own.

Over time, this can exacerbate achievement gaps and reinforce privilege. It's the classic "rich get richer" scenario, but with algorithms.

Flip it around, and consider what happens if schools, universities, and educators have access to differing levels of GenAI. What does it look like for a student whose work is assessed with GPT-3.5 compared to GPT-4? Efforts to "debias" AI have thus far only proven successful to an extent, and only for the more powerful models. For example, though it is far from perfect, GPT-4 exhibits less bias than GPT-3.5. So an educator or institution with the financial and technical resources to use a more powerful model will provide more sophisticated and potentially less biased feedback.

This is exacerbated by the fact that many educators and schools simply have not had the time to adjust to these technologies, and are being preyed upon by enterprising companies who aggressively push AI solutions. Many of these "AI for teachers" platforms build on top of GPT-3.5 or GPT-3.5 Turbo, as it would be more expensive to use GPT-4.

And that's only if the problem is resolved. A few months ago, Ryan Tannenbaum demonstrated the problem with GPT-3.5, 3.5 Turbo, and Claude 2. It's the same experiment that I replicated with GPT-4o, and will no doubt be replicable into the near future.

Who owns the contents of a student's brain?

I have huge issues with the cavalier approach to student intellectual property, extending back to before Generative AI was even in the mix. Plagiarism detection software, for example, is a multi-billion dollar industry built on top of student creativity and knowledge. I have never seen an education provider where students are permitted to "opt-in" to these platforms, and yet millions of pieces of their work are uploaded into the companies' databases daily. This has allowed these companies to grow and scale, with the volume of work compounding the usefulness and marketability of the software.

This problem is now exacerbated by AI. While plagiarism tools are no longer effective, it isn't stopping people from uploading student's IP into technology platforms. I've written about this before, but a lot of these AI tools are already built on pretty shaky ground when it comes to data provenance and consent. As educators, we have a duty of care to make sure student data isn't being misused or exploited.

So any use of AI in assessment needs to come with clear policies on data handling. No sharing student work with third-party AI providers without explicit, informed consent.

More appropriate uses of GenAI in assessment and feedback

While the use of GenAI for high-stakes, summative assessment is fraught with risks and limitations, as outlined in the previous sections, there are nevertheless several promising applications of this technology in education that warrant careful consideration and exploration. I also want to delineate between learning, assessment, feedback, and grading: these terms aren't interchangeable.

One area where GenAI can offer significant value is in providing low-stakes, formative feedback to students. By leveraging the natural language processing capabilities of models like GPT-4, educators can offer immediate, targeted feedback on specific aspects of student work, such as grammar, spelling, and punctuation. GenAI can also highlight areas for improvement in the structure, clarity, or coherence of a piece of writing, and offer prompts to encourage deeper reflection or analysis.

Crucially, however, such AI-generated feedback should be treated as a supplement to, rather than a replacement for, human feedback. As noted in my earlier post, Critic, Creator, Consumer, a balanced and thoughtful approach to GenAI involves recognising both its potential and its limitations. Educators should carefully review and contextualise the feedback provided by AI systems, and create opportunities for students to discuss, unpack, and apply that feedback in dialogue with their instructor.

Used judiciously, AI-generated feedback could encourage students to take greater ownership over their learning, engaging in self-assessment and revision of their work. It's also a workload-win. Teachers no longer have to take home piles of books and make grammatical or functional corrections that, in all likelihood, students will never read anyway.

GenAI has the potential to support adaptive learning and personalised support for students, though this is far from proven. By analysing patterns in student performance data, AI systems can identify specific skills or concepts that a student is struggling with, and recommend targeted resources or activities to address those individual learning needs: this is the basic premise of adaptive learning technologies including some of the features in Khan Academy's chatbot. This kind of "old school" AI based analytics could be coupled with GenAI to generate customised practice questions or prompts, adjusting the difficulty or complexity based on a student's demonstrated skill level. I'm yet to see this have a positive impact in practice, however.

As with all applications of GenAI in education, these adaptive learning systems must be designed and implemented with great care and attention to potential biases, limitations, and unintended consequences. They should be used to support and enhance, rather than replace, the role of human educators in diagnosing learning needs, building relationships, and guiding student growth.

Conclusions, but no clear answers

Fundamentally, the notion of using a language model (LLM) to grade student work is problematic, regardless of how sophisticated the prompt or input may be. While an LLM might generate feedback that appears thoughtful and thorough, it is essential to recognize that the model is producing a probabilistic output based on patterns in its training data, without the capacity for genuine reasoning or understanding. This holds true even if the input includes a detailed rubric, specific grading criteria, or sample student work with assessed comments. These additional contextual elements might help to anchor the LLM's response and make it seem more convincing, but ultimately, the output remains a product of statistical inference rather than true comprehension.

Arguing that "it's better if you use a better prompt" is a flawed premise because it fails to address the core limitations of LLMs in the context of evaluating student work. It's like suggesting that a spell checker can be used to assess the aesthetic qualities of a poem if given enough guidelines - while the spell checker may identify words that are commonly associated with poetic language, it lacks the appreciation for imagery, emotion, and figurative expression that a human reader would bring to the analysis.

In the same manner, an LLM may generate feedback that seems appropriate on the surface, but it lacks the deeper insights, critical thinking, and subjective judgment that a human educator brings to the grading process. We may be flawed, but at least we have the capacity to think, reason, and evaluate.

The use of AI in high-stakes assessment also raises concerns about fairness, accountability, and transparency: all key aspects of the Australian Framework for Generative AI in Schools along with other international education guidelines. While human graders may exhibit biases, they can be trained to recognise and mitigate them, whereas an LLM's biases are inherent to its training data and architecture, and may be more difficult to identify and address. The appearance of objectivity in AI-generated grades may mask underlying disparities and hinder efforts to ensure equitable evaluation. I would argue that using AI to generate results, numerical grades, or final, summative evaluations is actually contravenes the Framework

The use of LLMs for grading student work is a misapplication of the technology that fails to appreciate the fundamental differences between human and artificial intelligence. Rather than seeking to automate the assessment process, we should focus on leveraging AI to support and enhance human educators' abilities, while preserving the essential role of human judgment and expertise in evaluating student learning.

We need to begin by questioning why we grade work at all. Is assessment and grading simply a matter of competition, ranking, and placement? Do students actually need a letter or numerical grade at all? Or, as TEQSA asked recently in reference to the use of AI tools in assessment, are there other ways that students can demonstrate learning, and that educators can assess whether students have learned?

If nothing else, Generative AI is forcing us to have these difficult, sometimes uncomfortable conversations. Thanks to everyone who has joined in the discussion so far.

I regularly work with schools, universities, and faculty teams on developing guidelines and approaches for Generative AI. If you're interested in talking about consulting and PD, get in touch via the form below:

# data[].attributes.attachment_link.description
As a former secondary English teacher, senior examination assessor, and lecturer for initial teacher education, I understand the allure of using Generative AI (GenAI) for grading student work. We're all familiar with the workload of assessment and reporting. The idea of a tool that could save time and streamline the grading process is undeniably appealing....

# data[].attributes.attachment_link.readable_text
Bias is a natural inclination for or against an idea, object, group, or individual. It is often learned and is highly dependent on variables such as a person's socioeconomic status, race, ethnicity, education, gender, and religion. Several studies have been published about bias in generative artificial intelligence, including this one HERE, HERE, and my field report from 2022 titled " Interrogating AI Bias through Digital Art." "Bias in Generative AI" features a study that analyzed images generated by three popular AI tools - Midjourney, Stable Diffusion, and DALLE 2 - to investigate potential bias in AI generators.

Generative AI tools, like Midjourney can inadvertently perpetuate and intensify societal biases related to gender, race, and emotional portrayals in their images. Dr. Joy Buolamwini's "Unmasking AI" features a story of how the author uncovered "the coded gaze" - the evidence of encoded discrimination and exclusion in tech products. I decided to test this out using the very first computer-generated image I created in 1987. Back then, I used Deluxe Paint or D-Paint on an Amiga computer. For my experiment I uploaded my image to Midjourney and used the "Describe" command to create text prompts that depict the old CG image in new, different ways.

One thing the "Describe" command missed was the race of the child in my original image. I added one word to the prompt: Black. In all four of the initial thumbnails (and my original image) there are trees and other plants. But in the next round of thumbnails (with "Black" in the prompt) there is much less foliage. Also, the picket fence was replaced with cement walls... sometimes with markings or graffiti on them.

While generative AI has numerous benefits in creative fields, we must also remain aware and vigilant for potential drawbacks. The "Bias in Generative AI" study revealed systematic gender and racial biases in all three AI generators against women and African Americans. The study also uncovered more nuanced biases or prejudices in the portrayal of emotions and appearances.

It is important not only to consider who is shown in GenAI images, but in which light they are shown. Most of the images with the Black child (prompt) show rundown or poor/neglected surroundings but that wasn't in the original image from 1987 and it didn't show up in the initial Midjourney thumbnails. More important, I took note of how I felt when this was happening. Next, I imagined how someone who was less astute would feel. Would they feel less inclined to continue using the tool(s)?

In the initial image there is a gap in the white picket fence on the left side. The AI-generated version with the Black child shows a closed fence and the child seems to be looking for a way inside. His access is blocked. This last image visualizes the problem of inclusion and diversity in digital art. In a world where technology (esp. AI) continues to shape how we interact and learn, we must find ways to get more diverse people on the development side of the technology, to help inform how large language/training models are created and come up with better ways to counter bias in AI.

As our reliance on AI intensifies, it becomes imperative to ensure fairness and equity in the design and deployment of these technologies. We must prioritize the development of generative AI systems that are not only technologically advanced but are also shaped by an ethical commitment to inclusivity and equity. - Zhou et al. 2024

Also, we must also represent diverse cultures, perspectives, and experiences in digital art collections (ex. online, galleries, museums).

# data[].attributes.attachment_link.title
The Boy on the Tricycle: Bias in Generative AI

# data[].attributes.attachment_link.description
Bias is a natural inclination for or against an idea, object, group, or individual. It is often learned and is highly dependent on variables such as a person's socioeconomic status, race, ethnicity...

# data[].attributes.attachment_link.description
Given the right framing, AI tools can empower student agency and curiosity.

# data[].attributes.attachment_link.description
Given the right framing, AI tools can empower student agency and curiosity.

# data[].attributes.attachment_link.description
Please make sure your browser supports JavaScript and cookies and that you are not blocking them from loading. For more information you can review our Terms of Service and Cookie Policy.

# data[].attributes.attachment_link.readable_text
What is artificial intelligence (AI)? How does it work, and is it different than generative AI? We'll break it down while you consider the benefits and drawbacks of this evolving technology.

# data[].attributes.attachment_link.description
What Is AI? is a video from Common Sense Education. Use this video to help your students learn and thrive in our connected world.

# data[].attributes.attachment_link.title
Artificial Intelligence: Is It Plagiarism? - Lesson Slides

# data[].attributes.attachment_link.title
Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task – MIT Media Lab

# data[].attributes.attachment_link.description
 This study explores the neural and behavioral consequences of LLM-assisted essay writing. Participants were divided into three groups: LLM, Search Engine…

# data[].attributes.attachment_link.title
Deepfakes and Democracy | Common Sense Education

# data[].attributes.attachment_link.description
Common Sense is the nation's leading nonprofit organization dedicated to improving the lives of all kids and families by providing the trustworthy information, education, and independent voice they need to thrive in the 21st century.

# data[].attributes.attachment_link.description
What are the impacts of creating or sharing altered images without asking for consent? Check out Deepfakes & Consent, a free digital literacy and citizenship lesson plan by Common Sense Media. Teach students in 8 about responsible and healthy media and tech use.

# data[].attributes.attachment_link.title
Truth, Tricks, and AI: Learning to Verify Information

# data[].attributes.attachment_link.description
Students learn how to recognize misinformation in AI content and use fact-checking strategies (lateral reading) to verify what’s true.

# data[].attributes.attachment_link.description
Students comprehend the environmental impact of artificial intelligence (AI), including both its positive and negative effects.

# data[].attributes.attachment_link.title
The Unequal Burden of Data Centers - Kapor Foundation

# data[].attributes.attachment_link.description
An examination of the Environmental and Public Health Impacts on Communities in California

# data[].attributes.attachment_link.description
This lesson introduces students to the concepts of cognitive effort and cognitive debt and explores how AI tools can change the amount of thinking we do!

# data[].attributes.attachment_link.description
Two Truths and AI is a new series designed to help students sharpen their critical thinking skills around artificial intelligence. Each episode makes AI lite…

# data[].attributes.attachment_link.title
How do I fact-check AI search results? | Stanford CRAFT

# data[].attributes.attachment_link.description
Studies consistently show that students are using AI chatbots and search engines like ChatGPT and Perplexity. But we know that AI makes mistakes. So, how do we teach students not to get fooled? In this lesson, we’ll introduce students to the skill of lateral reading (Wineburg & McGrew, 2017) as one approach to fact-checking the information they get from AI.

# data[].attributes.attachment_link.title
Tips to becoming a world-class Prompt Engineer

# data[].attributes.attachment_link.title
Deepfakes | What is a Deepfake for Kids? | What is a deepfake and how do they work? | AI for Kids

# data[].attributes.attachment_link.title
[What is Vibe Coding_] _ Webinar, Slides 04_2026 .pdf