# data[].attributes.headline
Chat GPT in Education shared Padlet with opinions and resources

# data[].attributes.subject
Chat GPT in Education shared Padlet with opinions and resources

# data[].attributes.body
https://padlet.com/p_c_anthony/chatgpt-in-education-jm9scrmkfn9xzz35

# data[].attributes.body
https://docs.google.com/document/d/1V1drRG1XlWTBrEwgGqd-cCySUB12JrcoamB5i16-Ezw/edit#heading=h.y7vlxxluoxbv

# data[].attributes.attachment_link.title
AI Text Generators: Sources to Stimulate Discussion among Teachers

# data[].attributes.headline
Dr Sarah Eaton Guidance on incorporating AI ethically

# data[].attributes.subject
Dr Sarah Eaton Guidance on incorporating AI ethically

# data[].attributes.body
https://taylorinstitute.ucalgary.ca/teaching-with-AI-apps

# data[].attributes.attachment_link.provider_name
Taylor Institute for Teaching and Learning

# data[].attributes.attachment_link.readable_text
Artificial intelligence apps, such as ChatGPT, can be part of our educational toolbox just as dictionaries, calculators, and web searches are. If we think of artificial intelligence apps as another tool that students can use to ethically demonstrate their knowledge and learning, then we can emphasize learning as a process not a product.

Six thoughts on artificial intelligence and academic integrity (Eaton, 2022):

1. Using artificial intelligence for school work does not automatically equate to misconduct.

2. Artificial intelligence can be used ethically for teaching, learning, and assessment.

3. Trying to ban the use of artificial intelligence in school is not only futile, it is irresponsible.

4. Human imagination and creativity are not threatened by artificial intelligence.

5. Assessments must fit for purpose and should align with learning outcomes.

6. Artificial intelligence is not going anywhere. We must learn to work with new technology, not against it.

The following are considerations when thinking about artificial intelligent apps and your courses:
Engage:

Have open and honest conversations with your students about your expectations regarding artificial intelligence apps and their use in your courses. Start with a discussion on artificial intelligence literary and what that means in your course.

Some guiding questions for conversations with your students may include:

1. What do you know about artificial intelligent apps?

2. Have you used them before? If you have, in what ways?

3. How can you ethically use artificial intelligence apps to support your learning?

1. Co-creating class guidelines with your students regarding the use of artificial intelligence apps in your courses.

2. Encouraging students to acknowledge when they use artificial intelligence apps and acknowledge their efforts in doing so as there are no current citation standards.

3. Including an artificial intelligence discussion thread to create a space to continue the conversation with your students.

Explain:

Think about your current assessments through the lens of artificial intelligence. Consider how the course learning outcomes align with your assessments and the potential impact of artificial intelligence apps.

Guiding questions:

1. Are there small adjustments that can be made to your course design to be proactive with artificial intelligence apps?

2. How can artificial adjustments be included in conversations with your students or in discussions in small groups?

3. What type of assessments align best with your course learning outcomes? Could alternative assessments be an option?

1. Modeling how to use artificial intelligence apps ethically by creating an exemplar developed by an app for your students to critique.

2. Creating writing or discussion prompts that ask learners to apply a reading or video to their own lives or personal contexts that reflect current events.

3. Replacing short answers, essays, and reflections with alternative assessments such as mind maps, oral presentations, videos, or podcasts.

Explore:

Use these resources to inform and support your instructional design and artificial intelligence.

1. McMurtrie, B. (2023, January 5). Teaching: Will ChatGPT change the way you teach? Chronicle for Higher Education. https://www.chronicle.com/newsletter/teaching/2023-01-05?cid=gen_sign_in

2. Pavlik, J.V. (2023). Collaborating With ChatGPT: Considering the Implications of Generative Artificial Intelligence for Journalism and Media Education. Journalism & Mass Communication Educator, 1-10. https://doi.org/10.1177/10776958221149577

3. Susnjak, T. (2022). ChatGPT: The End of Online Exam Integrity? Cornell University. https://doi.org/10.48550/arxiv.2212.09292

Reference:
Eaton, S. (2022, December 9). Sarah's thoughts: Artificial intelligence and academic integrity. Learning, Teaching, and Leadership. https://drsaraheaton.wordpress.com/?s=ChatGPT

# data[].attributes.attachment_link.title
Teaching and Learning with Artificial Intelligence Apps

# data[].attributes.attachment_link.description
Artificial intelligence apps, such as ChatGPT, can be part of our educational toolbox just as dictionaries, calculators, and web searches are. If we think of artificial intelligence apps as another tool that students can use to ethically demonstrate their knowledge and learning, then we can emphasize learning as a process not a product.

# data[].attributes.headline
Sarah Eaton - Artificial Intelligence and Academic Integrity

# data[].attributes.subject
Sarah Eaton - Artificial Intelligence and Academic Integrity

# data[].attributes.body
https://bournemouth.cloud.panopto.eu/Panopto/Pages/Viewer.aspx?id=f4f33183-6242-4b5a-b865-af9600fac51e

# data[].attributes.attachment_link.title
Speaker Series: Artificial Intelligence and Academic Integrity - Insights for Teaching and Learning

# data[].attributes.attachment_link.description
Zoom Meeting ID: 85157672367 * Host: Dave Hunt (FLIE - HSS LT) * Meeting Start: 01/25/2023 @ 3:22 PM * Recording Start: 01/25/2023 @ 4:00 PM * Duration: 57 mins * Panelists: Tracey Webb, Oliver Moore-Oats, Robert (Bob) W Brennan, Dr Brenda McDermott, Steph Allen, Sarah Elaine Eaton, Dr Jason Wiens, Dave Hunt (FLIE - HSS LT)

# data[].attributes.headline
What to do about AI text generators? Next steps for educators - Anna Mills, College of Marin

# data[].attributes.subject
What to do about AI text generators? Next steps for educators - Anna Mills, College of Marin

# data[].attributes.body
https://www.youtube.com/watch?v=vl-6dzillr4

# data[].attributes.attachment_link.title
What to do about AI text generators? Next steps for educators - Anna Mills, College of Marin

# data[].attributes.body
https://www.turnitin.com/solutions/ai-writing

# data[].attributes.attachment_link.readable_text
Turnitin launches AI detection to help educators identify when AI writing tools such as ChatGPT have been used in students' submissions.

Academic integrity in the age of AI writing

Over the years, academic integrity has been both supported and tested by technology. Today, educators are facing a new frontier with AI writing and ChatGPT.

Here at Turnitin, we believe that AI can be a positive force that, when used responsibly, has the potential to support and enhance the learning process. We also believe that equitable access to AI tools is vital, which is why we're working with students and educators to develop technology that can support and enhance the learning process. However, it is important to acknowledge new challenges alongside the opportunities.
We recognize that for educators, there is a pressing and immediate need to know when and where AI and AI writing tools have been used by students. This is why we are now offering AI detection capabilities for educators in our products. Learn More About Our AI Detector

AI writing indicator keeps educators informed

Gain insights on how much of a student's submission is authentic, human writing versus AI-generated from ChatGPT or other tools.

AI writing reporting with powerful results

Robust reporting identifies AI-written text and provides information educators need to determine their next course of action. We've designed our solution with educators, for educators.

Check student essays as you do today

AI writing detection complements Turnitin's similarity checking workflow and is integrated with your LMS, providing a seamless, familiar experience.

Turnitin's AI writing detection capability available with Originality, helps educators identify AI-generated content in student work while safeguarding the interests of students.

Learn more about Originality

Turnitin AI Innovation Lab

Welcome to the Turnitin AI Innovation Lab, a hub for new and upcoming product developments in the area of AI writing. You can follow our progress on detection initiatives for AI writing, ChatGPT, and AI-paraphrasing.

Understanding the false positive rate for sentences of our AI writing detection capability

We'd like to share more insight on our sentence level false positive rate and tips on how to use our AI writing detection metrics.

Read more

Understanding false positives within our AI writing detection capabilities

We'd like to share some insight on how our AI detection model deals with false positives and what constitutes a false positive.

Read more

Trouble viewing? View the video on YouTube or adjust your cookie preferences.

Helping solve the AI writing puzzle one piece at a time

AI-generated writing has transformed every aspect of our lives, including the classroom. However, identifying AI writing in students' submissions is just one piece in the broader, complex, ever-evolving AI writing puzzle.

Read more

Teaching in the age of AI writing

As AI text generators like ChatGPT quickly evolve, our educator resources will, too. Curated and created by our team of veteran educators, our resources help educators meet these new challenges. They are built for professional learning and outline steps educators can take immediately to guide students in maintaining academic integrity when faced with AI writing tools.

The Turnitin Educator Network is a space to meet, discuss and share best practices on academic integrity in the age of AI.

Join the conversation here

Learn more about AI writing in our blog

Written by experts in the field, educators, and Turnitin professionals, our blog offers resources and thought leadership in support of students, instructors, and administrators. Dive into articles on a variety of important topics, including academic integrity, assessment, and instruction in a world with artificial intelligence.

Stay up to date with the latest blog posts delivered directly to your inbox.

Subscribe here

Turnitin AI tools in the news

Never miss an update or announcement. Visit our media center for recent news coverage and press releases.

Cheat GPT? Turnitin CEO Chris Caren weighs in on combating A.I. plagiarism | CNBC Squawk Box

Since the inception of AI-generated writing, educators and institutions are learning how to navigate it in the classroom. Turnitin's CEO Chris Caren joins 'Squawk Box' to discuss how it is being used in the classroom and how educators can identify AI writing in student submissions.

Trouble viewing? View the video on YouTube or adjust your cookie preferences.

Schools Ban ChatGPT amid Fears of Artificial-Intelligence-Assisted Cheating

U.S. educators are debating the merits and risks of a new, free artificial intelligence tool called ChatGPT, which students are using to write passable high school essays. So far, there isn't a reliable way to catch cheating. Matt Dibble has the story.

Trouble viewing? View the video on YouTube or adjust your cookie preferences.

Some U.S. schools banning AI technology while others embrace it | NBC Nightly News

ChatGPT, an artificial intelligence program, can write college-level essays in seconds. While some school districts are banning it due to cheating concerns, NBC News' Jacob Ward has details on why some teachers are embracing the technology.

Trouble viewing? View the video on YouTube or adjust your cookie preferences.

Testing Turnitin's New AI Detector: How Accurate Is It?

BestColleges

Artificial intelligence, it seems, is taking over the world. At least that's what alarmists would have you believe. The line between fact and fiction continues to blur, and recognizing what is real versus what some bot concocted grows increasingly difficult with each passing week.
Read more

The Sky Is Not Falling: Turnitin Urges Educators to 'Meet the Moment' of ChatGPT

THE Journal

In this episode of THE Journal Insider podcast, host and THEJournal.com editor Kristal Kuykendall welcomes two former teachers who have been working on AI writing tools at Turnitin, a plagiarism-detection software used by thousands of K-12 schools and institutions of higher education.
Read and listen

How ChatGPT will reshape the future of the high school essay

Fast Company

ChatGPT, an AI-powered "large language" model, is poised to change the way high school English teachers do their jobs. With the ability to understand and respond to natural language, ChatGPT is a valuable tool for educators looking to provide personalized instruction and feedback to their students.
Read more.

For press and media inquiries, contact us at press@turnitin.com

Contact us

Meet us in person to discuss AI writing detection

We are attending larger and smaller events to ensure we'll have the opportunity to discuss academic integrity in the age of AI writing with you in person. Here's a list of events that Turnitin has already participated in, or will be participating in this year. Looking forward to seeing you there!

MBO Digitaal Conferentie, September 21-22, Ulft, Netherlands
World Academic Summit, 26-28 September 2023, Sydney, Australia
Campus Innovation, September 27-29, Hamburg, Germany
tawiab 2023: Tagung wissenschaftliche Abschlussarbeiten und Hochschulschriften-Repositorien, September 28, Vienna, Austria
ACO-TEC 2023, September 28-29, Vienna, Austria
Digital Universities: MENA, 9-11 October, Riyadh, Saudi Arabia
EDUCAUSE Annual Conference, 9-12 October 2023, Chicago, USA

EDUtech Europe, October 10-12, Amsterdam, Netherlands
Digital Universities: Europe, 23-25 October 2023, Barcelona, Spain
THE Campus Live US, 8-9 November 2023, Los Angeles, USA
SURF Onderwijsdagen, November 14-15, Den Bosch, Netherlands
NCTE, 16-19 November 2023, Columbus, OH, USA
THE Campus Live UK, 6-7 December 2023, UK
Online Educa Berlin 2023 (OEB), November 22-24, Berlin, Germany

Stay up to date with the latest blog posts delivered directly to your inbox.

Subscribe here

# data[].attributes.attachment_link.title
Turnitin AI Innovation Lab | New AI tools from Turnitin

# data[].attributes.attachment_link.description
Turnitin is your resource for the latest AI tools to help teachers and students fairly and effectively utilize artificial intelligence in education.

# data[].attributes.headline
Should academia adapt or resist? -raises some points for discussion/questions about appropriate uses and how these might differ between staff/students. Good suggestions for use in teaching, but is it acceptable for staff to use this to create lessons/give feedback to students (dependent on if they cite it and what their policy is regarding student use in terms of fairness)?

# data[].attributes.subject
Should academia adapt or resist? -raises some points for discussion/questions about appropriate uses and how these might differ between staff/students. Good suggestions for use in teaching, but is it acceptable for staff to use this to create lessons/give feedback to students (dependent on if they cite it and what their policy is regarding student use in terms of fairness)?

# data[].attributes.body
https://hbsp.harvard.edu/inspiring-minds/chatgpt-and-ai-text-generators-should-academia-adapt-or-resist?cid=email%7Cmarketo%7C2023-02-09-inspiring-minds-digest-new%7C1482431%7Cinspiring-minds%7Ceducator%7Cinspiring-minds-article%7Cfeb2023&acctID=none&mkt_tok=ODU1LUFUWi0yOTQAAAGJ1Qi1MODbUcxresk1dJ4kvbh6YDe888tg0l4pDBCAbwDl956bPsp0T0UaLjgf6OrfA6b4fqD4fwzAD9U02nNIl6wrUNI3w4w7AuM-QCdjRw

# data[].attributes.attachment_link.readable_text
Those in the education sector have long expressed divergent opinions about technology innovations and their use in classrooms. The debate may be around something as simple as allowing students to use their laptops or smartphones during class, or it may center around an emerging artificial intelligence (AI) technology like ARISTO, a system that can pass (with a score of 90 percent) an eighth-grade-level science exam.

The COVID-19 pandemic forced even the most reluctant academic institutions to embrace the use of technology; most gradually understood that their mission was to educate new generations on the responsible use of technology rather than excluding it completely from the classroom.

Now, academia faces another technological evolution in the form of artificial intelligence text generators (AITGs). At the beginning of this year, we heard about an AITG called ChatGPT that can generate text by asking it questions (like a chatbot). But users quickly discovered that this AI is not just a chatbot; it can generate articles, summaries, essays, or code in seconds. The education-related use cases abound, especially for students, who can use ChatGPT to do their school assignments and examinations.

If used unethically, this technology could diminish their learning process and objectives, some have argued. While talking about its implications in education, academia took distance and started asking, Do we adapt or resist?

Since the pandemic, we have contributed to the technology debate by suggesting the adoption of a constructive and positive approach to pedagogy, in which it is better to concentrate on what innovations could bring to education rather than resisting change.

Here, we'll offer our own recommendations for approaching AITGs. But first, let's look more generally at some of the advantages and concerns surrounding this AI technology.

Opportunities ChatGPT May Bring to Education

The potential of ChatGPT presents intriguing opportunities for education. As most of us would agree, academia must continue to shift from unidirectional knowledge transfer to an active development of competencies, experience, social abilities, and technological agility. Access to information has democratized knowledge acquisition, and ChatGPT may synthesize the use of various sources, albeit not without a potential bias.

On January 21, the Financial Times published an article commenting on a paper published by a Wharton Business School professor that describes how the author, Christian Terwiesch, tested ChatGPT3 on an exam in a core course of his MBA program. The bot ChatGPT earned a solid grade (between B and B-) and outperformed most of the actual students in the course.

Obviously, these kinds of situations represent a challenge for universities and business schools; but instead of taking ChatGPT as a threat, educators and their institutions should nevertheless analyze its advantages and use them in favor of the student to achieve their educational objectives.

Here are five ways AITGs can be used in a participant-centered learning process.

1. Text generators like ChatGPT can boost our collective familiarity with AI and how to use it, a critical competency for our students and their futures. Industries move quickly, and new technologies push them to continue innovating. We must be the first facilitators of new technologies as they emerge and teach our students how to use them appropriately (technically and ethically). If not us, who?

2. AITGs can assist educators in preparing and reviewing sessions by providing them with additional resources or helping them create engaging educational content that can lead to a better learning experience for students. Educators and students already use search engines, citation and research management apps, spellcheckers, and data collection tools; the addition of tech like ChatGPT has the potential to develop written materials previously validated including scripts, examples, tests questions, or even cases analysis to be discussed in class.

3. AITGs can save educators time by automatically grading students' assignments or doing educators' repetitive work-for example, preparing announcements and instructions for assignments or exams; or providing feedback to students when making "recurrent" or common mistakes in solving their exercises; or preparing basic but customized guidelines for activities, such as how to structure a research thesis or how to solve an exercise. AITGs can also be effectively used to give students automatic feedback on their essays and texts.

4. AITGs can be used for training purposes. For example, students can use ChatGPT to emulate conversations and develop their language skills and abilities through conversational interactions with the chatbot.

5. ChatGPT could be used to improve engagement in online learning by increasing students' motivation in asynchronous sessions or activities. Students will find using an innovative tool to be exciting while discovering its potential use in day-to-day tasks, for example to write emails, send text messages, or even prepare a draft of a contract. It can also send automatic but customized feedback and instructions in online courses to help students stay on track (further increasing their engagement in the course).

Early Concerns About Using ChatGPT in Education

Educators understandably have concerns about the risks of this new technology on teaching and evaluating students. Some example concerns include the following:

Plagiarism or fraudulent use of AITGs in written content reflected in papers, essays, tests, and quizzes; identifying and handling unethical behavior in student content; and promoting ethical research with the use of reliable and valid sources of information.

Bias, limitations, and inaccuracy of AITGs in developing responses on certain topics, which could negatively affect the learning process and objectives for students and perpetuate stereotypes and other misinformation.

Students' overreliance on AITGs, causing them to miss out on important learning opportunities such as critical thinking, problem-solving, proper research techniques, and interaction with educators and peers.

Unfair or inaccurate evaluation of students' work using AITGs due to its inability to estimate creativity and originality, among other soft skills.

Should We Adapt or Resist? Recommendations for Academia

In the face of these abundant opportunities and risks, education institutions can respond to this new technology by either resisting or adapting. The resistors will treat AI as an enemy and ban it-as has happened with some educators prohibiting the use of laptops, mobile phones, and other technologies in the classroom.

But taking an adaptive response provides an opportunity to raise awareness among faculty about the importance of knowing and using technology in education. Further, we can step in to more adequately train our future generations of students to use AI and be better prepared for the challenges they will face in their careers.

"Instead of taking ChatGPT as a threat, educators and their institutions should analyze its advantages and use them in favor of the student to achieve their educational objectives."

Here are several recommendations for educators and academic institutions to adapt to this technology rather than resist it.

Help students develop their critical thinking skills by asking them to compare AI-produced content with reliable, valid sources of information.

Invest energy, resources, and creativity into providing an attractive and engaging learning environment for students-either in physical spaces or virtual settings.

Introduce more oral assignments and in-person exams into your curricula. Many Australian universities, for example, have already announced a return to in-person exams to protect the integrity of student assessments.

Increase the importance of in-class activities and participation as a significant element of the evaluation process and a relevant component of the final grade. In other words, shift the object of evaluation from the "product" to the "process" of student learning.

Adopt new or more ways to get students to develop and showcase their knowledge or learning: For example, through debates-in which students take a formalized position on a specific topic and have to defend their idea, perspective, or point of view. Or by introducing Art-Thinking methodology, which highlights the ways artists create to help students develop specific skills they need to enter the workforce and face ever-growing uncertainty.

Emphasize ethical use of AI and the importance of proper research and citation practices. For example, some universities include (or are reintroducing) mandatory courses in academic ethics, research integrity, and critical thinking for all students. Developing in future generations a sensibility on these topics is crucial for all levels of education.

At the institution level, train educators in the use and identification of AI-produced content despite the challenge it represents as an emergent technology.

While we are advocating for a more adaptive approach with these recommendations, some educators are choosing to take a more resistant approach. Examples of these more resistive reactions to ChatGPT include the following:

Trying to render AITG useless by asking more complex questions in examinations and assignments.

Asking for anecdotal or personal information that AI technology cannot provide-at the expense of learning from public knowledge and existing data on a topic.

Using software that blocks students' access to the rest of their system when working on examinations or assignments.

Fighting fire with fire: There is another AI in development called GPTZero that will specialize in finding AI-generated texts.

ChatGPT Can Complement Learning, Not Replace It

ChatGPT has academia buzzing, and many institutions are feeling compelled to "do something" about it quickly, perhaps even implementing policy changes or abrupt bans that may prove disruptive when a term is already in progress. But we're advocating a more adaptive and inclusive approach.

It is important for educators to consider how to address the use of AITGs in the classroom, ensuring that these tools are being used as a complement to, rather than a supplement for, traditional teaching methods. As facilitators of tools that can help students optimize their learning processes, we need to use and understand these new technologies to ensure students have a better learning experience.

It is still too early to know if this AITG is the next tipping point that will disrupt the human-digital interaction, like Google did some years ago or the introduction of the Gutenberg printing press did centuries ago. However, we do know the adaptation process is already unfolding in a few weeks rather than taking decades, while questions around this type of AI remain-like how it may make human life better, as time invested in producing writing content can potentially be significantly reduced.

Educators will need to think creatively about how to apply changes to their content, as well as the skills we give (and we want to give) our students. We can and should embrace AITGs like ChatGPT as a partner that helps us learn more, work smarter, and faster.

# data[].attributes.attachment_link.title
ChatGPT and AI Text Generators: Should Academia Adapt or Resist?

# data[].attributes.attachment_link.description
Rather than resist change, academia must adapt to AI technology in order to embrace the innovations it can bring to education. Here we explore the advantages and concerns surrounding artificial intelligence text generators like ChatGPT.

# data[].attributes.headline
TEQSA/CRADLE Deakin - ChatGPT, what do we need to know now?

# data[].attributes.subject
TEQSA/CRADLE Deakin - ChatGPT, what do we need to know now?

# data[].attributes.body
https://www.youtube.com/watch?v=mCCqf6tHI24

# data[].attributes.headline
ChatGPT & Academic Integrity - demo/video guide for students by Guy Curtis

# data[].attributes.subject
ChatGPT & Academic Integrity - demo/video guide for students by Guy Curtis

# data[].attributes.body
https://www.youtube.com/watch?v=zh8DHLReVa0

# data[].attributes.headline
CRITICAL AI: ADAPTING COLLEGE WRITING FOR THE AGE OF LARGE LANGUAGE MODELS SUCH AS CHATGPT: SOME NEXT STEPS FOR EDUCATORS (recommended practices/updates to practice)

# data[].attributes.subject
CRITICAL AI: ADAPTING COLLEGE WRITING FOR THE AGE OF LARGE LANGUAGE MODELS SUCH AS CHATGPT: SOME NEXT STEPS FOR EDUCATORS (recommended practices/updates to practice)

# data[].attributes.body
https://criticalai.org/2023/01/17/critical-ai-adapting-college-writing-for-the-age-of-large-language-models-such-as-chatgpt-some-next-steps-for-educators/

# data[].attributes.attachment_link.readable_text
Adapting College Writing for the Age of Large Language Models such as ChatGPT: Some Next Steps for Educators

By Anna Mills and Lauren M. E. Goodlad (licensed CC BY NC 4.0)

Large language models (LLMs) such as ChatGPT are sophisticated statistical models that predict probable word sequences in response to a prompt even though they do not "understand" language in any human-like sense. Through intensive mining, modeling, and memorization of vast stores of language data "scraped" from the internet, these text generators deliver a few paragraphs at a time which resemble writing authored by humans. This synthetic text is not directly "plagiarized" from some original, and it is usually grammatically and syntactically well-crafted.

From an academic integrity perspective, this means that "AI"-generated writing

1) is not easily identifiable as such to the unpracticed eye;

2) does not conform to "plagiarism" as that term is typically understood by teachers and students; and

3) encourages students to think of writing as task-specific labor disconnected from learning and the application of critical thinking.

Many teachers who assign writing are, thus, understandably concerned that students will use ChatGPT or other text generators to skip the learning and thinking around which their writing assignments are designed.

In the future, the producers of language models may offer tools for identifying texts generated by their systems. Such tools may arise independently, like a recent app that claims to identify ChatGPT's outputs. It is also possible that government regulators and other policy-making bodies will become involved in overseeing the use of LLMs in educational settings. New York City schools have already banned ChatGPT, and many experts argue that the abuse of LLMs could extend far beyond the impact on students.

As a teacher and textbook author, one of us (Anna Mills) has been collecting multiple perspectives on the topic for the Writing Across the Curriculum Clearinghouse. Another of us (Lauren Goodlad) is the chair of the Critical AI @ Rutgers initiative and the editor of Critical AI. Though both of us feel strongly that unsupervised use of LLMs for student assignments is detrimental to learning, we believe that, in the short run, a combination of the practices described below will effectively discourage students from submitting machine-generated writing as their own. At the very least, any student determined to use text generation will encounter significant obstacles.

In the long run, we believe, teachers need to help students develop a critical awareness of generative machine models: how they work; why their content is often biased, false, or simplistic; and what their social, intellectual, and environmental implications might be. But that kind of preparation takes time, not least because journalism on this topic is often clickbait-driven, and "AI" discourse tends to be jargony, hype-laden, and conflated with science fiction. (We offer a few solid links below.) In the meantime, the following practices should help to protect academic integrity and student learning. At least some of these practices might also enrich your teaching.

Common practices that can be updated in the current context

Encourage intrinsic motivation. Most educators already strive to make their assignments engaging, but it's worth emphasizing that students who feel connected to their writing will be less interested in outsourcing their work to an automated process.

Highlight how the writing process helps students learn. Make explicit that the goal of writing is neither a product nor a grade but, rather, a process that empowers critical thinking. Writing, reading, and research are entwined activities that help people to communicate more clearly, develop original thinking, evaluate claims, and form judgments.

Update academic integrity policies to make them explicit about the use of automated writing tools. Academic integrity policies and honor codes should specify what, if any, use of automated writing assistance is appropriate (teachers may wish to consult departmental or institutional policies).

Ask students to affirm that their submissions are their own work and not that of another person or of any automated system. This practice has long been used to deter plagiarism and can be adapted to include text generation. One of us asks students to add the following statement along with their initials when they turn in written work: "I certify that this assignment represents my own work. I have not used any unauthorized or unacknowledged assistance or sources in completing it including free or commercial systems or services offered on the internet."

Practices we recommend

Let students know that detectors for identifying "AI"-generated text exist and are improving. While these tools should not be relied upon as a "silver bullet," students should know that detection is possible. However, teachers who use these tools as a check on potential policy violations should bear in mind that the detectors may produce false positives and false negatives. Students should know that this technology is rapidly evolving: future detectors may be able to retroactively identify auto-generated prose from the past. No one should present auto-generated writing as their own on the expectation that this deception is undiscoverable.

Assign prompts that state-of-the-art systems such as ChatGPT are not good at. The below tasks are either impossible for ChatGPT to perform reliably, or require the student to supervise and edit in ways that entail significant expertise, rhetorical skills, and time. As such, students who are simply eager to earn a good grade with minimal effort will likely find such assignments difficult to automate. These requirements may also make assignments more robust in other ways.

Requirement for verifiable sources and quotations. ChatGPT currently fabricates sources and quotations (though it may occasionally hit on a real author or title). GPT-3 is even more prone to this "hallucination." Students using either model would need to find and input sources and quotations themselves.

Analysis of specifics from images, audio, or videos. Students would need to describe these kinds of media in detail in order to generate automated outputs about them.

Analysis of longer texts (too large for the limited windows for prompting automated systems). Although tools exist for summarizing longer texts, using them adds another obstacle to easy automation. Such programs may introduce errors that make automated text easier to detect.

Analysis that draws on class discussion. Assigning this criterion requires the student to input notes from class discussion, involving time and effort.

Analysis of recent events not in the training data for the system. Students would need to do their own research and then feed that information into the automated system.

Assignments that articulate nuanced relationships between ideas. Such assignments could entail comparing two passages that students themselves choose from two assigned texts. Students might be asked to explain a) why they chose these particular passages; b) how the chosen passages illuminate the whole of the texts from which they were excerpted; and then c) how the two passages compare according to instructions that bear on the course themes or content. LLMs usually cannot do a good job of explaining how a particular passage from a longer text illuminates the whole of that longer text. Moreover, ChatGPT's outputs on comparison and contrast are often superficial. Typically the system breaks down a task of logical comparison into bite-size pieces, conveys shallow information about each of those pieces, and then formulaically "compares" and "contrasts" in a noticeably superficial or repetitive way.

Assign in-class writing as a supplement to or launching point for take-home assignments. Students may be more likely to complete an assignment without automated assistance if they've gotten started through in-class writing. (Note: In-class writing, whether digital or handwritten, may have downsides for students with anxiety and disabilities).

Additional practices you might wish to undertake

Assign steps in the writing process and/or reflection on that process. Many instructors already include these practices in their approach to teaching writing. Note that ChatGPT can produce outputs that take the form of "brainstorms," outlines, and drafts. It can also provide commentary in the style of peer review or self-analysis. Nonetheless, students would need to coordinate multiple submissions of automated work in order to complete this type of assignment with a text generator.

Hold individual conferences on student writing or ask students to submit audio/video reflections on their writing. As we talk with students about their writing, or listen to them talk about it, we get a better sense of their thinking. By encouraging student engagement and building relationships, these activities could discourage reliance on automated tools.

If you are curious about the technology, test your own writing assignments. Once you sign up for an account, it is straightforward to test an assignment using ChatGPT by feeding in the instructions and other required information (such as a short text). Users of these models can learn to improve their outputs by prompting the model to add or revise. (Anna has compiled a set of examples).

Practices you might wish to undertake once you learn more about language models

Teach students about text generators and "AI." Students are more likely to misuse text generators if they trust them too much. The term "Artificial Intelligence" ("AI") has become a marketing tool for hyping products. For all their impressiveness, these systems are not intelligent in the conventional sense of that term. They are elaborate statistical models that rely on mass troves of data-which has often been scraped indiscriminately from the web and used without knowledge or consent. No matter how seemingly magical, text generators do not understand language (or anything else) the way that humans do. For the same reason, LLMs often mimic the harmful prejudices, misconceptions, and biases found in data scraped from the internet. (See the below resources for some preliminary introductions to the subject.)

Show students examples of inaccuracy, bias, logical, and stylistic problems in automated outputs. We can build students' cognitive abilities by modeling and encouraging this kind of critique. Given that social media and the internet are full of bogus accounts using synthetic text, alerting students to the intrinsic problems of such writing could be beneficial. (See the "ChatGPT/LLM Errors Tracker," maintained by Gary Marcus and Ernest Davis.) Note that teaching students about LLMs in this way is a different practice from using ChatGPT as a means of teaching students how to write (a practice that some are recommending but which we regard as having limited value). Since ChatGPT is good at grammar and syntax but suffers from formulaic, derivative, or inaccurate content, it seems like a poor foundation for building students' skills and may circumvent their independent thinking. The tool seems more beneficial for those who already have a lot of experience writing-not those learning how to develop ideas, organize thinking, support propositions with evidence, conduct independent research, and so on.

Join in discussions with fellow teachers, policymakers, and industry to explore regulations and tools that will help educators support student learning in the age of "AI"-text generators.

Practices we do not recommend

Requiring handwritten submissions. Writing by hand is difficult for many students, especially students with certain disabilities and students who use voice-to-text software. To be sure, some studies show that students remember things better when they take notes with pen and paper rather than laptops or other devices. Writing by hand might be offered as an option for some assignments.

Adopting surveillance tools. Some are advocating software that records the students' entire writing process-systems that have been shown to be biased and inaccurate. Such for-profit surveillance tools are highly intrusive and (like language models themselves) potentially exploitative. They may also be ineffective and inequitable.

Adopting systems trained to recognize specific student writing. In theory it is possible to determine the authorship of a text by training a system to recognize an individual student's patterns of word choice and syntax. Like the surveillance systems above, such technology is prone to unreliability, exploitation, and abuse.

Further resources on text generators
Note: Good journalism on language models is surprisingly hard to find since the technology is so new and the hype is ubiquitous. Here are a few reliable short pieces.

"ChatGPT Advice Academics Can Use Now" edited by Susan Dagostino, Inside Higher Ed, January 12, 2023

"University students recruit AI to write essays for them. Now what?" by Katyanna Quach, The Register, December 27, 2022

"How to spot AI-generated text"by Melissa Heikkilä, MIT Technology Review, December 19, 2022

The Road to AI We Can Trust, Substack by Gary Marcus, a cognitive scientist and AI researcher who writes frequently and lucidly about the topic. See also Gary Marcus and Ernest Davis, "GPT-3, Bloviator: OpenAI's Language Generator Has No Idea What It's Talking About" (2020).

The below academic article is now a classic:

A sample of resources on "AI" more generally (Updated 1/19)

Emily M. Bender, "On NYT Magazine on AI" (distinguished University of Washington linguist responds blow-by-blow to a misleading New York Times feature ).

Ruha Benjamin, "Assessing Risk, Automating Racism" Science (2019).

Frederico Bianchi et al. "Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale." Arxiv 2022.

Meredith Broussard, Artificial Unintelligence: How Computers Misunderstand the WorldMIT Press (2018),

Timnit Gebru, "Is Ethical AI Possible?" (podcast with co-author of "Stochastic Parrots" and director of the DAIR Institute).

Andrew Hundt et al. "Robots Enact Malignant Stereotypes." 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT 22). 743-756.

Frank Pasquale, New Laws of Robotics: Defending Human Expertise in the Age of AI.Harvard UP, 2020.

Billy Perrigo. "OpenAI Used Kenyan Workers on Less than $2 Per Hour to Make ChatGPT Less Toxic." Time January 18, 2023.

Meredith Whittaker, "The Steep Cost of Capture" Interactions 28.6 (December 2021): 50-55.

To share your ideas or offer advice please feel free to comment below (the comments are moderated) or write to one or both authors.

Anna Mills: armills@marin.edu

Lauren Goodlad: lauren.goodlad@rutgers.edu

In addition, the Modern Language Association and the College Conference on Composition and Communication have convened a task force on this topic (Anna is a member). Anyone who would like to contact the taskforce can write to Paula Krebs (MLA director) or Holly Hassel (CCCC director).

# data[].attributes.attachment_link.title
CRITICAL AI: Adapting College Writing for the Age of Large Language Models such as ChatGPT: Some Next Steps for Educators

# data[].attributes.attachment_link.description
Angel McNall photography Adapting College Writing for the Age of Large Language Models such as ChatGPT: Some Next Steps for Educators By Anna Mills and Lauren M. E. Goodlad (licensed CC BY NC 4.0) Large language models (LLMs) such as ChatGPT are sophisticated statistical models that predict probable word sequences in response to a prompt...

# data[].attributes.headline
OpenAI (creator of ChatGPT) have released an AI classifier to distinguish between AI and human-written text. Please note that classifications from this are not always accurate.

# data[].attributes.subject
OpenAI (creator of ChatGPT) have released an AI classifier to distinguish between AI and human-written text. Please note that classifications from this are not always accurate.

# data[].attributes.body
https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text/

# data[].attributes.attachment_link.readable_text
We're launching a classifier trained to distinguish between AI-written and human-written text.

Contributors

Michael Lampe, Joanne Jang, Pamela Mishkin, Andrew Mayne, Henrique Ponde de Oliveira Pinto, Valerie Balcom, Michelle Pokrass, Jeff Belgum, Madelaine Boyd, Heather Schmidt, Sherwin Wu, Logan Kilpatrick, Thomas Degry

# data[].attributes.attachment_link.title
New AI classifier for indicating AI-written text

# data[].attributes.attachment_link.description
We're launching a classifier trained to distinguish between AI-written and human-written text.

# data[].attributes.headline
Inside the post-ChatGPT scramble to create AI essay detectors

# data[].attributes.subject
Inside the post-ChatGPT scramble to create AI essay detectors

# data[].attributes.body
https://www.timeshighereducation.com/depth/inside-post-chatgpt-scramble-create-ai-essay-detectors?utm_source=newsletter&utm_medium=email&utm_campaign=editorial-daily&spMailingID=24150000&spUserID=MTAyMjQ5OTg5NTM5NAS2&spJobID=2170486892&spReportId=MjE3MDQ4Njg5MgS2

# data[].attributes.attachment_link.title
Inside the post-ChatGPT scramble to create AI essay detectors

# data[].attributes.attachment_link.description
Edtech giants and plucky start-ups are vying to create potentially lucrative tools to combat the use of AI in assessments, but will they cause more problems than they solve?

# data[].attributes.body
https://edition.cnn.com/2023/01/26/tech/chatgpt-passes-exams/index.html

# data[].attributes.attachment_link.title
ChatGPT passes exams from law and business schools | CNN Business

# data[].attributes.attachment_link.description
ChatGPT is smart enough to pass prestigious graduate-level exams -- though not with particularly high marks.

# data[].attributes.headline
What AI-generated lesson plans might mean for educators

# data[].attributes.subject
What AI-generated lesson plans might mean for educators

# data[].attributes.body
https://www.timeshighereducation.com/campus/what-aigenerated-lesson-plans-might-mean-educators?utm_source=newsletter&utm_medium=email&utm_campaign=campus-weekly&spMailingID=23873758&spUserID=MTAyMjQ5Njk3NjUyMQS2&spJobID=2153985085&spReportId=MjE1Mzk4NTA4NQS2

# data[].attributes.attachment_link.readable_text
Artificial intelligence (AI) is already changing education in Australia , whether we're ready for it or not. That was the takeaway from George Siemens' keynote address at the Australian Association for Research in Education conference in November. He went on to suggest that unless researchers and teacher educators are mindful, they may not have a seat at the table in deciding what the future of education will look like.

The prompt was: a lesson plan for English for teaching 16-year-old students about the idea of "juxtaposition" within Romeo and Juliet.

Romeo and Juliet, Shakespeare uses juxtaposition extensively to contrast the two lovers and their families, the Montagues and the Capulets. He also uses juxtaposition to contrast the tragic events of the play with the light-hearted moments.

3. Tell the students that they are going to be looking for examples of juxtaposition in Romeo and Juliet . Explain that they should pay attention to how the contrast created by the juxtaposition contributes to the mood and atmosphere of the play.

Have the students find additional examples of juxtaposition in other works of literature, or in everyday life. They can share their examples with the class and explain how the contrast created by the juxtaposition contributes to the overall effect .

Increasingly, design has become a part of education: design for learning, learning by design, thinking and so on. The epistemic fluency to design using computational tools in a way that enriches material life and human culture is at the root of all three areas.
This is an edited version of the post " The AI future for lesson plans is already here", which was published on the Australian Association for Research in Education blog, EduResearch Matters, on 29 November 2022.

# data[].attributes.attachment_link.title
What AI-generated lesson plans might mean for educators

# data[].attributes.attachment_link.description
Educators can use the issues, challenges and opportunities brought by artificial intelligence to think about teaching in terms of design and process rather than outcomes

# data[].attributes.headline
Alarmed by A.I. Chatbots, Universities Start Revamping How They Teach

# data[].attributes.subject
Alarmed by A.I. Chatbots, Universities Start Revamping How They Teach

# data[].attributes.body
https://www.nytimes.com/2023/01/16/technology/chatgpt-artificial-intelligence-universities.html?unlocked_article_code=KZlEMjqurKR12q1aI9prJZey7xVR1b3eqzzRxaYROoWPn57vsTnSIwm8iUAIG1S-RBqYiX2Wzs-JIVaTXmuEknFgM-yEcbuBq5Pu3gsFs6EW6hSOABh2cJLGs6Ph4syxgdmc1P-q6de4_cNHSbxQ2qHNU-5lKs30veZGUKu8Bcu1ToLkdRCu6rKSp66cNA8UL7l7bdU0HAuz-DCCojDsZy_imI91qMeu9Z6K31MgifluSadVrZJJUDL8lUgzhhzkH6667380RMQPrnskjdXd2xQdk7nN-k1InCTus-rxGY70JumYU5_LYYqCpzHbLhqjtsDpiJW2GoQYK8wdKk-l3ezEdYaquGQ_dClIVJDurJO4GgygRoZrXMeP&smid=share-url

# data[].attributes.headline
ChatGPT listed as author on research papers: many scientists disapprove

# data[].attributes.subject
ChatGPT listed as author on research papers: many scientists disapprove

# data[].attributes.body
https://www.nature.com/articles/d41586-023-00107-z?utm_source=Nature+Briefing&utm_campaign=2fd6fa2583-briefing-dy-20230120&utm_medium=email&utm_term=0_c9dfd39373-2fd6fa2583-47000119

# data[].attributes.attachment_link.readable_text
At least four articles credit the AI tool as a co-author, as publishers scramble to regulate its use.

The artificial-intelligence (AI) chatbot ChatGPT that has taken the world by storm has made its formal debut in the scientific literature - racking up at least four authorship credits on published papers and preprints.
doi: https://doi.org/10.1038/d41586-023-00107-z

# data[].attributes.attachment_link.title
ChatGPT listed as author on research papers: many scientists disapprove

# data[].attributes.attachment_link.description
At least four articles credit the AI tool as a co-author, as publishers scramble to regulate its use. At least four articles credit the AI tool as a co-author, as publishers scramble to regulate its use.

# data[].attributes.headline
Suggestions for how academics can use ChatGPT smartly (use it to create structure rather than content)

# data[].attributes.subject
Suggestions for how academics can use ChatGPT smartly (use it to create structure rather than content)

# data[].attributes.body
https://twitter.com/MushtaqBilalPhD/status/1621379333943083009

# data[].attributes.wish_content.attachment_props.text
ChatGPT is everywhere and everyone is using it.

But most academics don't know how to use it *smartly.*

Here's how to use ChatGPT intelligently (and ethically) for academic purposes:

# data[].attributes.wish_content.attachment_props.original_text
ChatGPT is everywhere and everyone is using it.

But most academics don't know how to use it *smartly.*

Here's how to use ChatGPT intelligently (and ethically) for academic purposes:

# data[].attributes.attachment_link.embed_code
ChatGPT is everywhere and everyone is using it.

But most academics don't know how to use it *smartly.*

Here's how to use ChatGPT intelligently (and ethically) for academic purposes:
— Mushtaq Bilal, PhD (@MushtaqBilalPhD) February 3, 2023

# data[].attributes.attachment_link.metadata.attachment_props_source.data.text
ChatGPT is everywhere and everyone is using it.

But most academics don't know how to use it *smartly.*

Here's how to use ChatGPT intelligently (and ethically) for academic purposes:

# data[].attributes.attachment_link.metadata.attachment_props.original_text
ChatGPT is everywhere and everyone is using it.

But most academics don't know how to use it *smartly.*

Here's how to use ChatGPT intelligently (and ethically) for academic purposes:

# data[].attributes.attachment_link.metadata.attachment_props.text
ChatGPT is everywhere and everyone is using it.

But most academics don't know how to use it *smartly.*

Here's how to use ChatGPT intelligently (and ethically) for academic purposes:

# data[].attributes.headline
Suggestions for using ChatGPT as a writing assistant (use it to speed up process rather than to fully create a product)

# data[].attributes.subject
Suggestions for using ChatGPT as a writing assistant (use it to speed up process rather than to fully create a product)

# data[].attributes.body
https://twitter.com/MushtaqBilalPhD/status/1622120418646675456

# data[].attributes.wish_content.attachment_props.text
How to use ChatGPT *smartly* as a personal writing assistant:

# data[].attributes.wish_content.attachment_props.original_text
How to use ChatGPT *smartly* as a personal writing assistant:

# data[].attributes.attachment_link.embed_code
How to use ChatGPT *smartly* as a personal writing assistant:
— Mushtaq Bilal, PhD (@MushtaqBilalPhD) February 5, 2023

# data[].attributes.attachment_link.metadata.attachment_props_source.data.text
How to use ChatGPT *smartly* as a personal writing assistant:

# data[].attributes.attachment_link.metadata.attachment_props.original_text
How to use ChatGPT *smartly* as a personal writing assistant:

# data[].attributes.attachment_link.metadata.attachment_props.text
How to use ChatGPT *smartly* as a personal writing assistant:

# data[].attributes.body
https://twitter.com/EmPittsDonahoe/status/1624095485345796096

# data[].attributes.wish_content.attachment_props.text
My students and I talked about ChatGPT in my first-year writing course this week. Some dispatches from the front lines of teaching in the age of AI: 🧵 1/

# data[].attributes.wish_content.attachment_props.original_text
My students and I talked about ChatGPT in my first-year writing course this week. Some dispatches from the front lines of teaching in the age of AI: 🧵 1/

# data[].attributes.attachment_link.embed_code
My students and I talked about ChatGPT in my first-year writing course this week. Some dispatches from the front lines of teaching in the age of AI: 🧵 1/
— Dr. Emily Pitts Donahoe (@EmPittsDonahoe) February 10, 2023

# data[].attributes.attachment_link.metadata.attachment_props_source.data.text
My students and I talked about ChatGPT in my first-year writing course this week. Some dispatches from the front lines of teaching in the age of AI: 🧵 1/

# data[].attributes.attachment_link.metadata.attachment_props.original_text
My students and I talked about ChatGPT in my first-year writing course this week. Some dispatches from the front lines of teaching in the age of AI: 🧵 1/

# data[].attributes.attachment_link.metadata.attachment_props.text
My students and I talked about ChatGPT in my first-year writing course this week. Some dispatches from the front lines of teaching in the age of AI: 🧵 1/

# data[].attributes.headline
Our Obsession with Cheating is Ruining Our Relationship with Students (focus on learning rather than AI detecting AI)

# data[].attributes.subject
Our Obsession with Cheating is Ruining Our Relationship with Students (focus on learning rather than AI detecting AI)

# data[].attributes.body
https://marcwatkins.substack.com/p/our-obsession-with-cheating-is-ruining?utm_source=direct&utm_campaign=post&utm_medium=web

# data[].attributes.attachment_link.readable_text
As the technology to generate writing with AI ramps up, so does the production of AI-detection tools that attempt to curtail "cheating" by using them. But the bevy of AI detection tools on the market aren't keeping up with the speed of AI's development. Because they are using older versions of GPT as the main component of their detection mechanism, they often misidentify human-composed text as AI-generated or confuse AI-generated text as human-generated. And while OpenAI is working on a watermarking scheme that uses cryptography to mark text generated by a transformer, even their lead engineer admits this would be easy to defeat by simply copying a generated output into a different language model.

We don't believe students want to learn. This profoundly misguided notion has broken the trust we've tried to establish with our students. Jonathan Malesic's recent op-ed in the New York Times reminds us all that our students want to learn, to be seen, to be heard. Like our students, we all thrive when we are open to learning new things and having our assumptions about the world challenged. Malesic's call for humility in this age of "knowingness" is a profound departure from a culture that views not knowing something as akin to failure:

Universities are factories of human knowledge. They're also monuments to individual ignorance. We know an incredible amount, but I know only a tiny bit. College puts students in classrooms with researchers who are acutely aware of all they don't know. Professors have a reputation for arrogance, but a humble awareness of the limits of knowledge is their first step toward discovering a little more.

To overcome careerism and knowingness and instill in students a desire to learn, schools and parents need to convince students (and perhaps themselves) that college has more to offer than job training. You're a worker for only part of your life; you're a human being, a creature with a powerful brain, throughout it.

One of the benefits of a college education is it teaches us how little we know about the world, about how vast and complex the range of knowledges are. Your students want to be there to be a part of that world, to occupy that space with their minds and ideas. College also invites you into a discussion about that knowledge, which tests your limits, allowing personal growth. When we take a stance on students and preemptively assume them to be cheaters, we are essentially denying them entry into this space.

Josh Elyer is correct. The current freak-out panic over ChatGPT mirrors the panic that drove institutions and faculty to plagiarism detection software in the first place. Decades later, plagiarism detection software has conditioned us to easily access a tool that we hope will prove to us a student's writing is their own. We don't think about the fact that this tool helped accelerate contract cheating and creates a culture of mistrust. We don't think about how the use of these tools has set up a situation in which companies are harvesting unfathomable amounts of data from our students. We mistrust our students' desire to learn in such a profound way that many don't give a second thought to using AI-detection software that tracks students' eye movements and may misflag female students of color at far higher rates than white males.

You can play around with a number of the AI detectors below. Sometimes they work. Sometimes they don't. Is that really the bar we want to use in so-called AI detection? I've included screenshots of the detectors mislabeling an AI-generated essay I created with a GPT-3 powered app in October. In each case, it labels the generated essay as human writing.
Huggingface's roBERTa powered Output Detector uses GPT-2 to try and detect AI IBM's Giant Language Model Testing Room (GLTR) also uses GPT-2 to try and detect AI
And then there are the waves of clones and eager developers rushing to sell terrified educators something to detect AI.
Crossplag decided to use roBERTa alongside their plagiarism detection and at least includes the caveat that the system doesn't work well enough to be deployed to institutions at this time. The newest one, GPTzero, also fails to detect the AI-generated text and likewise appears to use some form of GPT-2.
When we use AI to catch AI, we're making a decision to put our trust in systems that we do not fully understand under the vague promise that "at least this is something." Maybe that's what people need, but how long will it be until a faculty member takes one of these unreliable reports as gospel truth and brings an innocent student up on charges of academic misconduct that are false? Think of the damage this will further cause to the already frayed relationship between teachers and students.

I, too, have used plagiarism detection software in the past. Hundreds of my students have submitted thousands of essays through such services, knowing their words will be assessed against a large database to discern similarities to other essays. I have required the use of this service under a misguided belief that a rote and imperfect method of detecting cheating is more important than creating a culture of trust.

We're going to have to stop panicking and educate ourselves about the technology and think about what boundaries we'd like to put into place with our use of the tech and our students. While the current moral panic is about students using GPT to cheat, I imagine this is going to shift rapidly when folks realize that faculty can also use the technology to leave formative feedback on student essays. To echo Mike Sharples, "Students [will] employ AI to write assignments. Teachers use AI to assess and review them (Lu & Cutumisu, 2021; Lagakis & Demetriadis, 2021). Nobody learns, nobody gains."

We're going to have to search for moments of learning. One way of starting this exploration is to heed John Warner's call to stop automating the writing process. Students write predictable, formulaic essays and teachers leave similarly predictable, formulaic responses. What was learned and what was gained in such a system? The reasons why we've adopted this structure are based on convenience and labor. It's easier to teach the form of an essay than it is to teach a person the skills needed to develop as a writer. To do that, you need small classes, a reasonable teaching load, and well-compensated faculty.

# data[].attributes.attachment_link.title
Our Obsession with Cheating is Ruining Our Relationship with Students

# data[].attributes.attachment_link.description
AI detection isn't going to work-but that's not going to change our desire to have a tool to detect what work was written by a human vs. generated by an AI transformer.

# data[].attributes.headline
Postplagiarism - writing in the age of AI

# data[].attributes.subject
Postplagiarism - writing in the age of AI

# data[].attributes.headline
AI Text Generators and Teaching Writing: Starting Points for Inquiry

# data[].attributes.subject
AI Text Generators and Teaching Writing: Starting Points for Inquiry

# data[].attributes.body
https://wac.colostate.edu/repository/collections/ai-text-generators-and-teaching-writing-starting-points-for-inquiry/

# data[].attributes.attachment_link.readable_text
What do teachers who assign writing need to know about AI text generators? How should we change our pedagogical practices, given the recent advances in AI Large Language Models (LLMs) such as OpenAI's ChatGPT? How should teachers participate in shaping policies around these technologies in our departments, institutions, and society?
As teachers who assign writing, we need to reckon with the possibility that our students may use writing generators to produce essays or parts of essays. These tools do not produce writing copied from human sources; the AI origin appears to be not reliably detectable either by human readers or by software.
To shape our individual and institutional responses to this new technology, writing teachers and scholars need more information about the kinds and quality of AI-generated text we can expect in response to common types of essay prompts. For example, we may want to design prompts that AI text generators are unskilled at performing, or we may want to find ways to use these generators pedagogically. Either way, as faculty responsible for teaching writing as impactful, ethical intellectual activity, we need to know what AI generators are capable of. And we need to understand the forms of bias and error that emerge in AI-generated writing.

Shared Resources

Opportunities to Engage and Participate

The resources collected here serve as catalysts for inquiry, discussion and collaborative research as we respond to this major change in the kind of writing assistance available to our students.

Suggested Citation

Mills, Anna (Curator). AI Text Generators and Teaching Writing: Starting Points For Inquiry. 2022. Last Updated February 14, 2023.

Authorship and Credits

Anna Mills ( armills@marin.edu) teaches English at College of Marin and previously taught at City College of San Francisco for 17 years. She is the author of an Open Educational Resource (OER) textbook, How Arguments Work: A Guide to Writing and Analyzing Texts in College, which has been praised on OER review sites and used at over 45 colleges. She is the recipient of an Open Education Research Fellowship and currently serves as the English Discipline Lead for the Academic Senate of the California Community Colleges OER Initiative. Anna earned a master's degree from Bennington College in Writing and Literature with a focus on nonfiction writing. Her essays have appeared in journals such as The Writer's Chronicle, The Sun, and Salmagundi. Recently, she has focused on exploring how writing instructors can respond to the accessibility of large language models. Follow her at @EnglishOER.

# data[].attributes.attachment_link.title
AI Text Generators and Teaching Writing: Starting Points for Inquiry

# data[].attributes.attachment_link.description
What do teachers who assign writing need to know about AI text generators? How should we change our pedagogical practices, given the recent advances in AI Large Language Models (LLMs) such as OpenAI's ChatGPT? How should teachers participate in shaping policies around these technologies in our departments, institutions, and society?

# data[].attributes.headline
Canvas Commons module: Critical AI Literacy and Critical Assessment

# data[].attributes.subject
Canvas Commons module: Critical AI Literacy and Critical Assessment

# data[].attributes.body
https://lor.instructure.com/resources/455e6f86b9e2403ea59b7083dd7d8f56

# data[].attributes.attachment_link.title
Critical AI Literacy and Critical Assessment | Canvas Commons

# data[].attributes.headline
101 Creative Ideas to Use AI in Education (sample case studies)

# data[].attributes.subject
101 Creative Ideas to Use AI in Education (sample case studies)

# data[].attributes.body
https://docs.google.com/presentation/d/1wVgLWgeEvJm3fznlm0aV8ZiuWsW3o3aUQUCcvuM5vxQ/edit#slide=id.g2049b17e175_5_0

# data[].attributes.headline
A People's Guide to Artificial Intelligence

# data[].attributes.subject
A People's Guide to Artificial Intelligence

# data[].attributes.body
https://alliedmedia.org/wp-content/uploads/2020/09/peoples-guide-ai.pdf

# data[].attributes.headline
Policies related to ChatGPT and other AI Tools

# data[].attributes.subject
Policies related to ChatGPT and other AI Tools

# data[].attributes.body
https://docs.google.com/document/d/1WpCeTyiWCPQ9MNCsFeKMDQLSTsg1oKfNIH6MzoSFXqQ/edit#heading=h.oio9fphey5pp

# data[].attributes.attachment_link.title
Course Policies related to ChatGPT and other AI Tools

# data[].attributes.headline
Suggested additions to syllabus on use of AI

# data[].attributes.subject
Suggested additions to syllabus on use of AI

# data[].attributes.body
https://docs.google.com/document/d/1O1_uUvF8OYbleru5QyjjuNP_On7h5vaVQC2GaSQ315U/edit

# data[].attributes.body
https://csp.uber.space/phhd/rulesfortools.pdf

# data[].attributes.headline
Sample: Guidelines for the Use of Artificial Intelligence in University Courses

# data[].attributes.subject
Sample: Guidelines for the Use of Artificial Intelligence in University Courses

# data[].attributes.body
https://forogpp.files.wordpress.com/2023/02/guidelines-for-the-use-of-artificial-intelligence-in-university-courses-v4.3.1.pdf

# data[].attributes.headline
Finding and Building Open Educational Resources about AI Writing Assistance

# data[].attributes.subject
Finding and Building Open Educational Resources about AI Writing Assistance

# data[].attributes.body
https://www.youtube.com/watch?v=Ucp2ItG3Wsc

and slides: https://docs.google.com/presentation/d/12yZI0Wy1A-TE5IQirfIHZlE5wln6Fsjfj0ksq8GqRvw/edit#slide=id.g7ca92c065db60cef_1682

# data[].attributes.attachment_link.title
Finding and Building Open Educational Resources about AI Writing Assistance

# data[].attributes.headline
5 ways to change how students are graded

# data[].attributes.subject
5 ways to change how students are graded

# data[].attributes.attachment_link.readable_text
Universities and schools have entered a new phase in how they need to address academic integrity as our society navigates a second era of digital technologies, which include publicly available generative artificial intelligence (AI) like ChatGPT. Such platforms allow students to generate novel text for written assignments.

While many worry these advanced AI technologies are ushering in a new age of plagiarism and cheating, these technologies also introduce opportunities for educators to rethink assessment practices and engage students in deeper and more meaningful learning that can promote critical thinking skills.

We believe the emergence of ChatGPT creates an opportunity for schools and post-secondary institutions to reform traditional approaches to assessing students that rely heavily on testing and written tasks focused on students' recall, remembering and basic synthesis of content.

Cheating and ChatGPT

Estimates of cheating vary widely across national contexts and sectors.

Sarah Elaine Eaton, an expert who studies academic integrity, cautions cheating may be under-reported: she has estimated that at Canadian universities, 70,000 students buy cheating services every year.

How the recent launch of ChatGPT by OpenAI will impact cheating in both compulsory and higher education settings is unknown, but how this evolves may depend on whether or not institutions retain or reform traditional assessment practices.

Evading plagiarism detection software?

The ability of popular plagiarism detection tools to identify cheating using ChatGPT to generate assignments remains a challenge.

A recent study, not yet peer reviewed, found that 50 essays generated using ChatGPT produced sophisticated texts that were able to evade the traditional plagiarism check software.

Given that ChatGPT reached an estimated 100 million monthly active users in January, just two months after its launch, it is understandable why some have argued AI applications such as ChatGPT will spur enormous changes in contemporary schooling.

Policy responses to AI and ChatGPT

Not surprisingly, there are opposing views on how to respond to ChatGPT and other AI language models.

Some argue educators should embrace AI as a valuable technological tool, provided applications are cited correctly.

Others believe more resources and training are required so educators are better able to catch instances of cheating.

Still others, such as New York City's Department of Education, have resorted to blocking AI applications such as ChatGPT from devices and networks.

Forward-thinking assessment

The figure below depicts three critical elements of a forward-thinking assessment system. Although each element could be elaborated, our focus is in offering educators a series of strategies that will allow them to maintain academic standards and promote authentic learning and assessment in the face of current and future AI applications.

Teachers and university professors have relied heavily on "one and done" essay assignments for decades. Essentially, a student is assigned or asked to pick a generic essay topic from a list and submit their final assignment on a specific date.

Such assignments are particularly susceptible to new AI applications, as well as contract cheating - whereby a student buys a completed essay. Educators now need to rethink such assignments. Here are some strategies.
1. Consider ways to incorporate AI in valid assessment.
It's not useful or practical for institutions to outright ban AI and applications like ChatGPT.

AI has already been incorporated into some university classrooms. We believe AI technologies must be selectively integrated so that students are able to reflect on appropriate uses and connect their reflections to learning competencies.

For example, Paul Fyfe, an English professor who teaches about how humans interact with data describes a "pedagogical experiment" in which he required students to take content from text-generating AI software and weave this content into their final essay.

Students were then asked to confront the availability of AI as a writing tool and reflect on the ethical use and evaluation of language modes.
2. Engage students in setting learning goals.
Ensuring students understand how they will be graded is key to any good assessment system.

Inviting students to collaboratively establish learning goals and criteria for the task, with consideration for the role of AI software, would help students to evaluate and judge appropriate contexts in which AI can work as a learning tool.

Read more: Unlike with academics and reporters, you can't check when ChatGPT's telling the truth
3. Require students to submit drafts for feedback.
Although students should still complete essay assignments, research into academic integrity policy in response to generative AI suggests students should be required to submit drafts of their work for review and feedback. Apart from helping to detect plagiarism, this kind of "formative assessment" practice is positive for guiding student learning.

Feedback can be offered by the teacher or by students themselves. Peer- and self-feedback can serve to critically evaluate work in progress (or work generated by AI software).
4. Grade subcomponents of the task.
Students could receive a grade for each subcomponent - including their involvement in feedback processes. They would also be evaluated in relation to how well they incorporated and attended to the specific feedback provided.

The assignment becomes bigger than a final essay, it becomes a product of learning, where students' ideas are evaluated from development to final submission.
5. Move to more authentic assessments or include performance elements.
Good assessment practice involves an educator observing student learning across multiple contexts.

For example, educators can invite students to present their work, discuss an essay in a conference format or share a video articulation or an artistic representation. The aim here is to encourage students to share their learning through an alternative format. An important question to ask is whether or not you need the essay component at all? Is there a more authentic way to effectively assess student learning?

Authentic assessments are those that relate content to context. When students are asked to do this, they must apply knowledge in more practical settings, often making AI tools less helpful.

For help in rethinking assessment practices towards more authentic and alternative approaches, educators can consider taking the free course, Transforming Assessment: Strategies for Higher Education.

Improve benefits for students

Collectively, these suggestions may be more time-consuming, particularly in larger undergraduate classes.

But they do provide greater learning and synergy between forms of assessment that benefit students: formative assessment to guide teaching and learning, and "summative assessment," primarily used for grading and evaluation purposes.

AI is here and here to stay, and we must embrace it as part of our learning environment. Incorporating AI into how we assess student learning will yield more reliable assessment processes and valid and valued assessment outcomes.

# data[].attributes.attachment_link.title
ChatGPT and cheating: 5 ways to change how students are graded

# data[].attributes.attachment_link.description
Universities and schools have entered a new phase in how they need to address academic integrity as our society navigates a second era of digital technologies, which include publicly available generative artificial intelligence (AI) like ChatGPT. Such platforms allow students to generate novel text for written assignments.

# data[].attributes.headline
Noam Chomsky: The False Promise of ChatGPT

# data[].attributes.subject
Noam Chomsky: The False Promise of ChatGPT

# data[].attributes.body
https://www.nytimes.com/2023/03/08/opinion/noam-chomsky-chatgpt-ai.html

# data[].attributes.attachment_link.title
Opinion | Noam Chomsky: The False Promise of ChatGPT

# data[].attributes.attachment_link.description
The most prominent strain of A.I. encodes a flawed conception of language and knowledge.

# data[].attributes.headline
Australian Academic Integrity Network Generative Artificial Intelligence Guidelines (Creative Commons Licence) 

# data[].attributes.subject
Australian Academic Integrity Network Generative Artificial Intelligence Guidelines (Creative Commons Licence) 

# data[].attributes.headline
Teaching with ChatGPT: Examples of Practice - series of short videos

# data[].attributes.subject
Teaching with ChatGPT: Examples of Practice - series of short videos

# data[].attributes.body
https://www.youtube.com/playlist?list=PLAbF8wnSF-e-pzuN-XaSrdsBAZsglG8NO

# data[].attributes.attachment_link.title
Teaching with ChatGPT: Examples of Practice

# data[].attributes.attachment_link.description
Share your videos with friends, family, and the world

# data[].attributes.body
https://www.timeshighereducation.com/news/gpt-4s-launch-another-step-change-ai-and-higher-education?utm_source=newsletter&utm_medium=email&utm_campaign=editorial-daily&spMailingID=24969456&spUserID=MTAyMjQ5OTg5NTM5NAS2&spJobID=2194502673&spReportId=MjE5NDUwMjY3MwS2

# data[].attributes.attachment_link.title
GPT-4's launch 'another step change' for AI and higher education

# data[].attributes.attachment_link.description
New version of OpenAI's chatbot can interpret images and tutor students

# data[].attributes.headline
Supporting Academic Integrity: Ethical Uses of Artificial Intelligence in Higher Education Information Sheet

# data[].attributes.subject
Supporting Academic Integrity: Ethical Uses of Artificial Intelligence in Higher Education Information Sheet

# data[].attributes.body
https://drive.google.com/file/d/1oqMS4P2moRjssrBDev0BKPCYA53mvD-z/view

# data[].attributes.attachment_link.title
Supporting Academic Integrity- Ethical Uses of Artificial Intelligence in Higher Education Information Sheet.pdf

# data[].attributes.headline
We shouldn't fear ChatGPT in education - we need to work with it (Philosophy perspective)

# data[].attributes.subject
We shouldn't fear ChatGPT in education - we need to work with it (Philosophy perspective)

# data[].attributes.body
https://www.irishexaminer.com/opinion/commentanalysis/arid-41101504.html

# data[].attributes.attachment_link.authors[].name
Joel Walmsley, Department of Philosophy, UCC

# data[].attributes.attachment_link.title
We shouldn’t fear ChatGPT in education — we need to work with it

# data[].attributes.attachment_link.description
There is no point pretending ChatGPT doesn’t exist, and trying to prevent its use by implementing honour codes and harsher plagiarism penalties will not work either

# data[].attributes.headline
Cambridge UP launches ethical use of AI policy

# data[].attributes.subject
Cambridge UP launches ethical use of AI policy

# data[].attributes.body
https://www.businesswire.com/news/home/20230313005092/en/Cambridge-Launches-AI-Research-Ethics-Policy

# data[].attributes.attachment_link.readable_text
CAMBRIDGE, England--()--New guidelines will help researchers use generative AI tools like ChatGPT while upholding academic standards around transparency, plagiarism, accuracy and originality.

The rules are set out in the first AI ethics policy from Cambridge University Press and apply to research papers, books and other scholarly works.

They include a ban on AI being treated as an 'author' of academic papers and books published by Cambridge University Press.

The move provides clarity to academics amid concerns about flawed or misleading use of powerful large language models like ChatGPT in research, alongside excitement about its potential.

Mandy Hill, Managing Director for Academic at Cambridge University Press & Assessment, said: "Generative AI can enable new avenues of research and experimentation. Researchers have asked us for guidance to navigate its use.

"We believe academic authors, peer reviewers and editors should be free to use emerging technologies as they see fit within appropriate guidelines, just as they do with other research tools.

"Like our academic community, we are approaching this new technology with a spirit of critical engagement. In prioritising transparency, accountability, accuracy and originality, we see as much continuity as change in the use of generative AI for research.

"It's obvious that tools like ChatGPT cannot and should not be treated as authors.

"We want our new policy to help the thousands of researchers we publish each year, and their many readers. We will continue to work with them as we navigate the potential biases, flaws and compelling opportunities of AI."

R. Michael Alvarez , Professor of Political and Computational Social Science at the California Institute of Technology, said: "Generative AI introduces many issues for academic researchers and educators. As a series editor for Cambridge University Press, I appreciate the leadership the Press is taking to outline guidelines and policies for how we can use these new tools in our research and writing. I anticipate that we will be having this conversation about the opportunities and pitfalls presented by generative AI for academic publishing for many years to come."

Professor Alvarez and his Caltech collaborators use AI, including LLMs, to detect online harassment, trolling and abusive behaviour on social media platforms and in videogames such as Call of Duty, as well as to combat misinformation.

Professor Alvarez is co-editor of Quantitative and Computational Methods for Social Science, published by Cambridge University Press.

The Cambridge principles for generative AI in research publishinginclude that:

AI use must be declared and clearly explained in publications such as research papers, just as scholars do with other software, tools and methodologies.

AI does not meet the Cambridge requirements for authorship, given the need for accountability. AI and LLM tools may not be listed as an author on any scholarly work published by Cambridge.

Any use of AI must not breach Cambridge plagiarism policy. Scholarly works must be the author's own, and not present others' ideas, data, words or other material without adequate citation and transparent referencing.

Authors are accountable for the accuracy, integrity and originality of their research papers, including for any use of AI.

Each year Cambridge University Press publishes tens of thousands research papers in more than 400 peer-reviewed journals and 1,500 research monographs, reference works and higher education textbooks.

# data[].attributes.attachment_link.title
Cambridge Launches AI Research Ethics Policy

# data[].attributes.attachment_link.description
Cambridge launches ChatGPT ethics policy

# data[].attributes.headline
Cambridge UP: Ethical use of AI in research policy

# data[].attributes.subject
Cambridge UP: Ethical use of AI in research policy

# data[].attributes.body
https://www.businesswire.com/news/home/20230313005092/en/Cambridge-Launches-AI-Research-Ethics-Policy

# data[].attributes.attachment_link.readable_text
CAMBRIDGE, England--()--New guidelines will help researchers use generative AI tools like ChatGPT while upholding academic standards around transparency, plagiarism, accuracy and originality.

The rules are set out in the first AI ethics policy from Cambridge University Press and apply to research papers, books and other scholarly works.

They include a ban on AI being treated as an 'author' of academic papers and books published by Cambridge University Press.

The move provides clarity to academics amid concerns about flawed or misleading use of powerful large language models like ChatGPT in research, alongside excitement about its potential.

Mandy Hill, Managing Director for Academic at Cambridge University Press & Assessment, said: "Generative AI can enable new avenues of research and experimentation. Researchers have asked us for guidance to navigate its use.

"We believe academic authors, peer reviewers and editors should be free to use emerging technologies as they see fit within appropriate guidelines, just as they do with other research tools.

"Like our academic community, we are approaching this new technology with a spirit of critical engagement. In prioritising transparency, accountability, accuracy and originality, we see as much continuity as change in the use of generative AI for research.

"It's obvious that tools like ChatGPT cannot and should not be treated as authors.

"We want our new policy to help the thousands of researchers we publish each year, and their many readers. We will continue to work with them as we navigate the potential biases, flaws and compelling opportunities of AI."

R. Michael Alvarez , Professor of Political and Computational Social Science at the California Institute of Technology, said: "Generative AI introduces many issues for academic researchers and educators. As a series editor for Cambridge University Press, I appreciate the leadership the Press is taking to outline guidelines and policies for how we can use these new tools in our research and writing. I anticipate that we will be having this conversation about the opportunities and pitfalls presented by generative AI for academic publishing for many years to come."

Professor Alvarez and his Caltech collaborators use AI, including LLMs, to detect online harassment, trolling and abusive behaviour on social media platforms and in videogames such as Call of Duty, as well as to combat misinformation.

Professor Alvarez is co-editor of Quantitative and Computational Methods for Social Science, published by Cambridge University Press.

The Cambridge principles for generative AI in research publishinginclude that:

AI use must be declared and clearly explained in publications such as research papers, just as scholars do with other software, tools and methodologies.

AI does not meet the Cambridge requirements for authorship, given the need for accountability. AI and LLM tools may not be listed as an author on any scholarly work published by Cambridge.

Any use of AI must not breach Cambridge plagiarism policy. Scholarly works must be the author's own, and not present others' ideas, data, words or other material without adequate citation and transparent referencing.

Authors are accountable for the accuracy, integrity and originality of their research papers, including for any use of AI.

Each year Cambridge University Press publishes tens of thousands research papers in more than 400 peer-reviewed journals and 1,500 research monographs, reference works and higher education textbooks.

# data[].attributes.attachment_link.title
Cambridge Launches AI Research Ethics Policy

# data[].attributes.attachment_link.description
Cambridge launches ChatGPT ethics policy

# data[].attributes.headline
UK Whitepaper on A pro-innovation approach to AI regulation

# data[].attributes.subject
UK Whitepaper on A pro-innovation approach to AI regulation

# data[].attributes.body
https://www.gov.uk/government/publications/ai-regulation-a-pro-innovation-approach/white-paper#part-2-the-current-regulatory-environment

# data[].attributes.attachment_link.readable_text
Presented to Parliament by the Secretary of State for Science, Innovation and Technology by Command of His Majesty on 29 March 2023.

Command Paper Number: 815

© Crown copyright 2023

ISBN: 978-1-5286-4009-1

Correction slip

Correction:

Text currently reads in Annex C:

1. Do you agree that requiring organisations to make it clear when they are using AI would adequately ensure transparency?

2. What other transparency measures would be appropriate, if any?

3. Do you agree that current routes to contestability or redress for AI-related harms are adequate?

4. How could routes to contestability or redress for AI-related harms be improved, if at all?

[...]

[...]

S1. Which of the sandbox models described in section 3.3.4 would be most likely to support innovation?

Text should read:

1: Do you agree that requiring organisations to make it clear when they are using AI would improve transparency?

2: Are there other measures we could require of organisations to improve transparency for AI?

3: Do you agree that current routes to contest or get redress for AI-related harms are adequate?

4: How could current routes to contest or seek redress for AI-related harms be improved, if at all?

[...]

[...]

S1: To what extent would the sandbox models described in section 3.3.4 support innovation?

Date of correction: 4 July 2023

Ministerial foreword

The Rt Hon Michelle Donelan MP, Secretary of State for Science, Innovation and Technology

I believe that a common-sense, outcomes-oriented approach is the best way to get right to the heart of delivering on the priorities of people across the UK. Better public services, high quality jobs and opportunities to learn the skills that will power our future - these are the priorities that will drive our goal to become a science and technology superpower by 2030.

Artificial Intelligence ( AI) will play a central part in delivering and enabling these goals, and this white paper will ensure we are putting the UK on course to be the best place in the world to build, test and use AI technology. But we are not starting from zero. Having invested over £2.5 billion in AI since 2014, this paper builds on our recent announcements of £110 million for our AI Tech Missions Fund, £900 million to establish a new AI Research Resource and to develop an exascale supercomputer capable of running large AI models - backed up by our new £8 million AI Global Talent Network and £117 million of existing funding to create hundreds of new PhDs for AI researchers.

Most of us are only now beginning to understand the transformative potential of AI as the technology rapidly improves. But in many ways, AI is already delivering fantastic social and economic benefits for real people - from improving NHS medical care to making transport safer. Recent advances in things like generative AI give us a glimpse into the enormous opportunities that await us in the near future if we are prepared to lead the world in the AI sector with our values of transparency, accountability and innovation.

These risks could include anything from physical harm, an undermining of national security, as well as risks to mental health. The development and deployment of AI can also present ethical challenges which do not always have clear answers. Unless we act, household consumers, public services and businesses will not trust the technology and will be nervous about adopting it. Unless we build public trust, we will miss out on many of the benefits on offer.

That is why our white paper details how we intend to support innovation while providing a framework to ensure risks are identified and addressed. However, a heavy-handed and rigid approach can stifle innovation and slow AI adoption. That is why we set out a proportionate and pro-innovation regulatory framework. Rather than target specific technologies, it focuses on the context in which AI is deployed. This enables us to take a balanced approach to weighing up the benefits versus the potential risks.

We recognise that particular AI technologies, foundation models for example, can be applied in many different ways and this means the risks can vary hugely. For example, using a chatbot to produce a summary of a long article presents very different risks to using the same technology to provide medical advice. We understand the need to monitor these developments in partnership with innovators while also avoiding placing unnecessary regulatory burdens on those deploying AI.

To ensure our regulatory framework is effective, we will leverage the expertise of our world class regulators. They understand the risks in their sectors and are best placed to take a proportionate approach to regulating AI. This will mean supporting innovation and working closely with business, but also stepping in to address risks when necessary. By underpinning the framework with a set of principles, we will drive consistency across regulators while also providing them with the flexibility needed.

For innovators working at the cutting edge and developing novel technologies, navigating regulatory regimes can be challenging. That's why we are confirming our commitment to taking forward a key recommendation made by Sir Patrick Vallance to establish a regulatory sandbox for AI. This will bring together regulators to support innovators directly and help them get their products to market. The sandbox will also enable us to understand how regulation interacts with new technologies and refine this interaction where necessary.

Our approach relies on collaboration between government, regulators and business. Initially, we do not intend to introduce new legislation. By rushing to legislate too early, we would risk placing undue burdens on businesses. But alongside empowering regulators to take a lead, we are also setting expectations. Our new monitoring functions will provide a real time assessment of how the regulatory framework is performing so that we can be confident that it is proportionate. The pace of technological development also means that we need to understand new and emerging risks, engaging with experts to ensure we take action where necessary. A critical component of this activity will be engaging with the public to understand their expectations, raising awareness of the potential of AI and demonstrating that we are responding to concerns.

The framework set out in this white paper is deliberately designed to be flexible. As the technology evolves, our regulatory approach may also need to adjust. Our principles-based approach, with central functions to monitor and drive collaboration, will enable us to adapt as needed while providing industry with the clarity needed to innovate. We will continue to develop our approach, building on our commitment to making the UK the best place in the world to be a business developing and using AI. Responses to the consultation will inform how we develop the regulatory framework - I encourage all of those with an interest to respond.

Executive summary

Artificial intelligence - the opportunity and the challenge

1. Artificial intelligence ( AI) is already delivering wide societal benefits, from medical advances to mitigating climate change. For example, an AI technology developed by DeepMind, a UK-based business, can now predict the structure of almost every protein known to science. This breakthrough will accelerate scientific research and the development of life-saving medicines - it has already helped scientists to make huge progress in combating malaria, antibiotic resistance, and plastic waste.

2. The UK Science and Technology Framework sets out government's strategic vision and identifies AI as one of 5 critical technologies. The framework notes the role of regulation in creating the environment for AI to flourish. We know that we have yet to see AI technologies reach their full potential. Under the right conditions, AI will transform all areas of life and stimulate the UK economy by unleashing innovation and driving productivity, creating new jobs and improving the workplace.

4. While we should capitalise on the benefits of these technologies, we should also not overlook the new risks that may arise from their use, nor the unease that the complexity of AI technologies can produce in the wider public. We already know that some uses of AI could damage our physical and mental health, infringe on the privacy of individuals[footnote 10] and undermine human rights.[footnote 11]

5. Public trust in AI will be undermined unless these risks, and wider concerns about the potential for bias and discrimination, are addressed. By building trust, we can accelerate the adoption of AI across the UK to maximise the economic and social benefits that the technology can deliver, while attracting investment and stimulating the creation of high-skilled AI jobs.[footnote 12] In order to maintain the UK's position as a global AI leader, we need to ensure that the public continues to see how the benefits of AI can outweigh the risks.[footnote 13]

Our pro-innovation framework

9. We are taking a deliberately agile and iterative approach, recognising the speed at which these technologies are evolving. Our framework is designed to build the evidence base so that we can learn from experience and continuously adapt to develop the best possible regulatory regime. Industry has praised our pragmatic and proportionate approach.

10. Our framework is underpinned by 5 principles to guide and inform the responsible development and use of AI in all sectors of the economy:

Safety, security and robustness

Appropriate transparency and explainability

Fairness

Accountability and governance

Contestability and redress

11. We will not put these principles on a statutory footing initially. New rigid and onerous legislative requirements on businesses could hold back AI innovation and reduce our ability to respond quickly and in a proportionate way to future technological advances. Instead, the principles will be issued on a non-statutory basis and implemented by existing regulators. This approach makes use of regulators' domain-specific expertise to tailor the implementation of the principles to the specific context in which AI is used. During the initial period of implementation, we will continue to collaborate with regulators to identify any barriers to the proportionate application of the principles, and evaluate whether the non-statutory framework is having the desired effect.

12. Following this initial period of implementation, and when parliamentary time allows, we anticipate introducing a statutory duty on regulators requiring them to have due regard to the principles. Some feedback from regulators, industry and academia suggested we should implement further measures to support the enforcement of the framework. A duty requiring regulators to have regard to the principles should allow regulators the flexibility to exercise judgement when applying the principles in particular contexts, while also strengthening their mandate to implement them. In line with our proposal to work collaboratively with regulators and take an adaptable approach, we will not move to introduce such a statutory duty if our monitoring of the framework shows that implementation is effective without the need to legislate.

13. In the 2022 AI regulation policy paper,[footnote 15] we proposed a small coordination layer within the regulatory architecture. Industry and civil society were supportive of our intention to ensure coherence across the AI regulatory framework. However, feedback often argued strongly for greater central coordination to support regulators on issues requiring cross-cutting collaboration and ensure that the overall regulatory framework functions as intended.

14. We have identified a number of central support functions required to make sure that the overall framework offers a proportionate but effective response to risk while promoting innovation across the regulatory landscape:

Monitoring and evaluation of the overall regulatory framework's effectiveness and the implementation of the principles, including the extent to which implementation supports innovation. This will allow us to remain responsive and adapt the framework if necessary, including where it needs to be adapted to remain effective in the context of developments in AI's capabilities and the state of the art.

Assessing and monitoring risks across the economy arising from AI.

Conducting horizon scanning and gap analysis, including by convening industry, to inform a coherent response to emerging AI technology trends.

Supporting testbeds and sandbox initiatives to help AI innovators get new technologies to market.

Providing education and awareness to give clarity to businesses and empower citizens to make their voices heard as part of the ongoing iteration of the framework.

Promoting interoperability with international regulatory frameworks.

15. The central support functions will initially be provided from within government but will leverage existing activities and expertise from across the broader economy. The activities described above will neither replace nor duplicate the work undertaken by regulators and will not involve the creation of a new AI regulator.

16. Our proportionate approach recognises that regulation is not always the most effective way to support responsible innovation. The proposed framework is aligned with, and supplemented by, a variety of tools for trustworthy AI, such as assurance techniques, voluntary guidance and technical standards. Government will promote the use of such tools. We are collaborating with partners like the UK AI Standards Hub to ensure that our overall governance framework encourages responsible AI innovation (see part 4 for details).

17. In keeping with the global nature of these technologies, we will also continue to work with international partners to deliver interoperable measures that incentivise the responsible design, development and application of AI. During our call for views, industry, academia and civil society stressed that international alignment should support UK businesses to capitalise on global markets and protect UK citizens from cross-border harms.

18. The UK is frequently ranked third in the world across a range of measures, including level of investment, innovation and implementation of AI.[footnote 16] To make the UK the most attractive place in the world for AI innovation and support UK companies wishing to export and attract international investment, we must ensure international compatibility between approaches. Countries around the world, as well as multilateral forums, are exploring approaches to regulating AI. Thanks to our reputation for pragmatic regulation, the UK is rightly seen by international partners as a leader in this global conversation.

Part 1: Introduction

1.1 The power and potential of artificial intelligence

19. AI is already delivering major advances and efficiencies in many areas. AI quietly automates aspects of our everyday activities, from systems that monitor traffic to make our commutes smoother,[footnote 17] to those that detect fraud in our bank accounts.[footnote 18] AI has revolutionised large-scale safety-critical practices in industry, like controlling the process of nuclear fusion.[footnote 19] And it has also been used to accelerate scientific advancements, such as the discovery of new medicine[footnote 20] or the technologies we need to tackle climate change.[footnote 21]

20. But this is just the beginning. AI can be used in a huge variety of settings and has the extraordinary potential to transform our society and economy.[footnote 22] It could have as much impact as electricity or the internet, and has been identified as one of the 5 critical technologies in the UK Science and Technology Framework.[footnote 23] As AI becomes more powerful, and as innovators explore new ways to use it, we will see more applications of AI emerge. As a result, AI has a huge potential to drive growth[footnote 24] and create jobs.[footnote 25] It will support people to carry out their existing jobs, by helping to improve workforce efficiency and workplace safety.[footnote 26] To remain world leaders in AI, attract global talent and create high-skilled jobs in the UK, we must create a regulatory environment where such innovation can thrive.

21. Technological advances like large language models ( LLMs) are an indication of the transformative developments yet to come.[footnote 27] LLMs provide substantial opportunities to transform the economy and society. For example, LLMs can automate the process of writing code and fixing programming bugs. The technology can support genetic medicine by identifying links between genetic sequences and medical conditions. It can support people to review and summarise key points from lengthy documents. In the last 4 years, LLMs have been developed beyond expectations and they are becoming applicable to an increasingly wide range of tasks.[footnote 28] We expand on the development of LLM and other foundation models in section 3.3.3 below.

Box 1.1: Examples of AI opportunities

AI helps piece together the first complete image of a black hole
AI solves decades old protein-folding puzzle
An AI company based in the UK trained neural networks to predict the structures of proteins, solving a problem that had long stumped scientists. The predictions are advancing the field of structural biology: scientists have already used them to prevent antibiotic resistance,[footnote 30] advance disease research,[footnote 31] and accelerate the fight against plastic pollution.[footnote 32] As we find more uses for AI, it will rewrite scientific fields and change the way we learn about our world.
Deep learning AI could improve breast cancer screening Farming efficiency increased by AI robots AI helps accelerate the discovery of new medicines
Significant time and resources are currently needed to develop new and effective medicines. AI can accelerate the discovery of new medicines by quickly identifying potential biologically active compounds from millions of candidates within a short period.[footnote 35] Scientists may also have succeeded in using generative AI to design antibodies that bind to a human protein linked to cancer.[footnote 36]
AI is used in the fight against the most serious and harmful crimes
The Child Images Abuse Database[footnote 37] uses the powerful data processing capabilities of AI to identify victims and perpetrators of child sexual abuse. The quick and effective identification of victims and perpetrators in digital abuse images allows for real world action to remove victims from harm and ensure their abusers are held to account. The use of AI increases the scale and speed of analysis while protecting staff welfare by reducing their exposure to distressing content.
AI increases cyber security capabilities
Companies providing cyber security services are increasingly using AI to analyse large amounts of data about malware and respond to vulnerabilities in network security at faster-than-human speeds.[footnote 38] As the complexity of the cyber threat landscape evolves, the pattern-recognition and recursive learning capabilities of AI are likely to play an increasingly significant role in proactive cyber defence against malicious actors.

22. The concept of AI is not new, but recent advances in data generation and processing have changed the field and the technology it produces. For example, while recent developments in the capabilities of generative AI models have created exciting opportunities, they have also sparked new debates about potential AI risks.[footnote 39] As AI research and development continues at pace and scale, we expect to see even greater impact and public awareness of AI risks.[footnote 40]

24. We have made an initial assessment of AI-specific risks and their potential to cause harm, with reference in our analysis to the values that they threaten if left unaddressed. These values include safety, security, fairness, privacy and agency, human rights, societal well-being and prosperity.

25. Our assessment of cross-cutting AI risk identified a range of high-level risks that our framework will seek to prioritise and mitigate with proportionate interventions. For example, safety risks include physical damage to humans and property, as well as damage to mental health.[footnote 42] AI creates a range of new security risks to individuals, organisations, and critical infrastructure.[footnote 43] Without government action, AI could cause and amplify discrimination that results in, for example, unfairness in the justice system.[footnote 44] Similarly, without regulatory oversight, AI technologies could pose risks to our privacy and human dignity, potentially harming our fundamental liberties.[footnote 45] Our regulatory intervention will ensure that AI does not cause harm at a societal level, threatening democracy[footnote 46] or UK values.

Box 1.2: Illustrative AI risks

The patchwork of legal frameworks that currently regulate some uses of AI may not sufficiently address the risks that AI can pose. The following examples are hypothetical scenarios designed to illustrate AI 's potential to create harm.
Risks to human rights
Generative AI is used to generate deepfake pornographic video content, potentially damaging the reputation, relationships and dignity of the subject.
Risks to safety
An AI assistant based on LLM technology recommends a dangerous activity that it has found on the internet, without understanding or communicating the context of the website where the activity was described. The user undertakes this activity causing physical harm.

An AI tool assessing credit-worthiness of loan applicants is trained on incomplete or biased data, leading the company to offer loans to individuals on different terms based on characteristics like race or gender.
Risks to privacy and agency
Connected devices in the home may constantly gather data, including conversations, potentially creating a near-complete portrait of an individual's home life. Privacy risks are compounded the more parties can access this data.
Risks to societal wellbeing
Disinformation generated and propagated by AI could undermine access to reliable information and trust in democratic institutions and processes.
Risks to security
AI tools can be used to automate, accelerate and magnify the impact of highly targeted cyber attacks, increasing the severity of the threat from malicious actors. The emergence of LLMs enables hackers[footnote 48] with little technical knowledge or skill to generate phishing campaigns with malware delivery capabilities.[footnote 49]

1.3 A note on terminology

Terminology used in this paper:[footnote 50]

or or AI technologies: products and services that are 'adaptable' and 'autonomous' in the sense outlined in our definition in section 3.2.1.

AI supplier: any organisation or individual who plays a role in the research, development, training, implementation, deployment, maintenance, provision or sale of AI systems.

: any individual or organisation that uses an AI product.

AI life cycle: all events and processes that relate to an AI system's lifespan, from inception to decommissioning, including its design, research, training, development, deployment, integration, operation, maintenance, sale, use and governance.

AI ecosystem: the complex network of actors and processes that enable the use and supply of AI throughout the AI life cycle (including supply chains, markets, and governance mechanisms).

Foundation model: a type of AI model that is trained on a vast quantity of data and is adaptable for use on a wide range of tasks. Foundation models can be used as a base for building more specific AI models. Foundation models are discussed in more detail in section 3.3.3 below.[footnote 51]

Impacted third party: an individual or company that is impacted by the outcomes of the AI systems that they do not use or supply themselves.

Part 2: The current regulatory environment

26. The UK's AI success is, in part, due to our reputation for high-quality regulators and our strong approach to the rule of law, supported by our technology-neutral legislation and regulations. UK laws, regulators and courts already address some of the emerging risks posed by AI technologies (see box 2.1 for examples). This strong legal foundation encourages investment in new technologies, enabling AI innovation to thrive,[footnote 52] and high-quality jobs to flourish.[footnote 53]

Box 2.1: Example of legal coverage of AI in the UK and potential gaps

Discriminatory outcomes that result from the use of AI may contravene the protections set out in the Equality Act 2010.[footnote 54] AI systems are also required by data protection law to process personal data fairly.[footnote 55] However, AI can increase the risk of unfair bias or discrimination across a range of indicators or characteristics. This could undermine public trust in AI.

Product safety laws ensure that goods manufactured and placed on the market in the UK are safe. Product-specific legislation (such as for electrical and electronic equipment,[footnote 56] medical devices,[footnote 57] and toys[footnote 58]) may apply to some products that include integrated AI. However, safety risks specific to AI technologies should be monitored closely. As the capability and adoption of AI increases, it may pose new and substantial risks that are unaddressed by existing rules.

Consumer rights law[footnote 59] may protect consumers where they have entered into a sales contract for AI-based products and services. Certain contract terms (for example, that goods are of satisfactory quality, fit for a particular purpose, and as described) are relevant to consumer contracts. Similarly, businesses are prohibited from including certain terms in consumer contracts. Tort law provides a complementary regime that may provide redress where a civil wrong has caused harm. It is not yet clear whether consumer rights law will provide the right level of protection in the context of products that include integrated AI or services based on AI, or how tort law may apply to fill any gap in consumer rights law protection.

27. While AI is currently regulated through existing legal frameworks like financial services regulation,[footnote 60] some AI risks arise across, or in the gaps between, existing regulatory remits. Industry told us that conflicting or uncoordinated requirements from regulators create unnecessary burdens and that regulatory gaps may leave risks unmitigated, harming public trust and slowing AI adoption.

28. Industry has warned us that regulatory incoherence could stifle innovation and competition by causing a disproportionate amount of smaller businesses to leave the market. If regulators are not proportionate and aligned in their regulation of AI, businesses may have to spend excessive time and money complying with complex rules instead of creating new technologies. Small businesses and start-ups often do not have the resources to do both.[footnote 61] With the vast majority of digital technology businesses employing under 50 people,[footnote 62] it is important to ensure that regulatory burdens do not fall disproportionately on smaller companies, which play an essential role in the AI innovation ecosystem and act as engines for economic growth and job creation.[footnote 63]

29. Regulatory coordination will support businesses to invest confidently in AI innovation and build public trust by ensuring real risks are effectively addressed. While some regulators already work together to ensure regulatory coherence for AI through formal networks like the AI and digital regulations service in the health sector[footnote 64] and the Digital Regulation Cooperation Forum ( DRCF), other regulators have limited capacity and access to AI expertise. This creates the risk of inconsistent enforcement across regulators. There is also a risk that some regulators could begin to dominate and interpret the scope of their remit or role more broadly than may have been intended in order to fill perceived gaps in a way that increases incoherence and uncertainty. Industry asked us to support further system-wide coordination to clarify who is responsible for addressing cross-cutting AI risks and avoid duplicate requirements across multiple regulators.

Case study 2.1: Addressing AI fairness under the existing legal and regulatory framework

30. Government intervention is needed to improve the regulatory landscape. We intend to leverage and build on existing regimes, maximising the benefits of what we already have, while intervening in a proportionate way to address regulatory uncertainty and gaps. This will deliver a pro-innovation regulatory framework that is designed to be adaptable and future-proof, supported by tools for trustworthy AI including assurance techniques and technical standards. This approach will provide more clarity and encourage collaboration between government, regulators and industry to unlock innovation.

Case study 2.2: Adapting regulatory approaches to AI - AI as a medical device

Some UK regulators have led the way and proactively adapted their approaches to AI-enabled technologies.

As part of this work, the will develop guidance on the transparency and interpretability of AI as a medical device.[footnote 67] The will consider the specific challenges posed by AI in this context, drawing on the applicable AI regulation cross-sectoral principles and ethical principles for AI in health and social care to issue practical guidance on how to meet legal product safety requirements. The will work with other regulators such as the Information Commissioner's Office ( ICO) and the National Data Guardian to consider patients' data protection and trust in medical devices.

This work will provide manufacturers with clear requirements and guidance to attract responsible innovation to the UK.

Part 3: An innovative and iterative approach

3.1 Aims of the regulatory framework

31. Regulation can increase innovation by giving businesses the incentive to solve important problems while addressing the risk of harm to citizens. For example, product safety legislation has increased innovation towards safer products and services.[footnote 68] In the case of AI, a context-based, proportionate approach to regulation will help strengthen public trust and increase AI adoption.[footnote 69]

32. The National AI Strategy set out our aim to regulate AI effectively and support innovation.[footnote 70] In line with the principles set out in the Plan for Digital Regulation,[footnote 71] our approach to AI regulation will be proportionate; balancing real risks against the opportunities and benefits that AI can generate. We will maintain an effective balance as we implement the framework by focusing on the context and outcomes of AI.

33. Our policy paper proposed a pro-innovation framework designed to give consumers the confidence to use AI products and services, and provide businesses the clarity they need to invest in AI and innovate responsibly.[footnote 72] This approach was broadly welcomed - particularly by industry. Based on feedback, we have distilled our aims into 3 objectives that our framework is designed to achieve:

Drive growth and prosperity by making responsible innovation easier and reducing regulatory uncertainty. This will encourage investment in AI and support its adoption throughout the economy, creating jobs and helping us to do them more efficiently.

To achieve this objective we must act quickly to remove existing barriers to innovation and prevent the emergence of new ones. This will allow AI companies to capitalise on early development successes and achieve long term market advantage.[footnote 73] By acting now, we can give UK innovators a headstart in the global race to convert the potential of AI into long term advantages for the UK, maximising the economic and social value of these technologies and strengthening our current position as a world leader in AI.[footnote 74]

Increase public trust in AI by addressing risks and protecting our fundamental values.

Trust is a critical driver for AI adoption.[footnote 75] If people do not trust AI, they will be reluctant to use it. Such reluctance can reduce demand for AI products and hinder innovation. Therefore we must demonstrate that our regulatory framework (described in section 3.2) effectively addresses AI risks.

Strengthen the UK's position as a global leader in AI. The development of AI technologies can address some of the most pressing global challenges, from climate change to future pandemics. There is also growing international recognition that AI requires new regulatory responses to guide responsible innovation.

The UK can play a central role in the global conversation by shaping international governance and regulation to maximise opportunities and build trust in the technology, while mitigating potential cross-border risks and protecting our democratic values. There is also an important leadership role for the UK in the development of the global AI assurance industry,[footnote 76] including in auditing and safety.

We will ensure that the UK remains attractive to innovators and investors by promoting interoperability with other regulatory approaches and minimising cross-border frictions. We will work closely with global partners through multilateral and bilateral engagements to learn from, influence and adapt as international and domestic approaches to AI regulation continue to emerge (see part 6).

35. Government is taking wider action to ensure the UK retains its status as a global leader in AI, for example by taking forward Sir Patrick Vallance's recommendation relating to intellectual property law and generative AI.[footnote 77] This will ensure we keep the right balance between protecting rights holders and our thriving creative industries, while supporting AI developers to access the data they need.

3.2 The proposed regulatory framework

37. We have identified the essential characteristics of our regulatory regime. Our framework will be pro-innovation, proportionate, trustworthy, adaptable, clear and collaborative.[footnote 79]

Pro-innovation: enabling rather than stifling responsible innovation.

Proportionate: avoiding unnecessary or disproportionate burdens for businesses and regulators.

Trustworthy: addressing real risks and fostering public trust in AI in order to promote and encourage its uptake.

Adaptable: enabling us to adapt quickly and effectively to keep pace with emergent opportunities and risks as AI technologies evolve.

Clear: making it easy for actors in the AI life cycle, including businesses using AI, to know what the rules are, who they apply to, who enforces them, and how to comply with them.

Collaborative: encouraging government, regulators, and industry to work together to facilitate AI innovation, build trust and ensure that the voice of the public is heard and considered.

38. The framework, built around the 4 key elements below, is designed to empower our existing regulators and promote coherence across the regulatory landscape. The 4 key elements are:

Defining AI based on its unique characteristics to support regulator coordination (section 3.2.1).

Adopting a context-specific approach (section 3.2.2).

Providing a set of cross-sectoral principles to guide regulator responses to AI risks and opportunities(section 3.2.3).

The principles clarify government's expectations for responsible AI and describe good governance at all stages of the AI life cycle.

The application of the principles will initially be at the discretion of the regulators, allowing prioritisation according to the needs of their sectors.

Following this initial non-statutory period of implementation, and when parliamentary time allows, we anticipate introducing a statutory duty requiring regulators to have due regard to the principles.

Delivering new central functions to support regulators to deliver the AI regulatory framework, maximising the benefits of an iterative approach and ensuring that the framework is coherent (section 3.2.4).

3.2.1 Defining Artificial Intelligence

39. To regulate AI effectively, and to support the clarity of our proposed framework, we need a common understanding of what is meant by 'artificial intelligence'. There is no general definition of AI that enjoys widespread consensus.[footnote 80] That is why we have defined AI by reference to the 2 characteristics that generate the need for a bespoke regulatory response.

The 'adaptivity' of AI can make it difficult to explain the intent or logic of the system's outcomes:

AI systems are 'trained' - once or continually - and operate by inferring patterns and connections in data which are often not easily discernible to humans.

Through such training, AI systems often develop the ability to perform new forms of inference not directly envisioned by their human programmers.

The 'autonomy' of AI can make it difficult to assign responsibility for outcomes:

Some AI systems can make decisions without the express intent or ongoing control of a human.

40. The combination of adaptivity and autonomy can make it difficult to explain, predict, or control the outputs of an AI system, or the underlying logic by which they are generated. It can also be challenging to allocate responsibility for the system's operation and outputs. For regulatory purposes, this has potentially serious implications, particularly when decisions are made relating to significant matters, like an individual's health, or where there is an expectation that a decision should be justifiable in easily understood terms, like a legal ruling.

41. By defining AI with reference to these functional capabilities and designing our approach to address the challenges created by these characteristics, we future-proof our framework against unanticipated new technologies that are autonomous and adaptive. Because we are not creating blanket new rules for specific technologies or applications of AI, like facial recognition or LLMs, we do not need to use rigid legal definitions. Our use of these defining characteristics was widely supported in responses to our policy paper,[footnote 81] as rigid definitions can quickly become outdated and restrictive with the rapid evolution of AI.[footnote 82] We will, however, retain the ability to adapt our approach to defining AI if necessary, alongside the ongoing monitoring and iteration of the wider regulatory framework.

42. Below, we provide some illustrative examples of AI systems to demonstrate their autonomous and adaptive characteristics. While many aspects of the technologies described in these case studies will be covered by existing law, they illustrate how AI-specific characteristics introduce novel risks and regulatory implications.

Figure 1: Illustration of our strategy for regulating AI

Case study 3.1: Natural language processing in customer service chatbots

Adaptivity: Provides responses to real-time customer messages, having been trained on huge datasets to identify statistical patterns in ordinary human speech, potentially increasing personalisation over time as the system learns from each new experience.

Autonomy: Generates a human-like output based on the customer's text input, to answer queries, help customers find products and services, or send targeted updates. Operates with little need for human oversight or intervention.

Illustrative AI-related regulatory implication: Unintentional inclusion of inaccurate or misleading information in training data, producing harmful instructions or convincingly spreading misinformation.

Case study 3.2: Automated healthcare triage systems

Adaptivity: Predicts patient conditions based on the pathology, treatment and risk factors associated with health conditions from the analysis of medical datasets, patient records and real-time health data.

Autonomy: Generates information about the likely causes of a patient's symptoms and recommends potential interventions and treatments, either to a medical professional or straight to a patient.

Illustrative AI-related regulatory implication: Unclear liability for an AI triage system that provides incorrect medical advice, leading to negative health outcomes for a patient and affecting the patient's ability to obtain redress.

Case study 3.3: Text-to-image generators

Adaptivity: Uses large amounts of online content to learn how to create rich, highly specific images on the basis of a short text prompt.

Autonomy: Based on text input, these systems generate images that mimic the qualities of human-created art, with no ongoing oversight from the user.

Illustrative AI-related regulatory implication: Reproduction of biases or stereotyping in training data, leading to offensive language or content.

43. Industry, regulators, and civil society responded positively to our proposed definition, recognising that it supports our context-based and flexible approach to AI regulation. We will monitor how regulators interpret and apply adaptivity and autonomy when formulating domain-specific definitions of AI. Government will support coordination between regulators when we see potential for better alignment between their interpretations and use of our defining characteristics.

44. Active and collaborative horizon scanning will ensure that we can identify developments and emerging trends, and adapt our framework accordingly. We will convene industry, academia and other key stakeholders to inform economy-wide horizon scanning activity. This work will build on the activity of individual regulators.

3.2.2 Regulating the use - not the technology

45. Our framework is context-specific.[footnote 83] We will not assign rules or risk levels to entire sectors or technologies. Instead, we will regulate based on the outcomes AI is likely to generate in particular applications. For example, it would not be proportionate or effective to classify all applications of AI in critical infrastructure as high risk. Some uses of AI in critical infrastructure, like the identification of superficial scratches on machinery, can be relatively low risk. Similarly, an AI-powered chatbot used to triage customer service requests for an online clothing retailer should not be regulated in the same way as a similar application used as part of a medical diagnostic process.

46. A context-specific approach allows regulators to weigh the risks of using AI against the costs of missing opportunities to do so.[footnote 84] Regulators told us that AI risk assessments should include the failure to exploit AI capabilities. For example, there can be a significant opportunity cost related to not having access to AI in safety-critical operations, from heavy industry,[footnote 85] to personal healthcare (see box 1.1). Sensitivity to context will allow the framework to respond to the level of risk in a proportionate manner and avoid stifling innovation or missing opportunities to capitalise on the social benefits made available by AI.

47. To best achieve this context-specificity we will empower existing UK regulators to apply the cross-cutting principles. Regulators are best placed to conduct detailed risk analysis and enforcement activities within their areas of expertise. Creating a new AI-specific, cross-sector regulator would introduce complexity and confusion, undermining and likely conflicting with the work of our existing expert regulators.

3.2.3 A principles-based approach

48. Existing regulators will be expected to implement the framework underpinned by 5 values-focused cross-sectoral principles:

Safety, security and robustness

Appropriate transparency and explainability

Accountability and governance

Contestability and redress

49. The principles set out the key elements of responsible AI design, development and use, and will help guide businesses. Regulators will lead the implementation of the framework, for example by issuing guidance on best practice for adherence to these principles.

50. Regulators will be expected to apply the principles proportionately to address the risks posed by AI within their remits, in accordance with existing laws and regulations. In this way, the principles will complement existing regulation, increase clarity, and reduce friction for businesses operating across regulatory remits.

51. A principles-based approach allows the framework to be agile and proportionate. It is in line with the Plan for Digital Regulation,[footnote 86] the findings from the independent Taskforce on Innovation, Growth and Regulatory Reform,[footnote 87] the Regulatory Horizons Council's Closing the Gap report on implementing innovation-friendly regulation,[footnote 88] and Sir Patrick Vallance's Regulation for Innovation report.[footnote 89]

52. Since publishing the AI regulation policy paper,[footnote 90] we have updated and strengthened the principles. We have:

Reflected stakeholder feedback by expanding on concepts such as robustness and governance. We have also considered the results of public engagement research that highlighted an expectation for principles such as transparency, fairness and accountability to be included within an AI governance framework.[footnote 91]

Merged the safety principle with security and robustness, given the significant overlap between these concepts.

Better reflected concepts of accountability and responsibility.

Refined each principle's definition and rationale.

Principle: Safety, security, robustness
Definition and explanation
AI systems should function in a robust, secure and safe way throughout the AI life cycle, and risks should be continually identified, assessed and managed.

Regulators may need to introduce measures for regulated entities to ensure that AI systems are technically secure and function reliably as intended throughout their entire life cycle.
Rationale for the principle
Additionally, AI systems should be technically secure and should reliably function as intended and described. System developers should be aware of the specific security threats that could apply at different stages of the AI life cycle and embed resilience to these threats into their systems. Other actors should remain vigilant of security issues when they interact with an AI system. We anticipate that regulators may wish to consider the National Cyber Security Centre ( NCSC) principles for securing machine learning models when assessing whether AI actors are adequately prioritising security.[footnote 92]

When applying this principle, regulators will need to consider providing guidance in a way that is coordinated and coherent with the activities of other regulators. Regulators' implementation of this principle may require the corresponding AI life cycle actors to regularly test or carry out due diligence on the functioning, resilience and security of a system.[footnote 93] Regulators may also need to consider technical standards addressing safety, robustness and security to benchmark the safe and robust performance of AI systems and to provide AI life cycle actors with guidance for implementing this principle in their remit.

Principle: Appropriate transparency and explainability
Definition and explanation
AI systems should be appropriately transparent and explainable.

An appropriate level of transparency and explainability will mean that regulators have sufficient information about AI systems and their associated inputs and outputs to give meaningful effect to the other principles (for example, to identify accountability). An appropriate degree of transparency and explainability should be proportionate to the risk(s) presented by an AI system.
Rationale for the principle
Transparency can increase public trust,[footnote 96] which has been shown to be a significant driver of AI adoption.[footnote 97]

At a technical level, the explainability of AI systems remains an important research and development challenge. The logic and decision-making in AI systems cannot always be meaningfully explained in a way that is intelligible to humans, although in many settings this poses no substantial risk. It is also true that in some cases, a decision made by AI may perform no worse on explainability than a comparable decision made by a human.[footnote 98] Future developments of the technology may pose additional challenges to achieving explainability. AI systems should display levels of explainability that are appropriate to their context, including the level of risk and consideration of what is achievable given the state of the art.

Principle: Fairness
Definition and explanation
AI systems should not undermine the legal rights of individuals or organisations, discriminate unfairly against individuals or create unfair market outcomes. Actors involved in all stages of the AI life cycle should consider definitions of fairness that are appropriate to a system's use, outcomes and the application of relevant law.

Fairness is a concept embedded across many areas of law and regulation, including equality and human rights, data protection, consumer and competition law, public and common law, and rules protecting vulnerable people.

Regulators will need to ensure that AI systems in their domain are designed, deployed and used considering such descriptions of fairness. Where concepts of fairness are relevant in a broad range of intersecting regulatory domains, we anticipate that developing joint guidance will be a priority for regulators.
Rationale for the principle
In certain circumstances, AI can have a significant impact on people's lives, including insurance offers, credit scores, and recruitment outcomes. AI-enabled decisions with high impact outcomes should not be arbitrary and should be justifiable.

In order to ensure a proportionate and context-specific approach regulators should be able to describe and illustrate what fairness means within their sectors and domains, and consult with other regulators where multiple remits are engaged by a specific use case. We expect that regulators' interpretations of fairness will include consideration of compliance with relevant law and regulation, including:

1. AI systems should not produce discriminatory outcomes, such as those which contravene the Equality Act 2010 or the Human Rights Act 1998. Use of AI by public authorities should comply with the additional duties placed on them by legislation (such as the Public Sector Equality Duty).

2. Processing of personal data involved in the design, training, and use of AI systems should be compliant with requirements under the UK General Data Protection Regulation ( GDPR), the Data Protection Act 2018,[footnote 100] particularly around fair processing and solely automated decision-making.

3. Consumer and competition law, including rules protecting vulnerable consumers and individuals.[footnote 101]

4. Relevant sector-specific fairness requirements, such as the Financial Conduct Authority ( FCA) Handbook.

Principle: Accountability and governance
Definition and explanation
Governance measures should be in place to ensure effective oversight of the supply and use of AI systems, with clear lines of accountability established across the AI life cycle.

AI life cycle actors should take steps to consider, incorporate and adhere to the principles and introduce measures necessary for the effective implementation of the principles at all stages of the AI life cycle.

Regulators will need to look for ways to ensure that clear expectations for regulatory compliance and good practice are placed on appropriate actors in the AI supply chain, and may need to encourage the use of governance procedures that reliably ensure these expectations are met.

Regulator guidance on this principle should reflect that 'accountability' refers to the expectation that organisations or individuals will adopt appropriate measures to ensure the proper functioning, throughout their life cycle, of the AI systems that they research, design, develop, train, operate, deploy, or otherwise use.
Rationale for the principle
Doing so for actors in the AI life cycle is difficult, given the complexity of AI supply chains, as well as the adaptivity, autonomy and opacity of AI systems. In some cases, technical standards can provide useful guidance on good practices for AI governance.[footnote 103] Assurance techniques like impact assessments can help to identify potential risks early in the development life cycle, enabling their mitigation through appropriate safeguards and governance mechanisms.

Regulatory guidance should also reflect the responsibilities such life cycle actors have for demonstrating proper accountability and governance (for example, by providing documentation on key decisions throughout the AI system life cycle, conducting impact assessments or allowing audits where appropriate).

Principle: Contestability and redress
Definition and explanation
Where appropriate, users, impacted third parties and actors in the AI life cycle should be able to contest an AI decision or outcome that is harmful or creates material risk of harm.

Regulators will be expected to clarify existing routes to contestability and redress, and implement proportionate measures to ensure that the outcomes of AI use are contestable where appropriate.

We would also expect regulators to encourage and guide regulated entities to make clear routes (including informal channels) easily available and accessible, so affected parties can contest harmful AI outcomes or decisions as needed.
Rationale for the principle
The use of AI technologies can result in different types of harm and can have a material impact on people's lives. AI systems' outcomes may introduce risks such as the reproduction of biases or safety concerns.

People and organisations should be able to contest outcomes where existing rights have been violated or they have been harmed.

It will be important for regulators to provide clear guidance on this principle so that AI life cycle actors can implement it in practice. This should include clarifying that appropriate transparency and explainability are relevant to good implementation of this contestability and redress principle.

The UK's initial non-statutory approach will not create new rights or new routes to redress at this stage.

53. We anticipate that regulators will need to issue guidance on the principles or update existing guidance to provide clarity to business. Regulators may also publish joint guidance on one or more of the principles, focused on AI use cases that cross multiple regulatory remits. We are keen to work with regulators and industry to understand the best approach to providing guidance. We expect that practical guidance will support actors in the AI life cycle to adhere to the principles and embed them into their technical and operational business processes. Regulators may also use alternative measures and introduce other tools or resources, in addition to issuing guidance, within their existing remits and powers to implement the principles.

54. Government will monitor the overall effectiveness of the principles and the wider impact of the framework.[footnote 104] This will include working with regulators to understand how the principles are being applied and whether the framework is adequately supporting innovation.
Consultation questions:
1. Do you agree that requiring organisations to make it clear when they are using AI would improve transparency?

2. Are there other measures we could require of organisations to improve AI transparency?

3. Do you agree that current routes to contest or get redress for AI-related harms are adequate?

4. How could current routes to contest or seek redress for AI-related harms be improved, if at all?

5. Do you agree that, when implemented effectively, the revised cross-sectoral principles will cover the risks posed by AI technologies?

6. What, if anything, is missing from the revised principles?

Case study 3.4: Explainable AI in practice

The level of explainability needed from an AI system is highly specific to its context, including the extent to which an application is safety-critical. The level and type of explainability required will likely vary depending on whether the intended audience of the explanation is a regulator, technical expert, or lay person.

For example, a technical expert designing self-driving vehicles would need to understand the system's decision-making capabilities to test, assess and refine them. In the same context, a lay person may need to understand the decision-making process only in order to use the vehicle safely. If the vehicle malfunctioned and caused a harmful outcome,[footnote 105] a regulator may need information about how the system operates in order to allocate responsibility - similar to the level of explainability currently needed to hold human drivers accountable.

While AI explainability remains a technical challenge and an area of active research, regulators are already conducting work to address it. In 2021, the ICO and the Alan Turing Institute issued co-developed guidance on explaining decisions made with AI,[footnote 106] giving organisations practical advice to help explain the processes, services and decisions delivered or assisted by AI to the individuals affected by them.

The audience for an explanation of AI's outcomes will often be a regulator, who may require a higher standard of explainability depending on the risks represented by an application. The 's Project Glass Box work is addressing the challenge of setting medical device requirements that take into account adequate consideration of human interpretability and its consequences for the safety and effectiveness for AI used in medical devices.[footnote 107]

Case study 3.5: What the principles mean for businesses in practice

A fictional company, 'Good AI Recruitment Limited', provides recruitment services that use a range of AI systems to accelerate the recruitment process, including a service that automatically shortlists candidates based on application forms. While potentially useful, such systems may discriminate against certain groups that have historically not been selected for certain positions.

After the implementation of the UK's new AI regulatory framework, the Equality and Human Rights Commission () and the Information Commissioner Office ( ICO) will be supported and encouraged to work with the Employment Agency Standards Inspectorate ( EASI) and other regulators and organisations in the employment sector to issue joint guidance. The joint guidance could address the cross-cutting principles relating to fairness, appropriate transparency and explainability, and contestability and redress in the context of the use of AI systems in recruitment or employment. Such joint guidance could, for example, make things clearer and easier for Good AI Recruitment Limited by:

1. Clarifying the type of information businesses should provide when implementing such systems

2. Identifying appropriate supply chain management processes such as due diligence or AI impact assessments

3. Suggesting proportionate measures for bias detection, mitigation and monitoring

4. Providing suggestions for the provision of contestability and redress routes.

Good AI Recruitment Limited would also be able to apply a variety of tools for trustworthy AI, such as technical standards, that would supplement regulatory guidance and other measures promoted by regulators. In their published guidance regulators could, where appropriate, refer businesses to existing technical standards on transparency (such as IEEE 7001-2021), as well as standards on bias mitigation (such as ).

By following this guidance Good AI Recruitment Limited would be able to develop and deploy their services responsibly.

3.2.4 Our preferred model for applying the principles

55. Initially, the principles will be issued by government on a non-statutory basis and applied by regulators within their remits. We will support regulators to apply the principles using the powers and resources available to them. This initial period of implementation will provide a valuable opportunity to ensure that the principles are effective and that the wider framework is supporting innovation while addressing risks appropriately.

56. While industry has strongly supported non-statutory measures in the first instance, favouring flexibility and fewer burdens, some businesses and regulators have suggested that government should go beyond a non-statutory approach to ensure the principles have the desired impact.[footnote 108] Some regulators have also expressed concerns that they lack the statutory basis to consider the application of the principles. We are committed to an approach that leverages collaboration with our expert regulators but we agree that we may need to intervene further to ensure that our framework is effective.

57. Following a period of non-statutory implementation, and when parliamentary time allows, we anticipate that we will want to strengthen and clarify regulators' mandates by introducing a new duty requiring them to have due regard to the principles. Such a duty would give a clear signal that we expect regulators to act and support coherence across the regulatory landscape, ensuring that the framework displays the characteristics that we have identified.[footnote 109] One of the strengths of this approach is that regulators would still be able to exercise discretion and expert judgement regarding the relevance of each principle to their individual domains.

58. A duty would ensure that regulators retain the ability to exercise judgement when applying the principles in particular contexts - benefiting from some of the flexibility expected through non-statutory implementation. For example, while the duty to have due regard would require regulators to demonstrate that they had taken account of the principles, it may be the case that not every regulator will need to introduce measures to implement every principle. In having due regard to a particular principle, a regulator may exercise their expert judgement and determine that their sector or domain does not require action to be taken. The introduction of the duty will, however, give regulators a clear mandate and incentive to apply the principles where relevant to their sectors or domains.

59. If our monitoring of the effectiveness of the initial, non-statutory framework suggests that a statutory duty is unnecessary, we would not introduce it. Similarly, we will monitor whether particular principles cannot be, or are not being, applied in certain circumstances or by specific regulators because of the interpretation of existing legal requirements or because of technical constraints. Such circumstances may require broader legislative changes. Should we decide there is a need for statutory measures, we will work with regulators to review the interaction of our principles with their existing duties and powers.
Consultation questions:
7. Do you agree that introducing a statutory duty on regulators to have due regard to the principles would clarify and strengthen regulators' mandates to implement our principles while retaining a flexible approach to implementation?

8. Is there an alternative statutory intervention that would be more effective?

3.2.5 The role of individual regulators in applying the principles

60. In some sectors, principles for AI governance will already exist and may even go further than the cross-cutting principles we propose. Our framework gives sectors the ability to develop and apply more specific principles to suit their own domains, where government or regulators identify these are needed.

62. Not all principles will be equally relevant in all contexts and sometimes two or more principles may come into conflict. For example, it may be difficult to assess the fairness of an algorithm's outputs without access to sensitive personal data about the subjects of the processing. Regulators will need to use their expertise and judgement to prioritise and apply the principles in such cases, sharing information where possible with government and other regulators about how they are assessing the relevance of each principle. This collaboration between regulators and government will allow the framework to be adapted to ensure it is practical, coherent and supporting innovation.

63. In implementing the new regulatory framework we expect that regulators will:

Assess the cross-cutting principles and apply them to AI use cases that fall within their remit.

Issue relevant guidance on how the principles interact with existing legislation to support industry to apply the principles. Such guidance should also explain and illustrate what compliance looks like.

Support businesses operating within the remits of multiple regulators by collaborating and producing clear and consistent guidance, including joint guidance where appropriate.

64. Regulators will need to monitor and evaluate their own implementation of the framework and their own effectiveness at regulating AI within their remits. We understand that there may be AI-related risks that do not clearly fall within the remits of the UK's existing regulators.[footnote 110] Not every new AI-related risk will require a regulatory response and there is a growing ecosystem of tools for trustworthy AI that can support the application of the cross-cutting principles. These are described further in part 4.

65. Where prioritised risks fall within a gap in the legal landscape, regulators will need to collaborate with government to identify potential actions. This may include identifying iterations to the framework such as changes to regulators' remits, updates to the Regulators' Code,[footnote 111] or additional legislative interventions. Our approach benefits from our strong sovereign parliamentary system, which reliably allows for the introduction of targeted and proportionate measures in response to emerging issues, including by adapting existing legislation if necessary.[footnote 112]

66. The Sir Patrick Vallance review has highlighted that rushed attempts to regulate AI too early would risk stifling innovation.[footnote 113] Our approach aligns with this perspective. We recognise the need to build a stronger evidence base before making decisions on statutory interventions. In doing so, we will ensure that we strike the right balance between retaining flexibility in our iterative approach and providing clarity to businesses. As detailed in section 3.3.1, we will deliver a range of central functions, including horizon scanning and risk monitoring, to identify and respond to situations where prioritised risks are not adequately covered by the framework, or where gaps between regulators' remits are negatively impacting innovation.

Case study 3.6: Responding to regulatory policy challenges - self-driving vehicles

Some aspects of a new AI use case may sit outside regulators' existing remits, meaning they do not have a mandate to address specific harms or support a new product to enter the market.

The advent of self-driving vehicles highlighted such a regulatory and policy challenge. Where sophisticated AI-enabled software is capable of performing the designated driving task, existing regulatory structures - where responsibility for road safety is achieved by licensing human drivers - are not fit for purpose. This creates uncertainty regarding the development and deployment of self-driving vehicles that cannot be addressed by regulators alone.

To achieve the government's ambition to 'make the UK one of the best places in the world to develop and deploy self-driving vehicles technology',[footnote 114] manufacturers need clarity about the regulatory landscape they are operating in and the general public needs to have confidence in the safety, fairness and trustworthiness of these vehicles.

The government published its Connected & Automated Mobility 2025 report[footnote 115] to address this challenge, describing how the ecosystem could be adapted to spur innovation and secure the economic and social benefits of this technology.

The work of the UK's Centre for Connected and Autonomous Vehicles is an example of government acting to identify regulatory gaps, develop policy and build UK capabilities. A central monitoring and evaluation function, described below, will identify and assess gaps in the regulatory ecosystem that could stifle AI innovation so that government can take action to address them.

3.2.6 Guidance to regulators on applying the principles

67. The proposed regulatory framework is dependent upon the implementation of the principles by our expert regulators. This regulator-led approach has received broad support from across industry, with stakeholders acknowledging the importance of the sector-specific expertise held by individual regulators. We expect regulators to collaborate proactively to achieve the best outcomes for the economy and society. We will work with regulators to monitor the wider framework and ensure that this collaborative approach to implementation is effective. If improvements are needed, including interventions to drive stronger collaboration across regulators, we will take further action.

68. Our engagement with regulators and industry highlighted the need for central government to support regulators. We will work with regulators to develop guidance that helps them implement the principles in a way that aligns with our expectations for how the framework should operate. Existing legal frameworks already mandate and guide regulators' actions. For example, nearly all regulators are bound by the Regulators' Code[footnote 116] and all regulators - as public bodies - are required to comply with the Human Rights Act.[footnote 117] Our proposed guidance to regulators will seek to ensure that when applying the principles, regulators are supported and encouraged to:

Adopt a proportionate approach that promotes growth and innovation by focusing on the risks that AI poses in a particular context.

Consider proportionate measures to address prioritised risks, taking into account cross-cutting risk assessments undertaken by, or on behalf of, government.

Design, implement and enforce appropriate regulatory requirements and, where possible, integrate delivery of the principles into existing monitoring, investigation and enforcement processes.

Develop joint guidance, where appropriate, to support industry compliance with the principles and relevant regulatory requirements.

Consider how tools for trustworthy AI like assurance techniques and technical standards can support regulatory compliance.

Engage proactively and collaboratively with government's monitoring and evaluation of the framework.

Case study 3.7: What this means for businesses

Following implementation of the UK's new pro-innovation framework to regulate AI, we could expect to see joint guidance produced collaboratively by the Information Commissioner's Office ( ICO), Equality and Human Rights Commission (), Financial Conduct Authority ( FCA) and other relevant regulatory authorities. This would provide greater clarity on the regulatory requirements relevant to AI as well as guidance on how to satisfy those requirements in the context of insurance and consumer-facing financial services.

Under the proposed regulatory framework, AI Fairness Insurance Limited could be supported by new or updated guidance issued by regulators to address the AI regulatory principles. The company may also be able to follow joint regulatory guidance, issued as a result of collaboration between regulators, and use a set of tools provided by regulators, such as template risk assessments and transparency measures, and relevant technical standards (for example, international standards for transparency and bias mitigation). The collaboration between regulators and focus on practical implementation measures will guide the responsible deployment of AI Fairness Insurance Limited's AI product by making it easier for the company to navigate the regulatory landscape and address specific risks such as discrimination.

69. Further details about the implementation of the regulatory framework will be provided through an AI regulation roadmap which will be published alongside the government response to the consultation on this white paper.

3.3.1 New central functions to support the framework

70. Government has a responsibility to make sure the regulatory framework operates proportionately and supports innovation. Feedback to our proposals from businesses has been clear that the current patchwork of regulation, with relatively little in the way of central coordination or oversight, will create a growing barrier to innovation if left unaddressed. Responses from over 130 organisations and individuals to our 2022 policy paper highlighted the need for a greater level of monitoring and coordination to achieve the coherence and improved clarity we need to support innovation. Businesses, particularly small to medium sized enterprises, noted that regulatory coordination could improve business certainty and investment, resulting in more and better jobs in the sector.

71. Government therefore intends to put mechanisms in place to coordinate, monitor and adapt the framework as a whole. Further detail on these functions is set out below. Enhanced monitoring activity will allow us to take a structured approach to gathering feedback from industry on the impact of our regime as it is introduced. These mechanisms will supplement and support the work of regulators, without undermining their independence. Equally, such mechanisms are not intended to duplicate existing activities.

72. Delivering some functions centrally provides government with an overarching view of how the framework is working, where it is effective and where it may need improving. A central suite of functions will also facilitate collaboration by bringing together a wide range of interested parties, including regulators, international partners, industry, civil society organisations such as trade unions and advocacy groups, academia and the general public. Our engagement with these groups has highlighted the need for our proposed central functions. We will continue to convene a wide range of stakeholders to ensure that we hear the full spectrum of viewpoints. This breadth of engagement and collaboration will be integral to government's ability to monitor and improve the framework. The functions will identify and support opportunities for further coordination between regulators, resulting in greater clarity for businesses and stronger consumer trust.

73. We have identified a set of functions that will drive regulatory coherence and support regulators to implement the pro-innovation approach that we have outlined. These functions have been informed by our discussions with industry, research organisations, and regulators following the publication of the AI policy paper.

Box 3.1: Functions required to support implementation of the framework

Monitoring, assessment and feedback[footnote 118]
Activities Rationale Support coherent implementation of the principles Activities Rationale Cross-sectoral risk assessment Activities Rationale Support for innovators (including testbeds and sandboxes as detailed in section 3.3.4) Activities Rationale Education and awareness Activities Rationale Horizon scanning Activities Rationale Ensure interoperability with international regulatory frameworks Activities Rationale Consultation questions: [footnote 119]
9. Do you agree that the functions outlined in Box 3.1 would benefit our AI regulation framework if delivered centrally?

10. What, if anything, is missing from the central functions?

11. Do you know of any existing organisations who should deliver one or more of our proposed central functions?

12. Are there additional activities that would help businesses confidently innovate and use AI technologies?

12.1. If so, should these activities be delivered by government, regulators or a different organisation?

13. Are there additional activities that would help individuals and consumers confidently use AI technologies?

13.1. If so, should these activities be delivered by government, regulators or a different organisation?

14. How can we avoid overlapping, duplicative or contradictory guidance on AI issued by different regulators?

Box 3.2: Supporting coherence in risk assessment

Many AI risks do not fall neatly into the remit of one individual regulator and they could go unaddressed if not monitored at a cross-sector level. A central, cross-economy risk function will also enable government to monitor future risks in a rigorous, coherent and balanced way. This will include 'high impact but low probability' risks such as existential risks posed by artificial general intelligence or AI biosecurity risks.

A pro-innovation approach to regulation involves tolerating a certain degree of risk rather than intervening in all cases. Government needs the ability to assess and prioritise AI risks, ensuring that any intervention is proportionate and consistent with levels of risk mitigation activity elsewhere across the economy or AI life cycle.
What?
Establishing a central risk function will bring coherence to the way regulators and industry think about AI risk. It will also foster collaboration between government, regulators, industry and civil society to provide clarity for businesses managing AI risk across sectors.
How?
The central risk function will identify, assess, prioritise and monitor cross-cutting AI risks that may require government intervention.

The central risk function will bring together cutting-edge knowledge from industry, regulators, academia and civil society - including skilled computer scientists with a deep technical understanding of AI.

Given the importance of risk management expertise, we will seek inspiration and learning from sectors where operational risk management is highly developed. This will include looking for examples of how failures and near misses can be recorded and used to inform good practice.

Regulators will have a key role in designing the central risk framework and ensuring alignment with their existing practices. Where a risk that has been prioritised for intervention falls outside of any existing regulator's remit, the central risk function will identify measures that could be taken to address the gap (for example, updates to regulatory remits). The central risk function will also support smaller regulators that lack technical AI expertise to better understand AI risks.

Figure 2: Central risks function activities

Box 3.3: How a central monitoring and evaluation (M&E) function enables a proportionate, adaptable approach
What?
We will need to monitor the implementation of the framework closely to make sure that it is working as designed. We will monitor the regime to ensure it is pro-innovation, proportionate, adaptable, trustworthy, clear and collaborative - our desired characteristics.

Government's ability to access reliable, comprehensive data and insights for the purposes of monitoring the AI regulatory framework will be closely related to our work raising awareness and educating businesses and consumers on AI-related issues. It is important for our M&E data to be drawn from a wide range of sources, reflecting the full spectrum of views and including seldom heard voices from the general public. Raising awareness and educating stakeholder groups will help to ensure that the broader conversation is inclusive, informed and rigorous.

We will develop and monitor metrics that demonstrate whether the framework is working as intended. For example, the central M&E function will look at the effectiveness of the framework in mitigating unacceptable risks and assess whether the implementation of the principles by regulators is disproportionate or negatively affecting innovation.
How?
Insights from the M&E function will contribute to the adaptability of our framework by enabling government to identify opportunities for improvement so we can benefit fully from the flexibility we have built into our approach. Such iteration could include removing or amending existing regulation as well as updating the AI regulatory framework itself.

The range, sources and quality of the data that informs our monitoring and evaluation of the framework will be critical.

The M&E function will identify the metrics and data sources to help us measure how well the regime is working, both in terms of the framework's ability to mitigate risk but also to ensure that it is supporting innovation. It will bring together a wide range of views including industry, civil society groups and academia.
Consultation questions:
Crucially, we will work with regulators to identify how their work - including data collected from their own regulatory activities - can support our central M&E function in order to ensure the best outcomes for the whole economy.

15. Do you agree with our overall approach to monitoring and evaluation?

16. What is the best way to measure the impact of our framework?

17. Do you agree that our approach strikes the right balance between supporting AI innovation; addressing known, prioritised risks; and future-proofing the AI regulation framework?

75. Government will initially be responsible for delivering the central functions described above, working in partnership with regulators and other key actors in the AI ecosystem to leverage existing activities where possible. This is aligned with our overall iterative approach and enables system-wide review of the framework. We recognise that there may be value in a more independent delivery of the central functions in the longer term.

76. Where relevant activities are already undertaken by organisations either within or outside of government, the primary role of the central functions will be to leverage these activities and assess their effectiveness. Where this is not the case - for example, where new bespoke capabilities are needed to monitor and evaluate the operation of the framework as a whole - these functions will initially be established in government.

Case study 3.8: Building on a strong foundation of regulatory coordination

The growth of digital technologies requires regulators to coordinate and act cohesively. The Digital Regulation Cooperation Forum ( DRCF) has published its vision for a joined-up approach to digital regulation. It conducts cross-regulator horizon scanning for future technology and has issued detailed discussion papers on the benefits, harms and auditing of algorithms.

Regulators are also exploring ways to provide simpler 'shop fronts' for those they regulate, with the NHS AI and Digital Regulations Service offer already robustly tested with end users and now widely available.[footnote 123] DRCF regulators have a multi-agency advice service for digital innovators pilot underway, supported by government's Regulators' Pioneer Fund,[footnote 124] which aims to make it easier for firms operating across digital regulatory boundaries to do business.

Existing regulatory forums may need to be supplemented or adapted to successfully implement the cross-cutting principles. We will work in partnership with existing bodies as well as industry to improve and enhance regulatory coordination.

77. We are deliberately taking an iterative approach to the delivery of the regulatory framework and we anticipate that the model for providing the central functions will develop over time. We will identify where existing structures may need to be supplemented or adapted. In particular, we are focused on understanding:

Whether existing regulatory forums could be expanded to include the full range of regulators involved in the regulation of AI or whether additional mechanisms are needed.

What additional expertise government may need to support the implementation and monitoring of the principles, including the potential role that could be played by established advisory bodies.

The most effective way to convene input from across industry and consumers to ensure a broad range of opinions.

Consultation questions:
79. As the regulatory framework evolves and we develop a clearer understanding of the system-level functions that are needed, we will review the operational model outlined above. In particular, we will consider if a government unit is the most appropriate mechanism for delivering the central functions in the longer term or if an independent body would be more effective.

18. Do you agree that regulators are best placed to apply the principles and government is best placed to provide oversight and deliver central functions?

3.3.2 Government's role in addressing accountability across the life cycle

81. However, AI supply chains can be complex and opaque, making effective governance of AI and supply chain risk management difficult. Inappropriate allocation of AI risk, liability, and responsibility for AI governance throughout the AI life cycle and within AI supply chains could impact negatively on innovation. For example, inappropriate allocation of liability to a business using, but not developing, AI could stifle AI adoption. Similarly, allocating too much responsibility to businesses developing foundation models, on the grounds that these models could be used by third parties in a range of contexts, would hamper innovation.
Consultation questions:
82. We recognise the need to consider which actors should be responsible and liable for complying with the principles, which may not be the same actors who bear the burden under current legal frameworks. For example, data protection law differentiates between data controllers and data processors. Similarly, product safety laws include the concepts of producers and distributors. In the context of those specific legal frameworks, liability for compliance with various existing legal obligations is allocated by law to those identified supply chain actors. It is not yet clear how responsibility and liability for demonstrating compliance with the AI regulatory principles will be or should ideally be, allocated to existing supply chain actors within the AI life cycle.

L1. What challenges might arise when regulators apply the principles across different AI applications and systems? How could we address these challenges through our proposed AI regulatory framework?

L2.1 Do you agree that the implementation of our principles through existing legal frameworks will fairly and effectively allocate legal responsibility for AI across the life cycle?

L.2.2. How could it be improved, if at all?

L3. If you work for a business that develops, uses, or sells AI, how do you currently manage AI risk including through the wider supply chain? How could government support effective AI-related risk management?

3.3.3 Foundation models and the regulatory framework

86. Foundation models are an emerging type of general purpose AI that are trained on vast quantities of data and can be adapted to a wide range of tasks. The fast-paced development of foundation models brings novel challenges for governments seeking to regulate AI. Despite high levels of interest in the topic, the research community has not found a consensus on how foundation models work, the risks they pose or even the extent of their capabilities.[footnote 126]

87. Foundation models have been described as paradigm shifting and could have significant impacts on society and the economy.[footnote 127] They can be used for a wide variety of purposes and deployed in many already complex ecosystems. Given the widely acknowledged transformative potential of foundation models, we must give careful attention to how they might interact with our proposed regulatory framework. Our commitment to an adaptable, proportionate approach presents a clear opportunity for the UK to lead the global conversation and set global norms for the future-proof regulation of foundation models.

88. There is a relatively small number of organisations developing foundation models. Some organisations exercise close control over the development and distribution of their foundation models. Other organisations take an open-source approach to the development and distribution of the technology. Open-source models can improve access to the transformational power of foundation models, but can cause harm without adequate guardrails.[footnote 128] The variation in organisational approaches to developing and supplying foundation models introduces a wide range of complexities for the regulation of AI. The potential opacity of foundation models means that it can also be challenging to identify and allocate accountability for outcomes generated by AI systems that rely on or integrate them.

89. Our proposed framework considers the issues raised by foundation models in light of our life cycle accountability analysis, outlined in section 3.3.2 above. Given the small number of organisations supplying foundation models and a proportionately larger number of businesses integrating or otherwise deploying foundation models elsewhere in the AI ecosystem, we recognise the important role of tools for trustworthy AI, including assurance techniques and technical standards.

90. The proposed central functions described in section 3.3.1 will play an important role in informing our approach to regulating foundation models. The central risk function's proactive, rigorous monitoring of risks associated with foundation models and the horizon scanning function's identification of related opportunities will be critical to ensuring that we strike the balance needed as part of our proportionate, pro-innovation regulatory approach. It will be crucial to ensure that the proposed monitoring and evaluation function has access to the technical skills and capabilities needed to assess the impact that our framework has on the opportunities and risks presented by foundation models.

91. We recognise that industry, academia, research organisations and global partners are looking for ways to address the challenges related to the regulation of foundation models.[footnote 129] For example, we know that developers of foundation models are exploring ways to embed alignment theory into their models. This is an important area of research, and government will need to work closely with the AI research community to leverage insights and inform our iteration of the regulatory framework. Our collaborative, adaptable framework will draw on the expertise of those researchers and other stakeholders as we continue to develop policy in this evolving area.

92. The UK is committed to building its capabilities in foundation models. Our Foundation Model Taskforce announced in the Integrated Review Refresh 2023[footnote 130] will support government to build UK capability and ensure the UK harnesses the benefits presented by this emerging technology. Our proposed framework will ensure we create the right regulatory environment as we move to maximise the transformative potential of foundation models.

Case-study 3.9: Life cycle accountability for large language models

Large language models ( LLMs) are a type of foundation model.[footnote 131] The potential of LLMs goes beyond reproducing or translating natural language: LLMs also have the power to write software,[footnote 132] generate stories[footnote 133] through films and virtual reality,[footnote 134] and more.[footnote 135]

LLM s fall within the scope of our regulatory framework as they are autonomous and adaptable.

We are mindful of the rapid technological change in the development of foundation models such as LLMs and the new opportunities that they bring to applications including search engines, medical devices, and financial and legal services. However, LLMs also have limitations, for example, the models are not trained on a sense of truth,[footnote 136] so they can reproduce inconsistent or false outputs that seem highly credible.[footnote 137] Because they can be adapted to a wide variety of tasks downstream within an AI supply chain, any improvements or defects in a foundation model could quickly affect all adapted products.

The wide-reaching impact of LLMs through the AI supply chain - together with their general purpose and potential wide ranging application - means they are unlikely to be directly 'caught' within the remit of any single regulator. This makes effective governance and supply chain risk-management challenging where LLMs are involved. The AI regulatory framework's monitoring and evaluation function will therefore need to assess the impacts of LLMs. The cross-cutting accountability and governance principle will encourage regulators and businesses to find ways to demonstrate accountability and good governance in responsible LLM development and use.
Consultation questions:
However, we are mindful of the rapid rate of advances in the power and application of LLMs, and the potential creation of new or previously unforeseen risks. As such, LLMs will be a core focus of our monitoring and risk assessment functions and we will work with the wider AI community to ensure our adaptive framework is capable of identifying and responding to developments relating to LLM s.

F1. What specific challenges will foundation models such as large language models (LLMs) or open-source models pose for regulators trying to determine legal responsibility for AI outcomes?

F2. Do you agree that measuring compute provides a potential tool that could be considered as part of the governance of foundation models?

F3. Are there other approaches to governing foundation models that would be more effective?

3.3.4 Artificial intelligence sandboxes and testbeds

93. Government is committed to supporting innovators by addressing regulatory challenges that prevent new, cutting-edge products from getting to market. Barriers can be particularly high when a path to market requires interaction with multiple regulators or regulatory guidance is nascent. Sir Patrick Vallance's Digital Report recommends that government works with regulators to develop an AI sandbox to support innovators. At the Budget, government confirmed our commitment to taking forward this recommendation.[footnote 139]

94. The Information Commissioner's Office ( ICO) and the Financial Conduct Authority ( FCA) have already successfully piloted digital sandboxes in their sectors.[footnote 140] The FCA sandbox has worked with over 800 businesses and accelerated their speed to market by an estimated 40% on average.[footnote 141] Sandbox participation has also been found to have significant financial benefits, particularly for smaller organisations.[footnote 142] We have heard from regulators, including those with less experience of taking part in previous initiatives, that they are keen to participate in new AI sandboxes to support their regulated sectors.

95. Regulatory sandboxes and testbeds will play an important role in our proposed regulatory regime. Such initiatives enable government and regulators to:

Support innovators to get novel products and services to market faster, so they can start generating economic and social benefits.

Test how the regulatory framework is operating in practice and illuminate unnecessary barriers to innovation that need to be addressed.

Identify emerging technology and market trends to which our regulatory framework may need to adapt.

96. To deliver an effective sandbox, we would like to understand more deeply what service focus would be most useful to industry. We are considering 4 options:

Single sector, single regulator: support innovators to bring AI products to the market in collaboration with a single regulator, focusing on only one chosen industry sector.[footnote 143]

Multiple sectors, single regulator: support AI innovators in collaboration with a single regulator that is capable of working across multiple industry sectors.[footnote 144]

Single sector, multiple regulator: establish a sandbox that only operates in one industry sector but is capable of supporting AI innovators whose path to market requires interaction with one or more regulators operating in that sector.[footnote 145]

Multiple sectors, multiple regulators: a sandbox capable of operating with one or more regulators in one or more industry sectors to help AI innovators reach their target market. The DRCF is piloting a version of this model.[footnote 146]

97. We intend to focus an initial pilot on a single sector, multiple regulator sandbox. Recognising the importance of AI innovations that have implications in multiple sectors (like generative AI models), we would look to expand this capability to cover multiple industry sectors over time.

98. Initially, we envisage focusing the sandbox on a sector where there is a high degree of AI investment, industry demand for a sandbox, and appetite for improved collaboration between regulators to help AI innovators take their products to market. We invite consultation feedback on this proposal as well as suggestions for industry sectors that meet these criteria.

99. We would also like to build a deeper understanding of what service offering would be most helpful to industry. Some sandboxes offer supervised real-life or simulated test environments where innovators can trial new products, often under relaxed regulatory requirements.[footnote 147] In other scenarios, a team of technologists and regulation experts give customised advice and support to participating innovators over a number of months to help them understand and overcome regulatory barriers so they can reach their target market.[footnote 148] Our current preference is for the customised advice and support model, as we think this is where we can deliver benefits most effectively in the short term. We will explore options for developing a safe test environment capability at a later date, informed by our initial pilot work.
Consultation questions:
100. The implementation of an AI regulatory sandbox will also be closely informed by Sir Patrick Vallance's review into digital regulation and his recommendation to establish a multi-regulator sandbox.[footnote 149] The review sets out a number of design principles, which we will build into our pilot approach. This includes targeting such initiatives at start-ups and small to medium-sized businesses. As a matter of priority, we will engage with businesses to understand how such an approach should be designed and delivered to best support their needs.

S1. To what extent would the sandbox models described in section 3.3.4 support innovation?

S2. What could government do to maximise the benefit of sandboxes to AI innovators?

S3. What could government do to facilitate participation in an AI regulatory sandbox?

S4. Which industry sectors or classes of product would most benefit from an AI sandbox?

3.3.5 Regulator capabilities

101. Government has prioritised the ongoing assessment of the different capability needs across the regulatory landscape. We will keep this under close review as part of our ongoing monitoring and evaluation activity.

102. While our approach does not currently involve or anticipate extending any regulator's remit,[footnote 150] regulating AI uses effectively will require many of our regulators to acquire new skills and expertise. Our research[footnote 151] has highlighted different levels of capability among regulators when it comes to understanding AI and addressing its unique characteristics. Our engagement has also elicited a wide range of views on the capabilities regulators require to address AI risks and on the best way for regulators to acquire these.
AI expertise. Particularly:
103. We identified potential capability gaps among many, but not all, regulators, primarily in relation to:
Organisational capacity. A regulator's ability to:

Technical expertise in AI technology.[footnote 152] For example, on how AI is being used to deliver products and services and on the development, use and applicability of technical standards.[footnote 153]

Expertise on how AI use cases interact across multiple regulatory regimes.

Market intelligence on how AI technologies are being used to disrupt existing business models, both in terms of the potential opportunities and risks that can impact regulatory objectives.

Effectively adapt to the emergence of AI use cases and applications, and assimilate and share this knowledge throughout the organisation.

Work with organisations that provide assurance techniques (such as assurance service providers) and develop technical standards (such as standards development organisations), to identify relevant tools and embed them into the regulatory framework and best practice.

Work across regulators to share knowledge and cooperate in the regulation of AI use cases that interact across multiple regulatory regimes.

Establish relationships and communicate effectively with organisations and groups not normally within their remit.

104. In the initial phases of implementation, government will work collaboratively with key partners to leverage existing work on this topic. For example, the Digital Regulation Cooperation Forum ( DRCF) is already exploring ways of addressing capability gaps within its members.
Consultation questions:
105. There are options for addressing capability gaps within individual regulators and across the wider regulatory landscape, which we will continue to explore. It may, for example, be appropriate to establish a common pool of expertise that could establish best practice for supporting innovation through regulatory approaches and make it easier for regulators to work with each other on common issues. An alternative approach would be to explore and facilitate collaborative initiatives between regulators - including, where appropriate, further supporting existing initiatives such as the DRCF - to share skills and expertise.

19. As a regulator, what support would you need in order to apply the principles in a proportionate and pro-innovation way?

20. Do you agree that a pooled team of AI experts would be the most effective way to address capability gaps and help regulators apply the principles?

Part 4: Tools for trustworthy AI to support implementation

4.1 AI assurance techniques

106. Tools for trustworthy AI including assurance techniques and technical standards will play a critical role in enabling the responsible adoption of AI and supporting the proposed regulatory framework. Industry and civil society were keen to see a range of practical tools to aid compliance. Government is already supporting the development of these tools by publishing a Roadmap to an effective AI assurance ecosystem in the UK[footnote 154] and establishing the UK AI Standards Hub[footnote 155] to champion the use of technical standards.[footnote 156]

107. To assure AI systems effectively, we need a toolbox of assurance techniques to measure, evaluate and communicate the trustworthiness of AI systems across the development and deployment life cycle. These techniques include impact assessment, audit, and performance testing along with formal verification methods.

108. It is unlikely that demand for AI assurance can be entirely met through organisations building in-house capability. The emerging market for AI assurance services and expertise will have an important role to play in providing a range of assurance techniques to actors within the AI supply chain. There is an opportunity for the UK to become a global leader in this market as the AI assurance industry develops. This will enable organisations to determine whether AI technologies are aligned with relevant regulatory requirements.

4.2 AI technical standards

110. Assurance techniques need to be underpinned by available technical standards, which provide common understanding across assurance providers. Technical standards and assurance techniques will also enable organisations to demonstrate that their systems are in line with the UK's AI regulatory principles.

111. Multiple international and regional standards development organisations are developing, or have already released, AI-specific technical standards, addressing topics such as risk management, transparency, bias, safety and robustness. Accordingly, technical standards can be used by regulators to complement sector-specific approaches to AI regulation by providing common benchmarks and practical guidance to organisations.[footnote 157] Overall, technical standards can embed flexibility[footnote 158] into regulatory regimes and drive responsible innovation by helping organisations to address AI-related risks.[footnote 159]

112. The UK plays a leading role in the development of international technical standards, working with industry, international and UK partners.[footnote 160] The government will continue to support the role of technical standards in complementing our approach to AI regulation, including through the UK AI Standards Hub.

Box 4.1: Supporting a layered approach to AI technical standards

The government will complement its context-specific approach to AI regulation by proposing a proportionate 'layered approach' to applying available AI technical standards. This involves regulators identifying relevant technical standards and encouraging their adoption by actors in the AI life cycle to support the integration of the AI regulation principles into technical and operational business processes:

Layer 1: To provide consistency and common foundations across regulatory remits, in the first instance regulators could seek to encourage adoption of sector-agnostic standards which can be applied across AI use cases to support the implementation of cross-sectoral principles. For example, management systems, risk management, and quality standards[footnote 161] can provide industry with good practices for the responsible development of AI systems. The adoption of these standards should be encouraged by multiple regulators as tools for regulated entities to establish common good practices for AI governance.

Layer 2: To adapt these governance practices to the specific risks raised by AI in a particular context, regulators could look at encouraging adoption of additional standards addressing specific issues such as bias and transparency.[footnote 162] Such standards would act as tools for industry to operationalise compliance with specific AI regulation principles. As these standards will provide good practices for AI governance applicable to multiple sectors, regulators could complement these with sector-specific guidance.

For example, standards for bias mitigation could be promoted by the Financial Conduct Authority ( FCA) and the Equality and Human Rights Commission () as practical tools for providers of AI scoring models to identify and mitigate relevant sources of bias to ensure the fairness of the outcomes when the AI model is applied to financial services (credit scoring) and HR practices (candidate scoring) respectively.
Consultation questions:
Layer 3: Where relevant, regulators could encourage adoption of sector-specific technical standards to support compliance with specific regulatory requirements and performance measures.[footnote 163]

21. Which non-regulatory tools for trustworthy AI would most help organisations to embed the AI regulation principles into existing business processes?

Part 5: Territorial application

5.1 Territorial application of the regulatory framework

114. We will continue to engage devolved administrations, businesses, and members of the public from across the UK to ensure that every part of the country benefits from our pro-innovation approach. We will, for example, convene the devolved administrations for views on the functions we expect the government to perform and on the potential implications of introducing a statutory duty on regulators to have due regard to the principles.

5.2 Extraterritorial application of the regulatory framework

115. While we expect our principles-based approach to influence the global conversation on AI governance, we are not currently proposing the introduction of new legal requirements. Our framework will not therefore change the territorial applicability of existing legislation relevant to AI (including, for example, data protection legislation).

Part 6: Global interoperability and international engagement

6.1 Our regulatory framework on the world stage

116. Countries and jurisdictions around the world are moving quickly to set the rules that govern AI. The UK is a global leader in AI with a strategic advantage that places us at the forefront of these developments. The UK is ranked third in the world for AI publications and also has the third highest number of AI companies.[footnote 164] We want to build on this position, making the UK the best place to research AI and to create and build innovative AI companies. At the same time, we recognise the importance of working closely with international partners. As such, the UK's approach to both our domestic regulation and international discussions will continue to be guided by our ambition to develop AI frameworks that champion our democratic values and economic priorities.

117. In line with our domestic approach, we will focus on supporting the positive global opportunities AI can bring while protecting citizens against the potential harms and risks that can emanate across borders. We will work closely with international partners to both learn from, and influence, regulatory and non-regulatory developments (see examples in box 6.1). Given the complex and cross-border nature of AI supply chains, with many AI businesses operating across multiple jurisdictions, close international cooperation will strengthen the impact of our proposed framework.

118. We will promote interoperability and coherence between different approaches, challenging barriers which may stand in the way of businesses operating internationally. We will ensure that the UK's regulatory framework encourages the development of a responsive and compatible system of global AI governance. We will build our international influence, allowing the UK to engage meaningfully with like-minded partners on issues such as cross-border AI risks and opportunities.

120. Our international approach will include ensuring that proven, effective, and agreed upon assurance techniques and international technical standards play a role in the wider regulatory ecosystem. Such measures will also support cross-border trade by setting out risk management and AI governance practices that are globally recognised by trading partners, reducing technical barriers to trade and increasing market access. We will also use our world-leading innovation provisions in Free Trade Agreements to address the challenges innovators in AI may face and ensure that businesses are able to take advantage of the opportunities it presents.

121. In multilateral engagements, we will work to leverage each forum's strengths, expertise and membership to ensure they are adding maximum value to global AI governance discussions and are relevant to our democratic values and economic priorities.

Box 6.1: Examples of international engagement and collaboration
Multilateral AI engagement

The UK has played an active and leading role on the international AI stage and will continue to do so. Some (non-exhaustive) examples of activities are:
Bilateral AI engagement

OECD AI Governance Working Party (AI-GO): The UK is an active member of the OECD's Working Party on AI Governance (AIGO), which supports the implementation of the OECD's AI principles and enables the exchange of experience and best practice to advance the responsible stewardship of AI.[footnote 165]

Global Partnership on AI (GPAI): The UK is a key contributor to - and founding member of - the Global Partnership on AI (GPAI), which is an independent organisation consisting of 29 countries and a range of international experts. GPAI was launched in 2020 as the first international multilateral forum to focus solely on AI and the UK has played a significant role in shaping its development and influencing its agenda.[footnote 166]
At the 2022 GPAI Ministerial Summit in Japan, we demonstrated the scale of the UK's AI ambitions by announcing £1.2m of funding to develop a Net Zero Data Space for AI Applications (which will also support our Net Zero policy objectives).[footnote 167] This is in addition to the previous £1m investment to advance GPAI research on data justice (collaborating with The Alan Turing Institute and 12 pilot partners in low and medium income countries).

G7: The UK is actively engaged in the G7's work on AI and we are working closely with Japan - which holds the G7 Presidency for 2023 - to encourage greater international collaboration, support the development of consistent, proportionate and interoperable regulatory interventions, and champion the role of tools for trustworthy AI where appropriate.

Council of Europe Committee on AI (CAI): The UK holds a Bureau position and we are working closely with like-minded nations on the proposed Convention on AI, to help protect human rights, democracy and rule of law.[footnote 168]

UNESCO: The UK was actively involved in the development of the UNESCO Ethics of AI Recommendations and UK organisations have been supporting the development of implementation tools.[footnote 169]

Global standards development organisations: The UK will continue to work with international partners and global standards development organisations to develop and promote global technical standards for AI, including through the UK AI Standards Hub.[footnote 170] For example, the UK is playing a leading role in the International Organisation for Standardisation and International Electrotechnical Commission[footnote 171] (ISO/IEC) on 4 active AI projects.[footnote 172] Through the British Standards Institution (BSI),[footnote 173] we are also a member of the Open Community for Ethics in Autonomous and Intelligent Systems (OCEANIS).[footnote 174]

The UK is engaging with individual nations and jurisdictions as they develop regulatory and governance approaches to AI. These include the European Union (and its Member States), US, Canada, Singapore, Japan, Australia, Israel, Norway, and Switzerland, amongst many others. We will continue to maintain close dialogues to share information and knowledge, learn from and adapt our approach in collaboration with others, and work together to shape the international landscape.

Part 7: Conclusion and next steps

7.1 Conclusion and next steps

122. Our proportionate approach to regulating AI is designed to strengthen the UK's position as a global leader in artificial intelligence, harness AI's ability to drive growth and prosperity,[footnote 175] and increase public trust in these technologies. The approach we set out is proportionate, adaptable, and context-sensitive to strike the right balance between responding to risks and maximising opportunities.

123. The proposals set out in this document have been informed by the feedback we received from over 130 respondents as part of our call for views on our 2022 policy paper. We will continue to work closely with businesses and regulators as we start to establish the central functions we have identified. Ongoing engagement with industry will be key to our monitoring and evaluation. Feedback will ensure the framework can adapt to new evidence, future-proofing the UK's role as a leader in AI innovation and ensuring that we can take a leading role in shaping the global narrative on AI regulation.

124. Given the pace at which AI technologies and risks emerge, and the scale of the opportunities at stake, we know that there is no time to waste if we are to strengthen the UK's position as one of the best places in the world to start an AI company. In collaboration with regulators, we are already exploring approaches to implementing the framework and will scale up this activity over the coming months. We are committed to an adaptable, iterative approach that allows us to learn and improve the framework. Our sovereign parliamentary system enables us to deliver targeted and proportionate measures - including by adapting existing legislation if necessary - based on emerging evidence.[footnote 176] There are therefore aspects of our implementation work that will be delivered in parallel with the wider consultation set out in this white paper.

125. In the first 6 months following publication we will:

126. In the 6 to 12 months after publication we will:

Agree partnership arrangements with leading organisations and existing initiatives to deliver the first central functions.

Encourage key regulators to publish guidance on how the cross-sectoral principles apply within their remit.

Publish proposals for the design of a central M&E framework including identified metrics, data sources, and any identified thresholds or triggers for further intervention or iteration of the framework. This will be published for consultation.

Continue to develop a regulatory sandbox or testbed with innovators and regulators.

127. In the longer-term, 12 months or more after publication, we will:
Consultation questions:

Deliver a first iteration of all the central functions required to ensure the framework is effective.

Work with key regulators that have not published guidance on how the cross-sectoral principles apply within their remit to encourage and support them to do so.

Publish a draft central, cross-economy AI risk register for consultation.

Develop the regulatory sandbox or testbed drawing on insights from the pilot.

Publish the first monitoring and evaluation report. This will evaluate how well the cross-sectoral principles are functioning and the delivery of the central functions. Performance will be measured against our framework characteristics: pro-innovation, proportionate, trustworthy, adaptable, clear and collaborative. The report will also consider existing regulatory activity and the role of government in supporting this, including whether appropriate guidance (including joint guidance) has been issued. In the report, we will include considerations on the need for any iteration of the framework, including the need for statutory interventions.

Publish an updated AI Regulation Roadmap which will set out plans for the future delivery of the central functions. In particular, it will assess whether a central government team is the most appropriate mechanism for overseeing the central functions in the longer term or if a more independent body would be more effective.

22. Do you have any other thoughts on our overall approach? Please include any missed opportunities, flaws, and gaps in our framework.

Annex A: Implementation of the principles by regulators

A.1 Factors that government believes regulators may wish to consider when providing guidance/implementing each principle

Principle

Implementation considerations

Safety, security and robustness

We anticipate that regulators will need to:

1. Provide guidance about this principle including:

(i) considerations of good cybersecurity practices, such as the NCSC principles for the security of machine learning,[footnote 177] as a secured system should be capable of maintaining the integrity of information.

(ii) considerations of privacy practices such as accessibility only to authorised users and safeguards against bad actors.

2. Refer to a risk management framework that AI life cycle actors should apply. Models should be regularly reviewed over time as a mitigation strategy.

3. Consider the role of available technical standards, for example addressing AI safety, security, testing, data quality, and robustness (including, , * , * , and *) to clarify regulatory guidance and support the implementation of risk treatment measures.

Appropriate transparency and explainability

We anticipate that regulators will need to:

1. Set expectations for AI life cycle actors to proactively or retrospectively provide information relating to:

(i) the nature and purpose of the AI in question including information relating to any specific outcome,

(ii) the data being used and information relating to training data,

(iii) the logic and process used and where relevant information to support explainability of decision-making and outcomes,

(iiii) accountability for the AI and any specific outcomes.

2. Set explainability requirements, particularly of higher risk systems, to ensure appropriate balance between information needs for regulatory enforcement (for example, around safety) and technical tradeoffs with system robustness.

3. Consider the role of available technical standards addressing AI transparency and explainability (such as IEEE 7001, )[footnote 178] to clarify regulatory guidance and support the implementation of risk treatment measures.

Fairness

We anticipate that regulators will need to:

1. Interpret and articulate 'fairness' as relevant to their sector or domain,

2. Decide in which contexts and specific instances fairness is important and relevant (which it may not always be).

3. Design, implement and enforce appropriate governance requirements for 'fairness' as applicable to the entities that they regulate.

4. Where a decision involving use of an AI system has a legal or similarly significant effect on an individual, regulators will need to consider the suitability of requiring AI system operators to provide an appropriate justification for that decision to affected parties.

5. AI systems should comply with regulatory requirements relating to vulnerability of individuals within specific regulatory domains. Regulators will need to consider how use of AI systems may alter individuals' vulnerability, pursuant to their existing powers and remits.

6. Consider the role of available technical standards addressing AI fairness, bias mitigation and ethical considerations (for example, , *, ) to clarify regulatory guidance and support the implementation of risk treatment measures.

Accountability and governance

We anticipate that regulators will need to:

1. Determine who is accountable for compliance with existing regulation and the principles. In the initial stages of implementation, regulators might provide guidance on how to demonstrate accountability. In the medium to long term, government may issue additional guidance on how accountability applies to specific actors within the ecosystem.

2. Provide guidance on governance mechanisms including, potentially, activities in scope of appropriate risk management and governance processes (including reporting duties).

3. Consider how available technical standards addressing AI governance, risk management, transparency and other issues can support responsible behaviour and maintain accountability within an organisation (for example, , , *).

Contestability and redress

We anticipate that regulators will need to:

1. Create or update guidance with relevant information on where to direct a complaint or dispute for those affected by AI harms. Guidance should clarify existing 'formal' routes of redress offered by regulators in certain scenarios.

2. Clarify interactions with requirements of appropriate transparency and explainability, acting as pre-conditions of effective redress and contestability.

Annex B: Stakeholder engagement

B.1 Summary

In July 2022, we published a policy paper outlining our proposals for Establishing a pro-innovation approach to regulating AI.[footnote 179] We proposed a non-statutory framework underpinned by a set of cross-sectoral principles including transparency, safety, and security. The principles were intended to guide how regulators approach AI risks. We outlined our intention for the framework to be coherent, proportionate and adaptable, with regulatory coordination to reduce burdens on business and agility to keep pace with rapid technological advancements. Our proposals were designed to strengthen the UK's position as a global leader in AI by ensuring the UK is the best place to develop and use AI technologies.

We launched a call for views on the proposals outlined in our policy paper to capture feedback from stakeholders between July and September 2022. We received responses from over 130 different stakeholders. There were some clear themes amongst the responses, with stakeholders noting the importance of regulatory coordination and asking for further details on how this will be achieved.

The 2023 AI regulation white paper sets out our latest position based on the feedback we received. In particular, we have considered the need for new central functions to undertake activities such as system-wide risk monitoring and evaluation of the AI regulation framework.

We welcome feedback on our latest proposals and will actively engage stakeholders as part of a consultation running to 21st June. See Annex C for more details on how to contribute to this consultation.

B.2 Background

In July 2022, we opened a public call for views on our policy paper: Establishing a pro-innovation approach to regulating AI. We invited stakeholder views on how the UK can best set the rules for regulating AI in a way that drives innovation and growth while also protecting our fundamental values. Feedback was collected to inform the development of the white paper.

We welcomed reflections on our proposed approach and specifically invited views and supporting evidence on the following questions:

1. What are the most important challenges with our existing approach to regulating AI? Do you have views on the most important gaps, overlaps or contradictions?

2. Do you agree with the context-driven approach delivered through the UK's established regulators set out in this paper? What do you see as the benefits of this approach? What are the disadvantages?

3. Do you agree that we should establish a set of cross-sectoral principles to guide our overall approach? Do the proposed cross-sectoral principles cover the common issues and risks posed by AI technologies? What, if anything, is missing?

4. Do you have any early views on how we best implement our approach? In your view, what are some of the key practical considerations? What will the regulatory system need to deliver on our approach? How can we best streamline and coordinate guidance on AI from regulators?

5. Do you anticipate any challenges for businesses operating across multiple jurisdictions? Do you have any early views on how our approach could help support cross-border trade and international cooperation in the most effective way?

6. Are you aware of any robust data sources to support monitoring the effectiveness of our approach, both at an individual regulator and system level?

The call for views and evidence was open for 10 weeks, closing on 26 September 2022. In this period we met with 39 stakeholders to capture detailed feedback on our proposals. In total, we received responses from over 130 stakeholders. Stakeholders represented a range of perspectives, from start-ups to Big Tech, and included developers, deployers, and funders from across the AI life cycle. We also heard from researchers, regulators, lawyers, trade bodies and unions as well as representatives from the devolved administrations, local government, and wider public sector.

We have carefully analysed all the views and evidence submitted. We are grateful for the time and effort our stakeholders committed during this process, which has informed and strengthened our policy position as outlined in the white paper.

B.3 Responses

Overall, there was strong support for context specific regulation implemented by existing regulators and many noted that this approach would drive innovation. Stakeholders felt our proposals were a proportionate way to establish regulatory best practice in a fast-changing landscape. However, responses also asked for more practical detail, particularly around risk tolerance, compliance measures, and the overall coherence of the framework.

Our analysis found 6 overarching themes raised by stakeholders:

1. Articulating the intended societal benefits of AI is key to a future-proofed regulatory vision that works for citizens as well as businesses.

Government response: We have analysed our principles in consideration of both stakeholder feedback and our risk assessment work. The white paper clarifies the substance of the principles in section 3.2.3. Human rights and environmental sustainability are not explicitly named in the revised principles as we expect regulators to adhere to existing law when implementing the principles. We have emphasised the social benefits alongside the economic opportunities we intend to unlock with our pro-innovation approach to AI regulation.

2. Offering greater central clarity around the scope of the regime is critical to ensuring business confidence

A number of stakeholders praised our description of AI for capturing the distinct regulatory challenges that AI poses and our proposed characteristics were largely considered to be fit for purpose. There were some concerns that the definition was not 'user-friendly' on its own. While many felt that creating a more specific definition of AI would be difficult and some noted it could be unhelpful, there was clear appetite for further detail on how regulators will maintain a coherent definition of AI within and across sectors. Use cases were suggested as a means of illustrating AI technologies within scope.

Many stakeholders, especially from industry, were keen to see a clear and transparent risk management framework with assessment criteria. In particular, multiple stakeholders felt that it would be beneficial for central government or a central body to provide a clear description of what constitutes 'unacceptable risk'. Some suggested this could complement more detailed risk analysis by regulators to ensure a coordinated and coherent approach - as well as effectively identifying any gaps. Stakeholders indicated that greater clarity on risk would support business development and could also promote high standards, public trust, and the adoption of AI.

Government response: We stress-tested our proposed characteristics of AI against stakeholder feedback and found that concerns centred on how we would ensure coherence across sectors and regulators. We recognise a trade-off between the certainty provided by a blanket approach, such as a singular definition and central risk framework, and the agility enabled by sector-specific expertise, including regulator-refined definitions. Given the fast pace of technological development and stakeholder praise for a future-proofed approach, we have retained our core, defining characteristics for AI, see section 3.2.1. We have considered how regulators can be given the technical capability necessary to create clear definitions for AI in and across their sectors, see section 3.2.1. In section 3.3.1 of the white paper, we outline how new central functions will help identify conflicts or gaps in regulator definitions of AI. Acknowledging feedback that a central steer on 'acceptable' risk would provide business confidence and investment, we have proposed that centralised risk monitoring and horizon scanning would be key central functions.

3. A principles-based approach will enable regulation to keep pace with a fast-evolving technology

Stakeholders generally agreed that a principles-based approach implemented by regulators would offer a proportionate way to build best practice. Stakeholders felt the principles address the key risks that AI poses while allowing regulators to tailor approaches to their sectors. Stakeholders welcomed our use of the OECD principles as a means of promoting international alignment and interoperability.

While stakeholders recognised the benefits that a flexible non-statutory approach offers, some stakeholders were concerned that a non-statutory approach would be unenforceable. A few stakeholders suggested clarifying how AI regulation dovetails with existing legislation and defining thresholds for when our regime may shift to statutory implementation.

Government response: We appreciate the praise of our adaptation of the multilaterally agreed OECD principles. We further outline our international approach in the white paper, recognising that interoperability will help ensure that UK businesses can continue to innovate. While we continue with a non-statutory approach for initial implementation, reflecting on stakeholder concerns around enforceability, we anticipate that introducing a statutory duty to have due regard on regulators might be needed to strengthen the framework. A duty to have due regard to our cross-sector principles will provide a legislative incentive while maintaining flexibility for the framework to adapt to technological changes. We will monitor the implementation of the framework to assess whether it is effective without the need to implement a statutory duty and will also review responses to the white paper consultation.

4. Providing centralised coordination and oversight will be essential to regulatory coherence and horizon scanning

Stakeholders voiced concerns that regulators did not have the capability to ensure a coherent compliance process, especially for businesses operating across or between industry sectors or regulatory remits. Stakeholders reported expensive, time-consuming confusion when there was not clear regulatory ownership of a technology or issue. Some criticised communication and knowledge-sharing between regulators. One stakeholder explained that joint guidance had previously been very useful. Others suggested that regulators should have more stringent duties to collaborate to ensure consistency and shared best practice.

A number of stakeholders were supportive of a central coordination function for existing regulators, as opposed to a new regulator for AI. Many stressed the importance of a coordination function to aid navigation of trade-offs and conflicts (such as between the need to collect data to minimise bias and the need to refrain from collecting data in the interest of privacy). While many stakeholders stated the need for central coordination, many were solution-agnostic. Proposals included:

An expanded role for the DRCF. Some stakeholders suggested the DRCF was well-positioned to take on a coordination function but others questioned the DRCF's suitability. In particular, it was felt that the DRCF would require more capacity to fulfil a coordination role.

A new central body to undertake coordination. Stakeholders suggested establishing a new body, such as a 'Centre for AI Governance', to undertake functions such as: conducting cross-sector risk-mapping; conducting regulatory gap analyses and horizon scanning; monitoring the applicability of emerging AI standards; supplying training; and monitoring international approaches.

Appointing an existing regulator as 'lead regulator' for AI. Some stakeholders felt that regulators should have more incentives to work together and the entire regulatory landscape could learn from more advanced regulators.

Stakeholders stated the importance of clarifying regulator remits and addressing gaps, noting the fast pace of change for AI technologies. Some suggested that a coordination body should be responsible for a horizon scanning function that monitors and evaluates risks.

Government response: Building on reflections from stakeholders, we identified a small range of regulators with remits that are likely to be significantly affected by AI and conducted analysis of their capability to implement our policy paper proposals. We found varied readiness, with some regulators already demonstrating world-leading approaches to regulating AI and others asking for further support. Similarly, knowledge and information sharing mechanisms were not uniform across regulators and we identified a need for coordination mechanisms to streamline compliance processes for business and ensure regulation provides system-wide coverage of current and future opportunities. We considered multiple options for coordination functions, in line with stakeholder suggestions, and incorporated feedback into our analysis. We outline our proposals for central functions in section 3.3.1 in the white paper.

5. Streamlining liability and tailoring reporting obligations will be key to enabling responsible innovation

While stakeholders were strongly supportive of compliance and assurance as a means of facilitating public trust and the wider adoption of AI technologies, many were keen to limit the burden of reporting obligations, particularly for startups and SMEs. Industry stakeholders noted that the costs of reporting burdens would be passed onto consumers. Some stakeholders emphasised that the government should have a role in providing education and support for small businesses.

There was interest in regulatory sandboxes as a way to enable investment and establish best practice. Generally there was a strong appetite for industry-led solutions and a less burdensome or 'tick box' approach to compliance. Stakeholders were strongly supportive of standards as a way to drive accountability, adoption, and good consumer outcomes. Stakeholders suggested sector compliance templates and voluntary industry forums as ways to share knowledge and reduce the burden of establishing best practice.

Some stakeholders felt the paper lacked a position on liability and argued a clear allocation of legal responsibility would enable effective enforcement and unlock investment. More specifically, some stakeholders suggested that, when appropriate, targeting foundation models (often developed by larger organisations) would increase innovation and competition by reducing liability burdens on smaller companies. Stakeholders often suggested impact assessments could be used to help address liability issues at all stages of the AI life cycle.

Government response: We welcomed the thoughtful suggestions from respondents regarding innovative compliance measures. We noted the significant appetite for regulatory sandboxes and have outlined our proposals in the white paper, see section 3.3.4. We agree that reporting burdens should be proportionate and give detail on how we will continue to work with regulators to ensure compliance measures are streamlined. We acknowledge that regulation measures can affect competition and innovation by creating undue burdens on start-ups and SMEs. We are confident that regulators will oversee proportionate and innovation friendly measures in their remits, with a central function undertaking activity to streamline and ensure coherence. We recognise that liability is complicated by a complex AI value-chain that can incorporate many different actors in different roles. As such, we believe that regulators are best positioned to begin allocating liability in their sectors, adopting a context-based approach that builds on best practice. Our proposal setting out activities to be undertaken centrally will ensure that regulators' approaches to liability are proportionate, coherent across sectors, and supportive of innovation.

6. Establishing interoperability will be critical to ensuring an internationally competitive approach

Government response: In the white paper, we set out our vision for AI regulation to ensure that the UK is the best place to start and grow an AI business. We share stakeholder concerns on interoperability and plan to continue using our leading role in international forums such as the OECD, G7, and Council of Europe to promote pro-innovation approaches to regulation that capitalise on the potential social and economic benefits of AI while addressing the new risks the technology can pose. Our plan for international engagement, detailed in part 6, clarifies our approach with an emphasis on interoperability.

Annex C: How to respond to this consultation

We are inviting individuals and organisations to provide their views by responding to the questions set out in this consultation. The questions are listed below.

The consultation will be open for 12 weeks, until 21 June.

You can respond online via the consultation survey link here.

Our privacy statement is set out here.

If for exceptional reasons, you are unable to use the online system, for example because you use specialist accessibility software that is not compatible with the system, you may request and complete a word document version of the form.
By post
By email
evidence@officeforai.gov.uk

Questions:

The revised cross-sectoral AI principles

1. Do you agree that requiring organisations to make it clear when they are using AI would adequately ensure transparency?

2. What other transparency measures would be appropriate, if any?

3. Do you agree that current routes to contestability or redress for AI-related harms are adequate?

4. How could routes to contestability or redress for AI-related harms be improved, if at all?

5. Do you agree that, when implemented effectively, the revised cross-sectoral principles will cover the risks posed by AI technologies?

6. What, if anything, is missing from the revised principles?

A statutory duty to regard

7. Do you agree that introducing a statutory duty on regulators to have due regard to the principles would clarify and strengthen regulators' mandates to implement our principles, while retaining a flexible approach to implementation?

8. Is there an alternative statutory intervention that would be more effective?

New central functions to support the framework

9. Do you agree that the functions outlined in section 3.3.1 would benefit our AI regulation framework if delivered centrally?

10. What, if anything, is missing from the central functions?

11. Do you know of any existing organisations who should deliver one or more of our proposed central functions?

12. Are there additional activities that would help businesses confidently innovate and use AI technologies?

12.1. If so, should these activities be delivered by government, regulators or a different organisation?

13. Are there additional activities that would help individuals and consumers confidently use AI technologies?

13.1. If so, should these activities be delivered by government, regulators or a different organisation?

14. How can we avoid overlapping, duplicative or contradictory guidance on AI issued by different regulators?

Monitoring and evaluation of the framework

15. Do you agree with our overall approach to monitoring and evaluation?

16. What is the best way to measure the impact of our framework?

17. Do you agree that our approach strikes the right balance between supporting AI innovation; addressing known, prioritised risks; and future-proofing the AI regulation framework?

18. Do you agree that regulators are best placed to apply the principles and government is best placed to provide oversight and deliver central functions?

Regulator capabilities

19. As a regulator, what support would you need in order to apply the principles in a proportionate and pro-innovation way?

20. Do you agree that a pooled team of AI experts would be the most effective way to address capability gaps and help regulators apply the principles?

Tools for trustworthy AI

21. Which non-regulatory tools for trustworthy AI would most help organisations to embed the AI regulation principles into existing business processes?

Final thoughts

22. Do you have any other thoughts on our overall approach? Please include any missed opportunities, flaws, and gaps in our framework.

Legal responsibility for AI

L1. What challenges might arise when regulators apply the principles across different AI applications and systems? How could we address these challenges through our proposed AI regulatory framework?

L2.i. Do you agree that the implementation of our principles through existing legal frameworks will fairly and effectively allocate legal responsibility for AI across the life cycle?

L.2.ii. How could it be improved, if at all?

L3. If you are a business that develops, uses, or sells AI, how do you currently manage AI risk including through the wider supply chain? How could government support effective AI-related risk management?

Foundation models and the regulatory framework

F1. What specific challenges will foundation models such as large language models (LLMs) or open-source models pose for regulators trying to determine legal responsibility for AI outcomes?

F2. Do you agree that measuring compute provides a potential tool that could be considered as part of the governance of foundation models?

F3. Are there other approaches to governing foundation models that would be more effective?

AI sandboxes and testbeds

S1. Which of the sandbox models described in section 3.3.4 would be most likely to support innovation?

S2. What could government do to maximise the benefit of sandboxes to AI innovators

S3. What could government do to facilitate participation in an AI regulatory sandbox?

S4. Which industry sectors or classes of product would most benefit from an AI sandbox?

# data[].attributes.attachment_link.title
A pro-innovation approach to AI regulation

# data[].attributes.attachment_link.description
Presented to Parliament by the Secretary of State for Science, Innovation and Technology by Command of His Majesty on 29 March 2023. Command Paper Number: 815 © Crown copyright 2023 ISBN: 978-1-5286-4009-1 Correction: Text currently reads in Annex C : 1.

# data[].attributes.headline
University of Jyväskylä, Finland guidance policy on use of AI in studies

# data[].attributes.subject
University of Jyväskylä, Finland guidance policy on use of AI in studies

# data[].attributes.body
https://www.jyu.fi/jsbe/en/study/study-guide/administrative-rules-regulations-and-forms/the-use-of-ai-based-tools-e-g-gpt-in-jsbe-as-part-of-studies

# data[].attributes.attachment_link.readable_text
There has been much discussion about the use of language models in higher education, as they offer an obvious opportunity to cheat by writing essay answers with the help of AI or by rewriting plagiarized text. However, language models are likely to be integrated with word processor software in the future, and they also have applications that are less negative. Therefore, they should not be banned. Language models are a new technology, and much can be learned from their use.

JSBE recommends the following policy for the use of language models:

# data[].attributes.attachment_link.title
The use of AI-based tools (e.g. GPT) in JSBE as part of studies

# data[].attributes.attachment_link.description
There has been much discussion about the use of language models in higher education, as they offer an obvious opportunity to cheat by writing essay answers with the help of AI or by rewriting plagiarized text. However, language models are likely to be integrated with word processor software in the future, and they also have applications that are less negative.

# data[].attributes.headline
Resources compiled by Dr Perry Share, ATU / NAIN

# data[].attributes.subject
Resources compiled by Dr Perry Share, ATU / NAIN

# data[].attributes.body
https://qqi365-my.sharepoint.com/:w:/g/personal/cmarshall_qqi_ie/Ee09R713uFNKihC5bXnUNu8Bm-VwsbjuVpLqyemHKKKcxw?rtime=INIoOh4x20g

# data[].attributes.headline
Washington Post article about Turnitin False Positive

# data[].attributes.subject
Washington Post article about Turnitin False Positive

# data[].attributes.body
https://phys.org/news/2023-04-viewpoint-dont-fret-students-chatgpt.html

# data[].attributes.attachment_link.readable_text
Credit: Unsplash/CC0 Public Domain

Schools and universities are panicking about artificial intelligence (AI) and cheating. But AI presents far more significant threats to equity in education.

Fears of cheating typically arise from concerns about fairness. How is it fair that one student spends weeks laboring over an essay, while another asks ChatGPT to write the same thing in just a few minutes? Fretting about giving each student a "fair go" is essential to maintaining the idea of New Zealand as an egalitarian country.

But as with the myth of the "American dream," the egalitarian narrative of New Zealand masks more pernicious inequities like structural racism and the housing crisis, both of which have an outsized-and decidedly unfair-influence on today's students.

These persistent inequities dwarf the threat of cheating with AI. Instead of excessive hand wringing about cheating, educators would benefit from preparing for AI's other inequities, all of which are showcased in OpenAI's latest large language model (LLM): GPT-4.

GPT-4 is here, for a price

GPT-4, which has refined guardrails and more parameters than ChatGPT, is touted as safer and more accurate than its predecessors. But there's a catch. GPT-4 costs US$20 per month.

For some, that price will be inconsequential. But for those whose budgets have been squeezed thin by skyrocketing inflation, it may be a deal breaker. The democratizing potential of AI technology is here, but only if you can afford it.

This digital divide puts students and educational institutions in two camps. Those with enough resources to enjoy the benefits of AI tools. And those without the same financial flexibility who get left behind.

It may seem small now, but as the cost of AI tools increases, this digital divide could widen into an immense gulf. This should worry educators who have long been concerned about the ways unequal access to learning technologies creates inequity among students.

AI threatens Indigenous languages and data

AI tools also perpetuate the global dominance of English at the expense of other languages, especially oral and Indigenous languages. I recently spoke with a Microsoft executive who called these other languages "edge cases"-a term used to describe uncommon cases that cause problems for computer code.

But Indigenous languages are only a "problem" for AI tools because large language models learn from online data sets with little Indigenous content and an overwhelming amount of English content.

The dominance of English content online is not an accident. English rules the internet because centuries of British colonization and American cultural imperialism have made English the lingua franca of global capitalism, education and internet discourse. From this perspective, other languages aren't inferior to English; they just don't make as much money as English language content.

But Māori speakers are rightly wary of attempts to commodify their language. Too often, the commercialisation of Indigenous knowledge fails to benefit Indigenous people. That's why it's essential for Indigenous communities to maintain control over their own information, an idea known as Indigenous data sovereignty.

Without Indigenous data sovereignty, these billion-dollar tech companies could extract value from these so-called edge cases and then later decide to stop investing in them.

For educators, these threats are important because AI tools will soon be incorporated in Microsoft Office, search engines and other learning platforms.

At Massey University, where I teach, students can submit assignments in te reo Māori or in English. But if the AI writing tools compose better in English than in Māori, then they put Māori language learners at a disadvantage. And if Māori language students are forced to use tools that compromise Indigenous data sovereignty, that's a problem too.

Banning AI in education also creates inequities

Although it's tempting to ban AI in education-as some schools and academic journals and even some countries have already done-this too augments existing inequities. People with disabilities can benefit from communicating with AI tools. But like laptop bans from previous eras, AI bans deny students with disabilities access to important learning technologies.

Banning AI will also disadvantage multilingual students who may struggle to write in English. AI tools can help multilingual students learn important English language genres, structures, prose styles and grammar-all skills that contribute to social mobility. But banning AI penalizes these multilingual students.

Instead of banning AI, educators would be better off modifying their curricula, pedagogies and assessments for the AI tools that will soon become ubiquitous. But revisions like these take more time and resources, something school teachers and university educators have both been striking for recently. Teaching institutions must be prepared to invest not only in AI tools but also in the educators who are essential in helping students think critically about using them.

This article is republished from The Conversation under a Creative Commons license. Read the original article.

# data[].attributes.attachment_link.title
Viewpoint: Don't fret about students using ChatGPT to cheat-AI is a bigger threat to educational equality

# data[].attributes.attachment_link.description
Schools and universities are panicking about artificial intelligence (AI) and cheating. But AI presents far more significant threats to equity in education.

# data[].attributes.headline
Means. Motive, Opportunity: A Composite Narrative about Academic Misconduct

# data[].attributes.subject
Means. Motive, Opportunity: A Composite Narrative about Academic Misconduct

# data[].attributes.body
https://nationalcentreforai.jiscinvolve.org/wp/2023/03/27/means-motive-opportunity-a-composite-narrative-about-academic-misconduct/

# data[].attributes.attachment_link.readable_text
Academic misconduct seems to have been brought into sharp relief over the last few months with a huge amount of hyperbole around large language models, or "generative AI". Probably the most talked about was the arrival of ChatGPT3 just before Christmas 2022, social channels quickly lit up with discussions about how it will disrupt education, often with polarising views. But academic misconduct has always been a concern in education. The most common question is "why do students cheat?", but that just deflects from a range of other issues. It also diverts us from another question, "why do academics cheat?"

For several weeks I have been gathering anonymous stories from academics and students. In this post, I have sought to weave together some of the student and staff voices, and other non-sourceable data to create some understanding of how large language models and so-called generative AI have been used. However, as interviews continued, other topics in adjacent areas were discussed and presented in the research context too. These are where some of the voices may be viewed by some readers to have crossed a line into academic misconduct.

The term "composite" here refers to the narrative being constructed from multiple voices and other sources. This composite narrative aims to create an honest and open understanding of the cultural use of generative technologies, and those adjacent activities that were brought to light in the course of the research, without fear of prosecution for actual or perceived misconduct.

I want to start out by saying that I believe no one starts out to cheat or behave dishonestly. That may be seen as naive, but I do not want to work in a system that focuses on "policing behaviours". But I think it is worth looking at academic misconduct through a familiar police adage that writers of fictional crime seem to constantly employ "Means, Motivation, Opportunity".

First, here are two composite stories that use some of the accounts I collected during the research, and again, these are fictionalised, but pull together real voices.

Sam is a second-year student who is studying a STEM subject. They enjoyed the first year as none of it "counted", but as they started their second year, they feel more pressure as the work now "counts" toward their final degree classification. Added to this they work part-time in a local supermarket, but the job is part-time in name only some weeks as to make ends meet, they sometimes do more than 30 hours work. Sam feels that the academic environment they are studying in is highly competitive, they want to do well, and they are thinking of doing post-graduate studies, and feel they need the grades to do that. "Every assessment is important and judged" and in Sam's mind, they are all equally weighted. Sometimes lots of assessments from different modules come all at once. In January, with a deadline approaching, Sam started using the ChatGPT tool to expand some of their work in word length, and to make their drafts into better grammar. This felt like a good time-saving device and did not get flagged by a plagiarism detector. As weeks went on some of the assessments started to feel like "everything was coming at once", at this point Sam started thinking about using it to alleviate their workload. "I got an essay online, just a poor example really, I knew it would get flagged in the checker, but I also knew I could rewrite it quickly to suit the exact essay title, using ChatGPT, I did this by doing it in small sections where I had the references to back up the argument, only took me an hour to do the whole thing, and I did it for a module I didn't like, I wouldn't have done it for XXXX, I enjoy that."

Jamie is employed as a researcher at a university and also teaches undergraduates in a STEM-based subject. They became aware of ChatGPT through people talking about it on Twitter. Initially viewed as just another tool that they might have to learn, or go to staff development sessions on, they started seeing people posting up examples of outputs. Admittedly they considered most of the outputs they saw as mediocre, and in some specific examples wrong. But they started to take it a little more seriously when they saw people using it for things like grant applications. The first time they used it was for shortening an abstract for a paper to the required length (they reduced over 700 words to exactly 500), and "it was still readable and carried the point well". After the Christmas break and with a lot of teaching commitments upcoming they "used it to write an assignment brief and marking scheme. it took half an hour to finesse it, but it took the drudgery out of trying to formulate verbs and adjectives". As a new employee, they were also under pressure to complete a professional accreditation. "As a scientist, I don't particularly like writing reflections and feelings, I like that I can, with a few of the right prompts, get the chatbot to write a fictionalised reflection that looks about right, where's the harm, it just ticks a box".

Technology has provided students with lots of ways to access and share information, making it easier than ever to fall into accidental collusion or plagiarism, or actually just cheat. Essay mills offer students the chance to have their essays written for them by professionals, this is not often cheap, sometimes essays are more expensive depending on grade or level of risk for plagiarism detection. This means that students no longer need to spend time researching and writing their essays. Instead, they can simply pay for an essay and submit it as their own work. This presents a significant challenge for educators, as detecting this type of cheating can be incredibly difficult especially if the actual author has checked and tested it for plagiarism. But to get that level of essay can be expensive. Not all students will have the means. But in the case of Sam, they bypassed the expensive element - the good quality paid-for essay and just used a similar lower quality essay and used ChatGPT, which is currently free to everyone, gave them an accessible means to rewrite and improve it.

In the case of the academic, Jamie, they had to write a reflection on a piece of their teaching practice, they do not like writing in this way, and they decided that having access to ChatGPT would provide the means of doing something they did not want to. Jamie was clear that the work they submitted was "fictionalised" (their words).

For a lot of students, the pressure to succeed academically can be overwhelming, family, peers, friends, and actual or imagined future employers can all contribute to how a student experiences the pressure to succeed. Logistically, pressure on a student can occur when students perceive or experience assessment congestion, where there are multiple assignments due at the same time or exams are scheduled close together. And with financial pressures, this stress can be further compounded, especially if students also need to work while studying to support themselves. As a result, students may feel that they have no choice but to take some shortcuts in order to keep up with their workload and maintain their grades. It may be that academic misconduct is not necessarily malicious, but rather a desperate attempt to cope with mounting pressures. The motives for Sam were many, assessment congestion, perceptions of needing to compete and working while studying.

For the academic, Jamie, the motivation was less about work pressure and more around the perception of the work they had to submit. They felt the exercise was a "tick box" - the institution employed them to do research, and yes, they have to do a little teaching, but the research is the important thing. The university wants them to have this accreditation, but it is just a tick box, so they justify their approach to writing their reflections using a tool to generate fiction.

Opportunity

Poorly designed assessments can provide students with ample opportunity to engage in academic misconduct, or if the students can not relate the assessment to the real application of what they are learning. It is harder to design good authentic assessments that will not allow extensive use of a tool such as ChatGPT, or allow the purchase of essays online, but it can be done. The opportunity to engage in academic misconduct is greatly increased when we design assessments that don't promote critical thinking and problem-solving, or allow the students to demonstrate what they have learned in authentic ways.

When it comes to the opportunity for academics to move more along a continuum toward academic misconduct we need to look at the same issue - are we designing, in the case of Jamie, an authentic and relevant assessment for their accreditation? Certainly, on social media and at events we have seen academics talking about using ChatGPT to help them write their arguments to achieve various forms of professional accreditation in their teaching practice. In a discussion with Professor Sally Brown , Emerita Professor of Higher Education Diversity in Teaching and Learning at Leeds Beckett University, she saw issues even before the advent of the chatbot technology
"I think the trouble with [some] Fellowship applications is that people now adapt quite a formulaic approach, and there's a lot of coaching going on so the extent to which [some] fellowships are an individual achievement is sometimes stretched a bit"
The opportunity for people to provide a response in a way that will probably achieve their goal is much reduced when we don't design to assess effectively.

Means, Motive, and Opportunity

Means, motive, and opportunity are key elements to consider when we think about the assessment of students or our own professional accreditation. Staff and students have always had motive and opportunity. But the means have always been expensive. ChatGPT and similar tools have now lowered the bar to access the means. This coupled with increased workloads on staff and students, the pressure for some students of both work and academic achievement and the perception of assessment congestion and other logistical issues, have led to motives increasing. Finally, our design of how we assess or accredit gives students and staff the opportunity to use these tools in a way which ignores the learning process to a greater or lesser extent.

We need to ask ourselves serious questions.

How can we create assessments that promote genuine learning, critical thinking, and problem-solving, rather than encouraging students to find ways to cheat the system? Can you cheat if it is an authentic assessment (one where you would be performing a task after education and have access to the tools)? Are our own professional accreditation methods robust enough to negate the use of generative AI?

To what extent should we hold students accountable for academic misconduct when the pressures of assessment congestion and balancing work and getting an education may be driving their behaviours?

Are we modelling the right academic behaviours, we see chatbots being used for a range of academic activities and accreditation, why shouldn't students shortcut some of the work and go straight to the end goal?

If we can't control the Means, ChatGPT, is out of the bottle, and out of our hands - cost may eventually be a limiting factor but not at the moment- then we need to look at the other two aspects, motivation and opportunity. What can we do to reduce pressure on students and reengage them with learning, what can we do to take away the motivation to use these tools in ways that don't help their learning outcomes? Finally, we need to look at what and how we are assessing and how we do it - reducing the opportunity.

Over the course of January through mid-March confidential and sometimes anonymous text-based interviews took place through a range of media, including WhatsApp, Messenger, anonymous forms, and DMs on other social media, Approximately 40 students across STEM and humanities were included, and 27 academic, and academic support/professional services staff also contributed data. The author is grateful for their honesty and trust. .

Find out more by visiting our National centre for AI page to view publications and resources, join us for events and discover what AI has to offer through our range of interactive online demos.

For regular updates from the NCAI sign up to our mailing list.

Get in touch with the team directly at NCAI@jisc.ac.uk

# data[].attributes.attachment_link.title
Means, Motive, Opportunity: A Composite Narrative about Academic Misconduct - National centre for AI

# data[].attributes.attachment_link.description
Academic misconduct seems to have been brought into sharp relief over the last few months with a huge amount of hyperbole around large language models, or "generative AI". Probably the most talked about was the arrival of ChatGPT3 just before Christmas 2022, social channels quickly lit up with discussions about how it will disrupt education, [...]

# data[].attributes.headline
QQI Artificial Intelligence Week Playlist - 5 Webinars

# data[].attributes.subject
QQI Artificial Intelligence Week Playlist - 5 Webinars

# data[].attributes.body
https://www.youtube.com/watch?v=QUv2ti7ouH0&list=PLfoxtqBemVVXd178kjWZRGuR81j6eQq4V

# data[].attributes.attachment_link.title
A familiar, but strange time: Navigating Artificial Intelligence in higher education - Sarah Howard

# data[].attributes.headline
Slides from ENAI Webinar: Recommendations on Ethical Use of AI in Education

# data[].attributes.subject
Slides from ENAI Webinar: Recommendations on Ethical Use of AI in Education

# data[].attributes.headline
TEQSA Good Practice Guidelines for the use of Generative AI

# data[].attributes.subject
TEQSA Good Practice Guidelines for the use of Generative AI

# data[].attributes.body
https://www.teqsa.gov.au/sites/default/files/2023-04/aain-generative-ai-guidelines.pdf

# data[].attributes.body
https://www.unesco.org/en/artificial-intelligence/recommendation-ethics

# data[].attributes.attachment_link.title
Recommendation on the Ethics of Artificial Intelligence

# data[].attributes.body
https://www.irishtimes.com/opinion/2023/04/19/chat-gpt-poses-an-existential-threat-and-the-window-for-gaining-control-over-it-is-small/

# data[].attributes.attachment_link.readable_text
ChatGPT has pushed artificial intelligence into the public consciousness. Society is at a crossroads regarding AI's role in our future, forcing us to make societal decisions to avoid bad outcomes.

ChatGPT poses an existential danger, though not of the Terminator type. The threat is significant enough to demand regulation to ensure we use such powerful capabilities for societal benefit. Several efforts are under way to regulate this type of AI, but governments must act now: the window for effective regulation is relatively narrow.

Regulation will always lag behind innovation, especially in the tech sector. At present, there is no EU-wide regulation on AI-based technology in a wide range of products, such as self-driving cars, medical technology and surveillance systems. It may be early 2025 before rules being debated by the European Parliament come into force. That is far too late for products based on large language models such as ChatGPT. The Italian privacy regulator recently banned ChatGPT due to credible breaches of GDPR, and Spain and France have raised similar concerns. But more needs to be done.

[ AI could be the next industrial revolution - but what risks does it bring? ]

The developers of these technologies, including OpenAI, are aware of just some of the possible adverse outcomes. OpenAI hired a team of leading academics over a six-month period to assess the negative impact that GPT-4 might have. Among the outcomes, they found that it was able to find somewhere to make chemical weapons, write hate speech and find unlicensed guns online. Based on this evaluation, OpenAI amended GPT-4 before release. But this is just one limited set of problems. Clearly a huge range of as yet-undiscovered negative consequences remains.

Is ChatGPT inherently dangerous? Let's apply a literary notion of robotics to the question. Isaac Asimov, the science fiction author, devised a set of rules to be hard-wired into the software of robots, including a "First Law", stating that a robot may not injure a human being, or through inaction allow a human being to come to harm. Does ChatGPT abide by this law?

Currently, it has no real power to harm a human directly. Neither has it any notion of right or wrong, or the ability to make moral decisions. It is a generative AI system trained on unlabelled data. It can extrapolate from that data to generate responses to queries that it can then extend beyond its training to develop plausible, but not necessarily accurate, answers.

ChatGPT is based on GPT-4. No one alive can understand this system or explain why it generates responses to queries. This lack of explainability is profoundly worrying

ChatGPT is a data-driven system: it is created based on synthesising data and is only as powerful as the data on which it is based. Powerful and accurate data-driven AI systems require well-defined tasks within constrained environments, using carefully chosen data. For example, some areas of cancer diagnosis involve data-driven AI-based systems that outperform human experts and offer substantial societal benefit. By training this system in a constrained environment (detecting cancer in MRI images) using carefully chosen data - in this case, MRI images with and without cancer present - treatment outcomes can be improved.

ChatGPT satisfies none of these requirements, so it will remain a weak system with limited effectiveness without further modifications. It is trained on an undefined task within totally unconstrained environments, on random data accessible via the web.

Users can ask it any form of query, not a class of questions with well-defined responses - for example, "Tell me all Thai restaurants in Galway". It is already clear that ChatGPT makes errors, and that it can and has fabricated nonsense answers.

Critical to evaluating the danger of ChatGPT is that this system can only interact via text or voice; there is no inherent set of actions that a ChatGPT-based system may take. This is unlike autonomous systems that act in the world, such as self-driving cars, which are trained using reinforcement learning. Given input data from the car's sensors, the vehicle is trained to act - for example, drive straight or avoid a pedestrian.

ChatGPT is trained using language-based deep-learning algorithms, with no possible actions involved. As such, ChatGPT has no notion of acting in the world.

Despite this, it is a tool that needs to be regulated. The software driving autopilot systems for commercial aircraft is highly regulated. Contrast the virtually unregulated AI/software industry with other critical systems, such as airline autopilot software. These autopilot systems are explainable because the underlying mathematics, like the software implementing it, is clearly demonstrable.

[ The Irish Times view on artificial intelligence: promise and peril ]

But ChatGPT is based on GPT-4. No one alive can understand this system or explain why it generates responses to queries. This lack of explainability is profoundly worrying, especially if ChatGPT is used for crucial societal decision-making.

The network on which ChatGPT is based is only as good as the data on which it is trained, and ChatGPT is trained on unregulated data from the internet. How is an AI expected to create a theory of right or wrong, or the ability to make moral decisions, without some explicit guidance? This training process has zero regulation, so the outcomes are unregulated and can be dangerous. In addition, the data used may breach privacy laws, leading to decisions such as that of the Italian privacy regulator.

So how might ChatGPT, in its current incarnation, cause harm? Take its use as a search engine. Because we don't know how ChatGPT creates responses, there is no guarantee that the answers will be accurate, or might be based on data that is abusive or hateful - unlike the Google search engine, which is based on a PageRank algorithm with well-understood mathematical foundations. The possibility that we will replace search engines whose performance we understand with engines whose performance we don't understand is troubling.

The entire AI software industry needs better regulation. We already know about the damage done by Meta subsidiaries, such as the harm done by Instagram to young girls or Cambridge Analytica through its access to Facebook data. Deepfake technology also poses many problems.

The window for regulation is narrow. Governments must act before it shuts completely.
Prof Gregory Provan is a professor of computer and information technology at University College Cork and leads a Lero research spoke on blended autonomous vehicles

# data[].attributes.attachment_link.title
ChatGPT poses an existential threat, and the window for gaining control over it is small

# data[].attributes.attachment_link.description
How is an AI, with no notion of right or wrong, expected to make moral decisions without some explicit guidance? It can't

# data[].attributes.headline
ENAI Recommendations on the ethical use of Artificial Intelligence in Education

# data[].attributes.subject
ENAI Recommendations on the ethical use of Artificial Intelligence in Education

# data[].attributes.body
https://edintegrity.biomedcentral.com/articles/10.1007/s40979-023-00133-4

# data[].attributes.attachment_link.readable_text
Show authors

International Journal for Educational Integrity 19, Article number: 12 (2023) Cite this article

14k Accesses

2 Citations

114 Altmetric

Metrics

Rationale

Artificial Intelligence (AI) tools are constantly being released into the public domain. As with all new technological innovations, this brings a range of opportunities and challenges for education: primarily for educators and learners. There is an increasing interest in the academic community and beyond to use Artificial Intelligence in Education (AIED) to generate content. This presents opportunities and challenges for academic and research integrity.

The European Network for Academic Integrity (ENAI) is an international association gathering educational institutions and individuals interested in maintaining and promoting academic integrity. As the use of AI tools may not always be consistent with academic integrity, we consider it important to familiarise all education stakeholders with how to use AI tools responsibly and in accordance with academic integrity practices and values.

ENAI presents a set of recommendations with the aim of supporting academics, researchers and other educational stakeholders, including students' organisations, on the ethical use of AI tools. The recommendations focus on the importance of equipping stakeholders with the skills and knowledge to use AI tools ethically and the need to develop and implement relevant educational policies addressing the opportunities and challenges posed by AIED.

Process

To create these recommendations, we used our experience and knowledge, both about AI, and academic integrity. The authors collectively drafted the main structure and main ideas of the recommendation. The draft was circulated to the ENAI working group members, most of whom provided valuable comments and suggestions. Then the authors incorporated the comments and prepared the final wording of the document.

The field of artificial intelligence is advancing fast, and the ways that it disrupts education are changing from one week to the next. We consider these recommendations as a working document, and we invite everyone to provide feedback. There will be a special session at the European Conference on Ethics and Integrity in Academia 2023 in Derby, UK, dedicated to this document.

Background and context

Academic integrity (AI) can be defined as "compliance with ethical and professional principles, standards, practices, and a consistent system of values that serves as guidance for making decisions and taking actions in education, research and scholarship" (Tauginienė et al. 2018, p. 8).

Artificial Intelligence refers to systems that appear to have "intelligent behaviour by analysing their environment and taking actions - with some degree of autonomy - to achieve specific goals" (European Commission, 2018, p. 4).

AI-based tools can be used to transform, produce or generate any kind of content, such as text, images, art, music, or programming code. Different technologies, including machine learning and neural networks, are used to develop the capabilities of these tools.

Authorised and declared usage of AI tools is usually acceptable. However, in an educational context, undeclared and/or unauthorised usage of AI tools to produce work for academic credit or progression (e.g. students' assignments, theses or dissertations) may be considered a form of academic misconduct ("any action or attempted action that undermines academic integrity and may result in an unfair academic advantage or disadvantage for any member of the academic community or wider society" (Tauginienė et al. 2018, p. 9). Moreover, it is increasingly challenging to reliably distinguish AI-generated content from human-produced content.

The wide accessibility of AI may exacerbate existing types of academic integrity threats, such as essay and paper mills, fabrication and falsification of data, etc.

Current definitions of misconduct, such as contract cheating and plagiarism, may not explicitly include this type of misconduct. Therefore, we propose an umbrella definition for all types of unauthorised content generation, including contract cheating and inappropriate use of AI:

Unauthorised content generation (UCG) is the production of academic work, in whole or part, for academic credit, progression or award, whether or not a payment or other favour is involved, using unapproved or undeclared human or technological assistance.

AIED can be used for unauthorised content generation; however, the use of AIED is not automatically unethical. There can be differences between academic disciplines, education institutions, courses, types of assessment, cultures, regions, and countries as to what is considered acceptable use of AI and what is not.

While AI can threaten academic integrity, it also presents opportunities. AI multiplies users' abilities - in both good and bad ways. Therefore, students and educators should be guided on the benefits and limitations of AI tools in order to learn and use AI ethically and uphold academic integrity. Moreover, with the increasing automatisation of modern societies, they will likely use AI tools in their professional life. Therefore, they should be given opportunities to learn these skills during their education.

Recommendations on the ethical use of AIED

All persons, sources, and tools that influence the ideas or generate the content should be properly acknowledged. Consequently, when an AI tool is used, it should be acknowledged. The acknowledgement may be done in different ways, according to the context, the institutional policies or other requirements. When possible, the input given to the AI tool should be specified.

Appropriate use of services, sources, and tools that only influence the form is generally acceptable (e.g. proofreaders, proofreading tools, spelling checkers, thesaurus).

An AI tool cannot be listed as a co-author in a publication as it cannot take responsibility for the content and findings reported. The person (human being or legal entity) is always accountable for the content, whether or not it was generated by AI (see COPE Guidelines for Authorship and AI by Levene 2023).

The outputs of AI tools can include biased, inaccurate, or incorrect content that users should be aware of. This may be caused by bias in training data, algorithms, filters, etc.

It is important to include information about AI in education for all students and in training for teachers. If students do not have the opportunity to learn about the ethical use of AI, they will be more susceptible to engaging in inappropriate use of AI, which may constitute academic misconduct.

Students should be included and educated on the following:

Teachers should receive training on ethical use of AI including development of relevant learning outcomes, learning activities, and assessment strategies.

National guidance and institutional-level policies should be developed and/or reframed to include AI. National guidance should provide overarching advice on what institutions should include in their policies.

Institutional policies should:

Define default rules on when and how the students, teachers, researchers and other educational stakeholders are allowed to use different kinds of AI tools. There should be space for specific rules at course level. The policy and the rules should be clearly communicated to all stakeholders.

Guide the users on how to correctly and transparently acknowledge the use of AI tools in an assignment, dissertation, thesis, paper, article, book chapter, computer programme, graphic, artwork and other types of artefact.

Summary statement

There is no doubt that AI brings significant change to education. As with any other technology, it extends and enhances human abilities and may be used both in a positive and a negative way. ENAI urges national policymakers, institutions, and all individual members of the academic community to seek ways on the ethical use of AI and share best practices in order to benefit from the opportunities that AI brings to education and science.

References

Download references

Acknowledgements

We would like to acknowledge the contribution of the members of ENAI working groups, namely: Sue Hackett, Olumide Popoola, Debora Weber-Wulff, Teddi Fishman, Michael Draper, Shiva Sivasubramaniam, Lorna Waddington, Brenda Stoesz, Arash, Sandie Dann, Sarah Elaine Eaton, Salim Razi, Oraz Myradov.

Author information

Authors and Affiliations

1. Masaryk University, Brno, Czech Republic

Tomas Foltynek

2. Uppsala University, Uppsala, Sweden

Sonja Bjelobaba

3. Coventry University, Coventry, UK

Irene Glendinning

4. University of Wollongong in Dubai, Dubai, United Arab Emirates

Zeenath Reza Khan

5. European Network for Academic Integrity, Brno, Czech Republic

Rita Santos & Július Kravjar

6. European Students Union, Brussels, Belgium

Pegi Pavletic

Contributions

All the authors were included in the drafting of the document. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Sonja Bjelobaba.

Ethics declarations

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Reprints and Permissions

About this article

Cite this article

Foltynek, T., Bjelobaba, S., Glendinning, I. et al. ENAI Recommendations on the ethical use of Artificial Intelligence in Education. Int J Educ Integr 19, 12 (2023). https://doi.org/10.1007/s40979-023-00133-4
Download citation

# data[].attributes.attachment_link.title
ENAI Recommendations on the ethical use of Artificial Intelligence in Education - International Journal for Educational Integrity

# data[].attributes.attachment_link.description
Show authors International Journal for Educational Integrity 19, Article number: 12 (2023) Cite this article 14k Accesses 2 Citations 114 Altmetric Metrics Artificial Intelligence (AI) tools are constantly being released into the public domain. As with all new technological innovations, this brings a range of opportunities and challenges for education: primarily for educators and learners.

# data[].attributes.body
https://nationalcentreforai.jiscinvolve.org/wp/2023/05/11/generative-ai-primer/

# data[].attributes.attachment_link.readable_text
Publishing an intro to generative AI is a challenge as things are moving so quickly. However, we think things have now settled down enough for us to bring together information in a single place, to create a short primer. We aim to publish this as a more formal guide that will be updated regularly, but we are posting an initial version as a blog post to get feedback on whether it is useful and if there is other information you would like included.

Version 1.1 - May 22 2023

Table of Contents
2. An Introduction to the Generative AI Technology 2.2. Microsoft Bing Chat, Google Bard and Anthrompic's Claude 2.3. A summary of key capabilities, limitations, and concerns around ChatGPT and other Large Language Models 2.4 Image Generation 2.5 Microsoft Co-pilot and everyday AI 3. Impact of Generative AI on Education 3.1 Assessment 3.1.1 Guidance on advice to students 3.1.2 The role of AI detectors 3.2 Use in Learning and Teaching 3.2.1 Examples of use by students 3.2.2 Examples of use by teaching staff 3.2.3 Examples of uses to avoid 3.3 Adapting curriculum to reflect the use of AI in work and society.
1. Introduction

Generative AI tools such as ChatGPT are already having a significant impact on education. These tools are posing considerable challenges around assessment and academic integrity, but also present opportunities, for example saving staff time by helping with the creation of learning materials or presenting students with new tools to enhance the way they work. The impact of generative AI is being felt far beyond education, and it is already starting to change the way we work. This presents more challenges and opportunities, in making sure education prepares students for an AI-augmented workplace, and that assessments are authentic yet robust.

The primer is intended as a short introduction to generative AI, exploring some of the main points and areas relevant to education, including two main elements:

An introduction to Generative AI technology

The implications of Generative AI on education

2. An Introduction to the Generative AI Technology

Whilst we do not need a detailed technical understanding of the technology to make use of it, some understanding helps us understand its strengths, weaknesses, and issues to consider. In this section, we will be looking into the technology in a little more detail.

This is a fast-moving space, and the information here is likely to age quickly! This edition was written in May 2023, and we aim to update it regularly to take into account significant developments.

This guide starts by looking at AI text generators, also known as Large Language Models (LLMs).

2.1 ChatGPT

ChatGPT has grabbed most of the headlines since its launch in November 2022. It was created by a company called OpenAI, which started as a not-for-profit research organisation (hence the name) but is now a fully commercial company with heavy investment from Microsoft. It is available as a free version, plus a premium version at $20 a month, which provides faster, more reliable access, as well as access to its latest language models and features, including plugins, which changes its behaviour significantly.

ChatGPT is based on a machine learning approach called 'Transformers', first proposed in 2017, and is pre-trained on large chunks of the internet, which gives it the ability to generate text in response to user prompts, hence the name 'Generative Pre-trained Transformer'. Whilst OpenAI provided some information on the approach for training ChatGPT, they haven't so far, released any information about GPT4, the latest model released in early 2023.

In its standard mode, without plugins, ChatGPT works by predicting the next word given a sequence of words. This is important to understand, as it is not in any sense understanding your question and then searching for a result and has no concept of whether the text it is producing is correct. This leads it to be prone to producing plausible untruths or, as they are often known, hallucinations.

As it stands today, the free version of ChatGPT doesn't have access to the internet, so can't answer questions beyond its training data cut-off date of September 2021. Users paying for the ChatGPT Plus service have access to a version that can access the internet.

ChatGPT Plus customers also have access to plugins which extend ChatGPT's functionality. For example, a Wolfram plugin allows users to ask questions which are answered by Wolfram Alpha, which excels at mathematical and scientific information. Initial testing suggests this might resolve the issue of 'hallucination' in these domains. Many other plugins are available, and more are being developed.

OpenAI makes its service available to other developers, so many other applications make use of it, including many writing tools such as Jasper and Writesonic, as well as chatbots in popular applications such as Snapchat.

2.2. Microsoft Bing Chat, Google Bard and Anthrompic's Claude

Although ChatGPT has got most of the hype, there are other players in this space, and this number is likely to increase.

Microsoft has introduced Bing Chat, which is available for testing, and is based on Open AI's GPT-4. It's focused on searching for information and does have access to the internet. It takes your question, performs one or more web searches based on your question, and will then attempt to summarise and answer, giving references for the sites it uses.

Bard is Google's equivalent and is available for testing. Like Bing Chat, it can access the internet, but unlike Bing Chat, it doesn't provide references for the sites it's used to give its answers.

Claude is more similar to ChatGPT and is produced by Anthropic, and is likely to be built into many applications going forward.

At the moment, both Google Bard and Claude feel a long way behind ChatGPT in terms of capabilities, although at the time of writing, Google has just announced significant improvements, so this may change quickly.

2.3. A summary of key capabilities, limitations, and concerns around ChatGPT and other Large Language Models.

In considering generative AI, it's important not only to understand its capabilities but also its limitations. We also believe users should have a broader understanding of the societal impact of generative AI. Some of the key themes are summarised here:

It can generate plausible but incorrect information.

ChatGPT is only trained on information up until Sept 2021 (but those with the paid ChatGPT Plus service have access to a version that can access the internet)

Limited ability to explain the sources of information for its responses (this varies between Chatbots)

2.4 Image Generation

It's not all about text - image generation tools have made huge progress too, particularly with Midjourney, DALL-E 2 and Stable Diffusion.

As for the text generators, these have been trained on information scraped from the internet, and there is a lot of concern about the copyright of the training material.

These work in a similar way to text generators - the user gives a prompt and one or more variations of images are produced.

Image generation capabilities are being incorporated into general AI services, so Bing Chat, for example, can also generate images, using OpenAI's DALL-E 2.

2.5 Microsoft Co-pilot and everyday AI

There is a lot more to generative AI than just accessing it via the ChatGPT web service. The developers of major AI services such as OpenAI make their services available to other developers. One of the most significant announcements has been from Microsoft, who are incorporating generative AI across the Microsoft 365 tools, under the name ' Co-pilot'. At the time of writing (May 2023) the licensing terms haven't been confirmed, but it is likely that this will put generative AI directly into the hands of all our staff and students with access to Microsoft 365. Google have made similar announcements about their office tools.

2.6 Beyond Chatbots

Whilst the focus has been on chatbots and image generation, generative AI is finding it its way into a wide range of tools and services that can potentially be used in education. As an example, Teachermatic can create a range of resources for teachers, including lesson plans and quizzes, Gamma can create websites and presentations, and Curipod can create interactive learning resources. Many of these applications are built on tools provided by OpenAI and powered by the same technology as ChatGPT.

3. Impact of Generative AI on Education

The impact of generative AI in education is still unfolding and is likely to for some time to come. Key areas include assessment and academic integrity, its use in teaching and learning, use as a time-saving tool, and use by students.

Jisc has convened a number of working groups with representatives from universities and colleges to help us collate and present more detailed advice in these areas, particularly around sector-level advice, assessment and advice to students, and we'll share more information from these groups over the coming weeks.

Here we will now give an overview of the core themes.

3.1 Assessment

Initial discussions about generative AI have focused on assessment, with the concern that students will use generative AI to write essays or answer other assignments. This has parallels with concerns around essay mills. These concerns are valid, and whilst essays produced wholly by generative AI are unlikely to get the highest marks, their capability is improving all the time. The ability isn't just limited to essays. ChatGPT is also highly capable of answering multiple-choice questions and will attempt most forms of shorter-form questions. It will often fall short, especially when answers are highly mathematical, although this will not be obvious to the student using the chatbot service.

There are three main options, each with their own challenges. There is broad acceptance that 'embrace and adapt' is the best strategy in most instances.

Avoid
Revert to in-person exams where the use of AI isn't possible

This moves away from authentic assessment and creates many logistical challenges.

Outrun
Devise an assessment that AI can't do.

AI is advancing rapidly and given the time between the assessment being set and it being taken, AI might well be able to do the assignment when it is taken.

Embrace and adapt
Embrace the use of AI, discuss the appropriate use of AI with students, and actively encourage its use to create authentic assessments

Balancing authentic assessment and the use of generative AI with academic integrity is a challenge.

The immediate action is for all staff to engage with generative AI and try it themselves, learning how their assessments will be impacted. Alongside this, institutions will need to consider their strategic approach to AI, review, and update policies, and communicate guidance to students.

For higher education, this aligns with guidance provided by QAA, and for both higher education and further education, general guidance has been provided by the Department for Education. In addition, the Joint Council for Qualifications has produced guidance for protecting quality of qualifications.

3.1.1 Guidance on advice to students

We are seeing the first guidance to students appear. Getting the wording on this can be challenging, and is discussed further in a Jisc blog post ' Considerations on wording when creating advice or policy on AI use '. Our key messages are:

Don't try to 'ban' specific technology or AI as a whole.

Describe acceptable and expected behaviours and provide examples.

The detail of advice will vary between subjects, so supplement general advice with tailored advice for the subject.

A Jisc-convened working group is looking at best practices in this area and will be sharing its outcomes shortly.

3.1.2 The role of AI detectors

Discussions about academic integrity inevitably include the role of AI detection software. There are a number of tools in this space, but for most UK universities and colleges, Turnitin's newly released AI detection tool will be the obvious tool to consider. The key things are:

No AI detection tool can conclusively prove that text was written by AI

These tools will produce false positives.

The tools won't be able to differentiate between legitimate and other use of AI writing tools.

Best practice for using such tools is still being considered and developed across our sectors, and we will aim to provide examples as these develop.

A Jisc blog post explores the concepts and considerations around AI detection in more detail.

3.2 Use in Learning and Teaching

We are seeing a wide range of ideas for how to use generative AI in learning and teaching. It's worth remembering at this point that it is a fast-moving space and many of the tools are still under development or at the beta stage. Whilst exploration makes a lot of sense, it's also worth noting that individuals shouldn't feel the need to rush into using it in teaching - things are changing rapidly, but now is definitely the time to explore and start planning.

3.2.1 Examples of use by students

We welcome the fact that student voices have been brought into the discussion around generative AI, for example ' Generative AI: Lifeline for students or threat to traditional assessment?'. Jisc has run a number of student panels and will be reporting the findings in more detail in the near future.

Examples of uses by students.

To formulate ideas, for example creating essay structures

Generative AI tools are generally effective in producing outlines as a starting point for an assignment.

To provide feedback on writing

Generative AI will proofread and correct text for students, in a similar way to grammar tools. It will also provide feedback on style and content. Students will need clear advice on when this should be declared.

As a research tool

A good understanding of the tool and its limitations is crucial here, particularly the tendency for generative AI to give misinformation.

Generating images to include in assignments.

The best image generation tools come at a cost, and students need to be aware of copyright concerns.

In section 2.3 we noted that digital inequality is a concern. For example, students that pay for ChatGPT Plus get faster, more reliable access, as well as access to the latest model, and those that pay, for example, for MidJourney image generation will get more and better images than most of the free options. At the moment there are limited options for licensing these tools institutionally, but we expect this to change, and consideration should then be given to licensing options to avoid inequality.

3.2.2 Examples of use by teaching staff

If used appropriately, generative AI has the potential to reduce staff workload, for example by assisting with tasks such as lesson plans, learning material creation, and possibly feedback. The key words here are 'appropriately' and 'assisting'. In section 2.3 we noted some of the limitations of generative AI, such as incorrect information, so as things stand today, generative AI can assist by, for example, generating initial ideas, but its output should always be carefully reviewed and adapted.

Use of generative AI may be by directly using a general chat app such as ChatGPT, or maybe via an application built on top of generative AI, such as Teachermatic. In the former case in particular staff will need good prompting skills to make the most of generative AI.

Drafting ideas for lesson plans and other activities

The output may be factually incorrect or lack sound pedagogical foundations. Nonetheless, it may be a useful starting point.

Help with the design of quiz questions or other exercises.

Generative AI can quickly generate multiple-choice quizzes and assessment ideas, but they should be reviewed carefully as above.

Customising materials (simplifying language, adjusting to different reading levels, creating tailored activities for different interests)

Generally, when asked to customize material, generative AI won't introduce new concepts, and so is less likely to introduce factually incorrect information.

Providing custom feedback to students.

At the moment, generative AI should not be used to mark student work, but it can be a useful tool for assisting with personalized feedback.

3.2.3 Examples of uses to avoid

In general, you should avoid posting any work that isn't your own, and any personal information into ChatGPT, as it might well be used as part of the training data.

Things get a little more complex when considering tools built with ChatGPT, and it is almost certain that generative AI built into tools like Microsoft 365 will have the same data privacy protection as the rest of Microsoft 365.

Marking student work

There is no robust evidence of good performance, although ChatGPT will confidently do this if you ask.

Detecting whether work is written by AI

ChatGPT might claim it can detect whether it wrote the text, but it can't

Anything involving personal information

You should never put personal information into any system that your University or College hasn't got a proper contract in place with and made a full assessment of its data privacy policies etc. Generative AI services like ChatGPT are no exception.

3.3 Adapting curriculum to reflect the use of AI in work and society.

We are seeing a broad acknowledgement that work will change too, but understandably limited action around this at the moment.

OpenAI has done some initial research on the impact of LLMs and jobs and has estimated that 'around 80% of the U.S. workforce could have at least 10% of their work tasks affected by the introduction of LLMs, while approximately 19% of workers may see at least 50% of their tasks impacted.' A 2021 report by the Department for Business, Energy and Industrial Strategy looks more broadly at jobs affected by AI, both in terms of those likely to be automated and growth areas.

In the short term, we expect generative AI to be quickly adopted into courses where it quickly becomes the norm in the workplace. For example, visual arts make use of generative AI, and computer coding makes use of generative coding tools.

4. Regulation

Given the pace of change, regulation has struggled to keep up. In the UK, the government has published an AI white paper which aims to balance regulation and enabling innovation. The paper makes explicit reference to generative AI and large language models. The US government have produced a Bill of Rights and the EU's AI act is working its way through the European Parliament. The proposed UK approach will look to regulate the use of AI, rather than AI technology itself, through the work of existing regulators, against a common set of principles. As progress is made in this area we will endeavour to highlight any areas of particular concern for education.

5. Summary

Generative AI is progressing rapidly and is likely to have a significant impact on education for the foreseeable future. Keeping up with the advances is a challenge, and balancing authentic assessment and academic integrity is increasingly complex. Nonetheless, with care and an increase in staff and student knowledge, there are substantial gains to be made. This guide aimed to give a broad introduction to generative AI. Much more has been written on the topic, and for those that wish to explore further, we have included a range of resources for further reading.
Keeping updated:
To keep up to date with the work of Jisc's National Centre of AI in Tertiary Education, join our Jisc mail list: aied@jiscmail.ac.uk.
Further Reading:
Generative artificial intelligence in education - Department For Education (March 2023)

Maintaining Quality and Standards in the ChatGPT Era: QAA Advice on the Opportunities and Challenges Posed by Generative Artificial Intelligence - QAA (May 2023)

ChatGPT and Artificial Intelligence in higher education - Unesco (April 2023)

Artificial Intelligence (AI) Use in Assessments: Protecting the Integrity of Qualifications - JCQ (March 2023)
Relevant Jisc Blog Posts:
Staff and student use:
Generative AI: Lifeline for students or threat to traditional assessment? Means. Motive, Opportunity: A Composite Narrative about Academic Misconduct Considerations on wording when creating advice or policy on AI use
AI Detection:
AI writing detectors - concepts and considerations A short experiment in defeating a ChatGPT detector
Bias and other ethical considerations:
Hidden Workers powering AI Exploring the potential for bias in ChatGPT
Using Generative AI:
Getting started with ChatGPT Part 1 - Understanding ChatGPT How to Explore AI Image Generation
Change log

V1.1 - 22 May 2023

We've added a table of contents.

Section 2.1 (ChatGPT) has been updated to reflect the fact that all ChatGPT Plus users have access to plugins and web browsing.

We've added a CC BY-NC-SA license

# data[].attributes.attachment_link.title
A Generative AI Primer - National centre for AI

# data[].attributes.attachment_link.description
The primer is intended as a short introduction to generative AI, exploring some of the main points and areas relevant to education, including two main elements: an introduction to Generative AI technology and the implications of generative AI on education

# data[].attributes.headline
Creating effective assessment in context of ChatGPT

# data[].attributes.subject
Creating effective assessment in context of ChatGPT

# data[].attributes.body
https://www.timeshighereducation.com/campus/effective-assessment-practices-chatgptenabled-world?utm_source=newsletter&utm_medium=link-embed&utm_campaign=campus-weekly&spMailingID=25730347&spUserID=MTAyODg5ODQ1Nzk1NgS2&spJobID=2231992459&spReportId=MjIzMTk5MjQ1OQS2

# data[].attributes.attachment_link.readable_text
ChatGPT can provide step-by-step solutions to mathematical problems, write code and generate lesson plans, draft emails and produce stories, music lyrics and social media posts. Therefore, it is not surprising that many in academia are excited about the potential positive influence of ChatGPT on productivity and efficiency.

On the other hand, ChatGPT has triggered a lot of debate around integrity issues in higher education, including concerns about plagiarism and cheating in and outside the classroom. AI models designed for plagiarism detection are being developed. However, this problem naturally leads to an increase in faculty workload to deal with challenges in detecting AI usage in their students' work. Also, beyond modifying their course syllabus, instructors will need to be trained on new assessment practices in response to students' exposure to and usage of ChatGPT.

Through self-assessment or reflective writing, students could discuss their writing or thinking process. Additionally, peer evaluations or interactive assessment activities could be integrated into grading by engaging students in group discussions or other activities such as research and analysis in which students are expected to co-construct knowledge and apply certain skills. Instructors may consider placing an emphasis on assessing the process of learning rather than the outcome. Instead of grading a final writing product alone, instructors could grade multiple drafts and assess how a student's writing improved.

Lastly, instructors can use ChatGPT as a supplemental tool for designing assessment activities. Instructors may ask detailed and specific prompts of ChatGPT to generate a rubric for grading students' knowledge and skills, create multiple-choice quiz questions or come up with group discussion questions about course topics. However, when using these sources, it would be important to ensure they are accurate and to check how the information presented can be interpreted or if it contains any biased views.

Ongoing discussions about assessment methods in light of the growing influence of generative AI such as ChatGPT remind us to adopt more student-centric assessments that can create meaningful learning opportunities for individual students. Design assessments that can provide students with ample scope to apply their skills to analyse ideas and integrate concepts in order to deepen their knowledge.

assessing these cognitive skills , how to incorporate assessments of non-cognitive skills such as collaboration, communication and leadership into classroom learning needs further discussion. That is, instead of emphasising skills that are traditionally measured by standardised or psychometric tests on which AI can perform very well, we should emphasise skills that are useful and meaningful for human learning.

Tools such as ChatGPT do not necessarily generate factual information. Rather, they produce pieces based on the probability of potential relationships between words. Depending on the questions asked, the outputs generated by ChatGPT may be incorrect, outdated or illogical. It has a limited understanding of context, can potentially generate offensive or harmful content, has limited ability to fact-check or verify information, lacks empathy and has limited knowledge on certain topics (especially if the subject matter is complex). In such case, instructors must caution students about the limitations of such tools and encourage them to compare the outputs with other reliable sources to prevent the spread of misinformation and misconceptions.

Jonna Leeisdirector for research in education innovation in theCenter for 21st Century Universities at the Georgia Institute of Technology.
If you found this interesting and want advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.

# data[].attributes.attachment_link.title
Effective assessment practices for a ChatGPT-enabled world

# data[].attributes.attachment_link.description
Assessment methods that require students to produce authentic, novel and personalised responses can help educators stay ahead of the uncertainty and workload that AI writers create

# data[].attributes.headline
Upsides of ChatGPT (alternative to contract cheating, use as assistive tool/leveling the playing field)

# data[].attributes.subject
Upsides of ChatGPT (alternative to contract cheating, use as assistive tool/leveling the playing field)

# data[].attributes.body
https://theconversation.com/chatgpt-could-have-an-upside-for-universities-helping-bust-contract-cheating-by-ghostwriters-205004

# data[].attributes.attachment_link.readable_text
Since the launch of ChatGPT in late 2022, academics have expressed concern over the impact the artificial intelligence service could have on student work.

But educational institutions trying to safeguard academic integrity could be looking in the wrong direction. Yes, ChatGPT raises questions about how to assess students' learning. However, it should be less of a concern than the persistent and pervasive use of ghostwriting services.

Essentially, academic ghostwriting is when a student submits a piece of work as their own which is, in fact, written by someone else. Often dubbed "contract cheating," the outsourcing of assessment to ghostwriters undermines student learning.

Universities and other institutions have employed plagiarism-detection tools, such as Turnitin, in an effort to combat the more obvious forms of academic misconduct.

But contract cheating is increasingly commonplace as time-poor students juggle jobs to meet the soaring costs of education. And the internet creates the perfect breeding ground for willing ghostwriting entrepreneurs.

In New Zealand, 70-80% of tertiary students engage in some form of cheating. While most of this academic misconduct was collusion with peers or plagiarism, the emergence of artificial intelligence has been described as a battle academia will inevitably lose.

It is time a new approach is taken by universities.

Allowing the use of ChatGPT by students could help reduce the use of contract cheating by doing the heavy lifting of academic work while still giving students the opportunity to learn.

The risky business of ghostwriting

Universities have been cracking down on ghost writing to ensure quality education, to protect their students from blackmail and to even prevent international espionage.

Contract cheating websites store personal data making students unwittingly vulnerable to extortion to avoid exposure and potential expulsion from their institution, or the loss of their qualification.

Read more: ChatGPT is the push higher education needs to rethink assessment

Some researchers are warning there is an even greater risk - that private student data will fall into the hands of foreign state actors.

Preventing student engagement with contract cheating sites, or at least detecting students who use them, avoids the likelihood of graduates in critical job roles being targeted for nationally sensitive data.

ChatGPT as friend not foe

It is inevitable ChatGPT will increasingly become part of how students complete their assigned work. While changing the way assignments are completed - and assessed - there are a number of reasons why ChatGPT could also be harnessed as an educational tool.

ChatGPT still requires a certain level of engagement from students. They have to guide the AI through various stages of the research and writing process. By meticulously defining their research question, crafting precise prompts, critically assessing generated content and integrating it with their original thoughts, students retain control over their intellectual journey.

Read more: ChatGPT: Student insights are necessary to help universities plan for the future

Given the underworld associated with ghostwriting, artificial intelligence has the potential to bust the contract cheating economy. This would keep students safer by providing them with free, instant and accessible resources.

Using natural language processing and machine learning algorithms, ChatGPT fosters originality by offering students instant, personalised feedback. By stimulating creativity, broadening vocabulary and enhancing structural coherence, ChatGPT could cultivate an environment where students can flourish and develop their distinct style.

Finally, those who argue that artificial intelligence technologies like ChatGPT may contribute to the erosion of academic integrity overlook the game-changing potential of this sort of technology to refine citation practices. ChatGPT can provide students with style guides and citation generators. These tools can enable students to appropriately credit sources and circumvent plagiarism.

By inputting the relevant context, ChatGPT can assess the author backgrounds, considering cultural, political and ethical biases that may influence their views. In turn, ChatGPT can recommend alternative readings that offer a well-rounded array of viewpoints.

Levelling the playing field

Arguably, the most significant advantage of artificial intelligence tools lie in their potential to level the playing field for students.

Students from diverse backgrounds face several challenges, including navigating uncharted academic terrain, adapting to unfamiliar environments, and grappling with the pressures of independence. These obstacles are amplified for marginalised students and those attending underprivileged schools.

Read more: ChatGPT killed the student essay? Philosophers call bullshit

Academic integrity fundamentally hinges on promoting fairness in the educational process. However, ensuring equal access to resources and support for all students is a daunting task, particularly when confronted with large classes or students with varying academic preparedness.

ChatGPT can serve as a valuable tool in advancing academic integrity by granting all students access to the same resource for honing their writing skills and obtaining feedback on their work, irrespective of their backgrounds or academic prowess.

There needs to be more research on the learning opportunities offered by artificial intelligence programmes like ChatGPT. But it is here, and for a variety of reasons students are using it. Rather than banning ChatGPT and programmes like it, we should be using these tools to help students. In doing so, we would reduce the need for students to seek out other - potentially harmful - ways of completing their assessments.

# data[].attributes.attachment_link.title
ChatGPT could have an upside for universities - helping bust 'contract cheating' by ghostwriters

# data[].attributes.attachment_link.description
Educators have expressed concern about ChatGPT but it could be a tool to help stressed and time-poor students.

# data[].attributes.headline
QAA Advice on maintaining standards in ChatGPT era

# data[].attributes.subject
QAA Advice on maintaining standards in ChatGPT era

# data[].attributes.attachment_link.title
QAA Maintaining Quality and standards in the ChatGPT era May 2023

# data[].attributes.body
http://craft.stanford.edu/

# data[].attributes.attachment_link.description
Resources for teaching about artificial intelligence provided by the Stanford Graduate School of Education

# data[].attributes.headline
Office of Ed Tech, USA - AI and the Future of T&L recommendations

# data[].attributes.subject
Office of Ed Tech, USA - AI and the Future of T&L recommendations

# data[].attributes.body
https://tech.ed.gov/ai-future-of-teaching-and-learning/

# data[].attributes.attachment_link.readable_text
Share this Report

The U.S. Department of Education Office of Educational Technology's new policy report, Artificial Intelligence and the Future of Teaching and Learning: Insights and Recommendations, addresses the clear need for sharing knowledge, engaging educators, and refining technology plans and policies for artificial intelligence (AI) use in education. The report describes AI as a rapidly-advancing set of technologies for recognizing patterns in data and automating actions, and guides educators in understanding what these emerging technologies can do to advance educational goals-while evaluating and limiting key risks.

# data[].attributes.attachment_link.title
Artificial Intelligence and the Future of Teaching and Learning - Office of Educational Technology

# data[].attributes.attachment_link.description
The need for sharing knowledge, engaging educators, and refining technology plans and policies for artificial intelligence (AI) use in education.

# data[].attributes.headline
MacQuarie University proposed AI Literacy Framework

# data[].attributes.subject
MacQuarie University proposed AI Literacy Framework

# data[].attributes.attachment_link.readable_text
Artificial Intelligence (AI) literacy for staff and students will arguably be the next frontier in developing enhanced employability for many graduates. The widespread availability and use of Generative AI tools is shaping up to be a significant disruptor across many areas of human endeavour including study and work life.

Is AI literacy just modern ICT literacy?

AI literacy can be thought of as an extension of information and communications technology (ICT) literacy, information literacy and more broadly digital literacy, because it needs elements of all of these. However Generative AI tools have different and additional affordances compared to other ICT that make AI literacy an added area of expertise that needs to be considered.

What should AI literacy include?

Here I outline a possible AI literacy framework, along with some starter ideas for how educators may approach each element. This framework came about in considering what a student or member of the public would need to be able to effectively utilise emerging Generative AI tools. The framework is focused on a user perspective rather than a developer perspective because the majority of students will sit in the former category. It is perhaps less desirable at this early stage to add complexity pertaining to the development of AI technology that would only be relevant to a minority. Those developing AI tools would be better served by programs in computer science, data science and artificial intelligence. It is likely that as Generative AI becomes more embedded that a user-as-developer perspective should be added - as we see in the UNESCO digital literacy framework.
An AI literacy framework:
a) Ethical use of AI tools (why and when, including issues such as data ownership, privacy, algorithm transparency, legality, types of embedded bias, undisclosed plagiarism, equity, and hidden labour). Online courses such as " Humane technology" provides another example of possible topic coverage. Many of the ethical and legal issues are yet to be resolved. Educators can explore with students ethical issues from multiple perspectives, perhaps by facilitating class discussion. This includes guidelines for the ethical use of AI in education. Jointly arriving at a position for the use of generative AI tools in the local context, including guiding the using AI for study and research purposes, will help to minimise uncertainly for students and put them in a better position to be able to make informed decisions about use of AI tools in the future.

b) Knowledge of AI affordances ( Over 1000 AI tools are available and therefore awareness of the capabilities and limitations of various AI tools will help users choose appropriate tools for the job). Educators can collaborate with students to explore the capabilities and limitations of different tools relevant to the unit context, this could also include the key benefits and risks (e.g. hallucination, undisclosed plagiarism, embedded biases) associated with each tool. Students can be asked to undertake research on one or more tools. Building map of common features and limitations to AI tools and typical uses can then help students decide which tools may suit a given task.

c) Working effectively with AI tools (e.g. "prompt engineering" and prompt refining practices). Educators can leverage free, open access online resources that are appearing such the " Learn Prompting " online mini course. The educator could then work with students to develop unit specific examples and lead discussion on the effective use of tools relevant to the discipline, unit or assessment task context. As indicated earlier a 'user-as-developer' perspective may fit here when the maturity of the tools and their users develop further in the future.

d) Evaluation of AI output. Generative AI is known to hallucinate to produce plausible, but false information in its output (such as fake references) and so being able to evaluate the output for its quality is a key capability in making use of AI tools. Thinking, critique and evaluative judgement are important skills regardless. Educators can use frameworks such as the MQ library CRAAP model to work with students to identify the characteristics of quality AI generated responses.

e) Use and integration into practice. Generative AI can add value in study, personal and for professional work purposes (e.g. co-writing, ideation, creativity, code starters and code review). Building awareness of the changing patterns of work will also help students make informed career and development decisions. Educators can collaborate with students to explore how industry are adopting generative AI tools, how it is impacting workflows and productivity as well as how industry can navigate the emergent issues that these tools bring, such as by using an AI risk framework to evaluate technologies. This could also be done by leveraging existing industry links via work integrated learning activities and MQ's PACE arrangements. Dr Ali Amrollahi from MQBS provides an example of how a unit can be designed to teach a non-technical audience about how advanced technologies (such as Blockchain) can be deployed in professional contexts.

Where to next?

We are in the early stages of a potential AI driven revolution to the way many of us work and live. The potential benefits of well managed implementations of new technology can be big, but so are the pitfalls when such technology is done poorly (see Robodebt Scheme for an example of the latter). The 4th industrial revolution is underway and education institutions need to get on board to develop thoughtful and integrated approaches to educating students for this future as well as rethinking the design of education programs and operation of the institutions themselves.

Further reading on a review of AI literacy frameworks:

Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2, 100041. https://doi.org/10.1016/j.caeai.2021.100041

Join the conversation

We welcome your comments below about on this proposal for an AI literacy framework and how we can work with students around the use AI at tools. You can also contribute your ideas by emailing professional.learning@mq.edu.au.

Read other posts in the generative AI series. Recent posts include: comparing ChatGPT to a calculator and a search engine and the launch of Turnitin's new AI writing detection feature.

Found an 'AI generated' academic integrity breach? See this advice on how to gather evidence and report it to MQ for investigation.

Acknowledgements: Banner image: Modified image based on Stable Diffusion output with the prompt " view through magnifying glass to look at tiny robots. plain background." (24 March 2023). Further edited by M. Hillier. CC0 1.0 Universal Public Domain. Post last updated 11 May 2023.

# data[].attributes.attachment_link.title
A proposed AI literacy framework - TECHE

# data[].attributes.attachment_link.description
Exploring a possible Artificial Intelligence (AI) literacy framework for staff and students.