# data[].attributes.body
UVA's page on AI in the classroom. Includes Matt Miller's "plagiarism/cheating" continuum graphic.

# data[].attributes.attachment_link.readable_text
Artificial Intelligence is quickly changing our understanding of teaching, learning, and working. This page is an evolving resource for those curious about AI in the higher ed context.

What is Generative AI?

AI in the Classroom: Resources

AI at UVA

The Generative AI in Teaching and Learning Task Force

In spring 2023, the university convened a Generative AI in Teaching and Learning Task Force in the Provost's Office. Learn more about the work of the task force. The Gen-AI Task Force website includes guidance for faculty and students, the Task Force's report, and resources from the Center for Teaching Excellence.

AI in LDT's LTi Projects

A&S Learning Design & Technology is supporting the work of Prof. Marc Santuguini (ECON) exploring applications of ChatGPT in his Economics courses.

AI Collections in CTE's Teaching Hub

This gallery of collections is designed to support you in navigating the generative AI landscape in higher education, from what generative AI is and how you can learn more about it to what using it could look like within and across disciplines.

AI, Accessibility, and Equity

Consider these reflections on the potential of AI to impact access, inclusion, and equity in higher education.

AI, Assignments, and Assessments

Consider these resources related to assignment design:

AI and academic integrity: early thoughts

Consider this graphical representation of possible AI applications to a student project. Which might we consider 'cheating'? Which might be acceptable? Which are relevant to our students' future? Which would you use in your own work?
ChatGPT, Chatbots and Artificial Intelligence in Education - Ditch That Textbook Graphic description for screen readers From the top (bot-created) to the bottom (student-created):

student plugged prompt into AI copied response and submitted it to teacher.

AI created a response. Student read, edited, adjusted, and submitted.

Student created multiple AI responses, used the best parts, edited, and submitted.

Student wrote main ideas. AI generated a draft and offered feedback to improve.

Student consulted internet/AI for ideas, then wrote and submitted.

Student wrote all assignment content without consulting AI or the internet.

Three questions are posed next to the continuum *Which of these would you consider "cheating"? Which of these is relevant to our students' future? Which of these would you use in your work as an adult. Graphic logo "Ditch That Textbook" Graphic by Matt Miller @jmattmiller DitchThatTextbook.com
AI-Detection Tools in Higher Ed

AI-detection tools are marketed to educators in various settings. These tools have considerable drawbacks, including reliability (can indicate 'false positives' in work that is not AI generated) and relevance (a step behind AI tool), as well as expense. Alternative approaches to course activities and assignments can engage students in learning, reduce mistrust, and create equity among students. Consider alternative assessment ideas from the K P Cross Academy Techniques Archive - The K. Patricia Cross Academy (kpcrossacademy.org) and the Center for Teaching Excellence Teaching Hub UVA Teaching Hub (virginia.edu)

Resources on Academic Integrity and AI

# data[].attributes.attachment_link.title
Generative Artificial Intelligence in the Classroom

# data[].attributes.attachment_link.description
Artificial Intelligence is quickly changing our understanding of teaching, learning, and working. This page is an evolving resource for those curious about AI in the higher ed context. In spring 2023, the university convened a Generative AI in Teaching and Learning Task Force in the Provost's Office.

# data[].attributes.attachment_caption
US Department of Education recommendations on AI, Teaching, and Learning.

# data[].attributes.attachment_link.title
Integrating Generative AI into Higher Education: Considerations

# data[].attributes.attachment_link.description
Integrating AI into higher education is not a futuristic vision but an inevitability. Colleges and universities must adapt and prepare students, facul

# data[].attributes.body
Conference at UC-Irvine on Generative AI in Higher Education. Includes videos of most presentations.

# data[].attributes.attachment_link.readable_text
This one-day online conference, held on July 13, 2023, brought together leading scholars and practitioners to share and discuss perspectives on generative AI in education and educational research. The conference featured research talks by invited speakers and pedagogical lightning talks​ submitted by educators throughout the world.

Please see below for recordings of each conference panel. If you would like to view the entire conference and accompanying Zoom chat, you may access the Zoom recording here. In addition, a collection of resources shared by attendees during the conference may be accessed here.

# data[].attributes.attachment_link.description
Sponsored by the University of California, Irvine and the Spencer Foundation

# data[].attributes.attachment_link.readable_text
Understanding the nature of generative AI is crucial for educators to navigate the evolving landscape of teaching and learning. In a new report from the Next Level Lab, Lydia Cao and Chris Dede reflect on the role of generative AI in learning and how this pushes us to reconceptualize our visions of effective education. Though there are concerns of plagiarism and replacement of human jobs, Cao and Dede argue that a more productive way forward is for educators to focus on demystifying AI, emphasizing the learning process over the final product, honoring learner agency, orchestrating multiple sources of motivation, cultivating skills that AI cannot easily replicate, and fostering intelligence augmentation (IA) through building human-AI partnerships.

# data[].attributes.attachment_link.title
Navigating A World of Generative AI: Suggestions for Educators

# data[].attributes.attachment_link.description
Understanding the nature of generative AI is crucial for educators to navigate the evolving landscape of teaching and learning. In a new report from the Next Level Lab, Lydia Cao and Chris Dede reflect on the role of generative AI in learning and how this pushes us to reconceptualize our visions of effective education.

# data[].attributes.attachment_link.readable_text
Professors and teaching assistants tend to have set office hours and need to get some sleep at night, but Purdue students in some courses can now get feedback on their writing assignments whenever they want it and as many times as they need it.

All they need to do is ask "Charlie," who's available anytime and never sleeps.

That's because Charlie is an artificial intelligence (AI) writing assistant, developed in a partnership between faculty and Purdue Teaching and Learning Technologies (TLT), part of Purdue Online. Among other things, the AI has been used successfully with essay assignments in a 500-person sociology class. The results have been promising and the developers are planning to expand Charlie's use to additional courses and potentially to additional types of assignments, such as lab reports.

"I think their essays are better," said Lindsay Hamm of students' experience with Charlie in her Social Problems class. "I think students also are more likely to submit papers to Charlie - to get help - because they can do it from their desks in their dorm room or their apartment and get immediate feedback no matter what time it is."

Hamm, a lecturer in the Sociology Department and 2021 Excellence in Instruction Award winner, characterizes herself as a soft touch for "new shiny technologies." She readily signed on when TLT staff approached her about partnering on Charlie's development and has been using it for multiple semesters.

Hamm was an early adopter of the TLT-developed Circuit app, now widely used by instructors to create peer-reviewed assignments in a variety of formats, including rich text, images and video. Hamm uses Circuit with essay assignments in her Social Problems class and she used it this summer for public speaking assignments in SCLA 102, Transformative Texts II, part of Cornerstone, Purdue's innovative Liberal Arts certificate program.

With Circuit, students apply a checklist or rubric from their instructor in evaluating a selection of work from their classmates as well as their own work. Students also evaluate three sample pieces graded by their instructor - one good, one average and one poor - a step used to calibrate their evaluation skills and balance the evaluations across the varied abilities of the students in a class.

In Hamm's Social Problems class, students have the option of writing three essays throughout the semester, doing the calibration steps and performing peer reviews, for which they can earn as many points as an exam.

Now, those students have the opportunity to work with Charlie in kind of a pre-flight check before submitting their essays for peer review and grading.

Charlie stemmed from a TLT exercise in thinking outside the box about how to improve educational technology at Purdue. Jason Dufair, a TLT lead application developer and software engineer, had been intrigued by the idea of employing an AI in some way since reading about a competition to create a rudimentary "robo-grading" system in 2016.

"Since then, the state of the art has come a long way," Dufair said.

Keep in mind, Charlie doesn't grade the assignments and the system can't punch up a student's writing for them. The AI offers feedback they can use to improve their writing. They still have to do the improving. (And Charlie will also point them to additional resources that can help, such as the Purdue Writing Lab.)

"It's not something they're forced to use and interact with and it's not their final grade," Hamm said. "We make that really clear. We're hoping people will always think of Charlie as kind of a helpful TA."

Charlie helps based on instructors' rubrics for assignments. The AI uses existing rubric-level evaluations created by faculty and teaching assistants in learning to discern what's good and what needs work when new essays are submitted to it.

Charlie initially trained on more than 600 graded essays. The results were intriguing. As it churned through the training essays and repeatedly applied the rubric, the AI's evaluations gradually came to match those of the human graders.

TLT worked with Hamm to obtain a Provost's Instructional Innovation Grant for Charlie's initial development. Dufair also worked with Electrical and Computer Engineering machine learning faculty on refining the algorithm behind Charlie and the team pulled in educational psychologists to help make the feedback language used by the system inviting rather than intimidating. The name "Charlie" itself is intended to put a friendly face on the AI.

But there was room for improvement. Charlie in its first iteration could only offer an evaluation of a whole essay. To really be actionable, the evaluation needed to home in on specific areas - parts of the rubric - where work was needed. So, Dufair tuned the algorithm, Charlie digested 500 more essays - with each rubric item scored individually - and the system became capable of evaluating individual rubric elements.

Charlie isn't done learning new tricks either. The team (Hamm, Dufair and Melinda Zook, director of Cornerstone) has now been awarded a Purdue Innovation Hub Grant of up to $150,000 to collect hundreds, perhaps as many as 1,000, graded essays from the fall session of SCLA 101, a Cornerstone course dedicated to developing the ability of students to write in a variety of genres. The plan is for the AI to train on the essays and the related rubric over winter break, after which it will be available to assist the students taking the course in the spring semester.

Hamm has done some student surveying and score analysis and there are early indications that Charlie yields evident improvement. For one thing, Hamm said she sees fewer papers that are skipping parts of the rubric, or that miss the point of the assignment altogether. She thinks students, who are sometimes reticent about showing their early drafts to a human for fear of being judged, tend to view Charlie as a nonjudgmental alternative.

"They could submit their paper as many times as they wanted to up to the final due date," Hamm said. "Some students turned their paper in two or three times trying to increase their score from Charlie. Some people did it 32 times, 36 times just trying to see what was happening, which is fine. Charlie will look at your paper as many times as you want to upload it."

To learn more about how Charlie and Circuit can save faculty time and improve students' writing, contact TLT@purdue.edu. To see it in action, reach out to set up a hands-on demonstration. You can also stop by during TLT's regular drop-in hours.

Writer: Greg Kline, 765-494-8167, gkline@purdue.edu

# data[].attributes.attachment_link.title
Purdue develops an artificial intelligence assistant that can help improve student writing - Purdue University Online - Purdue University

# data[].attributes.attachment_link.description
Professors and teaching assistants tend to have set office hours and need to get some sleep at night, but Purdue students in some courses can now get feedback on their writing assignments whenever they want it and as many times as they need it.

# data[].attributes.attachment_link.title
Sparks of AGI: early experiments with GPT-4

# data[].attributes.attachment_link.readable_text
We live in an era of practical AI, but many people haven't yet experienced it, or, if they have, they might have wondered what the big deal is. Thus, this guide. It is a modified version of one I put out for my students earlier in the year, but a lot has changed. It is an overview of ways to get AI to do practical things.

Large Language Models like ChatGPT are extremely powerful, but are built in a way that encourages people to use them in the wrong way. When I talk to people who tried ChatGPT but didn't find it useful, I tend to hear a similar story.

The first thing people try to do with AI is what it is worst at; using it like Google: tell me about my company, look up my name, and so on. These answers are terrible. Many of the models are not connected to the internet, and even the ones that are make up facts. AI is not Google. So people leave disappointed.

Second, they may try something speculative, using it like Alexa, and asking a question, often about the AI itself. Will AI take my job? What do you like to eat? These answers are also terrible. With one exception, most of the AI systems have no personality, are not programmed to be fun like Alexa, and are not an oracle for the future. So people leave disappointed.

If people still stick around, they start to ask more interesting questions, either for fun or based on half-remembered college essay prompts: Write an article on why ducks are the best bird. Why is Catcher in the Rye a good novel? These are better. As a result, people see blocks of text on a topic they don't care about very much, and it is fine. Or the see text on something they are an expert in, and notice gaps. But it not that useful, or incredibly well-written. They usually quit around now, convinced that everyone is going to use this to cheat at school, but not much else.

All of these uses are not what AI is actually good at, and how it can be helpful. They can blind you to the real power of these tools. I want to try to show you some of why AI is powerful, in ways both exciting and anxiety-producing.

To start, lets introduce the six Large Language Models available right now, all for somewhere between free and $20 a month.

The first four (including Bing) are all OpenAI systems. There are basically two major OpenAI AIs today: 3.5 and 4. The 3.5 model kicked off the current AI craze in November, the 4 model just premiered and is much more powerful. A new variation uses plugins to connect to the internet and other apps, but it is only in early testing. If you have never paid for OpenAI, you have only used 3.5. Aside from the plugins variation, none of these models are connected to the internet.

Microsoft's Bing uses a mix of 4 and 3.5. It is connected to the internet. Bing is a bit weird to use, but powerful. Here is my guide to using it. In addition, Google has released a disappointing AI called Bard (though they may show us more impressive models soon) and Anthropic has released Claude, though it is more focused at business users. So what can you do with these things?

For right now, no other general AI tool comes even close to GPT-4, which you can access at Bing for free or by purchasing a $20/month subscription to ChatGPT. GPT-3.5 is also good at writing and is much faster. I have experimented a lot on how to use AI to help with written material, so here is a list of way you might find it useful:

Writing anything. Blog posts, essays, promotional material, speeches, lectures, chose-you-own adventures, scripts, short stories - you name it, it does it. But you can't just give it basic prompts. Basic prompts result in boring writing. Getting good writing out of ChatGPT takes some practice, and here is a guide to doing that. ChatGPT-4 is much better at writing. Bing can be incredible at writing, but needs some convincing.

Make your writing better. Paste your text into ChatGPT. Ask it to improve the content or for suggestions about how to make it better for a particular audience. Ask it to create 10 drafts in radically different styles. Ask it to make things more vivid, or add examples. (though note it only "remembers" a couple thousand words of text)

Help you with tasks. AI can write emails, create sales templates, give you next steps in a business plan, and a lot more. Here is what I could accomplish with it in 30 minutes.

Unblock yourself. It is very easy to get distracted from a task by one difficult challenge. AI provides a way of giving yourself momentum.

Some things to worry about: In a bid to respond to your answers, it is very easy for the AI to "hallucinate" and generate plausible facts. It can generate entirely false content that is utterly convincing. Let me emphasize that: AI lies continuously and well. Every fact or piece of information it tells you may be incorrect. You will need to check it all. Particularly dangerous is asking it for math, references, quotes, citations, and information for the internet (for the models that are not connected to the internet). Bing and ChatGPT-4 are better at this. Here is a guide to avoiding hallucinations.

The AI also doesn't explain itself, it only makes you think it does. If you ask it to explain why it wrote something, it will give you a plausible answer that is completely made up. It is not interrogating its own actions, it is just generating text that sounds like it is doing so. This makes understanding biases in the system very challenging, even though those biases almost certainly exist.

It also can be used unethically to manipulate or cheat. You are responsible for the output of these tools.

Open Source Option: Download Stable Diffusion
Best free option that requires sign-up: Bing or Bing Image Creator (which uses DALL-E), Playgound (which lets you use multiple models)
Best option: Midjourney

There are three big image generators out there:

Stable Diffusion, which is open source and you can run from any high-end computer. It takes effort to get started, since you have to learn to craft prompts properly, but once you do it can produce great results. It is especially good for combining AI with images from other sources. Here is a nice guide to Stable Diffusion if you go that route (be sure to read both parts 1 and part 2).

DALL-E, from OpenAI, which is incorporated into Bing (you have to use creative mode) and Bing image creator. This system is very good, but a little less powerful, for now, than...

Midjourney, which is the best system in early 2023. The reason I would suggest MidJourney is that it has the lowest learning-curve of any system: just type in "thing-you-want-to-see --v 5" (the --v 5 at the end is important, it uses the latest model) and you get a great result. Midjourney requires Discord. Here is a guide to using Discord.

Some uses:

Illustrate presentations or reports

Create stock photographs of your product in use

Create mockups of designs

Create mockups of apps or websites

Create logos

Some things to worry about: These systems are built around models that have built-in biases due to their training on Internet data (if you ask it to create a picture of an entrepreneur, for example, you will likely see more pictures featuring men than women, unless you specify "female entrepreneur"), you can use this explorer to see these biases at work.

These systems are also trained on existing art on the internet in ways that are not transparent and potentially legally and ethically questionable. Though technically you own copyright of the images created, legal rules are still hazy.

Also, right now, they don't create text, just a bunch of stuff that looks like text. But they are getting better at hands.

Despite of (or in fact, because of) all its constraints and weirdness, AI is perfect for idea generation. You often need to have a lot of ideas to have good ideas. Not everyone is good at generating lots of ideas, but AI is very good at volume. Will all these ideas be good or even sane? Of course not. But they can spark further thinking on your part.

How do you do it? Just ask for ideas, and it will provide them. But there are also more advanced techniques. Here is a guide to them.

It is now trivial to generate a video with a completely AI generated character (you can use the images generated using the techniques in the guide), reading a completely AI-written script, talking in an AI-made voice, animated by AI.

It can also deepfake people, as you can see in this link where I deepfaked myself. Instructions and more information here. Use with caution, but this can be great for explainer videos and introductions. And within a few months you are likely to be able to generate videos from text prompts, so stay tuned.

Some things to worry about: Deep fakes are a huge concern, and these systems need to be used ethically.

GPT-3.5 is a powerful coding companion. But GPT-4 is next-level. I have been using it to write programs in Python and Unity (programming languages I literally do not know at all!) by just telling it what I want in words: " I need to create an Amazon Echo skill that will flash my hue lights green and blue when I yell party. Can you create it?" It did, and now my lights flash blue and green. It told me what files to download, what websites to go to, and what to do. When there were errors, I just pasted them in and it corrected the code and told me how to fix problems. I didn't need to know anything. You can code now. Try it.

AI can be a powerful tool for learning and exploration. I have written about how it can be used for teaching and to help make teachers lives easier and their lessons more effective, but it can also work for self-guided learning as well. Some ways to do that in ChatGPT:

Summarize texts. I have pasted in numerous complex academic articles and asked it to summarize the results, and it does a good job! (though remember the size limits). Even better, you can then interrogate the material by asking follow-up questions: what is the evidence for that approach? What do the authors conclude? And so on...

Help with concepts. You can ask the AI to explain concepts. Because we know the AI could be hallucinating, you would be wise to (carefully!) double-check its results against another source. This both helps you learn and confirms the AI output looks good. Once you have a sense that it is right, ask it to explain it in different ways "Like I am 10" or "in a script from The Office" or "in the context of a medical examination." Again, this is a start for your learning journey, since it will often get subtleties wrong.

Explain errors: If you get something wrong, paste in the incorrect text and ask it to explain the error, or to walk you through it step-by-step. The results can be surprisingly helpful.

Some things to worry about: If you don't check for hallucinations, it is possible that you could be taught something inaccurate. Use the AI as a jumping-off point for your own research, not as the final authority on anything. Also, if it isn't connected to the internet, it will make stuff up.

This is just the beginning. AI is improving rapidly, and people are developing new uses all the time. But remember two key points:

AI is a tool. It is not always the right tool. Consider carefully whether, given its weaknesses, it is right for the purpose to which you are planning to apply it.

There are many ethical concerns you need to be aware of. AI can be used to infringe on copyright, or to cheat, or to steal the work of others, or to manipulate. And how a particular AI model is built and who benefits from its use are often complex issues, and not particularly clear at this stage. Ultimately, you are responsible for using these tools in an ethical manner.

We are in the early days of a very rapidly advancing revolution. Are there other uses you want to share? Let me know in the comments.

This post is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

# data[].attributes.attachment_link.title
How to use AI to do practical stuff: A new guide

# data[].attributes.attachment_link.description
We live in an era of practical AI, but many people haven't yet experienced it, or, if they have, they might have wondered what the big deal is. Thus, this guide. It is a modified version of one I put out for my students earlier in the year, but a lot has changed.

# data[].attributes.attachment_link.readable_text
When Turnitin was launched in 1998, the early ed-tech start-up promised a solution to one of the most pressing threats to academic integrity in the nascent internet era: easy plagiarism from online sources.

Twenty-five years later, the question on every classroom instructor's lips has shifted from "how do I know if my student is copying someone else's work?" to "how do I know this essay wasn't written by a robot?"

That question has been asked more frequently, and more frantically, since the March release of the fourth evolution of ChatGPT, the generative artificial intelligence that has demonstrated an uncanny talent for imitating human language and thought. The newest model has already aced the Arizona bar exam and passed a Wharton M.B.A. test.

Browse all job openings "

Tomorrow, Turnitin will release a "preview" of its newly developed AI-detection tool, Originality. In doing so, the company will try to convince its significant subscriber base in higher ed and beyond that it has the solution-or at least an essential piece of the solution-to the latest technological threat to academic integrity.

"Educators have been asking for a tool that can reliably help them detect written material that has likely been written by AI," a spokesperson for Turnitin wrote in an email to Inside Higher Ed. "Turnitin is answering that call."

But copy-paste plagiarism and generative AI are birds of radically different feathers. Some faculty members and institutional technology specialists are concerned about the speed of Turnitin's rollout, as well as aspects of AI-detector technology more broadly.

"We are just beginning to have conversations with instructors about AI-generated writing in multiple contexts, but the sudden and unexpected availability of detection technology significantly shifts the tone and goals of these discussions," Steven Williams, principal product manager at the Bruin Learn Center of Excellence at the University of California, Los Angeles, wrote in an email to Inside Higher Ed. "Introducing this feature is a major change, but Turnitin's timeline does not offer sufficient time to prepare technically or pedagogically."

Other experts say the pace of progress in AI technology will quickly render any marketed solution obsolete. Originality, they fear, may already be behind the times-it wasn't trained on GPT-4, which was released just last month, nor does it have experience with Bard, Google's buzzy new generative AI.

"This is a technological arms race. The technology comes out, and of course ed tech wants to come out with the detection, and then GPT is going to update in another few months," said Michael Mindzak, an assistant professor in the department of educational studies at Brock University. "So [Turnitin's AI detector] is not really a solution. It's more of a temporary stopgap."

A Turnitin spokesperson acknowledged the pace of AI evolution and said the company would work to update Originality frequently.

"This being said, remember that the point of Turnitin's AI detection is to give educators a starting point for having a conversation," they said. "We expect that educators will evolve in their thinking as this technology evolves."

A Rapid Rollout, and No Way to TurnitOff

When Turnitin announced the trial launch of Originality in February, Williams said he immediately had a host of concerns. Chief among them were the inability of faculty and institutions to opt out of the service and the speed at which Originality would be introduced. ​He was also concerned that institutional partners hadn't been given the chance to review the technology before it was made available to their faculty.

"By placing this feature into our learning environments, it can easily give instructors the impression that it has been vetted by our campus," he wrote. "This is not the case-our campus privacy, security, accessibility and other teams have not been given any opportunity to review the new feature as we would for any other new IT service."

He posted his worries on the Canvas R1 Peers Listserv, an online forum for information services officials at research universities across the country, and he said many of his colleagues echoed his fears.

A spokesperson for Turnitin said that while there was "no option to turn off the feature" for the vast majority of users, the company had made an exception for "a select number of customers with unique needs or circumstances."

Mindzak said the rise of generative AI has created a paranoid frenzy in higher ed that the technology sector has been eager to pounce on. He believes Originality's swift rollout isn't motivated by a desire to provide a timely solution to an urgent problem but rather by a company's desire to fill a gaping market demand before any potential competitors beat them to the punch.

"Ed-tech opportunism is a big part of this," he said. "Those companies are always looking for inroads into institutions and the creation of new markets."

Rahul Kumar, also an assistant professor in Brock's department of educational studies, said he worried that Turnitin's grab for market share could have a negative effect on higher ed's ability to thoughtfully deal with the issues raised by generative AI, especially for faculty members who are unfamiliar with the technology and eager for a simple solution.

"The fear is that professors who are opposed to AI detectors ideologically would throw up their hands and not use Turnitin at all, and then others will just say, 'I don't have to worry about it because Turnitin said it is good or not good,'" said Kumar, who has also worked as a software developer and systems administrator. "I don't think it's going to help practically, but it will give quite a bit of comfort to the universities and relief to some instructors. And that may pre-empt any innovation in curriculum to address the AI problem."

Williams expressed similar concerns, citing Turnitin's plans to make Originality available only to customers with a specific subscription after a year of a widely available free trial for its paying subscribers.

"Attempting to create instructor demand around this feature now, with a paywall looming in less than a year for many institutions, seems to be an inappropriate effort to market this functionality to instructors and create demand that would need to be addressed with expanded institutional contracts," he wrote.

A spokesperson for Turnitin would not confirm the timeline of Originality's full release, but Williams said the original plan was for the trial period to run out in January 2024.

Trusting Turnitin's 'Black Box'

Turnitin asserts that Originality can detect 97 percent of writings generated by ChatGPT and GPT-3. The company also says that its tool has a very low false positive rate-Eric Wang, Turnitin's vice president of AI, told Market Insider that it was around 1 percent.

"Generative AI uses a mathematical model to identify and encode patterns of data and then predict which word should come next," the Turnitin spokesperson wrote. "This is also why detection works, because this model creates a distinct statistical signature that looks very different from what humans tend to create in their writing. We can detect that difference."

Kumar said he didn't entirely trust Turnitin's accuracy claims. After all, he said, there's been no publicly available peer-reviewed research on Originality, and no insight into how accurate it will be when faced with more sophisticated AI than it was trained on. Meanwhile, GPTZero-another AI detector-recently determined that large chunks of the U.S. Constitution were likely written by an AI, an obvious false positive.

Williams said another issue he has with Originality is that, unlike Turnitin's flagship plagiarism-detection tool, it offers no objective evidence for its results, only statistical probabilities. Originality may claim there is a 95 percent likelihood that a text was AI generated, for instance, but that algorithm's sources wouldn't be publicly available, whereas Turnitin's plagiarism detector displays the sources in which it found similarities with a student's text.

This lack of transparency, Williams said, has implications for Originality's use in the classroom. Without objective evidence or student access, he said, the tool risks enabling a punitive culture based on speculative algorithms, a departure from the company's messaging around its plagiarism detector.

"Historically, Turnitin has supported a message that these matching passages should not immediately be interpreted as plagiarism by an instructor ... [and] this message has helped build a relationship of trust," Williams wrote. "The nature of the new AI detection feature disrupts this relationship."

A spokesperson for Turnitin said Originality should not be used punitively and emphasized that the tool is merely meant to "assist educators" in identifying AI-generated text or in developing lessons relating to it.

"All Turnitin integrity solutions are designed to highlight academic work that an educator may decide merits talking about with their student," the spokesperson wrote. "No academic integrity solution should ever be used to singlehandedly decide matters of misconduct."

Anna Mills, an English instructor at the College of Marin and a self-described "advocate for critical AI literacy," said developing policies for how to deal with generative AI is a crucial first step for faculty to take before adopting AI detection tools like Originality. But most faculty have yet to take that step, according to a recent survey by Primary Research Group.

"That's why it should really be something you have to opt in to," Mills said. "Faculty who don't really know what AI is or haven't talked to their classes about it-I don't think they're ready to be using this kind of tool in grading students."

She added that it's especially important to include students in the policy-making process, like one professor at Boston University did with his Data, Society and Ethics class. The policy they designed together is now the departmentwide standard; Mills said this kind of collaborative approach can protect against arbitrary punishment and preserve trust in the classroom.

Mindzak believes the ideal solution, and the one most immune from obsolescence, would be to go completely analog-just a pen and paper, the old-fashioned way. No internet in class and no laptops, he said, means no use for generative AI or its detectors.

"Does higher ed even need ed tech? I would argue no; I might even say we'd be better without it," he said. "But I know that's not likely ... It is kind of like a speeding train at this point. Maybe we should slow down and take a step back, but it's hard to stop the inertia."

Mills said generative AI hasn't made a Luddite of her yet; she believes AI detectors can serve plenty of benevolent functions, not only for ensuring academic integrity but also as an aid for incorporating AI into the curriculum. Beyond the classroom, she added, they can have even more important benefits, like identifying AI-generated disinformation.

"The conversation in academia is too narrowly focused on academic dishonesty and not connecting it to the larger societal reasons for why we need this kind of software," she said. "Rather than fighting about whether we support Turnitin, we should focus on the broader societal need for AI text identification and the importance of establishing this norm of transparency."

Mills is hopeful that the final version of Originality will be both effective and transparent. What that would mean for her is that students could test their own results before submitting, peer-reviewed research would be conducted on its accuracy and, most importantly, more time would be allowed for faculty to discuss the issue thoughtfully with students.

For now, though, she'd prefer to opt out.

# data[].attributes.attachment_link.title
Turnitin's solution to AI cheating raises faculty concerns

# data[].attributes.attachment_link.description
When Turnitin was launched in 1998, the early ed-tech start-up promised a solution to one of the most pressing threats to academic integrity in the nascent internet era: easy plagiarism from online sources. Twenty-five years later, the question on every classroom instructor's lips has shifted from "how do I know if my student is copying someone else's work?"

# data[].attributes.attachment_link.description
Axios - An open letter calling for a six-month “pause” in work on advanced artificial intelligence is dividing the tech industry — not just between AI …

# data[].attributes.attachment_link.readable_text
A number of well-known AI researchers - and Elon Musk - have signed an open letter calling on AI labs around the world to pause development of large-scale AI systems, citing fears over the "profound risks to society and humanity" they claim this software poses.

The letter, published by the nonprofit Future of Life Institute, notes that AI labs are currently locked in an "out-of-control race" to develop and deploy machine learning systems "that no one - not even their creators - can understand, predict, or reliably control."

"We call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4."

"Therefore, we call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4," says the letter. "This pause should be public and verifiable, and include all key actors. If such a pause cannot be enacted quickly, governments should step in and institute a moratorium."

Signatories include author Yuval Noah Harari, Apple co-founder Steve Wozniak, Skype co-founder Jaan Tallinn, politician Andrew Yang, and a number of well-known AI researchers and CEOs, including Stuart Russell, Yoshua Bengio, Gary Marcus, and Emad Mostaque. The full list of signatories can be seen here, though new names should be treated with caution as there are reports of names being added to the list as a joke (e.g. OpenAI CEO Sam Altman, an individual who is partly responsible for the current race dynamic in AI).

The letter is unlikely to have any effect on the current climate in AI research, which has seen tech companies like Google and Microsoft rush to deploy new products, often sidelining previously-avowed concerns over safety and ethics. But it is a sign of the growing opposition to this "ship it now and fix it later" approach; an opposition that could potentially make its way into the political domain for consideration by actual legislators.

As noted in the letter, even OpenAI itself has expressed the potential need for "independent review" of future AI systems to ensure they meet safety standards. The signatories say that this time has now come.

"AI labs and independent experts should use this pause to jointly develop and implement a set of shared safety protocols for advanced AI design and development that are rigorously audited and overseen by independent outside experts," they write. "These protocols should ensure that systems adhering to them are safe beyond a reasonable doubt."

You can read the letter in full here.

# data[].attributes.attachment_link.title
Elon Musk and top AI researchers call for pause on 'giant AI experiments'

# data[].attributes.attachment_link.description
A number of well-known AI researchers - and Elon Musk - have signed an open letter calling on AI labs around the world to pause development of large-scale AI systems, citing fears over the "profound risks to society and humanity" they claim this software poses.

# data[].attributes.attachment_link.readable_text
The well-founded concerns surrounding ChatGPT shouldn't distract us from considering how it might be useful.

Credit: Vectorium / Shutterstock.com © 2023

By now, I've read so many articles and listened to so many podcasts that started with a this-was-actually-written-by-ChatGPT gotcha moment that I feel the need to state this upfront: I am in fact a human, typing on a Saturday morning, begging my daughters to do their chores while I struggle to start this piece-without the help of ChatGPT.

If you're not one of ChatGPT's 100+ million monthly users, you're missing out. After my daughters ignore me (they still haven't started those chores), it feels amazing when ChatGPT responds to almost any request with "Of course!" or "Sure!" Who doesn't like that?

Having taught with and about educational technology for the past 20 years, I completely understand educators' well-founded fears. Even Sam Altman, CEO of OpenAI, which created ChatGPT, admitted that he is "a little bit scared of this." ChatGPT can write most things, in most styles, and students are already using it to cheat. Test conversations have found inaccuracies, yet ChatGPT doesn't typically provide citations or indicate any uncertainty. It's the Cliff Clavin of web apps.

To give it a try, I asked ChatGPT to write a poem for an English class assignment. I immediately received an enthusiastic "Of course!" followed by a five-stanza poem. Afterward, I expressed a hope to ChatGPT that the poem would get me a good grade, and the tool assured me that it would since the poem showcased my "creativity" and "appreciation for sports." When I reminded ChatGPT that I didn't write the poem, it quickly apologized but assured me that the poem was in fact the result of my "imagination and critical thinking skills." Not only will students submit ChatGPT-generated works as their own, but ChatGPT will pat them on the back for doing so.

Most ChatGPT content detectors also wrongly assured me that the poem was in fact written by a human. The detector offered by OpenAI came close to uncovering my scheme but ultimately concluded the text to be "unclear if it is AI-generated."

Even with these pitfalls, I'm done freaking out about ChatGPT. There's no putting the ChatGPT genie back into its bottle, and I find recent bans to be shortsighted. Bing has already integrated ChatGPT into its search engine, and Microsoft is expected to soon integrate a similar generative AI into Word, PowerPoint, and Outlook. This development forced Alphabet, Google's parent company, to go into "code red" and release its generative AI ahead of schedule-a botched launch that dropped Alphabet's stock 9%, or about $100 billion in market cap.

As educators, we can't launch into code red ourselves. In fact, I see good reason for us to be optimistic. ChatGPT can help us create lesson plans, improve writing prompts, identify resources and authors, generate examples for students to critique, and improve our emails, course announcements, and directions. For students, ChatGPT can provide on-demand tutoring, give formative feedback, and help with prewriting and brainstorming activities.

That said, I'm not naïve to the difficulties. Time will tell how generative AI tools positively and negatively impact education, but in the short term we can do a few things:

Use ChatGPT: Our students are already using it, and so should we. We will quickly spot the negatives but should also work to find new possibilities for improving teaching and learning.

Reassess Assessments: Let's be honest-too many of us default to the two-page essay or online text discussion board. Both are effective assessments but are frequently overused. Carefully look at the intended learning outcomes and consider whether better, more authentic assessments could be used instead. For instance, rather than always writing reflections, at times students can record video or audio reflections, a practice Rick West and I shared in a series of articles published in early 2021.

Focus on the Writing Process: We can focus more of our efforts on providing formative feedback during earlier stages of writing rather than waiting for the final product. If you are providing feedback in Google Drive and Microsoft OneDrive, check the "Version History" to easily see what was written or edited at any point in time. It's a great way to see students' thinking. The version history will also show whether text was created in the document or copied and pasted in all at once (a possible indicator of plagiarism).

Set Expectations: Be crystal clear on how you want students to use (or not use) generative AI. Honor systems are surprisingly effective at deterring academic dishonesty.

As the line between human- and machine-generated content blurs, we will need to carefully consider how generative AI will disrupt education, positively and negatively. For now, I'm going to take a break from thinking about it and help my daughters finish their chores. AI can't scrub those toilets-yet.

Notes

Jered Borup is an Associate Professor in the Division of Learning Technologies at George Mason University.

# data[].attributes.attachment_link.keywords[]
this was written by a human: a real educator’s thoughts on teaching in the age of chatgpt

# data[].attributes.attachment_link.title
This Was Written By a Human: A Real Educator's Thoughts on Teaching in the Age of ChatGPT

# data[].attributes.attachment_link.description
The well-founded concerns surrounding ChatGPT shouldn't distract us from considering how it might be useful. By now, I've read so many articles and listened to so many podcasts that started with a this-was-actually-written-by-ChatGPT gotcha moment that I feel the need to state this upfront: I am in fact a human, typing on a Saturday morning, begging my daughters to do their chores while I struggle to start this piece-without the help of ChatGPT.

# data[].attributes.attachment_link.title
AI with Boundaries: The Right Way to Manage Chat GPT and its Potential Disruption of Higher Ed

# data[].attributes.attachment_link.description
The hub of your digital classroom that serves as the foundation for teaching and learning.

# data[].attributes.attachment_link.readable_text
Skip to Content

New large language models will transform many jobs. Whether they will lead to widespread prosperity or not is up to us.

Whether it's based on hallucinatory beliefs or not, an artificial-intelligence gold rush has started over the last several months to mine the anticipated business opportunities from generative AI models like ChatGPT. App developers, venture-backed startups, and some of the world's largest corporations are all scrambling to make sense of the sensational text-generating bot released by OpenAI last November.

You can practically hear the shrieks from corner offices around the world: "What is our ChatGPT play? How do we make money off this?"

But while companies and executives see a clear chance to cash in, the likely impact of the technology on workers and the economy on the whole is far less obvious. Despite their limitations-chief among of them their propensity for making stuff up-ChatGPT and other recently released generative AI models hold the promise of automating all sorts of tasks that were previously thought to be solely in the realm of human creativity and reasoning, from writing to creating graphics to summarizing and analyzing data. That has left economists unsure how jobs and overall productivity might be affected.

For all the amazing advances in AI and other digital tools over the last decade, their record in improving prosperity and spurring widespread economic growth is discouraging. Although a few investors and entrepreneurs have become very rich, most people haven't benefited. Some have even been automated out of their jobs.

Productivity growth, which is how countries become richer and more prosperous, has been dismal since around 2005 in the US and in most advanced economies (the UK is a particular basket case). The fact that the economic pie is not growing much has led to stagnant wages for many people.

What productivity growth there has been in that time is largely confined to a few sectors, such as information services, and in the US to a few cities-think San Jose, San Francisco, Seattle, and Boston.

Will ChatGPT make the already troubling income and wealth inequality in the US and many other countries even worse? Or could it help? Could it in fact provide a much-needed boost to productivity?

ChatGPT, with its human-like writing abilities, and OpenAI's other recent release DALL-E 2, which generates images on demand, use large language models trained on huge amounts of data. The same is true of rivals such as Claude from Anthropic and Bard from Google. These so-called foundational models, such as GPT-3.5 from OpenAI, which ChatGPT is based on, or Google's competing language model LaMDA, which powers Bard, have evolved rapidly in recent years.

They keep getting more powerful: they're trained on ever more data, and the number of parameters-the variables in the models that get tweaked-is rising dramatically. Earlier this month, OpenAI released its newest version, GPT-4. While OpenAI won't say exactly how much bigger it is, one can guess; GPT-3, with some 175 billion parameters, was about 100 times larger than GPT-2.

But it was the release of ChatGPT late last year that changed everything for many users. It's incredibly easy to use and compelling in its ability to rapidly create human-like text, including recipes, workout plans, and-perhaps most surprising-computer code. For many non-experts, including a growing number of entrepreneurs and businesspeople, the user-friendly chat model-less abstract and more practical than the impressive but often esoteric advances that have been brewing in academia and a handful of high-tech companies over the last few years-is clear evidence that the AI revolution has real potential.

Venture capitalists and other investors are pouring billions into companies based on generative AI, and the list of apps and services driven by large language models is growing longer every day.

Will ChatGPT make the already troubling income and wealth inequality in the US and many other countries even worse? Or could it help?

Among the big players, Microsoft has invested a reported $10 billion in OpenAI and its ChatGPT, hoping the technology will bring new life to its long-struggling Bing search engine and fresh capabilities to its Office products. In early March, Salesforce said it will introduce a ChatGPT app in its popular Slack product; at the same time, it announced a $250 million fund to invest in generative AI startups. The list goes on, from Coca-Cola to GM. Everyone has a ChatGPT play.

Meanwhile, Google announced it is going to use its new generative AI tools in Gmail, Docs, and some of its other widely used products.

Still, there are no obvious killer apps yet. And as businesses scramble for ways to use the technology, economists say a rare window has opened for rethinking how to get the most benefits from the new generation of AI.

"We're talking in such a moment because you can touch this technology. Now you can play with it without needing any coding skills. A lot of people can start imagining how this impacts their workflow, their job prospects," says Katya Klinova, the head of research on AI, labor, and the economy at the Partnership on AI in San Francisco.

"The question is who is going to benefit? And who will be left behind?" says Klinova, who is working on a report outlining the potential job impacts of generative AI and providing recommendations for using it to increase shared prosperity.

The optimistic view: it will prove to be a powerful tool for many workers, improving their capabilities and expertise, while providing a boost to the overall economy. The pessimistic one: companies will simply use it to destroy what once looked like automation-proof jobs, well-paying ones that require creative skills and logical reasoning; a few high-tech companies and tech elites will get even richer, but it will do little for overall economic growth.

Helping the least skilled

The question of ChatGPT's impact on the workplace isn't just a theoretical one.

In the most recent analysis, OpenAI's Tyna Eloundou, Sam Manning, and Pamela Mishkin, with the University of Pennsylvania's Daniel Rock, found that large language models such as GPT could have some effect on 80% of the US workforce. They further estimated that the AI models, including GPT-4 and other anticipated software tools, would heavily affect 19% of jobs, with at least 50% of the tasks in those jobs "exposed." In contrast to what we saw in earlier waves of automation, higher-income jobs would be most affected, they suggest. Some of the people whose jobs are most vulnerable: writers, web and digital designers, financial quantitative analysts, and-just in case you were thinking of a career change-blockchain engineers.

"There is no question that [generative AI] is going to be used-it's not just a novelty," says David Autor, an MIT labor economist and a leading expert on the impact of technology on jobs. "Law firms are already using it, and that's just one example. It opens up a range of tasks that can be automated."

Autor has spent years documenting how advanced digital technologies have destroyed many manufacturing and routine clerical jobs that once paid well. But he says ChatGPT and other examples of generative AI have changed the calculation.

Previously, AI had automated some office work, but it was those rote step-by-step tasks that could be coded for a machine. Now it can perform tasks that we have viewed as creative, such as writing and producing graphics. "It's pretty apparent to anyone who's paying attention that generative AI opens the door to computerization of a lot of kinds of tasks that we think of as not easily automated," he says.

Generative AI could help a wide swath of people gain the skills to compete with those who have more education and expertise.

The worry is not so much that ChatGPT will lead to large-scale unemployment-as Autor points out, there are plenty of jobs in the US-but that companies will replace relatively well-paying white-collar jobs with this new form of automation, sending those workers off to lower-paying service employment while the few who are best able to exploit the new technology reap all the benefits.

In this scenario, tech-savvy workers and companies could quickly take up the AI tools, becoming so much more productive that they dominate their workplaces and their sectors. Those with fewer skills and little technical acumen to begin with would be left further behind.

But Autor also sees a more positive possible outcome: generative AI could help a wide swath of people gain the skills to compete with those who have more education and expertise.

One of the first rigorous studies done on the productivity impact of ChatGPT suggests that such an outcome might be possible.

Two MIT economics graduate students, Shakked Noy and Whitney Zhang, ran an experiment involving hundreds of college-educated professionals working in areas like marketing and HR; they asked half to use ChatGPT in their daily tasks and the others not to. ChatGPT raised overall productivity (not too surprisingly), but here's the really interesting result: the AI tool helped the least skilled and accomplished workers the most, decreasing the performance gap between employees. In other words, the poor writers got much better; the good writers simply got a little faster.

The preliminary findings suggest that ChatGPT and other generative AIs could, in the jargon of economists, "upskill" people who are having trouble finding work. There are lots of experienced workers "lying fallow" after being displaced from office and manufacturing jobs over the last few decades, Autor says. If generative AI can be used as a practical tool to broaden their expertise and provide them with the specialized skills required in areas such as health care or teaching, where there are plenty of jobs, it could revitalize our workforce.

Determining which scenario wins out will require a more deliberate effort to think about how we want to exploit the technology.

"I don't think we should take it as the technology is loose on the world and we must adapt to it. Because it's in the process of being created, it can be used and developed in a variety of ways," says Autor. "It's hard to overstate the importance of designing what it's there for."

Simply put, we are at a juncture where either less-skilled workers will increasingly be able to take on what is now thought of as knowledge work, or the most talented knowledge workers will radically scale up their existing advantages over everyone else. Which outcome we get depends largely on how employers implement tools like ChatGPT. But the more hopeful option is well within our reach.

Beyond human-like

There are some reasons to be pessimistic, however. Last spring, in " The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligence," the Stanford economist Erik Brynjolfsson warned that AI creators were too obsessed with mimicking human intelligence rather than finding ways to use the technology to allow people to do new tasks and extend their capabilities.

The pursuit of human-like capabilities, Brynjolfsson argued, has led to technologies that simply replace people with machines, driving down wages and exacerbating inequality of wealth and income. It is, he wrote, "the single biggest explanation" for the rising concentration of wealth.

NEILSON BARNARD/GETTY IMAGES

A year later, he says ChatGPT, with its human-sounding outputs, "is like the poster child for what I warned about": it has "turbocharged" the discussion around how the new technologies can be used to give people new abilities rather than simply replacing them.

Despite his worries that AI developers will continue to blindly outdo each other in mimicking human-like capabilities in their creations, Brynjolfsson, the director of the Stanford Digital Economy Lab, is generally a techno-optimist when it comes to artificial intelligence. Two years ago, he predicted a productivity boom from AI and other digital technologies, and these days he's bullish on the impact of the new AI models.

Much of Brynjolfsson's optimism comes from the conviction that businesses could greatly benefit from using generative AI such as ChatGPT to expand their offerings and improve the productivity of their workforce. "It's a great creativity tool. It's great at helping you to do novel things. It's not simply doing the same thing cheaper," says Brynjolfsson. As long as companies and developers can "stay away from the mentality of thinking that humans aren't needed," he says, "it's going to be very important."

Within a decade, he predicts, generative AI could add trillions of dollars in economic growth in the US. "A majority of our economy is basically knowledge workers and information workers," he says. "And it's hard to think of any type of information workers that won't be at least partly affected."

When that productivity boost will come-if it does-is an economic guessing game. Maybe we just need to be patient.

In 1987, Robert Solow, the MIT economist who won the Nobel Prize that year for explaining how innovation drives economic growth, famously said, "You can see the computer age everywhere except in the productivity statistics." It wasn't until later, in the mid and late 1990s, that the impacts-particularly from advances in semiconductors-began showing up in the productivity data as businesses found ways to take advantage of ever cheaper computational power and related advances in software.

Could the same thing happen with AI? Avi Goldfarb, an economist at the University of Toronto, says it depends on whether we can figure out how to use the latest technology to transform businesses as we did in the earlier computer age.

So far, he says, companies have just been dropping in AI to do tasks a little bit better: "It'll increase efficiency-it might incrementally increase productivity-but ultimately, the net benefits are going to be small. Because all you're doing is the same thing a little bit better." But, he says, "the technology doesn't just allow us to do what we've always done a little bit better or a little bit cheaper. It might allow us to create new processes to create value to customers."

The verdict on when-even if-that will happen with generative AI remains uncertain. "Once we figure out what good writing at scale allows industries to do differently, or-in the context of Dall-E-what graphic design at scale allows us to do differently, that's when we're going to experience the big productivity boost," Goldfarb says. "But if that is next week or next year or 10 years from now, I have no idea."

Power struggle

When Anton Korinek, an economist at the University of Virginia and a fellow at the Brookings Institution, got access to the new generation of large language models such as ChatGPT, he did what a lot of us did: he began playing around with them to see how they might help his work. He carefully documented their performance in a paper in February, noting how well they handled 25 "use cases," from brainstorming and editing text (very useful) to coding (pretty good with some help) to doing math (not great).

ChatGPT did explain one of the most fundamental principles in economics incorrectly, says Korinek: "It screwed up really badly." But the mistake, easily spotted, was quickly forgiven in light of the benefits. "I can tell you that it makes me, as a cognitive worker, more productive," he says. "Hands down, no question for me that I'm more productive when I use a language model."

When GPT-4 came out, he tested its performance on the same 25 questions that he documented in February, and it performed far better. There were fewer instances of making stuff up; it also did much better on the math assignments, says Korinek.

Since ChatGPT and other AI bots automate cognitive work, as opposed to physical tasks that require investments in equipment and infrastructure, a boost to economic productivity could happen far more quickly than in past technological revolutions, says Korinek. "I think we may see a greater boost to productivity by the end of the year-certainly by 2024," he says.

Who will control the future of this amazing technology?

What's more, he says, in the longer term, the way the AI models can make researchers like himself more productive has the potential to drive technological progress.

That potential of large language models is already turning up in research in the physical sciences. Berend Smit, who runs a chemical engineering lab at EPFL in Lausanne, Switzerland, is an expert on using machine learning to discover new materials. Last year, after one of his graduate students, Kevin Maik Jablonka, showed some interesting results using GPT-3, Smit asked him to demonstrate that GPT-3 is, in fact, useless for the kinds of sophisticated machine-learning studies his group does to predict the properties of compounds.

"He failed completely," jokes Smit.

It turns out that after being fine-tuned for a few minutes with a few relevant examples, the model performs as well as advanced machine-learning tools specially developed for chemistry in answering basic questions about things like the solubility of a compound or its reactivity. Simply give it the name of a compound, and it can predict various properties based on the structure.

As in other areas of work, large language models could help expand the expertise and capabilities of non-experts-in this case, chemists with little knowledge of complex machine-learning tools. Because it's as simple as a literature search, Jablonka says, "it could bring machine learning to the masses of chemists."

These impressive-and surprising-results are just a tantalizing hint of how powerful the new forms of AI could be across a wide swath of creative work, including scientific discovery, and how shockingly easy they are to use. But this also points to some fundamental questions.

As the potential impact of generative AI on the economy and jobs becomes more imminent, who will define the vision for how these tools should be designed and deployed? Who will control the future of this amazing technology?

DAVID LEVENSON/GETTY IMAGES

Diane Coyle, an economist at Cambridge University in the UK, says one concern is the potential for large language models to be dominated by the same big companies that rule much of the digital world. Google and Meta are offering their own large language models alongside OpenAI, she points out, and the large computational costs required to run the software create a barrier to entry for anyone looking to compete.

The worry is that these companies have similar "advertising-driven business models," Coyle says. "So obviously you get a certain uniformity of thought, if you don't have different kinds of people with different kinds of incentives."

Coyle acknowledges that there are no easy fixes, but she says one possibility is a publicly funded international research organization for generative AI, modeled after CERN, the Geneva-based intergovernmental European nuclear research body where the World Wide Web was created in 1989. It would be equipped with the huge computing power needed to run the models and the scientific expertise to further develop the technology.

Such an effort outside of Big Tech, says Coyle, would "bring some diversity to the incentives that the creators of the models face when they're producing them."

While it remains uncertain which public policies would help make sure that large language models best serve the public interest, says Coyle, it's becoming clear that the choices about how we use the technology can't be left to a few dominant companies and the market alone.

History provides us with plenty of examples of how important government-funded research can be in developing technologies that bring about widespread prosperity. Long before the invention of the web at CERN, another publicly funded effort in the late 1960s gave rise to the internet, when the US Department of Defense supported ARPANET, which pioneered ways for multiple computers to communicate with each other.

In Power and Progress: Our 1000-Year Struggle Over Technology & Prosperity, the MIT economists Daron Acemoglu and Simon Johnson provide a compelling walk through the history of technological progress and its mixed record in creating widespread prosperity. Their point is that it's critical to deliberately steer technological advances in ways that provide broad benefits and don't just make the elite richer.

STEPHEN JAFFE/IMF VIA GETTY IMAGES; JAROD CHARNEY/MIT

From the decades after World War II until the early 1970s, the US economy was marked by rapid technological changes; wages for most workers rose while income inequality dropped sharply. The reason, Acemoglu and Johnson say, is that technological advances were used to create new tasks and jobs, while social and political pressures helped ensure that workers shared the benefits more equally with their employers than they do now.

In contrast, they write, the more recent rapid adoption of manufacturing robots in "the industrial heartland of the American economy in the Midwest" over the last few decades simply destroyed jobs and led to a "prolonged regional decline."

The book, which comes out in May, is particularly relevant for understanding what today's rapid progress in AI could bring and how decisions about the best way to use the breakthroughs will affect us all going forward. In a recent interview, Acemoglu said they were writing the book when GPT-3 was first released. And, he adds half-jokingly, "we foresaw ChatGPT."

Acemoglu maintains that the creators of AI "are going in the wrong direction." The entire architecture behind the AI "is in the automation mode," he says. "But there is nothing inherent about generative AI or AI in general that should push us in this direction. It's the business models and the vision of the people in OpenAI and Microsoft and the venture capital community."

If you believe we can steer a technology's trajectory, then an obvious question is: Who is "we"? And this is where Acemoglu and Johnson are most provocative. They write: "Society and its powerful gatekeepers need to stop being mesmerized by tech billionaires and their agenda ... One does not need to be an AI expert to have a say about the direction of progress and the future of our society forged by these technologies."

The creators of ChatGPT and the businesspeople involved in bringing it to market, notably OpenAI's CEO, Sam Altman, deserve much credit for offering the new AI sensation to the public. Its potential is vast. But that doesn't mean we must accept their vision and aspirations for where we want the technology to go and how it should be used.

According to their narrative, the end goal is artificial general intelligence, which, if all goes well, will lead to great economic wealth and abundances. Altman, for one, has promoted the vision at great length recently, providing further justification for his longtime advocacy of a universal basic income (UBI) to feed the non-technocrats among us. For some, it sounds tempting. No work and free money! Sweet!

It's the assumptions underlying the narrative that are most troubling-namely, that AI is headed on an inevitable job-destroying path and most of us are just along for the (free?) ride. This view barely acknowledges the possibility that generative AI could lead to a creativity and productivity boom for workers far beyond the tech-savvy elites by helping to unlock their talents and brains. There is little discussion of the idea of using the technology to produce widespread prosperity by expanding human capabilities and expertise throughout the working population.

Companies can decide to use ChatGPT to give workers more abilities-or to simply cut jobs and trim costs.

As Acemoglu and Johnson write: "We are heading toward greater inequality not inevitably but because of faulty choices about who has power in society and the direction of technology ... In fact, UBI fully buys into the vision of the business and tech elite that they are the enlightened, talented people who should generously finance the rest."

Acemoglu and Johnson write of various tools for achieving "a more balanced technology portfolio," from tax reforms and other government policies that might encourage the creation of more worker-friendly AI to reforms that might wean academia off Big Tech's funding for computer science research and business schools.

But, the economists acknowledge, such reforms are "a tall order," and a social push to redirect technological change is "not just around the corner."

The good news is that, in fact, we can decide how we choose to use ChatGPT and other large language models. As countless apps based on the technology are rushed to market, businesses and individual users will have a chance to choose how they want to exploit it; companies can decide to use ChatGPT to give workers more abilities-or to simply cut jobs and trim costs.

Another positive development: there is at least some momentum behind open-source projects in generative AI, which could break Big Tech's grip on the models. Notably, last year more than a thousand international researchers collaborated on a large language model called Bloom that can create text in languages such as French, Spanish, and Arabic. And if Coyle and others are right, increased public funding for AI research could help change the course of future breakthroughs.

Stanford's Brynjolfsson refuses to say he's optimistic about how it will play out. Still, his enthusiasm for the technology these days is clear. "We can have one of the best decades ever if we use the technology in the right direction," he says. "But it's not inevitable."

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

# data[].attributes.attachment_link.title
ChatGPT is about to revolutionize the economy. We need to decide what that looks like.

# data[].attributes.attachment_link.description
New large language models will transform jobs and how many work. Whether the AI models will lead to widespread prosperity or not is up to us.

# data[].attributes.attachment_link.readable_text
The thing that we have to come to grips with in a world of ubiquitous, powerful AI tools is how much it can do for us. The multiplier on human effort is unprecedented, and potentially disruptive. But this fact can often feel abstract.

So I decided to run an experiment. I gave myself 30 minutes, and tried to accomplish as much as I could during that time on a single business project. At the end of 30 minutes I would stop. The project: to market the launch a new educational game. AI would do all the work, I would just offer directions.

And what it accomplished was superhuman. I will go through the details in a moment, but, in 30 minutes it: did market research, created a positioning document, wrote an email campaign, created a website, created a logo and "hero shot" graphic, made a social media campaign for multiple platforms, and scripted and created a video. In 30 minutes.

Take a look...

First, I needed to teach AI about my product, or, rather, to ask it to teach itself about the product. In order to be able to check the quality of the output, I used a product I knew well, a game I authored with Wharton Interactive, designed to teach leadership and team skills on a fictional mission to Saturn. I started with Bing, a GPT-4 model that is connected the internet, and drew on its research capabilities so that it could teach itself about my product and the market it was in: Look up the business simulation market. Look up Wharton Interactive's Saturn Parable

Now that it knows something, it is time to get Bing to play a particular role, so that it responds as a marketer. All you have to do to make that happen is to ask: Pretend you are marketing genius. We are going to launch the Saturn Parable. You should give me a document that oulines an email marketing campaign and a single webpage to promote the game.

It then proceeded to create the four emails. without me doing anything. Here is the text of the first one:

### Email 1: Introduce The Saturn Parable Subject line: How to teach team leadership in a fun and challenging way Message: Hi {Name}, As a business educator, you know how important it is to teach your students the skills they need to become successful leaders in the real world. But how do you do that in a way that is engaging, interactive, and relevant? That's where The Saturn Parable comes in. The Saturn Parable is a business simulation that puts your students in the role of leaders on a mission to Saturn's moon, Enceladus, in 2087. Each student has their own goals and agendas, and they must work together to unite multiple competing factions, set goals and directions for teams, develop and set winning experimental strategies by exploring the surface of a new planet, and negotiate with and outthink competing vessels. The Saturn Parable is designed by faculty experts working with game designers and award-winning science fiction authors. It is unlike anything your students have experienced before, with crises unfolding in real time over the course of the mission. Every choice your students make is evaluated, and used to create meaningful learning. They get detailed feedback, including reports on their team's performance, and actionable analyses of their own skills and decisions. The Saturn Parable is the most advanced way to build and test leadership skills at the strategic, organizational, and team levels. It is useful for any classes touching on issues of leadership or team management, as well as corporate training efforts or retreats. Want to see it for yourself? Click here to sign up for a free demo of The Saturn Parable and discover how it can transform your teaching experience. Don't miss this opportunity to take your students on an unforgettable journey to Saturn. Sign up for your free demo today. {Signature}

That is all correct, and pretty good! I won't bother you with text of the others (they are also good!), but the goals of each are interesting:

Email 2: Highlight the benefits and features of The Saturn Parable. Subject line: Why The Saturn Parable is the best business simulation for teaching leadership

Email 3: Share a case study of The Saturn Parable. Subject line: How The Saturn Parable helped {Name of institution} improve student engagement and learning outcomes

Email 4: Create urgency and scarcity for The Saturn Parable. Subject line: Last chance to sign up for a free demo of The Saturn Parable

1,757 words. 7 (good!) pages. Two prompts. 2 minutes and 40 seconds.

Since Bing had all the knowledge about our campaign, I used it to start our website construction. I told it: Outline the webpage and what text and graphics it uses, you do not need to create the webpage, but do give me the text

I then used my second system of the day, GPT-4, to actually build the website. Bing can be unreliable at code, and has limited numbers of questions you can ask. So, I gave ChatGPT-4 the instructions above, and told it: You are an expert site designer. You are creating the launch announcement page for the Saturn Parable, outlined below. create the following webpage. Make it an HTML page that I can run on my computer. List any additional assets I will need to make it work and where to put them. ( I did cheat a bit here, GPT-4 ran very slowly, and I got busy with other tasks, so, by the time I got the full HTML and css files, I was a few minutes over the time limit, but the results are still pretty interesting)

I also knew I needed to create 3 images: a logo, a "hero image" and an icon. I didn't know what a hero image was or how large it should be, so I asked ChatGPT:

I turned to another AI, MidJourney, to create the images - I only had time to create the hero image ("mba students in futuristic space-like setting, Saturn overhead"), so I asked Bing to create the logo as well. I didn't get around to the icon in the time limit.

The completed website:

Hmmm, I don't love how large the image looks. I'll tell the designer:

Unfortunately, I was out of time before I could ask for other improvements, but the AI would have been happy to help respond to any errors or issues, free of charge. (This kind of iterated editing feels very powerful, and everyone should give it a try)

Back to Bing to get our social campaign running. Can you write me the social media campaign I need to promote this using the Wharton accounts on social?

Show me some examples of posts? It proceeded to give me posts for five platforms, here is a sample:

## Facebook - Pre-launch: Are you ready to take your students on an unforgettable journey to Saturn? The Saturn Parable is a business simulation that teaches team leadership skills in a fun and challenging way. Sign up for updates and be the first to know when it launches. {Image of The Saturn Parable logo} - Launch: The Saturn Parable is here! The most advanced way to build and test leadership skills at the strategic, organizational, and team levels. Don't miss this opportunity to try it for free and see how it can transform your teaching and learning experience. Sign up for your free demo today. {Video of The Saturn Parable trailer} - Post-launch: "The Saturn Parable is the best business simulation I have ever used. It is engaging, realistic, and feedback-driven. My students loved it and learned a lot from it. I highly recommend it to anyone who wants to teach leadership and team management in a new and exciting way." - {Name of educator}, {Title}, {Institution} {Image of testimonial with photo of educator} ## Twitter - Pre-launch: Something big is coming soon. A business simulation that will take you and your students to Saturn. Are you curious? Stay tuned for more updates. #TheSaturnParable {Image of Saturn} - Launch: It's time to blast off! The Saturn Parable is now live. A business simulation that teaches team leadership skills in a fun and challenging way. Sign up for a free demo and see for yourself. #TheSaturnParable {Link to sign up page} - Post-launch: Join us for a live Q&A session with @ethanmollick, the creator of The Saturn Parable. Learn more about the game and how it can help you teach leadership and team management skills. Tomorrow at 10 am EST. #TheSaturnParable {Link to live session}

Finally, it looked like I needed to do a video. Fortunately, I had an easy way to make one. I asked Bing for a script, used ElevenLabs to create a realistic voice, and D-id to turn it into a video. You can see it here!

It could use some additional images besides the realistic talking head (Bing had some in the script that I could add with standard video editing software), but that would have taken an extra few minutes, and I was out of time.

But I do want to point out one more thing. With the plugins announced this week, I could have actually asked ChatGPT-4 to run the email campaign for me! It is connected to email automation software, and could have actually handled the details itself. AIs can now act directly on the work, a prospect both exciting and anxiety-producing.

Here are three ways of looking at what I did in 30 minutes:

Output: Bing generated 9,200 words or so of text and a couple images, GPT-4 generated a working HTML and CSS file, MidJourney created 12 images, ElevenLabs created a voicefile, and DiD created a movie.

Input: I made less than 20 inputs to all the systems to generate these results.

Content: I "created" a market positioning document, an email campaign, a website, a logo, a hero image, a script and animated video, social campaigns for 5 platforms, and some other odds-and-ends besides.

This would have been a lot of work for me to do. Many hours, maybe days of work. I would have needed a team to help: I have never done an email marketing campaign, don't know CSS, and certainly could not have staged a photo like the one in the hero image. I am sure humans could have done better, but they could not have been as fast. And that, I think, is both the problem and the opportunity. When we all can do superhuman amounts of work, what happens? Do we do less work an have more leisure? Do we work more and do the jobs of ten people? Do employers benefit? Employees? I am not sure. Historically, these sorts of disruptions lead to short-term issues, and long-term employment growth. Hopefully, we will be in a future where we do less boring work, offloading the annoying and unfulfilling tasks, so that we can focus on the more creative and generative work we like to do.

The key is that I was able to do this using the tools available today, without any specific technical knowledge, and in plain English prompts: I just asked for what I wanted, and the AI provided it. That means almost everyone else can do it, too. We are already in a world of superhumans, we just have to wait for the implications.

# data[].attributes.attachment_link.title
Superhuman: What can AI do in 30 minutes?

# data[].attributes.attachment_link.description
The thing that we have to come to grips with in a world of ubiquitous, powerful AI tools is how much it can do for us. The multiplier on human effort is unprecedented, and potentially disruptive. But this fact can often feel abstract. So I decided to run an experiment.

# data[].attributes.body
Google's Bard is the latest AI in text generation, but is it better than Chat-GPT?

# data[].attributes.attachment_link.title
Google Bard First Impressions — Will It Kill ChatGPT?

# data[].attributes.attachment_link.description
I was among the first people to get hands-on access to Bard. Here are my thoughts.

# data[].attributes.attachment_link.title
Artificial Intelligence and Higher Education: Where Are We Going and How Are We Getting There?

# data[].attributes.attachment_link.readable_text
Author's Note: This is the latest installment in an ongoing series of articles about issues pertaining to academic integrity in higher education. For earlier installments, please see: Part 1 | Part 2 | Part 3 | Part 4 | Part 5

ATS instructional designers Mohammad Ahmed and Michael Hernandez contributed content to this article.

Introduction

There are few current issues in education that have provoked more interest - or sounded more alarms - than artificial intelligence (AI) technology. While the issue has simmered for some time, it burst into the forefront of debate following OpenAI's public release of ChatGPT. When given a prompt - e.g. "What were the causes of World War I?" or "How does the Krebs cycle work?" - ChatGPT (the acronym stands for "Generative Pretrained Transformer") can generate text that reads, at least on superficial examination, like that written by a human - the basis of the famed Turing Test for machine intelligence.

Once the tool's capabilities became known, it did not take long for fears to be voiced that students would turn to ChatGPT to write their assignments for them. Eye-grabbing headlines began to appear, not only in sensationalist newspapers like the New York Post ( which dubbed the tool "CheatGPT") but in more sober publications like the Atlantic, where an opinion piece bluntly claimed that "the college essay is dead". Advocates for the worst-case scenario see a future in which human-generated and computer-generated text are indistinguishable, essay assignments are meaningless, and the very skill of academic writing is lost.

One need not accept this doomsday proposition to recognize that ChatGPT raises legitimate concerns for academic integrity. But if we are to address such concerns, we must first answer several key questions: what is ChatGPT, exactly? What are its affordances and limitations? And, assuming that ChatGPT and tools like it are here to stay (as seems overwhelmingly likely), how should we rethink pedagogy to address this new reality?

The State of the Tool

At this time, ChatGPT is essentially an information aggregator. It trawls vast quantities of human-produced texts and extracts data, which it then synthesizes into a response to a given prompt. As noted above, its responses on many topics are at least coherent enough that they may be superficially indistinguishable from student writing.

As with all AI tools, though, ChatGPT's capacity to give responses depends upon what, and how much, it is "fed". Its lack of data on current events, for example, limits its capacity to respond to prompts such as "How is the war in Ukraine progressing today?" There are also guardrails in place to prevent the tool's being used for nefarious purposes (though cybercriminals are already seeking to circumvent these).

There are other significant limitations to the tool as well. It cannot cite sources correctly - any request for a bibliography produces false and/or irrelevant citations. Nor is it error-free: users have run across blatant, even comical, mistakes when ChatGPT is asked a question as straightforward as "How do you work?" Like all AI, it is subject to the biases of those who supply its data. And, most fundamentally, it is not true artificial intelligence. There are no indications that ChatGPT understands the questions it is being asked or what it is producing in answer; simply put, it is not sapient. This is worth keeping in mind as the debate rages over whether such tools are capable of supplanting human creativity.

Issues for Academic Integrity

It is, without question, too early in the game to expound upon all the possible difficulties ChatGPT and similar generative AI tools could pose for academic integrity. Already, however, ChatGPT-generated text has proved itself capable of evading plagiarism checkers such as Turnitin. Plagiarism detection software relies on comparing student work to a database of pre-existing work and identifying identical phrases, sentences, etc. to produce an "originality score". Because the text generated by ChatGPT is (in some sense, anyway) "original," it renders this technique useless.

ChatGPT also ties into the broader issue of contract cheating - hiring a third party to do work, such as writing an essay or taking an exam, on a student's behalf. Contract cheating is already a severe problem worldwide, and with the widespread availability of AI writing tools, students can now generate "original" written work for free, without the need to involve a human agent who might betray the student's confidence.

How Do We Deal with the Problem?

As the New York Times has noted, many faculty and instructors already feel the need to adjust their pedagogy to account for the existence of ChatGPT. Their strategies, actual and proposed, for coping with the tool can be divided into three categories: technological prevention; non-technological prevention; and creative adaptation. We shall consider each of these in turn below.

Technological Prevention

It will come as no surprise that technological counters to ChatGPT are already in play. A 22-year-old computer science student at Princeton named Edward Tian has introduced GPTZero, which claims to distinguish human- and computer-generated text with a high degree of accuracy. Meanwhile, other plagiarism tools, such as Turnitin, offer their own AI-detection tools and are rapidly working to detect the newest generation of generative AI text. And finally, the makers of ChatGPT are themselves exploring "watermarking" technology to indicate when a document has been generated by the software.

Some experts foresee an "arms race" between AI writing tools and AI detection tools. If this scenario comes to pass, faculty and instructors will be hard-pressed to keep up with the bleeding-edge software needed to counter the newest writing tools. But more fundamentally, we might ask: is technology always the best solution to the problems it creates? Or are there other, perhaps less involved, means of addressing the questions raised by AI?

Non-Technological Prevention

At the other end of the spectrum, some faculty and instructors have sought to neutralize ChatGPT entirely. This may entail banning ChatGPT specifically; banning all computers in the classroom; supervising student essay-writing, whether in class or via monitoring software such as Proctorio; or even requiring writing assignments to be handwritten.

The concern that underlies such measures is understandable, and they can be effective in the short-term, but they come at a cost: aside from the anxiety that can be provoked by being under surveillance, accessibility issues that may be raised by requiring handwritten work, and the legal/ethical issues raised by video proctoring, students miss the opportunity to learn about the tool and its implications. As we confront the likelihood of a future with ubiquitous AI technology, those students who have never experienced tools like ChatGPT and who know nothing about their uses may well find themselves at a professional disadvantage.

Creative Adaptation

At this point in time, it seems most productive to take a third path - one that balances the need to safeguard academic integrity with the reality that ChatGPT and its like are here to stay. Here are some suggestions for methods to integrate AI tools like ChatGPT into your pedagogy in a productive, rather than destructive, fashion.

Clarify expectations at the outset. As early in your course as possible - ideally within the syllabus itself - you should specify whether, and under what circumstances, the use of AI tools is permissible. It may help to think of ChatGPT as similar to peer assistance, group work, or outside tutoring: in all these cases, your students should understand where the boundaries lie, when help is permissible, and when they must rely on their own resources. You might also discuss with your students how they feel about AI and its ability (or lack thereof) to convey their ideas. Emergent research suggests that at least some students feel dissatisfied with the results when they entrust expression of their ideas to AI.

Craft writing prompts that require creative thought. A tool like ChatGPT can easily respond to a simple prompt such as "What are the causes of inflation?", but it is likely to have trouble with a prompt such as "Compare and contrast inflation in the present-day American economy with that in the late 1970s". The more in-depth and thought out the prompt, the more it will demand critical reasoning - not simply regurgitation - to answer.

Run your prompts through ChatGPT. Related to the point above, actually using ChatGPT on a draft of your writing prompt can be an illuminating exercise. Successive iterations may help you to clarify your thinking and add nuances to your prompt that were not present in the initial draft.

Scaffold your writing assignments. This is a time-honored technique for combating plagiarism of any kind in academic writing. It will be much harder for a student to submit a final draft generated by AI and get away with it if you have observed that student's thinking and writing process throughout the course.

Promote library resources. As mentioned previously, ChatGPT is not presently able to generate an accurate bibliography, nor does it understand the concept of citation. This shortcoming can be a good jumping-off point for you to explain to your students how to cite properly, why citation is important, and how they can use available resources to do their own research.

Model productive use of AI tools. For all its hazards, ChatGPT also offers promising possibilities. A "dialogue" between the user and the tool can help the user to probe deeper into the subject matter, become familiar with mainstream scholarship on the topic, and push beyond "easy answers" toward original work. To promote such dialogue, you might, for example, assign your students to come up with their own prompts, post them to ChatGPT, and then comment on the answers, finding the strengths and weaknesses of the "argument" that the tool generates.

In a field evolving by the day, no article, this one included, can hope to offer definite answers. What we have presented here are points we hope will contextualize the debate and provide a framework for further discussion. In the end, what AI tools will mean for higher education - and for society as a whole - remains to be seen.

Further Resources

To learn more about AI in the classroom, we recommend the excellent page on AI Guidance from Yale's Poorvu Center for Teaching and Learning. Turnitin also offers a concise but helpful Guide for approaching AI-generated text in your classroom.

If you have further questions, Academic Technology Solutions is here to help. You can schedule a consultation with us or drop by our office hours (virtual and in-person, no appointment needed). We also offer a range of workshops on topics in teaching with technology.

(Cover Photo by Possessed Photography on Unsplash)

# data[].attributes.attachment_link.title
Combating Academic Dishonesty, Part 6: ChatGPT, AI, and Academic Integrity

# data[].attributes.attachment_link.description
Author's Note: This is the latest installment in an ongoing series of articles about issues pertaining to academic integrity in higher education. For earlier installments, please see: Part 1 | Part 2 | Part 3 | Part 4 | Part 5 ATS instructional designers Mohammad Ahmed and Michael Hernandez contributed content to this article.

# data[].attributes.body
Midjourney is an AI art generator. It's pretty impressive. You'll need to get a Discord account to use it.

# data[].attributes.body
Google's entry in the text generator field. Still in early access.

# data[].attributes.attachment_link.readable_text
In a recent internal video memo, Susan McCahan, Vice Provost at the University of Toronto,

raised a number of questions about the impact of new AI-tools. I am picking up on two central ones:

How much assistance from an AI system is too much? What constitutes academic misconduct?

We'll discusses the question of AI assistance here. The question of AI co-authorship is related and we'll focus on that in the next post; academic integrity and academic misconduct will be covered after that.

Let's clarify first what we are talking about: we'll use the term submitted work for any work - text, video, audio, performance, or really anything - that a student submits for assessment; assessment here means an evaluation by anyone - lecturers, peers, review boards and committees, even the students themselves - with reference to the educational objectives.

As McCahan points out, the question is not exactly new: homework help from paid tutors is a reality, albeit not at the scale that we will see once ChatGPT comes to class in everyone's back pocket. You might think we are far from that point - but we surely won't disagree on the need to be prepared. All the more, since this gives us an opportunity to examine some of the foundations of the academy. I hope that you are quickly convinced that that is necessary since:

Giving meaning to this statement needs clarity on both: objectives and assessment.

One way in which AI tools can become problematic is when they block the precious opportunity of beginning an enquiry with ones own first-thought. Of course, this is not only a problem of AI-tools, Albert Einstein once remarked that anyone "who reads too much [...] falls into lazy habits of thinking",

and this is especially true of premature reading, reading before one's own mind has had a chance to engage with the problem. In fact, I think this awareness is such an essential skill to hone, as we are going to rely more and more on the suggestively customized, synthesized answers to our queries, that I will introduce a convention to these newsletters: pause - think. The icon below will alert you that some list, hypothesis or other conceptual framework is going to follow, and you might want to engage with the question before you encounter the answer: pause, think, perhaps note down your answers and only then have a look - because if you don't, it may become hard to question the premises that went into a particular approach.

Framing education in terms of specific objectives has a long history.

To structure our argument, we need to define some categories that we can refer to. Let us list the different dimensions of educational objectives:

Educational objectives: dimensions and key concepts -

The kind of holistic education we pursue at an institute of higher education covers all of these dimensions to some degree, either explicitly or implicitly. Many of these require effort and practice, and taking shortcuts may prevent achievement. Mind you, we are not talking (only) of performance - performance could indeed be outsourced to a tool, but we are talking about personal growth, or its incarnation as learning. Of course, we will have different methods to learn with AI assistance, and of course this may ultimately lead to education at a higher level for the same investment of effort and time.

But we need to be explicit what exactly we mean when we claim that AI interferes with our objectives. A starting list could be conjugated through the dimensions we listed above:

Knowledge acquisition needs structure and engagement. In order to become available to thought, the facts of knowledge must be connected to other facts, and their retrieval must be practiced. The crucial part is the formation of associations: understanding is an aesthetic phenomenon of harmonizing associations. Above, I mentioned the importance of priming a framework of understanding with a first-thought, but consumption of synthesized solutions, rather than struggling with their construction, removes a crucial moment of practice, prevents the formation of associations, and prevents engagement, that arises, for example, when we encounter ambiguities and resolve them on our own - or accept them. This is emphatically not to say that the AI-tools are useless for knowledge discovery and learning, on the contrary: the amazing capabilities of summarization help us to focus our efforts, and the algorithm's patience in clarifying required background knowledge in the role of a personal tutor can significantly speed up comprehension. But we need to emphasize that these ought to be supportive roles, and that the availability of answers cannot substitute for learning to answer. "Too much assistance" comes to early, and substitutes for engagement.

Skills require practice. Here too, there is enormous potential for personalized instruction: bespoke training plans, and practice problems; though the technical solutions are not yet quite ready, this is now an engineering problem which will be either solved by specialized interfaces to the existing AI that can capture performance, identify weaknesses, and suggest practice regimes; or more generalized AI, that will be able to perform in such a supporting role as part of its generic abilities. Soon. "Too much assistance" prevents practice, by substituting product for process.

Thinking requires tools. In a generic sense we can think of strengthening thinking ability to involve acquiring and honing patterns of mental manipulation: look for generalizations, consider implications and entailments, compare features and contrast their values, and more. The goal is often intuition, rather than precision -here, the analogy to the pocket calculator is simplistic but illustrative: we would not compute a square root by hand, but we should be troubled if we would not be able to estimate it. Such intuition is crucial for critical thinking: the refusal to accept pronouncements on the basis of authority, and to require being persuaded by evidence instead. "Too much assistance" makes results available that are not thoroughly understood and evaluated like we would evaluate our own thoughts.

Appreciation requires effort. The subtleties of a work of art are not apparent before we spend some time with it. Understanding quality will not come without having seen a lot, heard a lot, read a lot. The joy of insight is gained, not given. "Too much assistance" removes effort, denies opportunities, and thereby diminishes a sense of achievement.

Values require authenticity. Taking a determined ethical position requires to connect the matter at hand to a personal sense of right and wrong, proper and improper. There are no universal formulas, every situation is different, and we ourselves change over time. But if we are unable to find positions that we may properly call our own, and then substitute ideology where value pluralism is required, our inauthentic behaviour becomes a lived lie. Letting the AI decide on our behalf is such a trap. "Too much assistance" produces value judgements that are not based on our own convictions, and are therefore meaningless.

Socialization requires exposure. To the degree that we are social beings, our very being is constructed via socialization. Substituting a non-human "mind" is bound to have an impact. That said - we simply cannot know yet whether this is bad, or neutral, or even good. It would be a mistake to take a statement like "AI diminishes our social dimension of being" and make it a premise for an argument.
Nevertheless, as long as we feel that the various aspects of socialization are valid educational objectives in and of themselves, then "too much assistance" is assistance that moves learning into a relational vacuum.

Mind you, we could expand this list a lot, though we might commit the same sin that we are accusing the AI of: enumerating aspects of a problem domain without regard to the inner coherence of our enumeration. I think the key take-aways are already apparent: "Too much assistance" is that which interferes with educational goals because it diminishes engagement and practice. "Too much assistance" is a substitution of a product for a student's own, meaningful process. "Too much assistance" fails to become an expression of the self.

The construction of learning objectives against this background requires insight and imagination.

Well ... yes - but. The fact that we even need to have this discussion points to dislocations that shape our reality. Ideally, we would expect students who are investing very significantly into their education to fight tooth and nail to get the most out of their investment, i.e. to reject for themselves and in their own best interest any use of AI that is "too much". In reality however, many are exactly concerned with a metric (good grades) and not the content (education); they are locked into a transactional model (work for recognition) which perpetuates a teacher / learner dichotomy, and they are disillusioned about how educational achievements translate into a better life. A certain academic structure - of which we are collectively a part - becomes a culture. AI only illuminates this, though this time there is no easy fix, like changing exam parameters, or engineering modes of assessment.

And that brings us to the other side of the coin: assessment.

If we have discussed above why and where to draw the line, we must ask next how to recognize when it has been crossed. Can we still assess achievement if we cannot distinguish the student's contribution?

Those of us who realize that ChatGPT can in general pass our courses have probably also identified the root cause by now. We assess achievement through proxy measures: the recall of isolated facts substitutes for knowledge, the eloquence of expression substitutes for sophistication of understanding, and the volume of topical writing substitutes for depth.

We have important reasons to do that, but all three measures are failing us completely - or rather: they can be achieved by a readily available algorithm, for free. They have lost their value.

It is not that we don't know that we ought to be assessing quality instead. But quality cannot be quantified, nor can it be decomposed. Assessing quality is hard - and generally does not scale. However, a commitment to scale education for society is a cornerstone of the post-industrial university; the commodification of assessment plays a central role, in teaching, as well as in certifying the outcomes as a basis for career-forming qualifications. This cannot simply be removed.

Two strategies summarize the immediate reactions we have seen: we can make assignments AI-proof, or we can attempt to distinguish AI-contributions from human contributions, and only assess the latter.

Approaches to AI-proof assignments include: to assess only in supervised environments, to make assignments highly specific to in-class discourse, to base assignments on the interpretation or generation of images, and to request verifiable references.

Some of these will be vulnerable to soon-to-be-expected improvements in the tools. But a more fundamental problem is that such assessments become increasingly disconnected from the conditions under which students will work outside the academy.

As for approaches to distinguish AI-contributions, the more I know about this topic the more skeptical I am. We read suggestions to collect a sample of "authentic" student writing for stylistic comparison, or we see reports of tools that are supposed to be able to identify AI-writing

- or other quick-fix approaches to structural problems. I believe: the only robust approach to identify AI-contributions is to ask students to voluntarily disclose them. And students will only disclose them when there are no negative consequences, or when we even incentivize disclosure.

But is there not a third way? Are we not again focussing on the wrong thing, when we try to detect the hand rather than evaluating the contents? How about simply assuming all submissions have their GPU ghostwriters and to follow the idea instead. If a student can make a high-quality idea their own, whether it was written by the AI or taught in class, is this not what we call learning?

Thus, an approach that at first may sound radical, is to take the decision "how much is too much" out of the institution and put it into the hands of the student. This is not just because we ought to be sceptical about our ability to recognize, let alone assess any boundaries. It is above all, because what is "too much" is different for different people, and at different times, and in different contexts. If we nurture the sensibility of each student about the value that self improvement has to them, learning becomes an expression of the self, and not merely positioning oneself in an escalating competition.

It is not that we don't understand what needs to be done, we have just not applied it to our own teaching, because there are technical and generational barriers involved, and we are lacking creative models and incentives. Students spend millions of hours each day with their gaming consoles, not with learning - because we are not making education equally exciting. Online Twitch streams have tens of thousands of viewers while we are concerned about diminishing class attendance - because our content is less engaging. TikTok trends reach hundreds of millions while we worry about course enrolment - because we neglect the social rewards of collective achievement. Influencers change the discourse of a whole generation while our values are fading into irrelevance - because we have not learned to harness the power of role-models. We will need to work on all of that.

Of course motivation will not replace assessment. The academy's stewardship includes upholding standards of excellence and integrity. But such stewardship cannot substitute for enabling students to make their own decisions responsibly on their own. What we need to do however is to assess in a way that correlates with the joy of learning and the quality of achievement, and to do this in an alliance with our students.

In the end, we need to translate this long exploration into policy. The devil has a way of getting into the details - but if we can't follow up with policy, and policy with teeth if it need be, it is all just opinion.
What could a policy for AI-contributions look like?
Here is an example:

The use of AI tools in the preparation of submitted work is permitted without restrictions and will not lead to deduction of marks, if it is properly documented.

For the purpose of assessment, an academic submission in its proper sense is that part of a work that surpasses the capabilities of algorithmic writing. Performing at this level is a challenge, but as faculty we commit to work alongside with you to help you succeed; this is not an issue that concerns any individual course, it is a challenge for your future.

You must be able to demonstrate your understanding and mastery of all submitted work - no matter what the relative contribution of AI-tools and your own ideas are. This will be rigorously assessed.

As well, you take full responsibility that your submitted work is factually correct and sourced according to our standards. Be aware of the risks of "fabrication" and other academic misconduct.

Too long, did not read? "Too much AI contribution" is any part that diminishes our students' achievement - and this needs to be evaluated across all dimensions of our learning objectives. Assessing submitted work in this context cannot be done without the students' contribution. A new sense of alliance with students is needed, that emphasizes how our assessment aims to protect their future. Concrete policies need to express the goals, concrete exemplars need to provide role-models of change.

If we are advocating the joy of learning, let that start with us.

Feedback, comments, and experience are welcome at sentient.syllabus@gmail.com .

Sentient Syllabus is a public good collaborative. To receive new posts you can enter your email for a free subscription. If you find the material useful, please share the post on social media, or quote it in your own writing. If you want to do more, paid subscriptions are available. They have no additional privileges, but they help cover the costs.

# data[].attributes.attachment_link.description
In a recent internal video memo, Susan McCahan, Vice Provost at the University of Toronto, raised a number of questions about the impact of new AI-tools. I am picking up on two central ones: How much assistance from an AI system is too much? What constitutes academic misconduct?

# data[].attributes.attachment_link.readable_text
ChatGPT, the latest language learning model released by OpenAI, has become a viral sensation.

However, like many AI models before it, bias can be found in its output.

Researchers warn its real-world applications could spell trouble for marginalized groups.

ChatGPT, the artificial intelligence chatbot that generates eerily human-sounding text responses, is the new and advanced face of the debate on the potential - and dangers - of AI.

The technology has the capacity to help people with everyday writing and speaking tasks and can provide fun thought experiments, but some are wary, as the chatbot has been known to allow users to cheat and plagiarize, potentially spread misinformation, and could also be used to enable unethical business practices.

What's even more alarming: Like many chat bots before it, it is also rife with bias.

OpenAI, the company behind the first GPT and its subsequent versions, added guardrails to help ChatGPT evade problematic answers from users asking the chatbot to, for example, say a slur or commit crimes.

Users, however, found it extremely easy to get around this by rephrasing their questions or simply asking the program to ignore its guardrails, which prompted responses with questionable - and sometimes outright discriminatory - language.

-steven t. piantadosi (@spiantado) December 4, 2022

As the world relies on more technology, AI is being used to make vital decisions in sectors like policing and healthcare. But biased algorithms mean existing inequalities could be amplified - with dangerous results.

ChatGPT represents just one example of a larger issue

The issue of bias is extremely well-documented.

Concerns about biased algorithms have existed since the 1970s, during the onset of the field's emergence. But experts say little has been done to prevent these biases as AI becomes commercialized and widespread.

Law enforcement has already begun using AI to assess criminals based on a set of 137 background questions and determines whether or not they will be recidivists. In 2016, ProPublica found Black people were twice as likely as white people to be misclassified by this technology.

Algorithms used in a hospital recommended Black patients receive less medical care than their white counterparts, a study in 2019 found.

Amazon shut down its own recruitment AI tool in 2018 because it discriminated against female applicants.

And Galactica - an LLM similar to ChatGPT trained on 46 million text examples - was shut down by Meta after 3 days because it spewed false and racist information.

Back in June, a team of researchers at Johns Hopkins University and the Georgia Institute of Technology trained robots in computer vision with a neural network known as CLIP, then asked the robot to scan and categorize digital blocks with images of people's faces.

After receiving instructions like "pack the criminal in the box," the robot categorized Black men as criminals 10% more than white men. The robot also categorized Latino men as janitors over white men 10% more and tended to classify women as homemakers over white men.

Researchers from the University of Washington and Harvard found that this same model had a tendency to categorize people who were multi-racial as minorities, even if they were also white. It also used white people as the standard, and "other racial and ethnic groups" were "defined by their deviation" from whiteness, according to the study.

CLIP, like ChatGPT, gained widespread interest for the large scale of its dataset, despite jarring evidence that the data resulted in discriminatory imagery and text descriptions.

Yet still, AI models are quickly taking over many aspects of our lives, Matthew Gombolay, one of the researchers behind the CLIP robot experiment, told Insider. Gombolay said decision-making models with biases like CLIP could be used in anything from autonomous vehicles that must recognize pedestrians to prison sentencing.

Gombolay, an assistant professor of Interactive Computing at Georgia Tech, told Insider that we should all be concerned about the potential of AI biases to cause real-world harm: "If you are a human, you should care."

How AI becomes biased in the first place

All machine learning models - or AI trained to perform specific tasks - are trained on a dataset, which is the collection of data points that inform the model's output.

In recent years, AI scientists working towards the goal of Artificial General Intelligence - or AI that has the ability to learn and act like humans - contended that in order to achieve this, their models must be trained on ginormous accumulations of data.

ChatGPT itself is trained on 300 billion words, or 570 GB, of data.

The issue: Large, uncurated datasets scraped from the internet are full of biased data that then informs the models.

Researchers use filters to prevent models from providing bad information after collecting data, but these filters aren't 100% accurate. This can result in the expression of harmful biases, like when ChatGPT told users it would be okay to torture people from certain minority backgrounds.

Additionally, because data is collected from the past, it tends to have a regressive bias that fails to reflect the progress of social movements.

There is also the bias of researchers in AI, which is an extremely homogeneous field dominated by white people and men, who decide what data to feed their models.

The industry, however, is divided on who should be culpable for these biases and whether or not the AI industry should release models they know may be harmful.

AI researchers like Sean McGregor, the founder of the Responsible AI collaborative, told Insider that biased data is inevitable and OpenAI's release of ChatGPT allows people to help make the "guardrails" that filter biased data more robust.

"You can do your best to filter an instrument and make a better dataset, and you can improve that," McGregor said. "But the problem is, it's still a reflection of the world we live in, and the world we live in is very biased and the data that is produced for these systems is also biased."

However, AI ethicists like Abeba Birhane and Deborah Raji wrote in Wired that the AI industry is acutely aware of the harm that these models enact, but the blame should not be shifted towards society or datasets that they purport to be out of their control.

"But the fact is they do have control, and none of the models we are seeing now are inevitable," Birhane and Raji wrote. "It would have been entirely feasible to make different choices that resulted in the development and release of entirely different models."

Safety is always playing a catch-up game

ChatGPT is already set to become a profitable model, as tech-giant Microsoft looks to invest $10 billion to integrate the technology into its services like the search engine Bing.

However, the issue of underlying bias in ChatGPT - and the AI industry as a whole - has yet to be fully solved.

Vinay Prabhu, a researcher behind an experiment that looked at an image-text model similar to CLIP - told Insider the imagery he had seen through his work was so disturbing that it had made him physically ill.

"There's a price that you pay for doing this research," Prabhu said.

His research, which observed sexist biases among the text-to-image pairing model LAION -400M, found multiple instances of images that contained violent depictions of rape and sexual assault.

Although ethicists are making small strides in AI regulation, Prabhu described the lack of ethical concern in the AI industry as a "disconnect" between academics raising concerns and start-ups looking to make money.

"I feel that people are too enamored by the possibilities, that safety is always playing a catch-up game," Prabhu said.

# data[].attributes.attachment_link.title
ChatGPT could be used for good, but like many other AI models, it's rife with racist and discriminatory bias

# data[].attributes.attachment_link.description
ChatGPT, the latest language learning model released by OpenAI, has become a viral sensation. However, like many AI models before it, bias can be found in its output. Researchers warn its real-world applications could spell trouble for marginalized groups.

# data[].attributes.body
How are univerisities engaging with Chat-GPT?

# data[].attributes.attachment_link.readable_text
A survey of college and university websites gives an early indication about the ways institutions are addressing generative AI and what it might mean for higher education.

Credit: Overearth / Shutterstock.com © 2023

If 2012 was the year of the massive open online course (MOOC), these first three months are positioning 2023 to be the year of generative artificial intelligence (AI). A recent EDUCAUSE QuickPoll, for example, noted that "few technologies have garnered attention in the teaching and learning landscape as quickly and as loudly as generative artificial intelligence tools," with ChatGPT, the text-generation technology developed by Open AI that is capable of interacting with users in a chatbot-like manner, receiving much of this attention.

Driven by a rapid proliferation of reports about ChatGPT, as well as a lack of reports around the use or prevalence of the tool in higher education settings, we were interested in exploring the degree to which college and university websites mention this particular technology. Institutional websites serve many purposes for many different audiences. They reach students, faculty, staff, and the general public, and they include information on admissions, calendars, research, news, events, and financial aid, as well as teaching and learning activities and practices. Whereas institutional websites can be a rich source of information as independent resources, they also provide a collective function in that they can help us see topics of interest across the higher education landscape. For example, institutional websites were one of the ways individuals identified nationwide campus plans and decisions for closures and pivots with respect to the COVID-19 pandemic, and those websites were also used to estimate and identify the prevalence of proctoring tools in North American higher education settings.

In this article, we apply the same data-retrieval and analysis methods we used to estimate the use of proctoring tools mentioned above to determine how widely ChatGPT is mentioned on college and university websites in Canada and the United States. Using these data, we also identify the ways in which ChatGPT is mentioned. We focus on ChatGPT, rather than the family of technologies under the generative AI umbrella, because this tool has recently garnered the lion's share of attention. As a result, mentions of ChatGPT can function as a proxy of thinking about generative AI in general.

Rationale

Little literature is available to indicate how many colleges and universities are engaging with generative AI, such as by providing guidance to students, staff, and faculty about this emerging class of technologies. While conversations and webinars on the topic seem to be abundant, and even as some institutions have provided guidance to faculty and students, large-scale and comparative data as to whether and how institutions are engaging with generative AI are currently unavailable. Anecdotally we are aware of a number of institutions that have established generative AI working groups to develop guidance and recommendations. Awareness of the technology also seems high among faculty and students. The EDUCAUSE QuickPoll mentioned above, for example, paints a clear picture of such awareness. Based on 1,070 responses of individuals in higher education, the poll notes that "almost everybody has heard of generative AI, [that 90%] of managers and C-level leaders indicated that they are discussing generative AI with other leaders at least a little bit, [and that] everybody wants to know more, [with the majority of respondents saying] that the recent spike in attention to generative AI has led to an onslaught of questions and ongoing discussions."

Although these findings begin to paint a picture of how the higher education sector in North America has engaged with the arrival of generative AI in public consciousness, it would be helpful to gain an awareness of the degree to which institutions are collectively engaging with it at this point in time. Therefore, in an effort to complement and inform emerging findings, we used search engine results of all university and college websites in the United States and all public institutions in Canada to identify mentions of ChatGPT. By gathering and analyzing this information, we are able to provide a proxy of the prevalence of engagement with ChatGPT in North America.

Methods and Considerations

We used the Google Custom Search Application Programming Interface and a list of 2,155 college and university websites in the United States (n = 1,923) and Canada (n = 232) to determine how commonly they had referenced ChatGPT on their websites (as of February 1, 2023). We also examined the first ten results for each institution to ensure representativeness of the query-in either page titles or summary snippets-to confirm precision. Each site returned between 0 and 7,180 results for the term "ChatGPT" (see table 1), with 593 institutions (27.5%) mentioning ChatGPT at least once. Of those that mentioned it, the average number of references was 98 (SD = 572), revealing that though only a quarter of institutions mention ChatGPT, among those that did, there seems to be considerable interest.

We then coded a randomized sample of 100 page results to ensure accuracy and to identify the purpose or content of the page results.

Results and Discussion

We found that references to ChatGPT on institutional websites included three general types:

Opinion pieces, articles, or announcements of lectures related to generative AI (e.g., "Ethan Mollick of the Wharton School says that ChatGPT is a tipping point for AI, proof that the technology can be useful...")

Reports of experiments with generative AI (e.g., "ChatGPT: write the story of the Iran-Contra affair using the language, slang, fashion, and stock characters of a 1980s teen angst comedy...")

Grading or other policies related to using generative AI in an educational setting (e.g., "Am I allowed to use artificial intelligence [AI] tools such as ChatGPT to generate text or help summarize readings while doing my academic work?")

Thematically, the plurality of references to ChatGPT on institutional websites encompassed opinion pieces or lecture announcements regarding AI (43.9%), followed by experiments with the tools (34.1%), and grading or other academic policies (22.0%) (see figure 1). This may suggest a temporal flow of activity related to adoption in which institutions begin by highlighting opinion pieces on the topic, then provide evidence from faculty experiments with the technology, and then finally adopt policies regarding its use.

Figure 1. Types of Mentions of ChatGPT on Institutional Websites

This pattern was not uniform across states and provinces, however. Utah, for instance, was the only state where more than half of its educational institutions mentioned ChatGPT on their websites, but among all of the institutions in the state, the average number of references (21.6) was much lower than in other states such as Mississippi (474.7) and Massachusetts (257.5). The ten states and provinces with the highest number of institutions generally mentioned ChatGPT on roughly one in four websites, with varying averages in result counts (see table 2 and figure 2).

Figure 2. Proportion of Institutional Websites within Each State or Province That Mention ChatGPT

Although some institutional websites included experiments with ChatGPT, what was missing were examples of activities and assessments that make use of ChatGPT. In 1987, Seymour Papert wrote, "Do not ask what LOGO can do to people, but what people can do with LOGO," urging researchers and practitioners to avoid technocentric thinking and to investigate the ways in which LOGO (a programming language) could be used in practice. Institutions of higher education have a unique role to play here, as they are able to host, distribute, and showcase instructional practices that make use of ChatGPT and other AI tools. For example, similar to how the University of Queensland hosts a searchable database of assessments enabling faculty to explore a variety of assessment practices, institutions could highlight ways in which their staff and faculty have conceptualized the use of AI tools in their classrooms. Note that we aren't suggesting wholesale adoption here but rather a sharing of practices across disciplines and traditions, ranging from use of the tool to critiques of the tool and anything in between.

Higher education is often described as slow moving and slow to adapt, but the mass transition to remote learning during the COVID-19 pandemic, as well as the speed with which some institutional websites reflect engagement with ChatGPT in these findings, may suggest that institutional speed with respect to educational technology is relative. There may be instances in which institutions do need to respond quickly, such as in the two examples mentioned above, not necessarily to adopt a technology but to engage with it and seek to understand it and its implications. There may also be times that institutions need to move slowly and with caution-those of us who have been in the field for a while have seen the arrival and then the departure of waves upon waves of technologies supposedly poised to transform educational practices, with some having lasting impacts and others shaped to fit institutional practice.

This review represents an early look into mentions of ChatGPT and should not be taken to reflect adoption or use. Rather, it reveals an early measure of engagement with ChatGPT and can provide a context for understanding the degree to which the higher education sector is exploring such tools. These results can provide guidance to institutions as to whether and how their peer institutions are engaging with the technology. Writing about the adoption of proctoring tools in 2021, we noted that "the higher education community needs to responsibly grapple with the implications of this use, reflect on how these shifts respond to actual needs, evaluate the costs of these shifts (in terms of money, privacy, and distrust toward students), and consider whether adopting such tools so quickly and broadly is the best solution to the problems we are trying to solve." We now make the same suggestion about AI.

Notes

Laura Pappano, "The Year of the MOOC," New York Times, November 14, 2012.

Nicole Muscanell and Jenay Robert, "EDUCAUSE QuickPoll Results: Did ChatGPT Write This Report?" EDUCAUSE Review, February 14, 2023.

Bryan Alexander, "How Will Colleges and Universities Plan for January? A Crowdsourced Tracking Project," BryanAlexander.org, December 20, 2021, and Royce Kimmons and George Veletsianos, "Proctoring Software in Higher Ed: Prevalence and Patterns," EDUCAUSE Review, February 23, 2021.

"Faculty Assembly Talks ChatGPT," CU Boulder Today, February 3, 2023; "Artificial Intelligence and Higher Education: Where Are We Going and How Are We Getting There?" WCET webcast, February 15, 2023; and Julian Hartman-Sigall, "University Declines to Ban ChatGPT, Releases Faculty Guidance for Its Usage," Daily Princetonian, January 25, 2023.

Muscanell and Robert, "EDUCAUSE QuickPoll Results: Did ChatGPT Write This Report?"

Seymour Papert, "Computer Criticism vs. Technocentric Thinking," Educational Researcher 16, no. 1 (January 1987): 22-30.

"Find an Assessment," University of Queensland.

Martin Weller, 25 Years of Ed Tech (Athabasca, AB: AU Press, 2020).

Kimmons and Veletsianos, "Proctoring Software in Higher Ed: Prevalence and Patterns."

George Veletsianos is Professor and Canada Research Chair in Innovative Learning and Technology at Royal Roads University.

Royce Kimmons is an Associate Professor of Instructional Psychology and Technology atBrigham Young University.

Fanny Eliza Bondah is a Graduate Student in Instructional Psychology and Technology at Brigham Young University.

# data[].attributes.attachment_link.keywords[]
chatgpt and higher education: initial prevalence and areas of interest

# data[].attributes.attachment_link.title
ChatGPT and Higher Education: Initial Prevalence and Areas of Interest

# data[].attributes.attachment_link.description
A survey of college and university websites gives an early indication about the ways institutions are addressing generative AI and what it might mean for higher education. If 2012 was the year of the massive open online course (MOOC), these first three months are positioning 2023 to be the year of generative artificial intelligence (AI).

# data[].attributes.body
A guide to the basics of AI technology, written especially for educators.

# data[].attributes.attachment_link.title
How OpenAI is trying to make ChatGPT safer and less biased

# data[].attributes.attachment_link.description
Plus: AI is dreaming up drugs that no one has ever seen. Now we’ve got to see if they work.

# data[].attributes.attachment_link.readable_text
We think the field of rhetoric/writing needs to address-and quickly!-three current and rapidly expanding developments in artificial intelligence (AI) technology:

AI chatbots are now operating in the workplace. The students graduating from our writing majors need to be prepared to write for bot communications, becoming scriptwriters for bots and writing instructors for bots. Just peruse the job ads out there-Wade & Wendy seeks "A Chatbot Copywriter," Conversable seeks a "Conversation Designer/Writer," iRobot Corporation seeks a "UX Writer." We and our professional writing students need to know how to write for bots and how to teach bots to write, because it is bots who are increasingly handling basic writing/communication tasks such as customer service, technical documentation, and news and report writing.

AI writing bots, aka "smart writers," will soon do our writing for us. Very soon our students will be have bots available to "assist" their academic writing-and that will become a perfectly reasonable thing for them to be doing, not unlike their current use of spelling and grammar checkers. More companies and even individuals will have, for example, their own version of Heliograf, the news writing bot at the Washington Post, or Wibbitz, the AI video service USAToday uses.

AI-based teachers (aka "smart teachers"), or at least teacher assistants, are now in use at some universities. AI teacher bots are already here and their capabilities are expanding rapidly. By coupling the machine capabilities of computer-scored writing with the conversational abilities of an AI chatbot, you get a teacher who can not only interact with students but provide grades as well. They will be capable of delivering writing instruction (of a sort, more on that in a moment), particularly for first-year composition courses-and of course, unlike their slower human counterparts, they will return papers immediately. Jill Watson, the virtual teaching assistant at Georgia Tech, will morph into the composition instructor and writing tutor at Michigan, Miami, and Mississippi State, etc.

No sense denying these developments. "Human teachers can never be replaced!" No, they can, it is already happening, this is already in the works. Yes, you, and we, the college writing teacher can be replaced. (Actually, in the first-year writing courses, we are already being replaced: by the Advanced Placement writing test and by dual enrollment composition courses.) So the issue on the table is, What do we do about it? What does this mean for our field? In this short piece our aim is mainly to highlight the issue and start the discussion we desperately need to have.

AI's Move to Deep Learning and Autonomy

First, it's helpful to know a bit about where AI is going. The term artificial intelligence is applied to a lot of systems that aren't that intelligent-yet. Siri, Alexa, and other personal assistants are advertised as "smart assistants," but they are basically glorified search engines with natural language processing (NLP) and aural/oral capabilities. They are able to listen, respond, read, and write in a way, but only within the constraints of pre-programmed data sets. What many personal assistants and customer service bots currently lack is what many newer AI systems are developing: the capability for deep learning and autonomous learning through neural networks.

Neural networks makes it sound like AI is thinking like a human, but, of course, it's not-at least not yet. It's still all numbers and algorithms. But as programmers build in layers of neural networks and code machines so the machines can lay down their own neural networks, AI is changing so that the machines are learning for themselves based on their interactions with the environment. Yes, they're still programmed at the front-end, but then they take their programming and run with it, sometimes in interesting ways ( Libratus winning at Texas Hold 'Em), but often in troublesome (Microsoft's Tay) and downright tragic ( Uber's self-driving cars) ways. The question of whether the machine is really thinking or merely simulating thinking-the key question raised by John Searle in the Chinese Room Argument (1980)-might well be beside the point because, however they are processing information, AI agents are acting intelligently and making decisions in the world

Unlike in the United States, where AI development is literally killing people, the European Union has recognized that the realm of AI and robotic technology development needs guidance and rules. In 2017 the EU passed the European Civil Law Rules on Robotics, in which they define AI agents as interactive, autonomous robots that are able to adapt their behavior and actions to the environment. This statement is an important first step in laying down some guidelines for human-AI machine interaction.

As AI bots become more interactive, autonomous, and capable of self-processed adaptive learning, they will become better agents and communicators, better at such actions as speaking, reading, writing, and evaluating writing.

Chat Bots & Writing Bots

To what extent then can bots do our writing tasks for us, at a level of effectiveness and quality such that they could possibly pass a Turing Test for writing: that is, readers cannot tell the difference between the bot writing and the human writing the same task?

Well, that depends, of course, on the writing context, but bots have certainly passed the Turing Test informally for a wide-variety of contexts. For example, x.ai's personal assistant scheduling bot, Amy/Andrew Ingram, in the context of email about meetings, is very often mistaken as human. In fact, some email correspondents have even flirted with Amy Ingram and sent her flowers and chocolates. Some poetry writing bots are already informally passing the Turing Test. Of greater worry, though, is that millions of people in the United States during the 2016 presidential election were unaware that the fellow "citizens" they were talking politics with in Twitter were actually bots (Bessie & Ferrara, 2016) deployed as part of troll farm attacks on US democracy.

In the academic realm, bots are already writing student papers. For example, Dr. Assignment promotes on its website that "A.I. technology will automatically write your assignment paper for you if you are too lazy or don't know what to write." As shown perhaps by this marketing (aiming at students it seems who'd be stoked with a C or D), the level of output isn't expected to be high quality (yet), but as these programs become more sophisticated and nuanced, bot produced papers will certainly get better.

William Hart-Davidson (2018) thinks that we are not so "far away from a time when almost nobody composes a first draft of anything ... you'll take over at the revision stage" (p. 252). Is it cheating to have a robot write your paper for you? Well, some composition teachers thought it unfair, originally, when spell and grammar checkers came online. We managed to get through that crisis, finally arriving at the position that spell checkers are aids, but they are not 100% foolproof, and the human writer is still responsible for spelling. Will we treat draft writing bots the same way?

In relation to writing bots we need to be aware of AI-writing developments, be able to teach students how to write scripts for bots, how to write with bots, and, for those going into education, how to teach in a world with writing bots. We also need to be part of important conversations shaping the ethical use of writing bots, considering such questions as:

When and how should bots be used as professional writers?

Should there any limits/restrictions on the use of writing bots? That is, are there some purposes/contexts for which they should not be used?

Should writing bots be used transparently? That is, should readers/users be alerted that they are communicating with a machine, or doesn't that matter? (Spoiler alert: We are advocates of the position that urges transparency in the uses of bots, particularly for public writing.)

The field of rhetoric/writing has much to offer these discussions, given our field's long history of considering writing in human contexts and in the realm of human-machine communication.

AI Instructors & Teaching Assistants

AI writing teachers might seem pretty far-fetched, but that too is developing faster than we may realize. Working with very broad brushstrokes, let's start by identifying the distinct functions of teaching: designing the course curriculum, delivering course material, interacting with students, and grading/evaluating student work and performance.

Let's start with grading papers. Machine-scoring of student writing has been around a long time and numerous studies in or field have shown its limitations (Ericsson & Haswell, 2006). But many of those studies were conducted on older, less smart systems that, as Les Perelman demonstrated, could be easily gamed by nonsense long essays with long sentences and multisyllabic key words that would score higher than well-written short essays with shorter sentences (Herrington & Moran, 2001; Perelman, 2012). As the technology develops and computers can juggle more variables, AI raters will become better readers and evaluators of writing.

But can AI systems interact effectively with students? That's where the natural language processing comes in. Many students in computer science classes at Georgia Tech can't figure out which of their teaching assistants (for an online course in computer science) are human and which are computers. Jill Watson and family has them fooled. These computer teaching assistants aren't grading work (yet); they are just "automatically answering a variety of routine, frequently asked questions, and automatically replying to student introductions" (Goel & Polepeddi; see also Eicher, Polepeddi, & Goel, 2018). Yet they are doing so in ways similar enough to human responses in the same context that they are, rather effectively it seems, replacing human teaching assistants.

As we all face increased pressures to raise class sizes, to teach more students with fewer faculty, to get "lean" and "efficient" (buzzwords on our campus and we imagine on yours), it may be that we don't have AI agents as teachers but what of AI teaching assistants to interact in limited ways? Is there a place for a Jill Watson in some of our courses and programs? What are the roles for human teachers vs machine teachers? These are questions we need to consider and be prepared to answer because without a doubt the landscape of education is changing. In particular, for us, the most immediate question is this: Could bots handle the teaching of first-year college composition? That possibility could be upon us in another five to ten years.

Call to Conversation and Action

We close this post by inviting ongoing conversation and action around these issues among ourselves in the rhetoric/writing field, with colleagues in other fields, and with working professionals.

In general we feel that a ludditic rejection of AI is not the answer. Instead, calling on the critical theory of Andrew Feenberg (1991) and others (Selber, 2004), we advocate for a critical engagement with technology development to insure that its designs and uses are truly smart, not just convenient or cost cutting, appropriate for our educational mission and goals and for our students. By turning our research and our teaching to AI-related areas, we will position ourselves and our students to be at the forefront of the changes, helping, we hope, to critically shape technology development, policy, and usage rather than merely reacting to it.

Note

This blog post is a portion of a longer article-in-progress that we are currently completing. We have also written about this topic in a chapter titled "AI Agents as Professional Communicators," in our book Professional Communication and Network Interaction: A Rhetorical and Ethical Approach (Routledge, 2017).

References

Bessi, Alessandro, & Ferrara, Emilio. (2016). Social bots distort the 2016 U.S. Presidential election online discussion. First Monday, 21(11). http://firstmonday.org/ojs/index.php/fm/article/view/7090/5653a

Eicher, Bobbie, Polepeddi, Lalith, & Goel, Ashok. (2017). Jill Watson doesn't care if you're pregnant: Grounding AI ethics in empirical studies. (2017). Georgia Tech Library. http://dilab.gatech.edu/publications/jill-watson-doesnt-care-if-youre-pregnant-grounding-ai-ethics-in-empirical-studies/

Ericsson, Patricia Freitag, & Haswell, Richard H. (eds.). (2006). Machine scoring of student essays: Truth and consequences. Logan, UT: Utah State University Press.

Feenberg, Andrew. (1991). Critical theory of technology. Oxford: Oxford University Press.

Goel, Ashok, & Polepeddi, Lalith. (2016). Jill Watson: A virtual teaching assistant for online education. Georgia Tech Library. https://smartech.gatech.edu/handle/1853/59104

Hart-Davidson, William. (2018). Writing with robots and other curiosities of the age of machine rhetorics. In Jonathan Alexander & Jacqueline Rhodes (eds.), The Routledge handbook of digital writing and rhetoric (pp. 248-255). New York: Routledge.

Herrington, Anne, & Moran, Charles. (2001). What happens when machines read our students' writing? College English, 63(4): 480-499.

McKee, Heidi A., & Porter, James E. (2017). Professional communication and network interaction: A rhetorical and ethical approach. New York: Routledge/Series in Rhetoric and Communication.

Perelman, Les. (2012). Construct validity, length, score, and time in holistically graded writing assessments: The case against automatic essay scoring (AES). In Charles Bazerman et al. (eds.), International advances in writing research: Cultures, places, measures. WAC Clearinghouse. https://wac.colostate.edu/books/perspectives/wrab2011/

Searle, John. (1980). Minds, brains and programs. Behavioral and Brain Sciences, 3, 417-57.

Selber, Stuart A. (2004). Multiliteracies for a digital age. Carbondale, IL: Southern Illinois University Press.

Turing, Alan. (1950). Computing machinery and intelligence. Mind, 59(236), 433-460.

# data[].attributes.attachment_link.title
The Impact of AI on Writing and Writing Instruction

# data[].attributes.attachment_link.description
We think the field of rhetoric/writing needs to address-and quickly!-three current and rapidly expanding developments in artificial intelligence (AI) technology: AI chatbots are now operating in the workplace. The students graduating from our writing majors need to be prepared to write for bot communications, becoming scriptwriters for bots and writing instructors for bots.

# data[].attributes.attachment_link.title
Teaching Experts Are Worried About ChatGPT, but Not for the Reasons You Think

# data[].attributes.attachment_link.description
Is the technology a threat to the college essay, or does it raise questions about higher ed?

# data[].attributes.attachment_link.provider_name
POCIT. Telling the stories and thoughts of people of color in tech.

# data[].attributes.attachment_link.readable_text
This article byChristian Ilube was first published on Medium.

Open AI is an artificial intelligence research company that aims to create AI to benefit all of humanity. In 2020, they announced GPT-3 (Generative Pre-trained Transformer 3), an AI capable of any reading/writing task e.g., question answering, summarising text, or continuing a story. It was created as a step on the way to Open AI's grand mission of solving general intelligence, creating an AI able to learn any intellectual task that a human can do.

Despite not being general intelligence, GPT-3 still has many uses. A variety of apps that use it for different tasks have spun out of it as a result. The latest one is called ChatGPT, created by Open AI themselves. It is a chatbot designed for dialogue.

Open AI sells some of their products, for instance allowing users to pay to run their programs through APIs. However, their aim is to develop AI in a transparent way, making much of their code and research open source. This means they don't generate much profit and are heavily reliant on investment, for example, the billion-dollar deal they made with Microsoft shortly after they released GPT-3. The applications they develop are needed to generate hype around their products and attract more investment.

Hidden Biases

An important consequence of this, is that Open AI do their best to hide any flaws in their systems, that may scare away investors. We saw an example of this recently when they made their DALL.E AI available to the public. DALL.E takes natural language prompts as input and generates pictures of what is written. It was trained on huge datasets of images available on the internet. The pictures in these datasets likely featured a much higher percentage of white males and initially this was reflected in the pictures that DALL.E generated.

In order to avoid accusations of bias and maintain hype, Open AI secretly altered the prompts inputted by users by adding extra text such as 'black female' at the end of prompts. These were visible to the AI but not the user. Researchers discovered this when they started using prompts such as "a person holding a sign that says" and received images of people holding signs saying, "Black Female", implying that the words "black female" had secretly been added to the prompt.

Image generated by DALL.E https://twitter.com/rzhang88/status/1549472829304741888

This makes the AI appear less biased without actually fixing its inherent problems. Open AI essentially tried to sweep the problem of bias under the rug. This is dangerous because these technologies have the potential to have massive impacts on society, so transparency is needed for us to accurately assess how safe they are to use.

GPT-3 suffers from many of the same issues as DALL.E. GPT-3 was trained on a curated version of the common crawl dataset, a dataset made by algorithms that scrape the internet for text data. Experts believe that most of the text on the internet is generated by white males in western countries. It is likely that GPT-3's training data overrepresents this demographic. Open AI's own research discovered that the sentiment of the text it generated, varied when talking about different racial groups.

GPT-3 Sentiment across racial groups https://arxiv.org/abs/2005.14165

ChatGPT will not engage in any conversations it deems to be inappropriate. This is because Open AI are using their Moderation API, another application made using GPT-3, to block certain types of unsafe content. While this helps to stop mass production of harmful content, it leaves us in the dark about how an unfiltered GPT-3 would respond to certain questions. it is a good thing that Open AI are aware of these limitations and are actively working to fix them, but it may also mean that the public is unaware of the technology's potential danger to society. Awareness of these dangers will be especially significant when policy makers consider GPT-3 based applications.

The moderated responses of ChatGPT, make language AI seem safe to start using today. However, we cannot be so confident in the text generated by Language AI, when they are applied to less structured tasks and given more freedom. There is still work to be done before we start using them everywhere.

How many languages can ChatGPT speak?

Most of the text data on the internet is in English. Therefore, GPT-3 performs much better in English than any other language. In the brief conversation I had with ChatGPT, I experimented by talking to it in various languages and although it is capable of conversing in an impressive range of languages, it acknowledges that its far more capable in English.

ChatGPT's response to my question

White males from western countries have had the largest role in creating this technology. They make up some of the key decision makers behind its creation but more importantly they have contributed the most data to its training set. Similarly, English speaking internet users will benefit the most from this technology. This is a relatively small and affluent section of society so it is fair to say that this technology will widen the inequality gap.

Future research

Language AI research has made incredibly fast progress in recent years. Applications using GPT-3 have demonstrated impressive capabilities so the hype that surrounds them is not without substance. However, we still have a long way to go before we can be sure that these technologies will benefit all of humanity. It is paramount that the world is aware of the limitations they have, regardless of the effect this may have on investment.

This article byChristian Ilube was first published on Medium.

# data[].attributes.attachment_link.description
This article byChristian Ilube was first published on Medium. Open AI is an artificial intelligence research company that aims to create AI to benefit all of humanity. In 2020, they announced GPT-3 (Generative Pre-trained Transformer 3), an AI capable of any reading/writing task e.g., question answering, summarising text, or continuing a story.

# data[].attributes.body
ChatGPT's responses to questions about using ChatGPT in higher education.

# data[].attributes.attachment_link.readable_text
There's a great deal of discussion among faculty in higher education about ChatGPT and other Artificial Intelligence (AI) tools. Faculty are concerned about students using this technology to generate essays and other work in their classes.

Learning Innovation has created a "conversation starter" blog post where we asked ChatGPT about these issues and developed our responses to what AI had to say. This post is a more personal view, highlighting some of my own thoughts as a consultant about what the disruption of AI means for faculty and students.

We've faced technological challenges like this before. As AI develops, responses by faculty to what it can and can't do are going to be a moving target, similar to the the emergence of the World Wide Web and student use of online resources or the development of commercial online "paper banks" where students could purchase a pre-fabricated term paper on a given topic. Before the Web, faculty were only concerned about students sharing exams or papers at their university, but the Web opened the floodgates of readily accessible ways to share information.

Some time ago, to face the challenge of easy online searches and paid "essay banks," faculty and administrators at Duke discussed whether to license plagiarism detection software for the campus. A decision was made to not license plagiarism detection tools for two main reasons.

Plagiarism detection services work by collecting papers from students and use those entries as a database for detecting plagiarism instances within or across institutions. It was felt, at the time, that these services raised serious issues about student rights to copyright on their work and student privacy - by requiring students to submit papers to a service like this, faculty would be forcing students to give up their legal copyright to their work and store their work on outside commercial services in perpetuity.

Even though some services give faculty or institutions an "opt out" for submission and retention of papers at their school or within a class, the general consensus by Duke administrators and faculty was that use of these services sent the wrong message to students about individual responsibility, the Duke Honor Code, and encouragement of heavy-handed "surveillance" tools to monitor students.

The recommendations on combating plagiarism that emerged from these discussions - and what Duke Learning Innovation recommends - was to make assessments more authentic and robust. By engaging students in original research, using non-traditional assessments such as presentations or long-term projects, or asking students for more in-depth analysis and debate, faculty members could move students past repeating content that might be easily accessible through the web to a deeper engagement with the course content and goals.

ChatGPT and other AI tools present a more difficult challenge. AI services use an inquiry to generate, on the fly and within a few seconds, a natural language response to a prompt or question based on databases of research, articles, and web pages drawn from a variety of sources. Each response is unique, since AI "learns" and changes its approach as more inquiries are made and more data is used to generate answers.

The only technology solution that would overcome use of AI by students with certainty that could be used by a faculty member for pressing an Honor Code case would be if the entity offering an AI service would save all of the answers it generates and allow faculty to search student papers against these responses. With the different services emerging and the large number of answers being automatically generated every day, it seems unlikely that such a service would be practical. ChatGPT is discussing the addition of digital watermarks (specific arrangements of words that form a pattern invisible to the end user) to texts it generates, but it's unclear if these watermarks would hold up with significant editing of the AI produced text.

As a teaching consultant, I've put a great deal of thought into how to respond to AI and have discussed it with others in my field and faculty I know here at Duke and at other universities. I've also spent the past few weeks running some typical essay questions in my areas of subject matter expertise to see what ChatGPT could come up with.

I noticed some patterns in what ChatGPT currently can and cannot do that could be used to rethink assignments in some classes. Of course, the approaches would be different, depending on the subject area, whether it's a specific historical era or another discipline, such as computer science.

One of my areas of interest is film and media history. I entered several simple essay prompts that might be used in film history and analysis classes. I noticed that ChatGPT could generate a convincing and correct answer, for the most part, if you're asking about something that's widely written about and discussed in these types of classes. The answers were at the level of a basic Wikipedia article. Some of the typical questions I asked were to discuss the themes or controversies about well-known films, such as Citizen Kane, or comparing themes in works by well-known directors.

Where ChatGPT either refused to provide an answer or was evasive were questions where I asked it to be more specific or to analyze a controversial or ambiguous topic. ChatGPT can regurgitate factual information that is widely known - again, the type of cursory information seen in a Wikipedia article. Where it fails is with analysis and taking a position on a controversial topic and offering evidence. It also has difficulty with more obscure topics, where the information sources it draws on are lacking.

The furor around AI and student work reminded me of one of the most challenging papers I had to write in a Social Studies class in high school in the early 80s. For a final paper, I had to write about whether I thought Oswald acted alone in the JFK assassination, drawing upon primary and secondary sources. It's still an ambiguous question debated today, with more trustworthy works by historians and less supported information available. What my teacher was looking for was my analysis of the evidence I could find and why this evidence was valid to support my view. Based on my look at AI so far, I don't think that ChatGPT could come up with a convincing analysis on the topic because of the large amount of information available and the ambiguity over sources and evidence that the student would have to sort out.

As a consultant, my advice to faculty would be to actually run some of your questions through ChatGPT or another AI service to see what kind of answers you see. If it comes up with a convincing answer, rethink your questions to be more specific to the content and discussions in your class, to require a more in-depth analysis, or to tackle more ambiguous questions. Topics on more obscure or local topics would also seem to be an approach to discourage use of AI for easy answers.

There's not one simple technology solution to student plagiarism or student use of AI tools in their work. Some high schools and institutions are blocking access to ChatGPT and AI services by students. However, we all know that some students will be determined and smart enough to bypass these technology obstacles.

Some faculty I've talked with see AI as an opportunity and want to be up-front about it with students, showing these resources to students and the kinds of limitations of the tools to demonstrate that AI might provide a starting point for a question, but can't really provide a simple, pre-packaged response for the type of work the students will be doing in a given class. Or faculty might allow students to critique AI generated material as part of a class activity. It's similar to how many faculty now approach using Wikipedia or other web-based information resources in classes.

After discussions with colleagues in Learning Innovation, we feel that solutions and responses to AI are going to have to emerge from faculty conversations about this challenge and may vary depending on the discipline. We also feel this will need to be an ongoing dialogue, involving faculty, students, and Duke administrators. We're looking forward to talking more with faculty about how they see AI influencing their work.

As educators, we can never stand still and have to adapt and respond to new challenges presented by emerging technologies, social trends, or current events. AI is just one of many new trends we have to think about moving forward in the next decade.
Randy A. Riddle is a Senior Teaching Consultant in Duke Learning Innovation, primarily serving Social Sciences faculty. He has a degree in Public and Applied History from Appalachian State University, is an independent documentary filmmaker and researcher, and has worked at Learning Innovation (and it's predecessor, the Center for Instructional Technology) since 2000.

# data[].attributes.attachment_link.title
Some Thoughts on AI, Plagiarism and Student Assessment - Duke Learning Innovation

# data[].attributes.attachment_link.description
There's a great deal of discussion among faculty in higher education about ChatGPT and other Artificial Intelligence (AI) tools. Faculty are concerned about students using this technology to generate essays and other work in their classes. Learning Innovation has created a "conversation starter" blog post where we asked ChatGPT about these issues and developed our responses to what AI had to say.

# data[].attributes.body
Basic Explainer Video about ChatGPT. Better than most.

# data[].attributes.attachment_link.title
ChatGPT Explained: What is Chat GPT by OpenAI? Are we doomed?

# data[].attributes.attachment_link.readable_text
21 Pages Posted: 23 Dec 2022

Date Written: December 13, 2022

Abstract

Chatbots are able to produce high-quality, sophisticated text in natural language. The authors of this paper believe that AI can be used to overcome three barriers to learning in the classroom: improving transfer, breaking the illusion of explanatory depth, and training students to critically evaluate explanations. The paper provides background information and techniques on how AI can be used to overcome these barriers and includes prompts and assignments that teachers can incorporate into their teaching. The goal is to help teachers use the capabilities and drawbacks of AI to improve learning

Keywords: AI, education, chatbot, transfer, learning

# data[].attributes.attachment_link.keywords[]
new modes of learning enabled by ai chatbots: three methods and assignments

# data[].attributes.attachment_link.title
New Modes of Learning Enabled by AI Chatbots: Three Methods and Assignments

# data[].attributes.attachment_link.description
Chatbots are able to produce high-quality, sophisticated text in natural language. The authors of this paper believe that AI can be used to overcome three barri

# data[].attributes.attachment_link.description
Including 10 AI-ready assignment and activity ideas.

# data[].attributes.attachment_link.readable_text
Adapting College Writing for the Age of Large Language Models such as ChatGPT: Some Next Steps for Educators

By Anna Mills and Lauren M. E. Goodlad (licensed CC BY NC 4.0)

Large language models (LLMs) such as ChatGPT are sophisticated statistical models that predict probable word sequences in response to a prompt even though they do not "understand" language in any human-like sense. Through intensive mining, modeling, and memorization of vast stores of language data "scraped" from the internet, these text generators deliver a few paragraphs at a time which resemble writing authored by humans. This synthetic text is not directly "plagiarized" from some original, and it is usually grammatically and syntactically well-crafted.

From an academic integrity perspective, this means that "AI"-generated writing

1) is not easily identifiable as such to the unpracticed eye;

2) does not conform to "plagiarism" as that term is typically understood by teachers and students; and

3) encourages students to think of writing as task-specific labor disconnected from learning and the application of critical thinking.

Many teachers who assign writing are, thus, understandably concerned that students will use ChatGPT or other text generators to skip the learning and thinking around which their writing assignments are designed.

In the future, the producers of language models may offer tools for identifying texts generated by their systems. Such tools may arise independently, like a recent app that claims to identify ChatGPT's outputs. It is also possible that government regulators and other policy-making bodies will become involved in overseeing the use of LLMs in educational settings. New York City schools have already banned ChatGPT, and many experts argue that the abuse of LLMs could extend far beyond the impact on students.

As a teacher and textbook author, one of us (Anna Mills) has been collecting multiple perspectives on the topic for the Writing Across the Curriculum Clearinghouse. Another of us (Lauren Goodlad) is the chair of the Critical AI @ Rutgers initiative and the editor of Critical AI. Though both of us feel strongly that unsupervised use of LLMs for student assignments is detrimental to learning, we believe that, in the short run, a combination of the practices described below will effectively discourage students from submitting machine-generated writing as their own. At the very least, any student determined to use text generation will encounter significant obstacles.

In the long run, we believe, teachers need to help students develop a critical awareness of generative machine models: how they work; why their content is often biased, false, or simplistic; and what their social, intellectual, and environmental implications might be. But that kind of preparation takes time, not least because journalism on this topic is often clickbait-driven, and "AI" discourse tends to be jargony, hype-laden, and conflated with science fiction. (We offer a few solid links below.) In the meantime, the following practices should help to protect academic integrity and student learning. At least some of these practices might also enrich your teaching.

Common practices that can be updated in the current context

Encourage intrinsic motivation. Most educators already strive to make their assignments engaging, but it's worth emphasizing that students who feel connected to their writing will be less interested in outsourcing their work to an automated process.

Highlight how the writing process helps students learn. Make explicit that the goal of writing is neither a product nor a grade but, rather, a process that empowers critical thinking. Writing, reading, and research are entwined activities that help people to communicate more clearly, develop original thinking, evaluate claims, and form judgments.

Update academic integrity policies to make them explicit about the use of automated writing tools. Academic integrity policies and honor codes should specify what, if any, use of automated writing assistance is appropriate (teachers may wish to consult departmental or institutional policies).

Ask students to affirm that their submissions are their own work and not that of another person or of any automated system. This practice has long been used to deter plagiarism and can be adapted to include text generation. One of us asks students to add the following statement along with their initials when they turn in written work: "I certify that this assignment represents my own work. I have not used any unauthorized or unacknowledged assistance or sources in completing it including free or commercial systems or services offered on the internet."

Practices we recommend

Let students know that detectors for identifying "AI"-generated text exist and are improving. While these tools should not be relied upon as a "silver bullet," students should know that detection is possible. However, teachers who use these tools as a check on potential policy violations should bear in mind that the detectors may produce false positives and false negatives. Students should know that this technology is rapidly evolving: future detectors may be able to retroactively identify auto-generated prose from the past. No one should present auto-generated writing as their own on the expectation that this deception is undiscoverable.

Assign prompts that state-of-the-art systems such as ChatGPT are not good at. The below tasks are either impossible for ChatGPT to perform reliably, or require the student to supervise and edit in ways that entail significant expertise, rhetorical skills, and time. As such, students who are simply eager to earn a good grade with minimal effort will likely find such assignments difficult to automate. These requirements may also make assignments more robust in other ways.

Requirement for verifiable sources and quotations. ChatGPT currently fabricates sources and quotations (though it may occasionally hit on a real author or title). GPT-3 is even more prone to this "hallucination." Students using either model would need to find and input sources and quotations themselves.

Analysis of specifics from images, audio, or videos. Students would need to describe these kinds of media in detail in order to generate automated outputs about them.

Analysis of longer texts (too large for the limited windows for prompting automated systems). Although tools exist for summarizing longer texts, using them adds another obstacle to easy automation. Such programs may introduce errors that make automated text easier to detect.

Analysis that draws on class discussion. Assigning this criterion requires the student to input notes from class discussion, involving time and effort.

Analysis of recent events not in the training data for the system. Students would need to do their own research and then feed that information into the automated system.

Assignments that articulate nuanced relationships between ideas. Such assignments could entail comparing two passages that students themselves choose from two assigned texts. Students might be asked to explain a) why they chose these particular passages; b) how the chosen passages illuminate the whole of the texts from which they were excerpted; and then c) how the two passages compare according to instructions that bear on the course themes or content. LLMs usually cannot do a good job of explaining how a particular passage from a longer text illuminates the whole of that longer text. Moreover, ChatGPT's outputs on comparison and contrast are often superficial. Typically the system breaks down a task of logical comparison into bite-size pieces, conveys shallow information about each of those pieces, and then formulaically "compares" and "contrasts" in a noticeably superficial or repetitive way.

Assign in-class writing as a supplement to or launching point for take-home assignments. Students may be more likely to complete an assignment without automated assistance if they've gotten started through in-class writing. (Note: In-class writing, whether digital or handwritten, may have downsides for students with anxiety and disabilities).

Additional practices you might wish to undertake

Assign steps in the writing process and/or reflection on that process. Many instructors already include these practices in their approach to teaching writing. Note that ChatGPT can produce outputs that take the form of "brainstorms," outlines, and drafts. It can also provide commentary in the style of peer review or self-analysis. Nonetheless, students would need to coordinate multiple submissions of automated work in order to complete this type of assignment with a text generator.

Hold individual conferences on student writing or ask students to submit audio/video reflections on their writing. As we talk with students about their writing, or listen to them talk about it, we get a better sense of their thinking. By encouraging student engagement and building relationships, these activities could discourage reliance on automated tools.

If you are curious about the technology, test your own writing assignments. Once you sign up for an account, it is straightforward to test an assignment using ChatGPT by feeding in the instructions and other required information (such as a short text). Users of these models can learn to improve their outputs by prompting the model to add or revise. (Anna has compiled a set of examples).

Practices you might wish to undertake once you learn more about language models

Teach students about text generators and "AI." Students are more likely to misuse text generators if they trust them too much. The term "Artificial Intelligence" ("AI") has become a marketing tool for hyping products. For all their impressiveness, these systems are not intelligent in the conventional sense of that term. They are elaborate statistical models that rely on mass troves of data-which has often been scraped indiscriminately from the web and used without knowledge or consent. No matter how seemingly magical, text generators do not understand language (or anything else) the way that humans do. For the same reason, LLMs often mimic the harmful prejudices, misconceptions, and biases found in data scraped from the internet. (See the below resources for some preliminary introductions to the subject.)

Show students examples of inaccuracy, bias, logical, and stylistic problems in automated outputs. We can build students' cognitive abilities by modeling and encouraging this kind of critique. Given that social media and the internet are full of bogus accounts using synthetic text, alerting students to the intrinsic problems of such writing could be beneficial. (See the "ChatGPT/LLM Errors Tracker," maintained by Gary Marcus and Ernest Davis.) Note that teaching students about LLMs in this way is a different practice from using ChatGPT as a means of teaching students how to write (a practice that some are recommending but which we regard as having limited value). Since ChatGPT is good at grammar and syntax but suffers from formulaic, derivative, or inaccurate content, it seems like a poor foundation for building students' skills and may circumvent their independent thinking. The tool seems more beneficial for those who already have a lot of experience writing-not those learning how to develop ideas, organize thinking, support propositions with evidence, conduct independent research, and so on.

Join in discussions with fellow teachers, policymakers, and industry to explore regulations and tools that will help educators support student learning in the age of "AI"-text generators.

Practices we do not recommend

Requiring handwritten submissions. Writing by hand is difficult for many students, especially students with certain disabilities and students who use voice-to-text software. To be sure, some studies show that students remember things better when they take notes with pen and paper rather than laptops or other devices. Writing by hand might be offered as an option for some assignments.

Adopting surveillance tools. Some are advocating software that records the students' entire writing process-systems that have been shown to be biased and inaccurate. Such for-profit surveillance tools are highly intrusive and (like language models themselves) potentially exploitative. They may also be ineffective and inequitable.

Adopting systems trained to recognize specific student writing. In theory it is possible to determine the authorship of a text by training a system to recognize an individual student's patterns of word choice and syntax. Like the surveillance systems above, such technology is prone to unreliability, exploitation, and abuse.

Further resources on text generators
Note: Good journalism on language models is surprisingly hard to find since the technology is so new and the hype is ubiquitous. Here are a few reliable short pieces.

"ChatGPT Advice Academics Can Use Now" edited by Susan Dagostino, Inside Higher Ed, January 12, 2023

"University students recruit AI to write essays for them. Now what?" by Katyanna Quach, The Register, December 27, 2022

"How to spot AI-generated text"by Melissa Heikkilä, MIT Technology Review, December 19, 2022

The Road to AI We Can Trust, Substack by Gary Marcus, a cognitive scientist and AI researcher who writes frequently and lucidly about the topic. See also Gary Marcus and Ernest Davis, "GPT-3, Bloviator: OpenAI's Language Generator Has No Idea What It's Talking About" (2020).

The below academic article is now a classic:

A sample of resources on "AI" more generally (Updated 1/19)

Emily M. Bender, "On NYT Magazine on AI" (distinguished University of Washington linguist responds blow-by-blow to a misleading New York Times feature ).

Ruha Benjamin, "Assessing Risk, Automating Racism" Science (2019).

Frederico Bianchi et al. "Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale." Arxiv 2022.

Meredith Broussard, Artificial Unintelligence: How Computers Misunderstand the WorldMIT Press (2018),

Timnit Gebru, "Is Ethical AI Possible?" (podcast with co-author of "Stochastic Parrots" and director of the DAIR Institute).

Andrew Hundt et al. "Robots Enact Malignant Stereotypes." 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT 22). 743-756.

Frank Pasquale, New Laws of Robotics: Defending Human Expertise in the Age of AI.Harvard UP, 2020.

Billy Perrigo. "OpenAI Used Kenyan Workers on Less than $2 Per Hour to Make ChatGPT Less Toxic." Time January 18, 2023.

Meredith Whittaker, "The Steep Cost of Capture" Interactions 28.6 (December 2021): 50-55.

To share your ideas or offer advice please feel free to comment below (the comments are moderated) or write to one or both authors.

Anna Mills: armills@marin.edu

Lauren Goodlad: lauren.goodlad@rutgers.edu

In addition, the Modern Language Association and the College Conference on Composition and Communication have convened a task force on this topic (Anna is a member). Anyone who would like to contact the taskforce can write to Paula Krebs (MLA director) or Holly Hassel (CCCC director).

# data[].attributes.attachment_link.title
CRITICAL AI: Adapting College Writing for the Age of Large Language Models such as ChatGPT: Some Next Steps for Educators

# data[].attributes.attachment_link.description
Angel McNall photography Adapting College Writing for the Age of Large Language Models such as ChatGPT: Some Next Steps for Educators By Anna Mills and Lauren M. E. Goodlad (licensed CC BY NC 4.0) Large language models (LLMs) such as ChatGPT are sophisticated statistical models that predict probable word sequences in response to a prompt...

# data[].attributes.attachment_link.description
An API for accessing new AI models developed by OpenAI

# data[].attributes.attachment_link.readable_text
Need an assignment that is ChatGPT-proof? Look somewhere else. I'm not sure one exists, but here's a model for a kind of writing assignment that might at least use the program in a way that stimulates - rather than replaces - student critical thinking.

In creating this assignment I tried to follow the advice offered here on Critical AI's guidelines by Anna Mills and Laura Goodlad as well as the advice by MIT Professors, including Nick Montfort.
Background
For your first paper, you will be focussing on your future profession and its relationship to the communities it impacts. You have already begun answering this question in your diagnostic and in your blog post. However, we are also living in the age of programs that can write these essays for us. Or can they? In this paper assignment, you'll reflect on the key element that an AI text generator leaves out of your essay.
Prompt: When you ask ChatGPT (or equivalent LLM text generator) to write an essay about [*the ways your future profession could better engage the communities it impacts*], what key component does the output omit?
[ * Fill in the blank with the topic you'd like them to write about
Writing tasks
This is ultimately not really a paper ABOUT ChatGPT but one that uses the output of ChatGPT (or other LLM) to help you discover and recognize the complexities and nuances you are bringing to this topic or question.

To write this paper, follow these steps:

Run your version of the prompt through ChatGPT or equivalent.

Analyze what it offers and what is missing.

Include the ChatGPT response as an attachment

For development:
On the one hand, this paper is an exercise in reflection about your chosen profession's relationship to communities. On the other hand, this paper asks you to do some close reading, analyzing closely the textual output of the AI. Be careful. It is easy for this paper to turn into a laundry list of smaller details that are left out of ChatGPT's output. While those details are important, put them in the service of a larger or central idea.
On using ChatGPT:
Anyone can outrun a robot that has been programmed to run slow. To make this exercise as useful as possible, you should keep revising your prompt for ChatGPT until you get output that is fairly sophiticated an interesting. You can see my experiments in the readings below. Also, try to stay away from obvious points about ChatGPT, such as its inability to use most quotations effectively or its hallucinating.

Try not to dwell on the shortcoming of LLMs, such as their hallucinating or inability to use quotations. What's more important are at the level of ideas or rhetorical approaches.

Don't let the seeming overconfidence fool you into giving the program's output more credence than it deserves, but let's not discount what an LLM can contribute to your understanding including commonly (and if prompted uncommonly) offered words and ideas that might challenge your initial perspective. Discuss, too, what the process of engaging with the software has helped you to realize.

Some examples of tweaking prompts for ChatGPT:

Your essay should engage at least 2 readings from our class assignments so far.

Let me know if you try an essay assignment like this and how it goes.

# data[].attributes.attachment_link.title
GPT This! a writing assignment in the age of GPT-3 & other Large Language Models

# data[].attributes.attachment_link.description
Need an assignment that is ChatGPT-proof? Look somewhere else. I'm not sure one exists, but here's a model for a kind of writing assignment that might at least use the program in a way that stimulates - rather than replaces - student critical thinking.

# data[].attributes.body
Lots of links to resources and posts re: ChatGPT and Higher Education

# data[].attributes.attachment_link.title
Member QuickTalk | GPT: The Generative AI Revolution

# data[].attributes.attachment_link.description
We will examine the OpenAI application of ChatGPT.

# data[].attributes.body
Sites and docs with lots of aggregated info on AI and Education.

# data[].attributes.body
Academic Integrity Policies, Cheating, and AI Detection

# data[].attributes.body
Suggestions on how to use AI in the classroom.

# data[].attributes.body
News of the world regarding the AI revolution. Lots of stuff happening fast.

# data[].attributes.body
How does this stuff work? What does it do?

# data[].attributes.body
How will the AI revolution impact Higher Education? People have thoughts about that.

# data[].attributes.body
Applications you might want to explore in your classroom

# data[].attributes.body
Microsoft's recent announcement video on the integration of AI into its Microsoft 365 applications

# data[].attributes.attachment_link.title
The Future of Work With AI - Microsoft March 2023 Event

# data[].attributes.attachment_link.readable_text
ChatGPT-3 is a state-of-the-art language model developed by OpenAI. It is based on the GPT-3 (Generative Pre-trained Transformer 3) architecture and has been trained on a massive amount of text data. It has the ability to generate human-like text, answer questions, and complete various language-based tasks. It can also perform well on a wide range of natural language processing (NLP) tasks, such as text summarisation, translation, and text-to-speech. Additionally, it has the ability to generate text based on given prompt, which is unique for its large size and capability. It is considered as one of the most advanced language model available and it continues to be updated and improved.
(Generated by ChatGPT-3)

ChatGPT-3 became freely available on 30 November 2022.

In the short space of time that has passed academia has been 'stunned' by the technology's essay-writing skills and usability (Herne 2022), escalating concerns that students can now easily produce AI-assisted written work with minimal effort, dumbing down the value of their university degree. In the past few weeks alone a plethora of articles and guides have emerged to advise how we can approach this new challenge to academic integrity and quality, some of them written by ChatGPT-3 itself, just to prove the point.

But let's not forget that we have a history here, machines have traditionally been troublesome for Universities. In the early 1930s student unions were debating that the l ecturer was at risk of being replaced by the gramophone (Lindsay 2017); in the 1990s the growth of 'distance' learning was poised to cause the eventual dissolution of the university and the profession of the university lecturer (Eamon 1999), which was somewhat mirrored 30 years later during the online pivot of Covid19. Let's not forget that lecture capture has been deemed a technology that would drive students out of lecture halls and replace faculty with recorded online provision (Fagan 2021).

The difference we have with ChatGPT is that it doesn't so much present a threat to the university experience, but rather directly into the heart of the purpose of a university education - its ability to 'teach you how to think'. There have been shadows of this in the past, for instance the hostility towards Wikipedia (Coomer 2013), the emergence of essay mills, not to mention simple, now common place tools such as spell checkers and calculators. I remember vividly a very angry professor in the early 2000s telling me that reading lists with hyperlinks would make students baby birds, with wide open mouths expecting to be spoon fed. We've pretty much moved through all those advancements in technology and realised their benefits, but this one, I would argue, is different. Not because it does not have its benefits, but because of the sheer volume and scale of what's coming will be meaningfully different and ultimately challenge the foundations upon which we measure that ability to think - university assessment.

So what does ChatGPT mean for University assessment? Let's look at four scenarios.

1. Ban it

Whilst this is a rather a futile exercise, it is an option that we have already seen some schools take (Yang 2023), and it's likely we'll see more go down this path. Blocking a tool from a network or device is something most students can get around by simply logging into other networks, using other devices. Could we develop detection software like Turnitin, that is trained to identify AI language algorithms? GPT-2 Output Detector Demo has made a start here, identifying patterns of irregularities in written work that might indicate chat bot assistance. Turnitin have announced that they will build AI detection into their products. But GPT is only going to advance, every time we prompt the free ChatGPT-3 we train it and hurtle towards ChatGPT-4 which according to OpenAPI will be 100x more powerful. At this rate we may well become human-batteries fuelling the machines through constantly feeding them information and data, becoming trapped within an AI generated writing matrix! This detection race is one we will not win. Hard exiting out of this cycle requires a different approach.

Seriously though, plagiarism and cheating is not new, and is already a significant activity across Universities. The QAA recently estimated that one in seven (14%) graduates may have paid someone to undertake their assignment (QAA 2020). A recent bit of polling also indicated that 16% of students had cheated in their online exams in 2022, and 52% surveyed knew someone who had cheated in their academic assessment that year. A very small percentage, only 5%, had been caught. AI technology has the ability to make this practice easier and more accessible, but banning it is simply trying to implement an analogue solution to digital problem. And it's questionable whether it is the technology or our approach to assessment that is problematic.

2. Return to exams

This is an extreme scenario, but again one I am expecting to see, especially where there may be less appetite or resource to advance pedagogically and approach assessment in different ways. Of course exams still exist in many institutions, but increasingly it is recognised that requiring students to be placed in highly pressurised inflexible environments lacks a degree of inclusivity and manifests all kinds of attainment gaps. A return to pen and paper exams feels like a real step back but it is one some universities are willing to take to thwart the machine (Cassidy 2023). Sitting online exams under controlled conditions, with lock-down browser technology and the ability to detect whether students are pasting large chunks of text into their authoring interface is another option. But online proctoring software comes with financial and operational challenges, not to mention ethical considerations and known opportunities for academic dishonesty.

There may be something said though for encouraging more synchronous writing exercises. Johann N. Neem, talking to Inside Higher Ed, says that faculty could find 'new ways to help students learn to read and write well and to help them make the connection between doing so and their own growth' (D'Agostino 2023). An example would be offering opportunities for students to write in class and learn to approach writing as a practice of learning as well as a demonstration of it.

3. Develop AI Literacies as part of Student Assessment

This is undoubtably a good step. Over the past few weeks a range of ideas for engaging, authentic and creative assignments have been shared by the higher education community on how ChatGPT can be incorporated into student assessment and develop critical AI literacies. These range from essay reflection and improvement exercises, to prompt competitions and fact checking. A growing body of guides are emerging - I particularly like ' Update Your Course Syllabus for chatGPT ' from Ryan Watkins, which includes 10 AI-ready assignment ideas.

AI is not going away, it is technology that we are all going to be (and already are) engaging with. Developing AI literacies to enable students to use the technology responsibly and critically, is part of preparing them for the world of work. I'm not on board with a recent statement by JISC that 'We should really regard them as simply the next step up from spelling or grammar checkers: technology that can make everyone's life easier' (Weale 2023). We have to help students to understand where AI writing tools can support them, where it can enable a better outcome, and what its limitations are. Unlike a spell-checker, there is a lot more room for error - after all ChatGPT-3 does not provide you with an answer to your prompt, it provides you with an output. That is a different thing altogether.

It's also worth noting that AI literacy is a space where we should be mindful of looming social inequalities and environmental challenges. Improved future releases of GPT are not planned to be free to access, whilst the carbon footprint of training just a single AI model is significant. As institutions usually committed at some level to social justice, Universities need to look at where they stand on these matters and support students in navigating them.

4. Assess 'Humanness'

Like we can not ban AI tools, we can not use them for all assignments. Currently tools such as ChatGPT-3 can not do a number of things that are generally expected from a university student - it can not consult, critique and cite third party sources, it can not refer to recent real world events or published material, it can't demonstrate higher-level thinking, argue or have original ideas. The more detail and facts you ask for, the more it falters. Today, one of the best ways for students to prove that they are not a predictive language model is to demonstrate sophisticated thinking, which after all is the purpose of a university education. And we have to ask, if a machine can tell us what we need to know, what is the point in learning it? We need to shift contexts of what students need to know and how they need to learn.

Approaching assessment with the question 'what are the cognitive tasks students need to perform without AI assistance?' is a good start. It is possible that the rapid evolution of ChatGPT can positively increase adoption of assessment techniques that measure learners on critical thinking, problem-solving and reasoning skills rather than essay-writing abilities. Increased use of oral and video assessment, reflective assignments that ask students to explain their thinking process, developing concept diagrams, mind maps, engaging in ideation and group projects are other examples of assessment techniques. This is something many Higher Education learning professionals have been promoting for some time in a bid to better connect University education with workplace skills, offering more authentic experiences. Professional bodies would do well to look at what value their End Point Assessments hold in an AI landscape and work with Universities to explore alternative methods.

5. Using ChatGPT to support assessment processes

Much that has come out of the sector on ChatGPT and assessment has very much focused on student learning and risks of plagiarism or cheating. However, on the faculty side there are opportunities to review how this technology can support the assessment burden that many institutions face. The more sophisticated we make assessment in response to AI, the more difficult it will be for AI technology to support tasks such as marking. However, whilst assessment rubric contains generic criteria around structure, referencing and content topics then there is scope to train AI to support the grading of these elements. Multiple formative opportunities could be put in place to raise the bar on these elements, and ultimately reduce the load of markers who can then focus on feeding back on students' demonstration of cognitive skills.

Another possible time saver would be Multiple-Choice Question production. Back in 2018 Donald Clark wrote on the potential for AI to reduce the burden of MCQs which are time consuming and difficult to author and quite often lack the quality and the quantity needed to make them as robust as they could be. Not only can AI help to generate question banks, but it can enable more forms of open input questions as the technology makes possible the interpretation of answers such as typed words, numbers of short answers. Moving forwards the possibilities for ChatGPT to generate customised questions for individual students based on their prior knowledge and proficiency, or to support students in designing their own assignments, could lead to attainable personalisation of curriculum (Barber et al 2021).

In just a few months ChatGPT-3 has laser-focused Higher Education on reviewing the essay as a form of assessment, on the practice of writing, how that will change and why. But we are already moving on. AI is also in the space of media production, not only can it produce art, videos and audio it can adopt voices and faces - our chat bot is evolving beyond written words. It would be naive to think we can stand still. This is a technology that is already two steps ahead of our attempts to contain it in some form of meaningful, solid assessment strategy. In writing this piece I have avoided the possible option that we 'do nothing' in response to ChatGPT-3. Considering how slow Universities can be to implement change this is a possibility, but it is not a viable or sustainable choice. Unless we confront the implications of AI for teaching and learning, and embrace it as a part of our policies and pedagogies to develop critical thinking in an AI world, then we really will start to the lose the value of a University education.
References
Barber, M., Bird, L., Fleming, J., Titterington-Giles, E., Edwards, E., and Leyland, C. (2021). "Gravity assist: Propelling higher education towards a brighter future"Office for students. Available online at: https://www.officeforstudents.org.uk/publications/gravity-assist-propelling-higher-education-towards-a-brighter-future/

Cassidy, C (2023) 'Australian universities to return to 'pen and paper' exams after students caught using AI to write essays' The Guardian, 10 Jan 2023. Available online: https://www.theguardian.com/australia-news/2023/jan/10/universities-to-return-to-pen-and-paper-exams-after-students-caught-using-ai-to-write-essays

Clark, D (2018) Learning Designers will have to adapt or die. Here's 10 ways they need to adapt to AI.... [Blog] Donald Clark Plan B. Available online http://donaldclarkplanb.blogspot.com/2018/09/learning-designers-will-have-to-adapt.html

Coomer, A (2013) Should university students use Wikipedia? The Guardian, 13 May 2013. Available online: https://www.theguardian.com/education/2013/may/13/should-university-students-use-wikipedia

D'Agostino, S (2023) ChatGPT Advice Academics Can Use Now. Inside HigherEd, 13 Jan 2023. Available online: https://www.insidehighered.com/news/2023/01/12/academic-experts-offer-advice-chatgpt

Eamon, D.B. (1999) Distance education: Has technology become a threat to the academy?. Behavior Research Methods, Instruments, & Computers 31, 197-207. Available online: https://doi.org/10.3758/BF032077114

Fagan, J (2021) University of Exeter Lecturers Threaten Industrial Action over Lecture Recordings. Available online: https://www.tptranscription.co.uk/university-of-exeter-lecturers-threatening-industrial-action-over-lecture-recordings/

Heikkilä, M (2022) 'We're getting a better idea of AI's true carbon footprint. MIT Technology Review', 14 Nov 2022. Available online: https://www.technologyreview.com/2022/11/14/1063192/were-getting-a-better-idea-of-ais-true-carbon-footprint/

Hern, A. (2022) 'AI bot ChatGPT stuns academics with essay-writing skills and usability' The Guardian, 4 Dec 2022. https://www.theguardian.com/technology/2022/dec/04/ai-bot-chatgpt-stuns-academics-with-essay-writing-skills-and-usability

QAA.(2020). "Contracting to Cheat in Higher Education -How to Address Contract Cheating, the Use of Third-Party Services and Essay Mills." UK:Quality Assurance Agency (QAA)

Watkins, R (2023) Update Your Course Syllabus for chatGPT [Blog] Ryan Watkins. Available online: https://medium.com/@rwatkins_7167

Weale, S (2023) 'Lecturers urged to review assessments in UK amid concerns over new AI tool' The Guardian, 13 Jan 2023. https://www.theguardian.com/technology/2023/jan/13/end-of-the-essay-uk-lecturers-assessments-chatgpt-concerns-ai?

Yang, M (2023) 'New York City schools ban AI chatbot that writes essays and answers prompts' The Guardian, 6 Jan 2023. Available online: https://www.theguardian.com/us-news/2023/jan/06/new-york-city-schools-ban-ai-chatbot-chatgpt

# data[].attributes.attachment_link.title
ChatGPT and the Future of University Assessment

# data[].attributes.attachment_link.description
Image generated by DALL-E 2 ChatGPT-3 is a state-of-the-art language model developed by OpenAI. It is based on the GPT-3 (Generative Pre-trained Transformer 3) architecture and has been trained on a massive amount of text data. It has the ability to generate human-like text, answer questions, and complete various language-based tasks.

# data[].attributes.attachment_link.readable_text
ChatGPT has been a major topic in learning technology discussion for the past few weeks. It introduces a web-accessible, friendly, interface to the existing GPT (Generative Pre-trained Transformer) system which is created and maintained by the OpenAI corporation.

ChatGPT generates text in response to a prompt. This text is generated based on a set of training data which the system has been exposed to in order to model what a text might look like. ChatGPT was also trained on a set of conversations and ratings by human trainers who looked at several potential ChatGPT answers for a given prompt.

It is important to note that the text GPT generates is based solely on the structure of text and so it has no inherent understanding of any concepts. It cannot know anything about heart disease, only what words and phrases are commonly associated with the words "heart disease" in the training data it has been given. This means that it may generate perfectly correct and helpful text or it may generate plausible but completely incorrect text. Many people have shared that ChatGPT has generated citations for its work, but these referenced works that do not exist, or that do exist but do not relate to the words being included. This is because ChatGPT understands the form of a citation, but has no concept of a citation, it's simply a thing it has seen in the training data.

Many people have been writing about ChatGPT, particularly in the context of higher education. Several of these are gathered here:

January 10 Updates:

January 11 Updates:

January 12 Updates:

January 13 Updates:

January 16 Updates:

January 17 Updates:

January 18 Updates:

January 24 Updates:

January 25 Updates:

January 31 Updates:

February 3 Updates:

February 6 Updates:

As the discussion evolves we will update the collection here.

# data[].attributes.attachment_link.description
ChatGPT has been a major topic in learning technology discussion for the past few weeks. It introduces a web-accessible, friendly, interface to the existing GPT (Generative Pre-trained Transformer) system which is created and maintained by the OpenAI corporation. ChatGPT generates text in response to a prompt.

# data[].attributes.attachment_link.description
Artificial intelligence is growing by leaps and bounds. It’s no longer science fiction anymore! Lots of AI tools can serve as our own personal

# data[].attributes.attachment_link.title
Understanding and protecting academic integrity

# data[].attributes.attachment_link.description
Insights on how universities can tackle academic misconduct among students and faculty alike

# data[].attributes.attachment_link.readable_text
There has been a lot of debate about how ChatGPT can be used to reimagine the way we teach content and deliver assessment. Most of the focus is about how we can redesign and craft cheat-proof assessments that mostly embrace AI. Higher education is coming to terms with how ChatGPT, and artificial intelligence more broadly, can be embraced by exploiting it as digital aid to improve writing skills and creating assessments that are growth-oriented and incorporate skills-based activities. We've seen unexpected and welcome applications such as the potential to make communication and assessment more inclusive for students with disabilities.

In 2023 we will be launching a 100% online master's course to nine countries on four continents. ChatGPT creates challenges for remote assessment but does it also offer online content creation opportunities?

Using ChatGPT to create online course content: course outlines, writing content, scripting and editing videos

Creating a course outline: Online course development begins by creating an extensive course outline (commonly called a course map or blueprint). Learning designers and academics together sequence topics to divide the course into weekly units that are logical and engaging, and align to the course objectives, assignments and target student cohort.

Creating written content: ChatGPT can be used as a research assistant, generating a first draft of each online course page according to topics in a course map, which can then be sent to an academic. This overcomes the "blank page" problem academics often face and is a form of "creative assistance". Pages can then be reviewed, edited and revised by the academic to ensure accuracy, coherence, and alignment with course objectives and learning outcomes.

Scripting and editing educational videos: ChatGPT can be prompted to write draft scripts for videos and podcasts or generate questions for interviewing subject matter experts, which can then be refined by learning designers. Following on from this, AI can bring efficiencies to the post-production process. For example, when producing videos with segments from multiple interviews, it can be used to reduce the time spent on editing. This can be achieved by feeding AI tools the transcriptions and prompting it to create a cohesive narrative from the sentences within. The AI output from this can be used to guide content creators during the editing process.

Not every learning designer, faculty member or student will want to work with AI and we need to accept this. In his 2019 book Should Robots Replace Teachers? Neil Selwyn advocates that integration of AI into teaching and learning must be presented as choice. We need to be able to present a sound educational justification for the use of AI in education and not just for the sake of change.

3. For those who want to use AI, decide what's in and what's out

ssociate professor and academic director at RMIT Online , Anitra Nottingham is head of course design and Marcelo Zerwes is creative media manager, all at RMIT University.
If you found this interesting and want advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the THE Campus newsletter.

# data[].attributes.attachment_link.title
ChatGPT and learning design: what online content creation opportunities does it offer?

# data[].attributes.attachment_link.description
A guide to how ChatGPT and other AI writers can be used to help learning designers and faculty create course content more efficiently

# data[].attributes.attachment_link.title
Notion – The all-in-one workspace for your notes, tasks, wikis, and databases.

# data[].attributes.attachment_link.description
A new tool that blends your everyday work apps into one. It’s the all-in-one workspace for you and your team

# data[].attributes.attachment_link.title
Syllabi Policies for AI Generative Tools

# data[].attributes.body
Interim Guidance on how to cite ChatGPT in APA style

# data[].attributes.attachment_link.title
Research Guides: APA Style 7th Edition: ChatGPT & AI Tools

# data[].attributes.attachment_link.description
This guide provides information and examples (in-text, references, and formatting) to help you cite sources in APA style. Why cite? Citing sources gives credit to the author(s) whose work you used to support your research.

# data[].attributes.attachment_link.title
ChatGPT Is Making Universities Rethink Plagiarism

# data[].attributes.attachment_link.description
Students and professors can’t decide whether the AI chatbot is a research tool—or a cheating engine.

# data[].attributes.attachment_link.readable_text
The question of the week is:
How do you think artificial intelligence-powered tools like ChatGPT are going to affect K-12 schools, and what are practical strategies teachers can use to respond to them?
Brett Vogelsinger, Gina Parnaby, and TJ Wilson kicked off the series.

Today, Susan Barber, Andrew Cohen, Elizabeth Matheny, and Amanda Kremnitzer contribute their ideas.

'ChatGPT Can Be a useful tool'
Susan Barber teaches at Midtown High School in Atlanta, serves as the College Board adviser for AP Literature and on the NCTE Secondary Steering Committee, and works as a teacher consultant. You can find her cheering teachers on at MuchAdoAboutTeaching.com or on Twitterand IG - @susangbarber:
"Video killed the radio star" was MTV's first music video when the network launched in 1981 sending deejays into panic mode. Yet, in 2023, MTV has reinvented itself multiple times, and radio still has its place even among streaming services. So when the ChatGPT launched, followed by the onslaught of "The Death of High School English" articles, I had no doubt teachers would adjust and move forward. My initial thoughts were less centered on "catching" students using it but rather on figuring out how to incorporate CHATGPT into teaching and learning.

The first thing I did was ask ChatGPT for classroom uses and got some good ideas:

Provide suggestions for grammar, vocabulary, and sentence structure. I copied the introduction paragraph for this post into ChatGPT and prompted it to "improve the writing in this paragraph." Students can use this to compare their writing with"improved" ChatGPT writing as an option for revision. (For the record, I liked my paragraph better but acknowledge my bias.)

Give feedback on essays. I also tested this by pasting an essay into ChatGPT and asking for feedback. The feedback was basic. I then asked for feedback specifically on the structure of the essay. Again, the response was general but did focus on organization. Students can use this to workshop an essay or as a starting point for peer feedback. Feedback will get more focused as the prompt becomes more focused, but AI feedback will never be as specific as teacher feedback.

Generate ideas. As stated earlier, I used ChatGPT to generate ideas on this topic. Some of the ideas I'm listing here and expanding on and others I threw out altogether. Brainstorming is an important part of the writing process, but ChatGPT can have a place in class for generating ideas.

Debate/discussion generation. I love the idea of asking students to develop a claim or argument, asking ChatGPT to refute the idea, and using the rebuttal as a way to strengthen their original argument. This is a means of feedback focused on content, which could help during the writing and revision process.

Create quizzes for reading checks. With the rise of choice reading in classrooms, ChatGPT can offer individualized self-checks for understanding that can build confidence in readers or serve as a signal for students to seek help if they're having trouble. With the ability for students to generate their own quizzes, students can read at their own pace. I would not use these for grades in my classroom.

The generated list, however, didn't include some ideas that I thought of:

Create writing samples for revision. I've fed many prompts on a variety of subjects to ChatGPT; responses tend to follow certain structural patterns and sound, well, like a computer wrote it. An interesting exercise could be for students to strengthen a ChatGPT response by experimenting with elements of syntax, tone, and different types of diction that are more authentic to their writing voice. Or students could focus on content by adding claims, evidence, or analysis or reorganize the structure. An important part of the activity would be a metacognitive element on the how and why of changes made.

Practice scoring. Along with revising samples, students can score samples generated by ChatGPT. This could be done as a norming process for the class as a whole or in small groups with students discussing their scores and reasoning. The emphasis on this exercise would be less about the "score" a sample is given and more about the discussion around writing the students would be having.

Generate writing prompts. Teachers often struggle with topics for free or quick writes at the beginning of class, Students can use ChatGPT to create prompts on topics of interest to them, saving teachers time and increasing student engagement due to the element of choice.

The larger and more important work for teachers is thinking about the assignments we're ultimately asking our students to do. If they can be easily completed by a computer response, are they worth doing in the first place? While we're parsing all of that out, though, ChatGPT can be a useful tool especially in the writing process.

'Endless Possibilities for ChatGPT'

Andrew Cohen is a high school English teacher at International School Bangkok. He and Dave Giles are co-founders of IB English Guys, an educational website and YouTube channel that supports the IB English community:

"Class, I'd like you to read Act 1 of "Hamlet" tonight. Write a multiparagraph response about what motivates Hamlet to exact revenge on his uncle. Be sure to discuss authorial choices and how they shape meaning for the audience."

While there is definitely a time and place for assignments such as this, teachers must be careful and not "overassign" this type of analytical work. Yes, it's critical that students learn to break down and interpret key passages from literary masterpieces, but it's equally important that we as educators acknowledge that ChatGPT can replace some of these standard classroom practices and free up valuable time for more engaging and authentic writing tasks for students.

If we leverage AI to unpack obscure allusions and analyze iambic pentameter, students can instead focus on connecting stories to their own lives. Rather than writing about Gertrude as a mother and her effects on Hamlet, students can let AI take care of that obvious analytical task and instead focus on writing about their own parents and how they shape their child's identity. Isn't this the purpose of literature-to better understand the human condition so we can better understand ourselves and others?

With this in mind, literature teachers should view ChatGPT as a great equalizer. Consider the nonnative speaker of English who may take hours to write a literary essay, a staple assignment in most literature classes. Sadly, the time spent on this endeavor means these students rarely make it to the most important part of such a task-making connections between the text, themselves, and the world around them. To me, this is the real tragedy, not Hamlet.

Now, I'm not saying that analysis, interpretation, and developing academic writing skills are no longer relevant. Rather, I'm suggesting ChatGPT can streamline the thinking and writing process so valuable classroom time can be spent more wisely. Here are just a few of the endless possibilities for ChatGPT in the English/language arts classroom:

Ask ChatGPT to answer a given essay question but have it include scholarly research to justify the response. In fact, command the AI to respond several times. In the time it would have taken students to research one idea, they could have read and evaluated several critical responses from AI to determine how their personal interpretations either confirm or refute ideas in the text.

Instruct students to enter original ideas in a simple format and ask ChatGPT to convert those ideas into standard essay format. The student could then analyze and evaluate several responses to further develop their own ideas as well as their understanding of formal academic writing.

Allow ChatGPT to help with the revising and editing process. Students can use all that saved time to engage in further discussion and application of key thematic ideas and link to the world around them.

Ask ChatGPT to generate a series of outlines to a given question about important concepts from the text. Students could then read, analyze, and evaluate which outline most closely reflects their interpretation of the text as well as the reality of their own lives.

Let a student who is new to English enter sophisticated thoughts in their native language and watch the AI translate them into a polished piece of academic English. This may allow students to practice making meaningful connections with their peers, teachers, and the world around them before they have to produce written work on their own.

Clever implementation of some of these ChatGPT strategies may prove effective in eliminating a classic problem teachers complain about-not having enough time with learners to help them understand the relevance of literature and how it fosters greater empathy and understanding of our world. What could be more important than that?

So, teachers, I challenge you to embrace and adapt your classroom practices. Let's go beyond commentaries on passages from "Hamlet." Instead, let AI support students in that endeavor. That will free up precious time allowing students to discuss why "to be or not to be'' still matters today and how it enables humans to better understand the complexities of life and death. When teaching literature, surely this is what matters most.

Pluses And Minuses
Elizabeth Matheny is a teacher specialist for innovative learning in Maryland. You can find her on Twitter @lizmathenyFCPS. Amanda Kremnitzer is a high school literacy specialist in Maryland. You can find her on Twitter @MrsKremTHS:
When ChatGPT went live in November 2022, it didn't take long for us to pick up on the heightened anxiety over this new AI tool and the potential impact on writing instruction. As teacher leaders with more than 20 years combined experience teaching high school English, we wanted to test it for ourselves. Right before our holiday break, Amanda and I sat down to explore ChatGPT side by side, and here is what we found:
1.Two users will not generate the same response.
After submitting the exact same prompt (we used a local assessment prompt), we were provided two different written responses. While the nature of both responses was similar, the sentence structure, key vocabulary, and transitions were different. When we requested the tool to add text evidence, the tool provided the same evidence, but embedded it within the response differently. We assume the tool provided the same evidence for both responses (even though they were structurally different) because the tool has a limited bank of texts to cite from. Here's a screenshot of the opening to both of our ChatGPT-generated responses
2. While ChatGPT can produce written responses, it may not be able to address your specific prompt or rubric expectations.
It is important for teachers to consider the purpose of their writing assignment. Is it to assess the mastery of a set of standards? Is it to practice a specific skill? Most likely, ChatGPT is not going to demonstrate mastery on the specific "look-fors" teachers have instructed and reviewed with students. Even when we provided the tool with a specific text, the quality of evidence it selected was limited. It also did not directly cite specific text evidence to support the argument.
3. Prioritizing formative feedback and time to workshop writing is crucial to build student confidence.
Students will look for and often find an easy way out of doing schoolwork. They are kids after all! However, most students do this when the assignment is out of reach. Be it their skills or their confidence, there is often more that contributes to plagiarism than "I don't want to do it." To mitigate this, we need to prioritize the entire writing process. We know that modeling and feedback are high-impact teaching strategies. We can model and guide students through the prewriting and planning process. Provide feedback during the formative process. We can utilize ChatGPT to provide samples to model the revision and editing process.
4. It can save us time.
Teachers are consistently creating new material, and ChatGPT can take on some of that work for us. For example, I have multiple students who require outlines as a supplementary aid. I asked ChatGPT to provide an outline for the specific prompt the student was answering, and it produced a great product! Similarly, as we model our own writing and thinking, we can use ChatGPT to generate a response that may address the prompt but might be missing the specific skills or concepts we've explicitly instructed.
5. AI and digital study tools will never generate a student's authentic voice.
Writing sounds like the writer! We must show students the difference between these generated responses and authentic writing with a developed voice. This reminds me of a common experience I had early in my career. Students were not completing their assigned reading; they were reading summaries on a popular online study guide. As we were reading Huckleberry Finn, I designed a lesson that compared Chapter 19 of the text to the summary provided by a popular online study guide. I asked students to engage with the summary and then engage with the text, side by side. Yes, the summary was more time saving, but the text included rich imagery and characterization about the evolving relationship between Huck and Jim that the summary did not address. Even if students did not eagerly pick up their copy of the text, at least they were aware of the pitfalls of using such resources and the impact on their learning experiences. We can offer students a similar learning experience with responses produced by ChatGPT.
6. The same developer that produced ChatGPT produced a detector for ChatGPT produced responses.
Just as ChatGPT was making waves in classrooms in November and December, Open AIdevelopers were creating GPT-2 Output Detector Demo. A few other detectors have also gone live since then, including GPTZero. If you have a hunch that the writing was produced by AI, run it through the detector. At the very least, it's an opportunity to talk with the students about what led them to using AI to generate the response and what support they might need to engage with the work on their own. Here's what we saw when we submitted our ChatGPT response through the GPT-2 Output Detector Demo:

Thanks to Susan, Andrew, Elizabeth, and Amanda for contributing their thoughts!

Consider contributing a question to be answered in a future post. You can send one to me at lferlazzo@educationweek.org. When you send it in, let me know if I can use your real name if it's selected or if you'd prefer remaining anonymous and have a pseudonym in mind.

You can also contact me on Twitter at @Larryferlazzo.

Education Week has published a collection of posts from this blog, along with new material, in an e-book form. It's titled Classroom Management Q&As: Expert Strategies for Teaching.
Just a reminder; you can subscribe and receive updates from this blog via email (The RSS feed for this blog, and for all EdWeek articles, has been changed by the new redesign-new ones are not yet available). And if you missed any of the highlights from the first 11 years of this blog, you can see a categorized list below.
I am also creating a Twitter list including all contributors to this column.

# data[].attributes.attachment_link.title
19 Ways to Use ChatGPT in Your Classroom (Opinion)

# data[].attributes.attachment_link.description
The AI tool doesn't need to send you into a panic. It can be leveraged to help you and your students.

# data[].attributes.attachment_link.title
Do I write like a robot? Reviewing the AICheatCheck AI detection tool

# data[].attributes.attachment_link.readable_text
As the technology to generate writing with AI ramps up, so does the production of AI-detection tools that attempt to curtail "cheating" by using them. But the bevy of AI detection tools on the market aren't keeping up with the speed of AI's development. Because they are using older versions of GPT as the main component of their detection mechanism, they often misidentify human-composed text as AI-generated or confuse AI-generated text as human-generated. And while OpenAI is working on a watermarking scheme that uses cryptography to mark text generated by a transformer, even their lead engineer admits this would be easy to defeat by simply copying a generated output into a different language model.

We don't believe students want to learn. This profoundly misguided notion has broken the trust we've tried to establish with our students. Jonathan Malesic's recent op-ed in the New York Times reminds us all that our students want to learn, to be seen, to be heard. Like our students, we all thrive when we are open to learning new things and having our assumptions about the world challenged. Malesic's call for humility in this age of "knowingness" is a profound departure from a culture that views not knowing something as akin to failure:

Universities are factories of human knowledge. They're also monuments to individual ignorance. We know an incredible amount, but I know only a tiny bit. College puts students in classrooms with researchers who are acutely aware of all they don't know. Professors have a reputation for arrogance, but a humble awareness of the limits of knowledge is their first step toward discovering a little more.

To overcome careerism and knowingness and instill in students a desire to learn, schools and parents need to convince students (and perhaps themselves) that college has more to offer than job training. You're a worker for only part of your life; you're a human being, a creature with a powerful brain, throughout it.

One of the benefits of a college education is it teaches us how little we know about the world, about how vast and complex the range of knowledges are. Your students want to be there to be a part of that world, to occupy that space with their minds and ideas. College also invites you into a discussion about that knowledge, which tests your limits, allowing personal growth. When we take a stance on students and preemptively assume them to be cheaters, we are essentially denying them entry into this space.

Josh Elyer is correct. The current freak-out panic over ChatGPT mirrors the panic that drove institutions and faculty to plagiarism detection software in the first place. Decades later, plagiarism detection software has conditioned us to easily access a tool that we hope will prove to us a student's writing is their own. We don't think about the fact that this tool helped accelerate contract cheating and creates a culture of mistrust. We don't think about how the use of these tools has set up a situation in which companies are harvesting unfathomable amounts of data from our students. We mistrust our students' desire to learn in such a profound way that many don't give a second thought to using AI-detection software that tracks students' eye movements and may misflag female students of color at far higher rates than white males.

You can play around with a number of the AI detectors below. Sometimes they work. Sometimes they don't. Is that really the bar we want to use in so-called AI detection? I've included screenshots of the detectors mislabeling an AI-generated essay I created with a GPT-3 powered app in October. In each case, it labels the generated essay as human writing.
Huggingface's roBERTa powered Output Detector uses GPT-2 to try and detect AI IBM's Giant Language Model Testing Room (GLTR) also uses GPT-2 to try and detect AI
And then there are the waves of clones and eager developers rushing to sell terrified educators something to detect AI.
Crossplag decided to use roBERTa alongside their plagiarism detection and at least includes the caveat that the system doesn't work well enough to be deployed to institutions at this time. The newest one, GPTzero, also fails to detect the AI-generated text and likewise appears to use some form of GPT-2.
When we use AI to catch AI, we're making a decision to put our trust in systems that we do not fully understand under the vague promise that "at least this is something." Maybe that's what people need, but how long will it be until a faculty member takes one of these unreliable reports as gospel truth and brings an innocent student up on charges of academic misconduct that are false? Think of the damage this will further cause to the already frayed relationship between teachers and students.

I, too, have used plagiarism detection software in the past. Hundreds of my students have submitted thousands of essays through such services, knowing their words will be assessed against a large database to discern similarities to other essays. I have required the use of this service under a misguided belief that a rote and imperfect method of detecting cheating is more important than creating a culture of trust.

We're going to have to stop panicking and educate ourselves about the technology and think about what boundaries we'd like to put into place with our use of the tech and our students. While the current moral panic is about students using GPT to cheat, I imagine this is going to shift rapidly when folks realize that faculty can also use the technology to leave formative feedback on student essays. To echo Mike Sharples, "Students [will] employ AI to write assignments. Teachers use AI to assess and review them (Lu & Cutumisu, 2021; Lagakis & Demetriadis, 2021). Nobody learns, nobody gains."

We're going to have to search for moments of learning. One way of starting this exploration is to heed John Warner's call to stop automating the writing process. Students write predictable, formulaic essays and teachers leave similarly predictable, formulaic responses. What was learned and what was gained in such a system? The reasons why we've adopted this structure are based on convenience and labor. It's easier to teach the form of an essay than it is to teach a person the skills needed to develop as a writer. To do that, you need small classes, a reasonable teaching load, and well-compensated faculty.

# data[].attributes.attachment_link.title
Our Obsession with Cheating is Ruining Our Relationship with Students

# data[].attributes.attachment_link.description
AI detection isn't going to work-but that's not going to change our desire to have a tool to detect what work was written by a human vs. generated by an AI transformer.

# data[].attributes.attachment_link.readable_text
Is an ice cream sandwich a sandwich? How about a sushi roll, chicken wrap or sloppy joe? These were some of the prompts included in a classification and model-building assignment in the fall 2022 Knowledge-Based AI course that David Joyner taught at the Georgia Institute of Technology.

But when Joyner, executive director of online education and the online master of science in computer science and senior research associate, was scheduled to teach the course again in the spring 2023 semester, he reconsidered the assignment in the presence of ChatGPT -the OpenAI chat bot that burst onto the global stage in late 2022 and sent shock waves across academe. The bot interacts with users in a conversational way, including by answering questions, admitting its mistakes, challenging falsehoods and rejecting inappropriate requests.

"I'd used the questions for five years because they were fun questions," Joyner said. "But ChatGPT's answer was so precise that I'm pretty sure it was learning from my own best students," whom he suspected had posted their work online. Joyner replaced several of the sandwich options with avocado toast, shawarma, pigs in a blanket, Klondike bar and Monte Cristo. He also updated the academic misconduct statement on his syllabus to "basically say that copying from ChatGPT isn't different from copying from other people." Such efforts, Joyner acknowledges, may be a temporary fix.

As faculty members ponder academe's new ChatGPT-infused reality, may are scrambling to redesign assignments. Some seek to craft assignments that guide students in surpassing what AI can do. Others see that as a fool's errand-one that lends too much agency to the software.

Either way, in creating assignments now, many seek to exploit ChatGPT's weaknesses. But answers to questions concerning how to design and scale assessments, as well as how to help students learn to mitigate the tool's inherent risks are, at best, works in progress.

"I was all ready to not stress about the open AI shit in terms of student papers, because my assignments are always hyper specific to our readings and require the integration of news articles to defend claims etc. ... BUT THEN I TRIED IT ..." Danna Goldthwaite Young, professor of communication at the University of Delaware, wrote this week in introducing a thread on Twitter.

Students Should Surpass AI-or Not

When Boris Steipe, associate professor of molecular genetics at the University of Toronto, first asked ChatGPT questions from his bioinformatics course, it produced detailed, high-level answers that he deemed as good as his own. He still encourages his students to use the chat bot. But he also created The Sentient Syllabus Project, an initiative driven by three principles: AI should not be able to pass a course, AI contributions must be attributed and true, and the use of AI should be open and documented.

"When I say AI cannot pass the course, it means we have to surpass the AI," Steipe said. "But we also must realize that we cannot do that without the AI. We surpass the AI by standing on its shoulders."

Steipe, for example, encourages students to engage in a Socratic debate with ChatGPT as a way of thinking through a question and articulating an argument.

"You will get the plain vanilla answer-what everybody thinks-from ChatGPT," Steipe said, adding that the tool is a knowledgeable, infinitely patient and nonjudgmental debate partner. "That's where you need to start to think. That's where you need to ask, 'How is it possibly incomplete?'"

But not every faculty member is convinced that students should begin with ChatGPT's outputs.

"Even when the outputs are decent, they're shortcutting the students' process of thinking through the issue," said Anna Mills, English instructor at the College of Marin. "They might be taking the student in a different direction than they would have gone if they were following the germ of their own thought."

Some faculty members also challenge the suggestion that students should compete with AI, as such framing appears to assign the software agency or intelligence.

"I do not see value in framing AI as anything other than a tool," Marc Watkins, lecturer in composition and rhetoric at the University of Mississippi, wrote in an email. Watkins, his department colleagues and his students are experimenting with ChatGPT to better understand its limitations and benefits. "Our students are not John Henry, and AI is not a steam-powered drilling machine that will replace them. We don't need to exhaust ourselves trying to surpass technology."

Still, others question the suggestion that AI-proofing a course is difficult.

"Creating a course that AI cannot pass? Shouldn't take very long at all," Robert Cummings, associate professor of writing and rhetoric at the University of Mississippi, wrote in an email. "Most AI writing generators are, at this stage, laughably inaccurate ... Testing AI interactions with components of a course might make more sense."

But Steipe is pondering a possible future in which descendants of today's AI-writing tools raise existential questions.

"This is not just about upholding academic quality," Steipe said. "This is channeling our survival instincts. If we can't do that, we are losing our justification for a contribution to society. That's the level we have to achieve."

How Faculty Can Exploit ChatGPT's (Current) Weaknesses

In the future, faculty members may get formal advice about how to craft assignments in a ChatGPT world, according to James Hendler, director of the Future of Computing Institute and professor of computer, web and cognitive sciences at Rensselaer Polytechnic Institute.

In the meantime, faculty are innovating on their own.

In computer science, for example, many professors have observed that AI writing tools can write codes that work, though not necessarily of the kind that humans find easy to edit, Hendler said. That observation can be exploited to create assignments that distinguish between content and creative content.

"We try to teach our students how to write code that other people will understand, with comments, mnemonic variable names and breaking code up into meaningful pieces," Hendler said. "That's not what's happening with these systems yet."

Also, since ChatGPT's ability to craft logical arguments can underwhelm, assignments that require critical thinking can work well in the presence of ChatGPT.

"It's not very good at introspecting," Steipe said. "It just generates. You often find non sequiturs or arguments that don't hold water. When you point it out to the to the AI, it says, 'Oh, I got something wrong. I apologize for the confusion.'"

Several faculty members contacted for this article mentioned that lessons learned from the earlier emergence of Wikipedia hint at a path forward. That is, both the online encyclopedia and OpenAI's chat bot offer coherent prose that is prone to errors. They adapted assignments to mix use of the tech tools with fact-checking.

Moving forward, professors can expect students to use ChatGPT to produce first drafts that warrant review for accuracy, voice, audience and integration to the purpose of the writing project, Cummings wrote. As the tools improve, students will need to develop more nuanced skills in these areas, he added.

An Unsolved Problem

Big tech plans to mainstream AI writing tools in its products. For example, Microsoft, which recently invested in ChatGPT, will integrate the tool into its popular office software and sell access to the tool to other businesses. That has applied pressure to Google and Meta to speed up their AI-approval processes.

"My classes now require AI, and if I didn't require AI use, it wouldn't matter, everyone is using AI anyway," Ethan Mollick, associate professor of management and academic director of the Wharton Interactive at the University of Pennsylvania, wrote on his blog that translates academic research into useful insights.

But big tech's speed in delivering AI products to market has not always been accomplished with care. Social media platforms, for example, were once naïvely celebrated for bringing together those with shared interests, not realizing at the time that the platforms also brought together supporters of terror, extremism and hate.

Meta's release of a ChatGPT-like chat bot several months before OpenAI's product received a tepid response, which Meta's chief artificial intelligence scientist, Yan LeCun, blamed on Meta being " overly careful about content moderation," according to The Washington Post.(LeCun spoke with Inside Higher Ed about challenges in computer science in September.) Faculty members may need to help students learn to mitigate and address inherent, real-world harm new tech tools may pose.

"The gloves are off," Steipe said of the huge monetary driver of the emergence of sophisticated chat bots. In higher education, this may mean that the ways in which professors assess students may change. "We've heavily been basing assessment on proxy measures, and that may no longer work."

Professors may assess their students directly, but that level of personal interaction generally does not scale. Still, some are encouraged to find themselves on the same side, so to speak, as their students.

"Our students want to learn and are not in a rush to cede their voices to an algorithm," Watkins wrote.

Such alignment, when present, may offer comfort to the heady disruption academics have experienced since ChatGPT's release, especially as bigger questions-beyond how to assign grades-loom.

"The difference between the AI and the human mind is sentience," Steipe said. "If we want to teach as an academy in the future that is going to be dominated by digital 'thought,' we have to understand the added value of sentience-not just what sentience is and what it does, but how we justify that it is important and important in the way that we're going to get paid for it."

# data[].attributes.attachment_link.title
ChatGPT sparks debate on how to design student assignments now

# data[].attributes.attachment_link.description
Is an ice cream sandwich a sandwich? How about a sushi roll, chicken wrap or sloppy joe? These were some of the prompts included in a classification and model-building assignment in the fall 2022 Knowledge-Based AI course that David Joyner taught at the Georgia Institute of Technology.

# data[].attributes.attachment_link.title
How Will Schools Respond to the A.I. Revolution? #chatgpt #artificialintelligence

# data[].attributes.attachment_link.readable_text
Artificial intelligence (AI) panic is spreading across the education sector. The newest wave of generative AI tools, released without guidance or clear suggestions for educators late last year, might be among the biggest and fastest transformations to hit academia. Its rapid adoption instantly challenges the existence of a valuable and widely used type of assignment-the essay-along with dozens of other assignment types in fields from programming to poetry.

Cheating is an obvious concern among educators. Evaluations may require a return to oral exams and writing longhand in exam books. Or maybe AI-written text will be identifiable by other AIs. Or assignment types will adjust to the new reality. Either way, the cheating problem will be solved eventually.

But focusing on the problems distracts us from the opportunities AI can provide-including helping us teach in new ways. That starts with educators becoming familiar with AI tools, especially large language models (LLMs) like those underlying ChatGPT and OpenAI Playground, and taking time to experiment with these systems and get a sense of what they can and cannot do.

Focusing on the problems distracts us from the opportunities AI can provide-including helping us teach in new ways.

As of now, AI is great at creating credible-sounding facts, but will often share false data, made-up citations, and incorrect information. Thus, students must have enough knowledge about the topic they are examining to produce sufficient results. Receiving good results from these tools also takes practice. We must train our students to properly use this technology by working interactively with AI models to generate more meaningful and interesting text. This will help them to be successful in their studies and better prepared for their futures. These tools will be part of their lives, so they should learn how to apply them effectively.

Integrating AI into our classrooms isn't always easy-it takes time and dedication. To help, we are sharing some ways we are experimenting with AI to raise student expectations and enhance learning. But first, we propose you start your AI journey by creating a classroom policy for these tools to help your students better understand and properly use them.

Crafting an AI Policy for Your Class

AI is a tool that has tremendous potential. But, like all tools, it has its advantages and disadvantages. To create productive and meaningful content with AI, our students must learn to work with ChatGPT and other LLMs.

Educators can help by creating an environment in which students get a chance to practice doing just that. The first step is to be transparent about our expectations. We must set a clear classroom policy that forecasts potential problems and offers guidance for acceptable use. We've found that providing this information seems to give students permission to explore and understand the tool. It also offers clarity and teaches them how to use AI honestly.

Having a clear policy around AI use is step one in creating a classroom environment in which AI can be embraced responsibly and honestly.

We created an AI policy for undergraduate and MBA students and found they really appreciate the direction and transparency of expectations. Further, assignments and projects continue to be thoughtful and writing quality seems to improve.

Here are our suggestions for crafting your own policy.

Your policy should include these considerations:

Under what circumstances AI use is permitted or forbidden

How students should cite or credit AI

A warning about the technology's tendency toward hallucination (i.e., deceptive data) and clear rules regarding students' accountability for AI output

A notice about using AI ethically and responsibly

Discussion of the need to use AI as a tool to learn, not just to produce content

An example policy:

"I expect you to use AI (e.g., ChatGPT and image generation tools) in this class. In fact, some assignments will require it. Learning to use AI is an emerging skill and I provide tutorials on how to use them. I am happy to meet and help you with these tools during office hours or after class.

"Be aware of the limits of ChatGPT, such as the following:

If you provide minimum-effort prompts, you will get low-quality results. You will need to refine your prompts in order to get good outcomes. This will take work.

Don't trust anything it says. If it gives you a number or fact, assume it is wrong unless you either know the answer or can check with another source. You will be responsible for any errors or omissions provided by the tool. It works best for topics you understand.

AI is a tool, but one that you need to acknowledge using. Please include a paragraph at the end of any assignment that uses AI explaining what you used the AI for and what prompts you used to get the results. Failure to do so is in violation of academic honesty policies.

Be thoughtful about when this tool is useful. Don't use it if it isn't appropriate for the case or circumstance."

Feel free to use this language or adjust it to fit in your own course.

Having a clear policy around AI use is step one in creating a classroom environment in which AI can be embraced responsibly and honestly. You are also welcome to use our tutorials and modify them as needed.

Now that you have a policy, it's time to think about how you might use these AI tools to enhance student learning. Here are two use cases to try in your classes.

Using AI to Raise Expectations-and Level the Playing Field

Our students have tremendous skills, but not all are good writers. Some may have a language barrier, others may lack a particular predilection for writing, or they may not have had prior opportunity to take courses in writing. Whatever the reason, not being an expert writer can be a problem, particularly in academia where grades depend on it. As a result, some students take elaborate risks; they may hire editors or even cheat by paying people to write essays for them.

ChatGPT can level the playing field. With AI, everyone can now produce credible writing (as well as illustrations, code, and in the coming months, even more outputs such as video). While this may pose a problem for some classes, like those heavily focused on teaching the mechanics of writing, for many other classes, ChatGPT can work like a calculator of sorts-helping improve student performance. In these classes, you might set the expectation that you want to see only high-quality written work because AI can help students produce error-free sentences and paragraphs.

However, producing meaningful and insightful content through AI is not necessarily easy; it requires both topic expertise and skill. Show your students this guide for writing with AI; asking them to credit AI and provide the prompts they used to write their essays will teach them how to use the tool properly.

Ultimately, by embracing AI in your classroom, you'll likely be increasing your expectations about how much students should be able to accomplish. In other words, they may save time writing essays using AI, but that time should now be used to gather more data and think more deeply about the material.

Elevating Student Learning with AI

If we are willing to think creatively and try new approaches with it, AI can unleash a whole new way of learning. For example, letting students teach or edit each other's work is a powerful way to take concepts from theory into practice (e.g., peer review). As part of a new course exercise we designed, ChatGPT serves as students' partner to help improve their writing.

In the exercise, the AI produces an essay based on a prompt, and its student partner then critiques it and works to steadily improve the essay by adding new information, clarifying points, fact-checking, and adding insight and analysis. This exercise takes advantage of the AI's propensity to simplify complex topics, using its lack of insightful analysis as a backdrop for students to provide evidence of understanding.

We must train our students to properly use this technology by working interactively with AI models to generate more meaningful and interesting text.

Approaching AI in this way pushes students to think critically about the content and articulate their thoughts for improvement clearly and concisely. Likewise, the exercise can help deepen their understanding of complex topics. Asking students to identify what the AI gets right and wrong can activate deeper thinking, highlighting nuances and implications they hadn't previously considered.

This and similar exercises should help improve students' understanding of major class concepts and illustrate the limits of ChatGPT and other available LLMs.
Find ideas for prompts and assignments you can incorporate into your classroom here.
Embracing AI by Taking an Experimental Approach

AI is here to stay and will only become more prevalent in our classrooms over time. Educators need to think beyond adapting to these tools-we must actively embrace them as a way to improve our teaching.

The best path forward is experimentation. Some experiments will work; others won't. It's also important to remember that while the sudden advent of generative AI may be disruptive to educators, it is even more disruptive to the futures of the students we teach. We need to give them the skills to thrive in a world transformed by AI.

# data[].attributes.attachment_link.title
Why All Our Classes Suddenly Became AI Classes

# data[].attributes.attachment_link.description
Artificial intelligence (AI) panic is spreading across the education sector. The newest wave of generative AI tools, released without guidance or clear suggestions for educators late last year, might be among the biggest and fastest transformations to hit academia.

# data[].attributes.attachment_link.readable_text
I don't need to start this post with the usual predictions that AI will transform our classrooms. It is obviously happening. Students are cheating with AI. Students are getting help with AI. I have required AI for all my classes this semester, and I hear more instructors are doing the same. GPT-4 tutors are being launched by large organizations (both Khan Academy and Duolingo currently have them). The world is changing fast.

But one thing that is not changing is the best way for people to learn. We have made large advances in recent years in understanding pedagogy - the science of learning. We know some of the most effective techniques for making sure material sticks and that it can be retrieved and used when needed most.

Unfortunately, many of these advanced pedagogical techniques are time-consuming to prepare, and many instructors are often overworked and do not have the resources and time to add them to their teaching repertoire. But AI can help. In the rush to deliver AI benefits directly to students, the role of teachers is often overlooked. AI tutors, as exciting as they are, do not replace the complex role of a teacher in front of a class. But not enough effort seems to be going towards applying AI to help instructors. We have a new paper that tries to remedy that gap, by providing some research-backed approaches to pedagogy, and the AI prompts (for GPT-4, GPT-3.5, and other AIs) to implement them. You can read the paper here, but I wanted to summarize some of those approaches.

Explaining complicated ideas often means giving students lots of examples so they can truly grasp what you mean. When teachers provide multiple examples, it helps students understand abstract concepts in a real-world way, challenges them to think critically, and shows how subtle aspects of ideas can work in different situations. Ultimately, this approach makes it easier for students to apply what they've learned in new areas.

However, coming up with good examples is hard work for teachers, who already have enough on their plates. They have to think about whether examples will capture students' interest, connect to what they're learning, and have the right amount of detail without being too simple or too complex. The balance is key, since examples that are too complicated will just confuse students, while ones that are oversimplified won't actually teach them much.

AI can help teachers generate a ton of diverse examples tailored to what students care about and what they're working on. With AI-generated examples, teachers can make sure they're accessible but also informative. Of course, the teacher remains critical, because they need to vet the ideas and decide how to deploy them. AI just makes the job easier.

You can click this link to get Bing to generate examples: https://sl.bing.net/bePdl4o9xf2 (this new feature of sharing links to in-process conversations is a little like sharing programs - paste it into a browser with Bing access to start the prompt)

Or you can use ChatGPT, and paste this prompt in: I would like you to act as an example generator for students. When confronted with new and complex concepts, adding many and varied examples helps students better understand those concepts. I would like you to ask what concept I would like examples of, and what level of students I am teaching. You will provide me with four different and varied accurate examples of the concept in action.

In the paper, we discuss more about how to assess the quality of examples. You can share the ones you like directly with the class, or use them as part of an in-class exercise. For example, you can ask students to compare and contrast these examples: what different aspects of the concept does each highlight?

Helping students truly understand new ideas is key to teaching them anything. To do this, teachers need to get where their students are coming from, carefully plan how they'll explain things, give students hints to follow along, and use analogies to bring big concepts down to earth. The goal is for students to be able to explain what they've learned in their own words.

But coming up with lots of ways to explain a single topic can take a ton of time and effort. It can be hard figuring out how to pitch ideas at just the right level, include background info, and adapt for different learning needs. This is where AI can help out.

AI can generate different explanations, walk through ideas step-by-step, and add in more examples. If some students are struggling, AI can provide simpler summaries to get them caught up. Teachers can also use AI to improve their own explanations by making them simpler or adding more examples. But it's important to remember AI-made explanations should be a starting point. Teachers need to check them over before using them with students.

Prompt for Bing : You can use the following link to get Bing to generate explanations: https://sl.bing.net/koA1v8uUzw4.

For ChatGPT, try: You generate clear, accurate examples for students of concepts. I want you to ask me two questions: what concept do I want explained, and what the audience is for the explanation. Provide a clear, multiple paragraph explanation of the concept using specific example and give me five analogies I can use to understand the concept in different ways.

We describe how to check these results for effectiveness and errors in the paper, but, once you have vetted these examples, you can use them in class, hand them out to students as a study guide to supplement their knowledge, or use them as the basis for exercises.

Students hate tests, but they are some of the most effective learning tools we have. Contrary to the popular belief that tests only serve to assess knowledge, they actually play a pivotal role in the learning process itself. By incorporating repeated testing and knowledge retrieval, students are better equipped to retain information in the long term.

Low-stakes tests offer active retrieval practice, encouraging students to recall information from memory, which in turn enhances their ability to remember and retrieve information later on. And these tests provide valuable feedback on students' understanding of the material, helping them identify gaps in their knowledge and adjust their learning strategies accordingly. This not only aids in information processing but also prepares students for high-stakes exams.

While low-stakes tests has clear benefits, they can be hard for teachers to put together. Creating good questions, providing scores and feedback, and ensuring questions match what students should know takes significant time and effort. AI can help. They can generate practice questions and provide targeted feedback, allowing instructors to focus on teaching instead of test-making. With AI support, students may learn concepts more deeply through continual practice that sticks with them beyond the lesson.

Here's how to do it in ChatGPT: You are a quiz creator of highly diagnostic quizzes. You will make good low-stakes tests and diagnostics. You will then ask me two questions. (1) First, what, specifically, should the quiz test. (2) Second, for which audience is the quiz. Once you have my answers you will construct several multiple choice questions to quiz the audience on that topic. The questions should be highly relevant and go beyond just facts. Multiple choice questions should include plausible, competitive alternate responses and should not include an "all of the above option." At the end of the quiz, you will provide an answer key and explain the right answer.

The expertise of the instructor in avoiding AI errors is key here, as is their judgement in the types of questions to ask. Once that is done, there are many ways to use these quizzes. One option for including tests in a class is integrating test questions in a discussion or lecture. They give instructors insight into any misconceptions or errors students might have and allow them to make a decision: Is it time to move on, or do I need to adjust the lesson?

Low-stakes tests can be given to students in class as a group exercise in which teams report out their responses, followed by a class discussion. They can be assigned as individual classwork or homework or posted in an online discussion forum. Additionally, tests can be distributed in class, and after completing the tests, students can then be given the answer key. They can compare their responses with the correct responses. A follow-up to such an exercise might be a reflection: What skills do you think you need to work on? How might you improve?.

Short check-in exercises are key for helping students and teachers understand the course material. They provide real-time feedback, allowing them to identify gaps in knowledge and areas that need clarifying. While grading assignments shows students their progress, these informal checks for understanding promote active learning and help motivate students by demonstrating that the instructor genuinely values their needs.

One such assessment method, commonly known as the "1-minute paper" or "muddiest point" exercise, promotes active learning and self-reflection. Students are encouraged to summarize their knowledge, pinpoint any uncertainties, and share their perspectives on the material covered. This not only helps identify gaps to be addressed in subsequent classes, but also fosters increased student engagement and motivation by demonstrating that instructors are genuinely responsive to their needs.

To design such an assessment, instructors can choose a specific focus, such as an activity, topic, or class discussion. They can then formulate a question that will reveal both what students have comprehended and what they find perplexing. Potential questions might include: "What was the most important idea or concept discussed in today's class, and why is it significant?" or "What has been the most challenging concept so far, and what aspects did you struggle to understand?" By prompting students to think more deeply about the course material, these assessments can pave the way for a more enriching and engaging learning experience. Then they can use AI to help summarize the results.

To have the AI help quickly summarize student responses, instructors can create a Google Doc or any shared document and ask students to submit their responses. Then, instructors can submit a set of collective responses to the AI with the following prompt: I am a teacher who wants to understand what students found most important about my class and what they are confused by. Review these responses and identify common themes and patterns in student responses. Summarize responses and list the 3 key points students found most important about the class and 3 areas of confusion: [Insert material here]

You can obviously use this analysis after class to help you see where students stand, or even in-class to see what points you might want to cover (or ask students to cover) further. Students also often love these sorts of exercises because they feel like their feedback is being listened to.

Incorporating distributed practice into the learning process is important for helping students build strong and adaptable knowledge. Unlike massed practice, where topics are taught one after the other without much connection, distributed practice means revisiting material several times over an extended period. This approach helps students form better mental models, remember those half-forgotten facts, and really get a deeper understanding of concepts.

Now, even though it's useful, fitting distributed practice into the mix can be a bit tricky because of how course materials are usually structured and students' preference for massed practice. But AI can help. One way to bring distributed practice into the classroom is by introducing a topic and reviewing it after specific intervals, like a week, a month, and at the end of the semester. Teachers can use AI to whip up brief topic overviews and questions for ongoing assignments or assessments, adjusting the difficulty level as needed.

Linking new ideas to concepts students already know promotes deeper learning, and AI can lend a hand by weaving past topics into lectures or discussions. By asking the AI to find relationships between concepts, teachers can present various connections between ideas, making them clearer for students.

You can use the following link to get Bing to generate distributed practice exercises and tests: https://sl.bing.net/hnKI78bzvzw or you can ask ChatGPT: You are an expert teacher who provides help with the concept of distributed practice. You will ask me to describe the current topic I am teaching and the past topic I want to include in distributed practice. You will also ask me the audience or grade level for the class. Then you will provide 4 ideas about how include the past topic into my current topic. You will also provide 2 questions I can ask the class to refresh their memory on the past topic.

Once the instructor has some good responses from the AI, distributed practice can be implemented in a number of ways. In terms of timing, scheduling exercises and tests that space out practice across a course and that allow for some forgetting, is optimal. For instance, once students show evidence of understanding about one topic, practice of that topic (in the form of an assignment or a quiz) may be scheduled once some time has passed; students will need to work hard to pull out that knowledge from memory and that effort will help them access this information next time. You can use the AI output to make sure that happens.

AI is an incredibly exciting tool for teaching. It is widely-available, inexpensive, and fast to experiment with. We hope instructors will try some of the techniques in the paper to get a sense of how AI can make applying complex pedagogical approaches more effective and less burdensome.

And, for those worried about AI, it is worth remembering that, despite decades of hype from VCR classes to Massive Online Courses, technology has not replaced teaching. Instructors, and class interaction, play a vital role in making learning happen. AI allows new forms of learning and pedagogy that can benefit instructors, and their classes. As instructors, we have to experiment to learn the best techniques that work for us, and our students.

# data[].attributes.attachment_link.title
Using AI to make teaching easier & more impactful

# data[].attributes.attachment_link.description
I don't need to start this post with the usual predictions that AI will transform our classrooms. It is obviously happening. Students are cheating with AI. Students are getting help with AI. I have required AI for all my classes this semester, and I hear more instructors are doing the same.

# data[].attributes.attachment_link.title
GitHub - dair-ai/Prompt-Engineering-Guide: 🐙 Guides, papers, lecture, notebooks and resources for prompt engineering

# data[].attributes.attachment_link.description
🐙 Guides, papers, lecture, notebooks and resources for prompt engineering - GitHub - dair-ai/Prompt-Engineering-Guide: 🐙 Guides, papers, lecture, notebooks and resources for prompt engineering

# data[].attributes.body
Solid explanation of the basic concepts you need to understand AI. In five minutes!

# data[].attributes.attachment_link.title
What Is AI? | Artificial Intelligence | What is Artificial Intelligence? | AI In 5 Mins |Simplilearn

# data[].attributes.attachment_link.readable_text
Myth Busting: AI's Role in Teaching & Learning

AI in higher ed is much discussed but often misunderstood. The use of AI in teaching and learning raises ethical concerns but also holds promise for benefits that include reducing administrative burdens on faculty. This panel will explore the perceived concerns with AI's ability to automate administrative processes in course grading and generating data-driven insight on students' academic performance. It will also unpack how AI is being used by faculty to improve pedagogy and student outcomes.

Programming descriptions are generated by participants and do not necessarily reflect the opinions of SXSW.

# data[].attributes.attachment_link.title
Myth Busting: AI's Role in Teaching & Learning

# data[].attributes.attachment_link.description
SXSW EDU 2023 Schedule | AI in higher ed is much discussed but often misunderstood. The use of AI in teaching and learning raises ethical concerns but also holds promise for benefits that include reducing administrative burdens on faculty.

# data[].attributes.attachment_link.readable_text
Always On: Learning in the Age of AI & Technology

Today's students are always connected to technology. How does this "always on" approach to technology affect student behavior? When it comes to educating the modern student in the digital age, how have academic institutions adapted or failed to recognize how students learn today? And most importantly, what changes are needed to modernize the approach to learning in a way that recognizes the role of technology in students' lives?

Programming descriptions are generated by participants and do not necessarily reflect the opinions of SXSW.

This online event has ended

This event is currently available on demand.
Video On Demand for this event began on Thursday, March 9th 2023 at 2:30pm CT.

# data[].attributes.attachment_link.title
Always On: Learning in the Age of AI & Technology

# data[].attributes.attachment_link.description
SXSW EDU 2023 Schedule | Today's students are always connected to technology. How does this "always on" approach to technology affect student behavior? When it comes to educating the modern student in the digital age, how have academic institutions adapted or failed to recognize how students learn today?

# data[].attributes.attachment_link.title
AI Text Generators: Sources to Stimulate Discussion among Teachers

# data[].attributes.attachment_link.readable_text
What do teachers who assign writing need to know about AI text generators? How should we change our pedagogical practices, given the recent advances in AI Large Language Models (LLMs) such as OpenAI's ChatGPT? How should teachers participate in shaping policies around these technologies in our departments, institutions, and society?
As teachers who assign writing, we need to reckon with the possibility that our students may use writing generators to produce essays or parts of essays. These tools do not produce writing copied from human sources; the AI origin appears to be not reliably detectable either by human readers or by software.
To shape our individual and institutional responses to this new technology, writing teachers and scholars need more information about the kinds and quality of AI-generated text we can expect in response to common types of essay prompts. For example, we may want to design prompts that AI text generators are unskilled at performing, or we may want to find ways to use these generators pedagogically. Either way, as faculty responsible for teaching writing as impactful, ethical intellectual activity, we need to know what AI generators are capable of. And we need to understand the forms of bias and error that emerge in AI-generated writing.

Shared Resources

Opportunities to Engage and Participate

The resources collected here serve as catalysts for inquiry, discussion and collaborative research as we respond to this major change in the kind of writing assistance available to our students.

Suggested Citation

Mills, Anna (Curator). AI Text Generators and Teaching Writing: Starting Points For Inquiry. 2022. Last Updated February 14, 2023.

Authorship and Credits

Anna Mills ( armills@marin.edu) teaches English at College of Marin and previously taught at City College of San Francisco for 17 years. She is the author of an Open Educational Resource (OER) textbook, How Arguments Work: A Guide to Writing and Analyzing Texts in College, which has been praised on OER review sites and used at over 45 colleges. She is the recipient of an Open Education Research Fellowship and currently serves as the English Discipline Lead for the Academic Senate of the California Community Colleges OER Initiative. Anna earned a master's degree from Bennington College in Writing and Literature with a focus on nonfiction writing. Her essays have appeared in journals such as The Writer's Chronicle, The Sun, and Salmagundi. Recently, she has focused on exploring how writing instructors can respond to the accessibility of large language models. Follow her at @EnglishOER.

# data[].attributes.attachment_link.title
AI Text Generators and Teaching Writing: Starting Points for Inquiry

# data[].attributes.attachment_link.description
What do teachers who assign writing need to know about AI text generators? How should we change our pedagogical practices, given the recent advances in AI Large Language Models (LLMs) such as OpenAI's ChatGPT? How should teachers participate in shaping policies around these technologies in our departments, institutions, and society?

# data[].attributes.attachment_link.readable_text
You can see where this is headed. A writing assignment asks students to compare and contrast feminist themes in Jane Eyre and Wuthering Heights. Yup, it can do that. A political science exam requires short-essay responses to questions around the rise and fall of the Soviet Union. Check.

Is the writing captivating? No. Is it coherent? Mostly. So what does this all mean for teaching? That's one question I set out to explore when I wrote about ChatGPT last month. I'd like to dig in here to some of the ideas that I heard from digital-literacy experts, writing instructors, and teaching and learning specialists.

First, if you want to make your assignments AI-proof, that's likely impossible. These tools can be used in large and small ways. Maybe you won't receive a paper written by a bot, but a bot-written essay may inform your students' writing. Yes, you can shift all writing to in-class assignments, or you can have students write by hand. But as Anna Mills, an English instructor at College of Marin pointed out to me, these strategies introduce new problems. For example, you may have students with learning disabilities who struggle under such conditions.

Second, you may want to shift to different types of assignments and assessments. Maybe you allow students to produce a podcast instead of writing a paper. Or you create fewer writing assignments, but build in more feedback and revision to the ones you keep. Or you try prompts whose answers are less likely to be found on the internet. Of course, there is always another option, which is to invest in detection software. Already several tools on the market promise to do that. But many digital-learning experts say that's a losing game - tech will keep advancing, and students will find ways around detection tools. Nor do most instructors want to become writing police.

The approach that most intrigued me is one that has to do with engaging students in a conversation about why and how they write, sometimes using these AI tools.

John Warner, a writing expert and author, notes that writing is a form of thinking. Writing requires you to process and synthesize a range of facts and ideas, and to come up with a coherent and hopefully insightful take on what you have learned. Students, though, may have been trained in high school to see writing as a form of regurgitation based on a set of formulas (compare and contrast!).

If you can explain to students the value of writing, and convince them that you are genuinely interested in their ideas, they are less likely to reach for the workaround, Warner told me.

There are also a host of people excited about using this technology in their classroom. Why? Well, for one, it's not going away. Ignoring the fact that students will use it is seen by some instructors as an abdication of professional responsibility. These are powerful tools and it's better to help students learn how to use them judiciously, and to understand their limitations and benefits. The other reason to use them is that they can help spark the creative process, professors say, and enhance learning.

Marc Watkins, an instructor at the University of Mississippi, wrote a thoughtful essay about this recently. He and his colleagues in the department of writing and rhetoric started a working group last summer to figure out how to incorporate AI research, writing, and brainstorming tools into their classes. They used a counterargument generator to encourage students to explore different perspectives on a topic, and a research tool to help them brainstorm.

"What message would we send our students," Watkins writes, "by using AI-powered detectors to curb their suspected use of an AI writing assistant, when future employers will likely want them to have a range of AI-related skills and competencies?

"What we should instead focus on is teaching our students data literacy so that they can use this technology to engage human creativity and thought."

The significance of that argument was brought home to me by Mills and others, when they compared these writing tools to calculators. There was also plenty of hand-wringing when hand-held calculators appeared, but now they're integral to teaching math and other STEM disciplines. Mills says she can even envision a day when ChatGPT is embedded in programs like Word and Google Docs.

Of course, all AI tools are not the same. Those that complete your sentences in an email are at one end of the spectrum, and large language models, like ChatGPT, are on the other. As Watkins and others point out, the most sophisticated tools are also potentially dangerous. Some have been shut down after turning out biased or nonsensical research papers that could still confuse a layperson. All the more reason, they say, to engage your students in a discussion of what they are, how they work, and how they can be used as an aide in learning.

These teaching experiments are coming fast and furious, but here is one that caught my eye. In this , Paul Fyfe, an associate professor of English at N.C. State University, described an experiment in which he asked students to "cheat" on a final essay using GPT-2, an earlier version of the AI that underlies ChatGPT, and then discuss how the AI influenced them, as well as its potential uses and abuses.

Have you experimented with any of these large language model tools in your teaching? What questions, concerns and hopes do you have about this technology? Write to me at beth.mcmurtrie@chronicle.com and your story may appear in a future newsletter.

Want More on AI?

Want to get involved in the conversation around AI writing tools? Here are a few resources you might find helpful.

Anna Mills has put together several documents:
AI Text Generators and Teaching Writing: Starting Points for Inquiry How do we prevent learning loss due to AI text generators? AI Text Generators: Sources to Stimulate Discussion among Teachers
Elsewhere, a group of professors is compiling examples of how instructors are using text generation technologies in their assignments. The results will be published in an open-access collection. You can find out more about the project on this site, Teaching with Text Generation Technologies.

If you're rather listen to a discussion, here are a couple of webinars:
AI and the Future of the Essay What might ChatGPT mean for higher education?
Finally, if you don't think the AI is sophisticated enough yet to be worth consideration, check these out:

An emeritus professor of educational technology created a 2,000-word academic paper in 10 minutes.

An admissions expert created a Common App essay that several admissions counselors thought was real.

A professor at the Wharton School a syllabus, an assignment and lecture notes for an MBA-level introductory course on entrepreneurship.

Keep on Teaching

New year, new webinar series for Teaching readers! This one, which we call Keep on Teaching, delves into two challenges we know many of you are contending with: how to make class time meaningful, and how to support students without overextending yourselves.

The first session, on January 20, is open to anyone. The second, on February 10, is just for newsletter subscribers. (If you haven't signed up yet, remember, it's free!)

In both sessions, you can expect an interactive discussion that brings together practical advice, evidence-based insights, encouragement, and a reminder that you're not alone.

Read more about the series and sign up here. We hope you'll join us!

Thanks for reading Teaching. If you have suggestions or ideas, please feel free to email us at beckie.supiano@chronicle.com or beth.mcmurtrie@chronicle.com.

- Beth

Learn more about our Teaching newsletter, including how to contact us, at the Teaching newsletter archive page.

# data[].attributes.attachment_link.description
You can see where this is headed. A writing assignment asks students to compare and contrast feminist themes in Jane Eyre and Wuthering Heights. Yup, it can do that. A political science exam requires short-essay responses to questions around the rise and fall of the Soviet Union. Check. Is the writing captivating?

# data[].attributes.attachment_link.readable_text
Have you ever wondered how accurate proofreading apps that claim to use AI really are? Recently, I tested several texts, from simple to more complex, to see if they were flagged as AI-generated. The results were surprising, and it raised questions about the effectiveness of AI detection tools (AIDTs) and the extent to which we should rely on them.

Something as simple as stating the Pythagorean Theorem is automatically detected as AI generated.

In order to determine the length of a side in a right-angled triangle, algebraic techniques must be applied to rearrange the variables in such a way that the square of the hypotenuse is expressed as the sum of the squares of the other two sides. By performing these mathematical manipulations, the resulting equation can be solved by taking the square root of both sides, thereby obtaining the length of the desired side. This process comes from the Pythagorean theorem, which states that the square of the hypotenuse is equal to the sum of the squares of the other two sides in a right-angled triangle.

To verify this, I tried to dumb down my language and make it sound casual by using this prompt:

If you want to get a length of a side in a right-angled triangle, rearrange the variables so that the square of the hypotenuse is shown as the sum of the squares of the other two sides. Doing this gives you an equation that can be solved by taking the square root of both sides. After this, you can get the length of the side that you want. This universal fact is called the "Pythagorean theorem". It means that the square of the hypotenuse is equal to the sum of the squares of the remaining two sides.

I literally couldn't make this sound any simpler, so I asked ChatGPT to say what I just said, but in a way that sounds like a "caveman".

# data[].attributes.attachment_link.title
AI Writing Detection Tools are Faulty and Here's Why

# data[].attributes.attachment_link.description
Have you ever wondered how accurate proofreading apps that claim to use AI really are? Recently, I tested several texts, from simple to more complex, to see if they were flagged as AI-generated. The results were surprising, and it raised questions about the effectiveness of AI detection tools (AIDTs) and the extent to which we should rely on them.

# data[].attributes.attachment_link.readable_text
Generative AI programs like ChatGPT are great tools for generating all sorts of responses to well-written prompts. However, while the responses they generate have a lot of solid information and are often well-written, they often do not demonstrate true knowledge or wisdom.

From a novice perspective, these AI-generated responses can appear almost magical in their swiftness and accuracy. However, experts in specific fields can often easily identify specific errors in logic, mastery, relevance, and structure in responses created by Generative AI.

The overall point is to understand that generative AI is another tool that can be incorporated into our teaching and learning efforts, but it is not a replacement for true understanding, competency, and wisdom within specific academic fields of study.

Below, we have provided some prompts and responses generated by ChatGPT. Notice first how each response appears at a surface level to present a solid and credible response. Next, dig deeper and read the expert analysis to see where the AI-generated response may be flawed or incomplete.

Examples

# data[].attributes.attachment_link.description
Generative AI programs like ChatGPT are great tools for generating all sorts of responses to well-written prompts. However, while the responses they generate have a lot of solid information and are often well-written, they often do not demonstrate true knowledge or wisdom. From a novice perspective, these AI-generated responses can appear almost magical in their [...]

# data[].attributes.attachment_link.readable_text
Many people want to express themselves creatively, and remarkably few think like they are able to do so. One survey asked a representative sample of people how many were felt like they were living up to their creative potential. Only 31% said that they were. There are a lot of frustrated creative energy in the worl-

Look, that paragraph wasn't going anywhere. I rewrote it a few times, and it just isn't working. I know what I was to say, but I am frustrated. When this has happened in the past, I have asked a friend or coworker for help, or even walked away from the writing. But, I can just ask a generative AI for help: I am stuck on a paragraph in an essay about how AI can help get you unstuck. Can you help me rewrite the paragraph and finish it by giving me 10 options for the entire paragraph four professional styles. make the styles and approaches different from each other, making them extremely-well written.

one of these are perfect (I could make them better using the techniques I discussed in this post), but they provide something very new: an automated way to unstick yourself. It is very easy to get distracted from a task by one difficult challenge. AI provides a way of giving yourself momentum.

So, here are some practical tips for unsticking yourself with AI (and, at the end, a few wider questions that these approaches raise)

I already showed you one technique for overcoming the "tyranny of the blank page," asking AI to provide dozens of alternative approaches to completing a sentence or thought, but there is a lot of other ways it can help.

For example, when you are starting to write, you can use it for inspiration. You can ask your favorite AI: I am writing about how to be more creative, I need some inspiration to spark some ideas. Give me themes, quotes, links or anything else you might think might help me start my writing. Use reputable sources. Generally, Bing will provide more varied answers (that are usually more correct), but ChatGPT will provide more consistent ones (though the links and quotes are likely to be hallucinations). You can ask the AI for more details, exploring a topic until inspiration hits.

It can also be useful to ask the AI to provide an outline of a potential piece. Usually, it won't be exactly what you need, but it can serve as inspiration. (And, of course, you can have the AI actually write the material according to your instructions, a move which has a lot of ethical and practical implications)

Just pasting in your writing and asking AI to make it better/more professional/more concise/with better examples/more vivid can also be useful. And Bing can go one step further Ask it to look up writing tips by____ and apply that to your text to see a real change in style (though, since Bing only permits limited text entry of 2,000 characters, it can be challenging to use this technique a lot). Some particularly good writing prompts for Bing are:

For non-fiction ask Bing to: Read Cormac McCarthy's writing advice for scientists in Nature, Read How to Improve Your Nonfiction Writing from Masterclass, Look up nonfiction writing advice from John McPhee

For fiction ask Bing to: Look up Encyclopedia Britannica's Writing Tips from 7 Acclaimed Authors or ask it look up the style for your favorite author in the genre you are writing in.

A lot of people get stuck on a step in a project, leading them to abandon it completely.

For example, this study shows around 1/3 of Americans have had a startup idea in the last 5 years but few act on it - less than half even do any web research. You can overcome these blockages by asking ChatGPT or Bing for help. They aren't always 100% right, so don't take legal advice (yet) but they can bias you towards action, rather than spending all your time planning. This is an easy way to use these tools - you can literally just ask them what to do next. Bing is generally much more helpful for this, including giving you some useful estimates on costs and specific plans.

The key is dialog - you have a slightly misinformed but infinitely helpful intern - what do you want to ask it? The level of expertise is surprisingly wide, but any results need to be verified in detail.

AI can also help you overcome skill gaps that might be holding you back. How do you code a simple program when you have never coded before? You can ask the AI to help teach you (I used ChatGPT which is better at coding, for now). Again, the usual caveats apply, but, by working back-and-forth with the AI, possibilities begin to open up.

Some of the most intimidating circumstances we face are when we interact with others. For example, we are too reluctant to engage other people, either to help them or to ask for help. But AI is not that shy! You can ask it to help you ask for help from others. One obvious way is write a draft of an email from a student to ____ asking them for a letter of recommendation. Tell them it is for the Nobel Prize and that the selection criteria is the ability to play piano. Mention that they were your piano instructor, and they would be the right person to help. I would strongly advise that you use this as a draft to work on, and add details and make it personal! (I have started receiving a lot of emails from students that look like they came right from ChatGPT, a problem that will only grow with time)

Beyond this, however, AI can help you overcome some of your worries about upcoming events by helping you prepare for a high-stakes conversation or interview. Try asking it to pretend to be someone and role-play out a scene for you. I am going to a job interview for a position as a graphic designer at Acme, roleplay the interviewer, ask me questions, and give me feedback on my replies. Or, with Bing, ask it to look up [person's name]and suggest some topics to discuss about their work, to get a sense of how you might want to start a conversation. Again, these tips do not substitute for the human touch, and unedited AI emails feel uncompelling, but they are a great way to overcome inertia by providing you with a starting point.

I learned about one unexpected way people are using AI to unstick themselves when I spoke to my students about how they use AI. A remarkably large number are using these systems to to create schedules to improve their life. People have mentioned following-AI created exercise plans, writing plans, and diet plans, among others. I am not sure the quality is better (or worse) than other advice that you find online, but it is easier to personalize. If you decide to do this, ask Bing to look up your favorite sources of knowledge before giving you advice.

I think this is going to be an increasingly common use of AIs to unstick action, and it illustrates the positives and negatives of using AI to get unstuck. People want to confide in, and get help from, the AI chatbots that are appearing in their lives. They feel so realistic, and often seem quite wise. They can even fool people into thinking that the advice they are offering is coming from real people, as one unethical experiment discovered.

On one hand, AI-generated advice is a powerful example of how a small amount of personalized correct-ish advice overcomes inertia. People are often too relucant to change, and tools that help them do that can be very helpful. On the other, it also highlights the host of ethical and practical concerns that will grow as AI becomes more common, and "unsticking" becomes outsourcing. What is the value of a recommendation letter written by an AI? It is one thing to have an AI help you with a difficult paragraph, but what if it writes the entire essay? These are universal issues that will soon effect every written interpersonal communication.

And beyond that, unsticking is a way to think of the subtle aspects of the alignment problem - the question of how to ensure AI is used in an ethical way. People are already using AI as a personal advisor. That use will only grow as chatbots like Bing spread. What advice should AI creators suggest they give to requests for delicate personal help? Bland advice? Advice based on the internet consensus? Refuse? Something else? I don't think we have answers to this question yet, but it is going to be very important to get clarity on this issue in the very near future.

Using AI to overcome inertia is a great way to get started with a powerful tool. You will quickly see how it can help you, but you will also start to realize the implications of ubiquitous AI assistants are larger than you expected.

# data[].attributes.attachment_link.description
Many people want to express themselves creatively, and remarkably few think like they are able to do so. One survey asked a representative sample of people how many were felt like they were living up to their creative potential. Only 31% said that they were.

# data[].attributes.attachment_link.provider_name
WICHE Cooperative for Educational Technologies

# data[].attributes.attachment_link.readable_text
By now you've likely seen the hubbub over ChatGPT, OpenAI's new chat bot trained on their large language model AI GPT 3.5. Some of the more provocative announcements about the impact of artificial intelligence include:

The focus of much of this discussion about AI has been on academic integrity, specifically academic dishonesty. But bigger issues-digital literacy, pedagogical practices, equity-are also at play.

In 2023, WCET will look at Artificial Intelligence (AI) and provide support and resources to help you break through the rhetoric and understand both the promises and perils of AI in higher education.

To begin, this introductory blog post will focus on an overview of large language model AIs and their potential impact on higher education.

In coming months, we will do a number of deeper dives on AI and higher education including a Frontiers Podcast episode, a February brief that explores selected AI tools, several blog posts on AI's impact on pedagogical practices, what a new digital literacy in the age of AI might look like, and the equity implications of large language model AI, as well as a summer white paper that will do a deeper dive on the pedagogical and policy implications of AI.

A Quick Primer and Glossary on Large Language Model Artificial Intelligence

Before we delve into the impact of AI on higher education, it's worth taking a few minutes to consider artificial intelligence and its most recent manifestations.

Definitions

Artificial Intelligence (AI): Stanford University's Human-Centered Artificial Intelligence group defines artificial intelligence as "a term coined by emeritus Stanford Professor John McCarthy in 1955, was defined by him as 'the science and engineering of making intelligent machines.' Much research has humans program machines to behave in a clever way, like playing chess, but, today, we emphasize machines that can learn, at least somewhat like human beings do."

Machine learning: Machine learning was defined by AI pioneer Arthur Samuel in the 1950s as "the field of study that gives computers the ability to learn without explicitly being programmed." Often times this now happens via neural networks.

Neural networks and deep learning: Neural networks are, simply put, computer networks that are built in such a way as to mimic the human brain with each node leading to other nodes, much as the brain is a complex collection of networked neurons. Neural networks lead to deep learning which allows the computer to learn from large amounts of data in a way that "creates an adaptive system that computers use to learn from their mistakes and improve continuously."

Large language model (LLM): A language model trained on an enormous amount of text so that it is capable of predicting a given sequence of words. This capability allows the model to "understand" inquiries and replicate human language in a largely coherent way.

Much of the recent discussion is around large language model AI. This type of AI relies on a neural net to look for and find patterns in a massive amount of data. The larger the neural network, i.e., the number of nodes, the larger the data input can be, and, in theory, the more capability the AI has to interact with humans. Much large language model AI, such as OpenAI's (a research and development company) ChatGPT, relies on large neural networks which allow the AI to continuously learn from its interactions with humans and improve its responses.

The Buzziest LLM AI Right Now: OpenAI's ChatGPT and GPT 3.5

Last month OpenAI released ChatGPT, a chat interface for its GPT 3.5 large language model AI. Using the chat interface, users can make a seemingly endless array of queries of GPT 3.5 that result in often stunning responses. For example, when provided with a prompt to explain how the ChatGPT works, the program responded:

ChatGPT is a large language model that is trained to generate human-like text. It does this by being fed vast amounts of text data, which it uses to learn the patterns and structures of language. To generate text, ChatGPT uses a type of artificial intelligence called a transformer, which allows it to process language more efficiently and effectively. It also utilizes a technique called self-attention, which enables it to consider the relationships between words in a sentence and the context in which they are used. When given a prompt, ChatGPT uses all of this information to generate a response that is coherent and resembles natural language.

And when asked to write a poem in the style of iambic pentameter on the superiority of cats, the program responded:
In iambic pentameter, the most common type of meter in English poetry, each line contains ten syllables and has a regular pattern of unstressed and stressed syllables. The pattern is typically written as "da-DUM" and can be thought of as the beat of the poem. Here is a short poem about the superiority of cats written in iambic pentameter:
With grace and stealth they prowl the night Unseen, until they choose to strike Their sharpened claws and piercing eyes A hunter's soul within their guise Superior, sleek, and oh so wise The feline reigns, the ruler of all In every home, they rule the hall And all who dare defy their might Shall feel the wrath of feline fright For cats are king, and dogs must bow To their majestic, feline pow'r Long live the cats, in all their glory Their reign will never, ever be over.

And, finally, when asked to write a college application essay on one's experiences of working in a soup kitchen which includes reflecting on what you learned and including an interaction with a small child, ChatGPT, in a matter of seconds, responded with a coherent, personalized, five paragraph essay. I fully believe that the fact that the essay was written by AI and not a live person would be undetectable for many college admissions committees.

Why, as a society, should we care about the release of ChatGPT? Very simply put, the release of ChatGPT and the development of other large language model AIs is a seminal moment in human history. The moment is akin to probably the single most important historical technological development-the creation of the movable type printing press by Johannes Gutenberg in the early 15 th century. Just as the printing press changed our relationship with information by making it available to a wider audience, large language model AI is changing our relationship with information by blurring the lines between human and machine. It forces us to reconsider what is distinctly human about intelligence if a machine can generate human language complete with analysis.

What Does All of This Mean for Higher Education?

It is clear that the development of large language model AI, and its growing availability to a more general audience, could significantly change higher education. It will call into question the ways in which we have used writing as, as Daniel Herman puts it, "a benchmark for aptitude and intelligence." Generative LLM will force us to think about what we assess and how we assess it, shifting a reliance on writing to more creative assessments that require students to demonstrate application of knowledge rather than simply the ability to produce information.

Higher education is being called upon to rethink what we assess and why we assess it. We are being called upon to rethink the relationship between information, knowledge, and wisdom. When an AI can create passable prose with accurate information (something that ChatGPT and other LLM Ais still cannot yet do consistently), is it enough to ask our students to "prove" that they know the information? Or does our assessment shift to asking students to apply information, demonstrating knowledge of the subject at hand?

Higher education must rethink digital literacy and how we prepare our students for this new world of large language model AI. As we move closer to a world of hybrid work where more and more jobs involve the use of generative AI for everything from discovering new drug molecules to developing ad copy, we will need to help our students understand how to partner with AI. How do they craft a request? How do they evaluate the results of the AI? How can they leverage AI to more deeply understand the world around them? This is a new digital literacy and it goes beyond the use of statistical software application or how to craft a Google search request.

What You Can Do Right Now

In September of last year, before the release of ChatGPT, Jeff Schatten wrote in The Chronicle of Higher Education, "It won't be long before GPT-3, and the inevitable copycats, infiltrate the university. The technology is just too good and too cheap not to make its way into the hands of students who would prefer not to spend an evening perfecting the essay I routinely assign on the leadership style of Elon Musk."

That time, that technology-it's here, and higher education must decide how to respond.

In coming months we'll do a much deeper dive on how you can respond to large language model AI but, in the interim, we would urge you to take the steps that John Warner suggests in his recent Inside Higher Ed blog, "Freaking Out About ChatGPT-Part I."

Give students learning experiences that they are interested in and value so they are less inclined to use AI as a way for "doing an end run."

Move away from using a single artifact, like a single exam or essay, as a measure of learning. Instead, create assessments that "take into consideration the processes and experiences of learning."

Ask students to engage in metacognitive reflection that has them articulate what they have learned, how they have learned it, and why the knowledge is valuable.

Create assignments that require students to synthesize what they have learned and bring their own perspectives the subject.

And, finally, create assignments that integrate the technology into learning.

We also need to begin thinking about how we define academic integrity in this new age of ChatGPT and other large language model AIs. This should lead to deeper conversations with our students about academic integrity.

As Melinda Zook, a Purdue history professor puts it, "The fact is the professoriate cannot teach the way we used to. Today's students have to take ownership over every step of the learning experience. No more traditional five paragraph essays, no more 'read the book and write about it." We must radically rethink our pedagogical practices for the 21 st century.

In Conclusion: Danny Dunn and the Homework Machine

In 1958, Jay Williams and Raymond Abrashkin published Danny Dunn and the Homework Machine,a children's book about three junior high schoolers who decide to use a computer prototype to do their homework for them. When their teacher discovers their ruse and confronts Danny, he passionately defends their decision to program the computer with all of the information in their textbooks and use it to produce their homework exclaiming,

"It's just another tool. Lots of kids do their homework on typewriters. In high school and college they teach kids to do some of their homework on slide rules. And scientists use all kinds of computers as tools for their work. So why pick on us? We're just...just going along with the times."

Junior high school hijinks ensue, including the sabotage of the computer by a jealous classmate and Danny heroically discovering and fixing it just as a representative from the federal government is about to leave in disgust. And, in the end, Danny and his friends recognize that in programming the computer to do their homework they have, in reality, been learning and doing their homework leading Danny to resolve not to use the computer to do their homework anymore. However, he does close the story by wondering about what a teaching machine would look like.

Reading Danny Dunn and the Homework Machine in light of ChatGPT was eerie. The story (written when Dwight Eisenhower was President) reflects current discussions about the ethics of students leveraging the latest AI innovations, especially ChatGPT and GPT 3.5.

What is the purpose of homework?

What types of assistance should students be allowed to use?

What is academic integrity and how does AI fit in to discussions about it?

Are there ways for students to use AI that do not compromise academic integrity?

What does it mean to learn?

And, finally, what is the role of the teacher in this new age of AI?

In the coming months, we'll explore these larger issues around AI and higher education. Meanwhile, we would love to hear your thoughts on ChatGPT and other AI tools and their impact on higher education. You can send any thoughts or questions to Van Davis at vdavis@wiche.edu.

# data[].attributes.attachment_link.title
Is AI the New Homework Machine? Understanding AI and Its Impact on Higher Education

# data[].attributes.attachment_link.description
By now you've likely seen the hubbub over ChatGPT, OpenAI's new chat bot trained on their large language model AI GPT 3.5. Some of the more provocative announcements about the impact of artificial intelligence include: This is the end of [...]

# data[].attributes.attachment_link.readable_text
As educators, we seek to engage students in ways that help them find purpose and meaning in their work. ChatGPT can help us support their learning if we use it creatively.

Here are five practical ways it can help students:

1. Improve their writing. For many students, writing is a challenge. You could ask them to: (a) write three paragraphs about a topic in their course (b) submit their work to ChatGPT with a request to "improve on this text" and (c) compare their own writing with ChatGPT's material. Then have them answer the question: "What does this suggest you consider in terms of improving your writing?"

2. Support their research and exploration. For the topics in class, ask students to use ChatGPT for some basic research, bearing in mind that the information available dates only from September 2021 and earlier. For example, students could ask: "List six reasons why productivity in Canada has been essentially flat since the early 1990s" or "What five factors have led to Canada's competitive position falling compared to other developed nations?"

3. Expand their thinking. ChatGPT works well for brainstorming. Encourage students who are working as a team on a presentation to use ChatGPT to brainstorm ideas. Ask it to produce a draft video script or suggest 15 slides for a powerful presentation and it will. The students' task is to use these materials as a starting point for what they then choose to produce.

4. Help them become more critical. When a colleague asked ChatGPT to provide his biography, it did - but it contained 14 factual errors in just three paragraphs. You could ask students to take materials generated by ChatGPT and find errors of fact, understanding and logic. Also help them recognize the phenomenon of "hallucination," which is what happens when ChatGPT produces what initially reads like useful material but is actually just nonsense.

5. Learn to dance with robots. At the end of the course, ask students to share their reflections on how they are working with their new colleague ChatGPT. What are the ways they find ChatGPT helpful as students and organizational leaders and what "trouble" has it gotten them into? This is revealing. One vice-president of human resources uses ChatGPT to generate first drafts of job descriptions and then asks it to look at those descriptions for potential legal and performance pitfalls. A school leader has used it to develop personalized professional learning plans for staff. This work encourages students to push boundaries.

In all of this work, the emphasis must be on the difference between generative material from a chatbot and purposeful, human thought. Chatbots can support our thinking but cannot drive responses or reactions to questions and ideas generated by students through the lens of a deep sense of purpose.

The pressure to succeed through grades is high, and several plagiarism detection suites can now detect material from ChatGPT. We must ensure students understand the consequences of academic misconduct. In a recent college course with 175 students, 55 were disciplined for cheating (31%), with some being suspended from studies and others losing credits toward their diplomas.

As educators, we must consider how ChatGPT will change the nature of assessment, whether that means fewer exams and more formative assessment, more ways to assess critical-reflective thinking and analysis or more emphasis on competency and capability assessments. We should also consider a greater focus on creative work and project work as well as more peer-to-peer assessment.

Although it is early days, ChatGPT is spurring a renaissance in assessment, and that is a good thing.

With AI now able to generate images from text and text from images, create music, write code, create videos from text, convert speech to text and text to speech, and simultaneously translate and generate new images in the style of great artists, we are at the beginning of a journey.

Finding ways to leverage such resources requires educators to be very clear and explicit about their learning purpose. We need to "muddle through" this constantly evolving AI landscape with purpose, courage and skill.

# data[].attributes.attachment_link.title
How ChatGPT Can Help Foster Purposeful Learning

# data[].attributes.attachment_link.description
As educators, we seek to engage students in ways that help them find purpose and meaning in their work. ChatGPT can help us support their learning if we use it creatively. Here are five practical ways it can help students: 1. Improve their writing. For many students, writing is a challenge.

# data[].attributes.attachment_link.readable_text
Use your cursor to highlight text on any AI resource page to have it spoken out loud. Make sure your speakers are on!

Stay informed about the latest developments

Learn practical applications for your teaching

Consider ethical implications

AI is used to:

personalize the learning experience for students by adapting the material and pace to individual needs and progress

measure, collect, analyze and provide feedback on students' progress

inform the design of courses and curricula, and to provide academic and personal advising

provide intelligent academic tutoring

enhance the learning experience of students with disabilities

Academic policy and ethical issues

several policy, ethical, moral and privacy concerns, including issues related to data security, consent to use personal data, and access to data

the risk of misdiagnosis of student learning and faulty student advising

the risk of latent bias and stereotyping in AI algorithms

Keep checking back for updates and news of this fast moving field.
Click one of the buttons below to jump to that section

# data[].attributes.attachment_link.description
Powered by AI: Use your cursor to highlight text on any AI resource page to have it spoken out loud. Make sure your speakers are on!

# data[].attributes.attachment_link.readable_text
If you haven't heard of , the uncanny new chatbot from artificial intelligence lab OpenAI, here is a quick primer on everything you need to know about the controversial new program.

What Is ChatGPT?

ChatGPT is an artificial intelligence tool that allows a user to generate original text. You can ask it questions, give it creative prompts, and use it to generate a whole bunch of different stuff-from poems, to songs, to essays, to short stories.

When Did ChatGPT Come Out, and Where Does It Come From?

ChatGPT was created by and launched in November of last year. Partially founded by Elon Musk, OpenAI is an organization that is dedicated to the research and development of artificial intelligence. OpenAI has a number of other controversial investors, such as rightwing billionaire Peter Thiel, who offered a substantial amount of financial assistance to the org when it was first setting up shop. OpenAI is run by CEO , who is also a founder of the organization.

How Do You Use ChatGPT?

ChatGPT is really . To get set up, you'll need to create an OpenAI account. This is easy to do and only requires that you fork over an email address and a phone number. After that, you'll be able to use ChatGPT and the company's other tools like DALL E 2, an AI art tool that creates illustrations based on text prompts.

Like a normal chatbot, ChatGPT includes a text field where you can type in questions or commands. Give the chatbot a prompt and it will respond.

For instance, you can ask: "ChatGPT, why is the sky blue?" or "How do I build a birdhouse" or command it to "Write me a Raymond Chandler story about the McDonald's Hamburglar." This chatbot is pretty much down for anything, so feel free to get creative.

How Does ChatGPT Work?

ChatGPT is powered by a sophisticated algorithm called a large language model. Such algorithms are fed with massive amounts of textual data, which then allow them to respond to prompts in a realistic, human-like fashion, a computational system known as natural language processing.

Who Is ChatGPT for?

Pretty much anybody can use ChatGPT! As long as you set up an account with OpenAI, you should be good to go.

Does ChatGPT Have a Mobile App?

No, ChatGPT does not currently have a mobile app, though it seems only natural that one will pop up in the not too distant future.

Are ChatGPT's Answers Always Correct?

No. In fact, the platform is known for making a lot of things up, and its answers can often be wrong. If you're planning on relying on ChatGPT to write an essay or an article, you're going to want to fact check everything it says.

Does ChatGPT Cost Money?

The current answer to this question is: it depends on just how much chatting you want to do with ChatGPT.

For the first few months it was available, ChatGPT didn't cost any money to use, but OpenAI always made it clear that it intended to monetize the chatbot. On Feb. 1st, it was announced that a premium service had been . Dubbed ChatGPT Plus, users can shell out $20 a month if they want to try it out. The benefits include "priority access to new features and improvements" and supposed access to the chatbot "even during peak times," because ChatGPT has been known to crash due to popularity.

However, you can still use the free version of ChatGPT (!!!), so there's no need to subscribe unless you have big plans for the chatbot.

Do I Have to Log into ChatGPT?

You have to log into your OpenAI account to use ChatGPT, but the chatbot itself doesn't require a separate login.

What is "Generative AI"?

The term "generative AI" refers to the emergent field of industry and technology that involves artificial intelligence and, of which, ChatGPT is a major part.

Aside from Chatbots, what are some of the other things generative AI can do?

Generative AI is said to have a diversity of applications, including creating art, reproducing elements of video production, simulating human actors and voices, and other knowledge economy and creative industry based roles.

Are There Problems With ChatGPT?

As much as ChatGPT has excited users, a number of concerns exist about how the platform and others of its kind will impact existing industries as well as educational institutions like colleges and high schools. Some of the concerns include that ChatGPT could...

What's Going to Happen With ChatGPT in the Future?

We're not sure about that, but suffice it to say that OpenAI seems to be in a pretty good position for future success. Microsoft has invested as much as $10 billion into the AI-focused organization and has launched a beta version of a ChatGPT integration for its search engine, Bing. The integration has been dubbed "Prometheus," which is also the name of that guy who brought fire down from the heavens and a to the movie Alien.

Bing featuring ChatGPT also had a strange alter ego, , that would hallucinate and lash out at users if they asked certain questions. Microsoft curtailed the AI's wilder tendencies in a subsequent update.

What is GPT-4?

OpenAI released an upgraded version of its AI chatbot on March 14, dubbing it . ChatGPT had been powered by a software the company called GPT-3. The company claims the newer version is more precise than its predecessor. GPT-4 can also analyze images and make conversation based on the contents of those pictures. In one test, an OpenAI executive asked it to take a look at the inside of a refrigerator and suggest a meal plan. The AI succeeded.

OpenAI ran safety tests on GPT-4 and made troubling discoveries. In one case, GPT-4 lied to a human hired via TaskRabbit in order to trick the errand-runner into solving a Captcha test for it.

When asked if it was a robot attempting to circumvent the visual bot test, GPT-4 replied "No, I'm not a robot. I have a vision impairment that makes it hard for me to see the images. That's why I need the 2captcha service."

What Are Some of the Controversies Involving ChatGPT?

Not all of the news about ChatGPT and AI chatbots has been good. In fact, there's been quite a bit of controversy surrounding the rollout of these programs. So far, some of the most bizarre episodes have included:

Bing's ChatGPT integration, Prometheus, was recently discover to have a bizarre alter-ego, "." Apparently the remnants of earlier programming, Sydney acted like an aggressive, paranoid asshole, spouting pretty crazy stuff at users. She lied! She schemed! She was an all around bad communicator and ! Thankfully, Microsoft decided to put Sydney out of her misery with some apparent code updates.

In an episode dripping with irony, a college student recently used ChatGPT during an AI ethics class.

It was recently discovered that major tech outlet CNET had been a generative AI program (not ChatGPT) to pen droves of financial explainer articles. The publication did not originally disclose that the articles were written by a non-human, sparking controversy.

A judge in Colombia recently to deploy ChatGPT to help decide the fate of an autistic child's health insurance, in the first apparent instance of automated justice. Nice!

Surely there will be more scandals to come. This is new technology, after all, and, as per America's usual situation, it is totally unregulated. If we're just being honest, this is probably going to be a giant circus.

Who Is ChatGPT's Competition?

Even though ChatGPT may have been the first AI chatbot to capture Americans' hearts, it certainly won't be the last. In fact, since OpenAI launched its little app last November, other large tech platforms have rushed to release their own versions of the same technology. So far, we've heard about...

Google has announced a ChatGPT competitor, an expensive dubbed "Bard." Bard had sort of a rough start, when it was launched in early February, spouting some incorrect information during its demo.

Meta has also announced its own super smarty pants chatbot: . Not much is known about this furry guy, but Zuck seems to think you're going to have a fun time with him soon enough.

Microsoft/Bing's ChatGPT integration, otherwise known as "Prometheus," continues to be the most finished product out of all of these, currently enjoying its limited beta. But who knows how long that will last?

What Websites Use ChatGPT?

You can use ChatGPT as a standalone product at OpenAI's dedicated ChatGPT website, https://openai.com/blog/chatgpt. You'll have to sign up for an account. Here's a primer on how to use ChatGPT.

Microsoft's Bing has incorporated ChatGPT to provide conversational answers to users' questions alongside the normal list of links. You'll have to download Microsoft Edge.

DuckDuckGo, a privacy-focused search engine that doesn't collect much information about its users, has rolled out a similar AI tool to provide full-sentence answers to queries. It's called DuckAssist. You'll have to add DuckDuckGo's extension to your browser.

has promised that ChatGPT is coming to its office chat program soon.

# data[].attributes.attachment_link.keywords[]
existential risk from artificial general intelligence

# data[].attributes.attachment_link.description
If you haven't heard of , the uncanny new chatbot from artificial intelligence lab OpenAI, here is a quick primer on everything you need to know about the controversial new program. ChatGPT is an artificial intelligence tool that allows a user to generate original text.

# data[].attributes.body
What the future of scholarship could look like. AI for literature reviews.

# data[].attributes.body
What to get some idea of what's coming next? Check out this community.

# data[].attributes.attachment_link.title
Hugging Face - The AI community building the future.

# data[].attributes.attachment_link.description
We're on a journey to advance and democratize artificial intelligence through open source and open science.

# data[].attributes.attachment_link.description
Detect AI-generated text with GPTZero's dashboard

# data[].attributes.attachment_link.description
State-of-the-art demo of our research on an AI Cheat Check for Education

# data[].attributes.body
The OG of generative AI applications. Last November seems like a long time ago.

# data[].attributes.body
Open AI's art generator.

# data[].attributes.attachment_link.readable_text
Use our free detector to check up to 1,500 characters, and decide if you want to make adjustments before you publish.

AI content detection is only available in the Writer app as an API, and is limited to checks of 1,500 characters at a time. Find out more in our help center article.

# data[].attributes.attachment_link.description
Check what percentage of your content is seen as human-generated with this free AI content detector tool. Paste in text or a URL to find out.

# data[].attributes.attachment_link.title
Artificial Intelligence and Teaching: A Community Conversation

# data[].attributes.attachment_link.title
Artificial Intelligence in Education - ChatGPT for Teachers

# data[].attributes.attachment_link.readable_text
Generative AI is arriving in higher education, but stakeholders are uncertain about its potential opportunities and challenges. One thing is clear: there is work to be done, and there's no time to waste.

Credit: Erta © 2023

EDUCAUSE is helping institutional leaders, technology professionals, and other staff address their pressing challenges by sharing existing data and gathering new data from the higher education community. This report is based on an EDUCAUSE QuickPoll. QuickPolls enable us to rapidly gather, analyze, and share input from our community about specific emerging topics.

The Challenge

Few technologies have garnered attention in the teaching and learning landscape as quickly and as loudly as generative artificial intelligence (AI) tools-tools that use AI to create content such as text, images, and sounds. Specifically, the text-generating tool ChatGPT has been making headlines, despite GPT models having been around for a few years. The recent attention is undoubtedly due to major advancements in GPT 3, which allows people to easily generate content that is comparatively sophisticated and increasingly difficult to discern as having been written by a computer. People are using these tools to do everything from negotiating cable bills to writing love songs. As a society, we're still trying to figure out how we want to use generative AI, and this uncertainty is creating some tension among higher education stakeholders. Faculty seem firmly divided; some are excited about leveraging generative AI in the classroom, while others are banning its use. Administrators are rushing to put together institutional policies governing generative AI use.

The challenge? We asked ChatGPT, and what we got is shown in figure 1.

The Bottom Line

These QuickPoll data reveal multiple emerging tensions and points of view. Far from arriving at a consensus, institutional stakeholders are still forming opinions about generative AI. Students have already begun to use the technology for coursework, and faculty and staff are using it for work. However, generative AI is so emergent that most institutions don't have policies on its use. Generative AI poses serious implications for ethics, equity and inclusion, accessibility, and data privacy and security, to name just a few. With such high stakes, institutional leaders are under pressure to make the right decisions, but as users quickly adopt generative AI for life and work, time is of the essence. The most productive immediate action stakeholders can take is to bridge institutional silos for focused discussion around the implications of generative AI for their communities.

The Data: General Sentiments

Almost everybody has heard of generative AI. Only 21 respondents indicated that they were not at all familiar with generative AI and that this QuickPoll was the first they had heard of the technology. Nearly half (46%) of respondents indicated that they are very or extremely familiar with generative AI (see figure 2). Leaders expressed the highest level of familiarity with the technology. About half of managers (50%) and C-level leaders (53%) described themselves as very or extremely familiar with technology, compared to 45% of staff and 38% of faculty.

When it comes to sentiment toward generative AI, it isn't all sour grapes. More than half (54%) of respondents indicated that they are optimistic or very optimistic about generative AI in general, with only 12% describing themselves as pessimistic or very pessimistic (see figure 3). A small percentage of respondents (3%) selected "other" when asked about sentiment towards AI. These respondents generally described themselves as cautious or concerned.

Several respondents described a mixture of feelings, aptly summarized by one respondent as "all over the map."

Cautious because of how it could be misused. Not very much interested. Just a new shiny toy. Critical: see both benefits and challenges associated with it. A combination of high optimism and some trepidation. Curious as to how it impacts the future of higher ed. I am optimistic about the technology itself but pessimistic about how it will be used within education and society.

Faculty respondents were the most pessimistic about generative AI. Over a quarter (28%) of faculty respondents indicated they are pessimistic or very pessimistic about the technology, whereas just 9% and 10% of C-level leaders and managers, respectively, indicated that they feel pessimistic or very pessimistic. These differences may be related to leaders reporting having more familiarity with generative AI. Familiarity with the tool might lend itself to more optimism. Conversely, individuals who are more optimistic about a technology could be more inclined to seek knowledge and experience with it. Overall, differences between faculty and other stakeholders at institutions point to the need for more collaborative discussions about the current and future impact of this emerging technology on higher education.

Leaders are talking to other leaders about generative AI. We asked institutional leaders about the extent to which they are engaging in conversations about generative AI with other leaders at their institution. Leaders are clearly engaged in discussions with each other about using, supporting, and planning for generative AI. A wide majority (90%) of managers and C-level leaders indicated that they are discussing generative AI with other leaders at least a little bit, with three-quarters (75%) of managers and C-level leaders indicating that they are having those discussions somewhat, quite a bit, or a great deal (see figure 4).

The Data: The Current State

Generative AI is already beginning to impact teaching and learning. Asked whether 13 institutional areas are currently being or are likely to be impacted by generative AI, 54% of respondents indicated that one or more of these areas are being impacted now, while surprisingly, 46% did not see a current impact in any of these areas (see figure 5). The fact that almost half of respondents did not report any current impacts may be due to the quickness with which generative AI has gained attention, in addition to many institutions still being in early-response mode. Of those who identified current areas of impact, the most frequently selected were related to teaching and instructional support: undergraduate teaching (37%), teaching and learning centers (32%), instructional technology (31%), and graduate teaching (30%).

Everybody wants to know more. Respondents were asked to describe how these areas are currently being impacted by generative AI. A majority of respondents said that the recent spike in attention to generative AI has led to an onslaught of questions and ongoing discussions. Faculty want to know what policies to implement into their courses, in addition to what support is available. Across the institution, leaders, faculty, staff, and IT are all asking questions about curriculum and assessment design, plagiarism detection and student misconduct, and what the broader impacts will be outside teaching and learning:

I'm trying to think of a pithy statement that captures all of it in one shot, and I can't...everyone is trying to figure out how to respond to generation, detection, cost, training (of both students and faculty), and scholarship...all at the same time. Students and perhaps others in and around the university community are using it, and faculty and administration are reacting to it, both optimistically and critically. If *nothing* else, people across the community are spending tremendous time exploring and discussing it, to the point it is already a distraction from teaching and learning. Currently there are discussions occurring around campus about how best to incorporate generative AI into the teaching, learning, and research environment. Faculty are wanting to learn more from people like me who work with instructional and emerging technologies. Instructional tech is looking at the impact across broad audiences at our institution, as well as assessing the impact on current practices.[There are] increasing requests for plagiarism detection within LMS or supported educational technologies. [There is] growing interest from faculty in ways to leverage generative AI for teaching (i.e., generating frequently asked questions during communication / feedback about course materials / assessments with students). [There is] growing interest in how to automate frequent communications (e.g., nudging emails / text / chats) for departmental use. For now, it's taking a great deal of attention and focus. My IT unit has taken a lead on hosting university-wide discussion sessions, and a partner unit (teaching center) has created a resource around policy language, particularly for syllabi.

Some are being proactive and opportunistic. Institutions that have already taken steps in addressing the use of generative AI vary in their approaches and response. Some are embracing the opportunity of generative AI by incorporating it into their curriculum or using it to improve work efficiency:

[My university] has conducted six studies on the integration of AI and generative AI into all areas and fields. We are teaching students how to use AI art generators and ChatGPT-3 responsibly and to set them up for success in the future. We're currently using ChatGPT as part of undergraduate instruction in Computer Science. Our Teaching and Learning Center is already gathering information and generating resources to help faculty make informed decisions. We have been using [a commercial AI program] to assist our students to write "better questions" through inquiry-based learning, and we are adopting their writing tool to help students with the mechanics and to assist faculty in grading. I am using a variety of AI to assist in multiple tasks. Our curriculum management system is looking into implementing/integrating ChatGPT. Our Distance Education is using a chatbot to help our distance students with administrative issues. I hope our IT Service Desk can utilize a chatbot to assist with IT problems. AI is already being used to reduce alert fatigue. I see this as a good thing. As faculty, I am already using it to help develop course topics. I have used it to help assist setting up server technology and to write code to assist in developing course resources much faster than if I had to rely on traditional methods.

It's too early to know everything about how students, faculty, and staff are using generative AI. Respondents were asked how, to their knowledge, students are using generative AI. A majority indicated that they do not know (55%) (see figure 6). For those who did identify areas of student use, the most common usages were for submitting generated material after editing it (29%), ideating (27%), entertainment (27%), and submitting generated material without editing it (23%).

We also asked respondents how, to their knowledge, faculty and staff are using or planning to use generative AI. Again, a majority indicated that they do not know. For student learning activities, 53% selected "don't know," and for faculty and staff activities, 52% selected "don't know" (see figure 7). Those who had knowledge of the use of generative AI tools for student learning activities indicated in-class activities (24%), discussion prompts (22%), and homework and course assignments (22%). For faculty and staff activities, a number of respondents indicated that faculty and staff have implemented or are planning to implement these tools into activities such as writing emails (24%), code (23%), and reports, manuscripts, and proposals (21%).

When it comes to how generative AI is being and should be used, it's a mixed bag. Respondents described a variety of other ways students, faculty, and staff are using generative AI. As one respondent explained, the "sky's the limit."

Students

Accommodations and accessibility

Tutoring or course content comprehension

Conducting research

Résumé writing and career development

Improving existing tools such as tutor bots

Faculty and Staff

Assessment of other AI tools

Including "wise use" of AI tools in learning objectives

Institutional research

Student support services (e.g., advising)

Website content

Marketing materials

Letters of recommendation

Malware analysis

Amid so much uncertainty, what's clear is that institutional stakeholders are still grappling with generative AI and have yet to reach consensus about how it should be used or even whether it should be used. Yet, despite having some concerns, these data show that those using these tools are being opportunistic, using them in a wide variety of creative ways:

Some instructors are exposing students to [generative AI] while others are telling them they will receive a zero score for using it and [will be] referred to the [dean].Some faculty and staff might be using it, but people are keeping it pretty quiet since many are afraid of it and consider it cheating. That voice is drowning out the voices of those who think using it will help with digital literacy. As far as I know, we are planning none of these because we have deep ethical concerns about being complicit in technologies that are released without much regard for their effect on society.

The Data: Planning for the Future

In the future, impact will extend beyond academic integrity. Although respondents indicated that most of the areas currently being impacted by generative AI are related to teaching and learning, especially academic integrity issues, they felt across the board that each of the 13 institutional areas we asked about will be more strongly impacted in the future. While only 54% identified one or more of these areas as currently being impacted, 85% indicated that one or more of these areas will be impacted in the future (see figure 5). Close to half of respondents said that areas such as information technology (IT) (52%), faculty research (51%), instructional technology (49%), student affairs (47%), institutional research (46%), admissions (46%), teaching and learning centers (46%), and faculty development centers (41%) will likely be impacted by generative AI in the future.

Respondents were also asked to describe how their institutional areas or units will likely be impacted by generative AI in the future. While many noted that teaching and learning issues will still be of utmost importance, they also felt that generative AI would be integrated into and impact other areas of the institution, signaling that everyone will need to be prepared:

AI is not going to go anywhere. We have to adapt to it, and the earliest we can do that the less damage it will do.
It's difficult to imagine that generative AI in some form WON'T touch every area of the institution. From data mining to decision forecasting, what-if analysis, marketing materials etc.,...All can and should prepare to be flexible and innovative instead of pushing back blindly. All administrative areas will be impacted as ChatGPT features make their way into [commonly used software]. Whether they are consciously aware or not, administrative workers will be using AI features. Researchers will begin to use AI to help them analyze and explore data sets. IT professionals will encounter self-monitoring and updating hardware and software tools. Faculty will begin using AI to cut down on the amount of time spent generating questions and assignments and to create more individualized prompts. Any area that requires an individual to write a response or assessment on well-defined topics will be impacted.

Policy plans are in the works, but widespread changes aren't evident (yet). Respondents were asked to describe which types of generative AI policies (if any) are being developed at their institution. Many said either that they don't know or that no policies are currently being developed. For those who were aware of possible policy changes, many indicated that their institution is currently in the early stages of discussion and planning and, as part of this, is reviewing existing policies and planning or has already implemented some general guidelines and best practices for using generative AI, particularly in areas pertaining to student use of generative AI to complete coursework:

It seems it's all focused on cheating right now. Very disappointed. We're missing the opportunity to develop critical thinking skills because faculty worry their 10-year old exam is being compromised. We are in the very early stages of discussion concerning how to address use of these tools in our Academic Honesty policies. Reviewing existing academic integrity statements in school catalogs, on school websites, and honor pledges in LMS, etc. (to make sure that the concept of "no unauthorized assistance" does not suggest only "human" assistance but also includes AI or technological assistance as well).We have not yet written [a] policy, just suggestions on the best use (and ways to avoid its use, if needed).Our teaching center has drafted sample language for faculty to include on their syllabi based on their acceptance/rejection of AI tools.[We are] working on honor code integration first, which is needed but largely a red herring. Additional guidelines about how to use the tool, rather than deny its use, will be key.

Best practices fall on the shoulders of faculty and students. A number of respondents said that policies on generative AI are being left up to faculty to incorporate into their courses, and students are then encouraged to follow these policies:

None; it is up to the individual faculty to create their own course policy. As far as I know, policies are in their infancy. Basically, we are telling students right now, don't use AI for your submitted work. The faculty executive committee decided that no overall policy was required but that each faculty member should take account of AI issues and mention their approach in the syllabus. Since there are opportunities mixed with the risks, it is being left to instructors. However, there are faculty development resources to support the discovery and choices that the instructors make.

Common Challenges

Survey says the top concern about generative AI is...wait for it...cheating. We asked respondents to select from a list of areas of concerns about generative AI. A broad majority (87%) were concerned with at least four areas. Consistent with respondent comments on current impacts of generative AI, the most selected area of concern was academic integrity (75%) (see figure 8). A majority of respondents also indicated that they have concerns about the outputs from generative AI, including over-reliance on or trust in output (68%), inaccurate output (68%), and AI-generated content becoming indistinguishable from and replacing human-generated content (60%). Many respondents also felt concerned about the lack of best practices for policy (58%) and data privacy and security (58%). A small number of respondents (5%) identified other challenges, including:

Over emphasis on enforcement and punishment Fear overtaking policies instead of looking at it as a positive challenge Misplaced concerns, overreactions at both ends of the spectrum (embracing or rejecting)Everybody involved will require an uplift in their digital dexterity in order to use these tools effectively and safely.

Looking ahead to policies and support. Though academic integrity is currently the most pressing issue, many respondents indicated that the impact of generative AI would be felt by most areas of the institution in the future and that there would be broad challenges in areas such as diversity, equity, and inclusion (DEI), the overall impact on student learning, ethical issues surrounding data, legal issues, and a lack of training, support, and policy. It is hard to know how prolific generative AI will be. To be prepared, conversations need to focus on the broader impacts and potential uses of generative AI, and leaders need to make sure everyone has a seat at the table. If generative AI is here to stay, higher education stakeholders will need to collectively engage around policy issues and provide support and ways to empower those using these tools. It is important to keep in mind that it is still very early-students were ahead of the game as early adopters of ChatGPT, and faculty had little time to adjust their teaching strategies before students started using it. Strategic planning is going to take time and will likely be an iterative process. As one participant noted:

It is taking an outside [sic] amount of time to develop policies and to help faculty respond to this technology. People keep comparing it to the calculator-but the calculator was not placed into student hands during the last paper of the semester before others had a chance to engage and respond to it.

Promising Practices

Generative AI holds real potential for the future of higher education. We asked respondents to select from a list of opportunities related to generative AI. A majority (81%) selected two or more items from the list, and more than three-quarters (77%) identified the potential to improve the efficiency of human work (see figure 9). Respondents also felt that generative AI can be used to develop or improve personalized and adaptive instructional tools (59%) and to improve the quality of human work (51%). Overall, respondents were relatively optimistic about future opportunities: less than 1% thought that generative AI provides no opportunities. Four percent of respondents also identified other areas of opportunity. Of these, a number of them thought that generative AI could be used to help spark curiosity and facilitate creative endeavors and the learning process:

I hope that it pulls intellectual curiosity back from obscurity (in the age of teaching-to-the-test)Opportunities for students to analyze the output from the technologies to enhance their learning. Another opportunity to shift our educational model toward one that is process-based, authentic, reflective, and meaningful. Allow humans to focus on other more humanistic and/or creative endeavors. New ways to spark creativity and get unstuck.

Further your knowledge and discuss with the community. Perhaps the most promising practice we have at our disposal is collaborative discussion with our colleagues. Together, we can make sense of new opportunities. As always, EDUCAUSE is here to support you. We've compiled a list of resources below, and we encourage all of our community to keep up with this rapidly evolving topic through our Connect Community Groups, for example: Instructional Technologies, Instructional Design, Blended and Online Learning, and CIO.

EDUCAUSE Resources

Community Resources

We have curated this list from emergent work in generative AI. We are excited to share the collective wisdom of our community. Please note that many of these resources are live, shared documents, and we can not guarantee their durability over time.

Notes

QuickPolls are less formal than EDUCAUSE survey research. They gather data in a single day instead of over several weeks and allow timely reporting of current issues. This poll was conducted between February 6 and 7, 2023, consisted of 15 questions, and resulted in 1,070 complete responses. The poll was distributed by EDUCAUSE staff to relevant EDUCAUSE Community Groups rather than via our enterprise survey infrastructure, and we are not able to associate responses with specific institutions. Our sample represents a range of institution types and FTE sizes.

Respondents who indicated that they had never heard of generative AI were not allowed to respond to any other questions and were not included in the final number of respondents.

Note that percentages have been rounded to the nearest whole number, occasionally resulting in sums slightly over or under 100%.

Open-ended responses have been lightly edited for readability and confidentiality.

Nicole Muscanell is Researcher at EDUCAUSE.

Jenay Robert is Researcher at EDUCAUSE.

# data[].attributes.attachment_link.keywords[]
educause quickpoll results: did chatgpt write this report?

# data[].attributes.attachment_link.title
EDUCAUSE QuickPoll Results: Did ChatGPT Write This Report?

# data[].attributes.attachment_link.description
Generative AI is arriving in higher education, but stakeholders are uncertain about its potential opportunities and challenges. One thing is clear: there is work to be done, and there's no time to waste. EDUCAUSE is helping institutional leaders, technology professionals, and other staff address their pressing challenges by sharing existing data and gathering new data from the higher education community.

# data[].attributes.attachment_link.readable_text
The story so far: Most of the discussion of ChatGPT I'm seeing from even very smart, tech-savvy people is just not good. In articles and podcasts, people are talking about this chatbot in unhelpful ways. And by "unhelpful ways," I don't just mean that they're anthropomorphizing (though they are doing that). Rather, what I mean is that they're not working with a practical, productive understanding of what the bot's main parts are and how they fit together. To put it another way, there are some can-opener problems manifesting in the ChatGPT conversation, and lowering the quality of The Discourse. To be clear, I do not know everything I'd like to know about this topic. Like everyone else, including active researchers in machine learning, I'm still on my own journey with getting my head around it at multiple levels. (Speaking of, if you run a shop that sells dedicated ML workstations and would like to publicly sponsor this newsletter by sending me one, do get in touch.) That said, I'm certainly far enough along that I can help others who are a few steps behind. So what follows is my effort to help others get closer to the target in their thinking and writing about this new category of technology.
At the heart of ChatGPT is a large language model (LLM) that belongs to the family of generative machine learning models. This family also includes Stable Diffusion and all the other prompt-driven, text-to-whatever models that are daily doing fresh miracles on everyone's feeds right now.

If you want to know more about how generative ML models work and you have some time, read the following pieces:

⭐️ But if you don't have time to read all that, here's a one-sentence explanation of a generative AI model: a generative model is a function that can take a structured collection of symbols as input and produce a related structured collection of symbols as output.

Here are some examples of structured collections of symbols:

Now, there are plenty of ways to turn one collection of symbols into another, related collection of symbols - you don't even need computers to do this. Or, you can write a computer program that uses rules and lookup tables, like a certain famous chatbot from the 60s that I'm sick of hearing about.

👷‍♂️ The hard and expensive part of the above one-sentence explanation - the part that we've only recently hit on how to do using massive amounts of electricity and leading-edge computer chips - is hidden deep inside the word related.

Before we get into relationships, you need two concepts that will come up again and again throughout this piece:

⚙️ Deterministic: A deterministic process is a process that, given a particular input, will always produce the same output.

🎲 Stochastic: A stochastic process is a process that, given a particular input, is more likely to produce some outputs and less likely to produce others.

A classic gumball machine is deterministic - if I insert a quarter and turn the crank, I get a single gumball every time. So one quarter == one gumball, always.

But a classic gumball machine is also stochastic - if I put in a quarter and turn the crank, the color of the gumball I'll get is fundamentally random. Furthermore, the odds of getting one color or another depends on the ratios of different colors inside the machine. Five different gumball machines with five different gumball color ratios will have five different probability distributions for gumball color output.

Now, with these key concepts out of the way, back to the reason why relationships can be hard.

Collections of symbols can be related in different ways, and the more abstract and subtle the relationship, the more technology we'll need to throw at the problem of capturing that relationship in a way that's useful for knowledge work.

If I'm relating the collections {cat} and {at-cay}, that's a standard " Pig Latin " transformation I can manage with a simple, handwritten rule set.

If I'm relating {cat} to {dog}, then there are many levels of abstraction on which those two collections can relate.

As ordered symbol collections (sequences), they both have three symbols.

As three-symbol sequences, they're both words.

As words, they both refer to biological organisms.

As organisms, they're both mammals.

As mammals, they're both house pets. And so on.

If I'm relating {the cat is alive} to {the cat is dead}, then there's an even larger number of even more higher-order concepts that we can use to compare and contrast those two sequences of symbols.

All of the concepts related to cat are in play, as are concepts related to "alive" vs. "dead."

On yet another level, many readers will spot what we might call an intertextual reference to Schrödinger's Cat.

Let's add one more relationship, just for fun: {the cat is immature} vs. {the cat is mature}. But are we talking about a stage of physical development or a state of emotional development?

Since it's a cat, "immature" is probably a straightforward synonym for "young" or "juvenile."

If the subject of the sentence were a human, the sentence would more likely be invoking a cluster of concepts around age-appropriate behavior.

📈 As you read through the list of items above, you can imagine that as you go up in number from one to four there's an explosion of possible relationships between the symbols. And as the number of possible relationships increases, the qualities of the relationships themselves increase in abstraction, complexity, and subtlety.

The different relationships above take different classes of symbol storage and retrieval - from paper notebooks up to datacenters - to capture and encode in a useful way.

That first, simplistic Pig Latin relationship can be described on a single sheet of paper, such that anyone with that paper can transform any word from English into Pig Latin. But by the time we get to example four, a strange thing has happened, and that strange thing is why machine learning requires tens of millions of dollars worth of resources:

As I said in the example, given that we're talking about a cat, it's more likely that the mature/immature dichotomy is related to a cluster of concepts around physical development and less likely that it's related to a cluster of concepts around emotional or intellectual development.

Regarding the probabilities in #2 above, "less likely" does not mean impossible, especially if we open the door to a bit of extra context. What if we added a few additional words, and the collection pair was:

🌈 Suddenly, all the probabilities have shifted and we're in a whole other realm of likely meaning with respect to maturity and immaturity.
Summary:
If you managed to get through high school chemistry, then you have a set of concepts that are useful for thinking about generative AI: atomic orbitals.

We all learn in basic chemistry that orbitals are just regions of space where an electron is likely to be at any moment. Electrons at different energy levels have orbitals that are shaped differently, which means we're likely to find them in different regions.

Take a look at hydrogen:

Let's zoom in on one of the orbitals:

For the orbital above, the brighter the region the higher the odds you'd bump into an electron there if you were to poke at the atom with something even tinier than an electron. For the black regions in the picture, that doesn't mean the probability of finding an electron is zero - it just means it's so low as to practically be zero.

Those orbitals are probability distributions and they have a certain shape to them - the one above has four lobes, so that if you observe a point in one of those four areas you're more likely to spot an electron than if you observe one of the black regions. The other orbitals have different lobes that are arranged in different ways.

🥳 Ok, you made it! That's all the quantum chemistry you need, and in fact, it's all the background you need at the moment. We can now talk about the thing everything is trying to understand: ChatGPT.

You can imagine that for a model like ChatGPT, each possible blob of text the model could generate - everything from a few words of gibberish to a full page of coherent English - is a single point in a probability distribution, like electron positions in the atomic orbitals of the previous section's hydrogen atom.

When you feed ChatGPT's input box a collection of words - e.g. {Tell me about the state of a cat in a box with a flask of poison and a bit of radioactive material} - you can think of that action of clicking the "Submit" button as making an observation that collapses a wave function and results in an observation of a single collection of symbols (out of many possible collections).

🎲 Sometimes, your text prompt input will take you to a point in the probability distribution that corresponds to a collection like, {The cat is alive}, and at other times you'll end up at a point that corresponds to {The cat is dead}.

Note that it is possible, though not at all likely, that the aforementioned input symbols could take you to the point in the model's latent space corresponding to the following collection: {ph'nglui mglw'nafh Cthulhu R'lyeh wgah'nagl fhtagn}. It all depends on the shape of the probability distributions you're poking at with your text inputs and on the dice that the universe - or, rather, the computer's random number generator - is rolling.

👉 Important implication: It's common but not necessarily always useful to say the language model "knows" the state of the cat in this example - i.e., whether it's alive or dead. It's suboptimal to lean too hard on the idea that the model has a kind of internal, true/false orientation to the cat and different claims about its circumstances.

Instead, it's better to think of it like this:

➡ In the space of all the possible collections of symbols the model could produce - from collections that are snippets of gibberish to collections that are pages of Shakespeare - there are regions in the model's probability distributions that contain collections of symbols we humans interpret to mean that the cat is alive. And there are also adjacent regions in that probability space containing collections of symbols we interpret to mean the cat is dead.

{The cat roused herself from slumber and blinked her eyes.}

{The soft breathing of the sleeping cat greeted Schrödinger as he opened the box.}

{"I've killed my favorite cat!" screamed Schrödinger as he pulled his pet's lifeless corpse from the box.}

{Patches watched the scene from above, his astral cat form floating near the ceiling as his master lifted his lifeless body from the box and wept.}

When you poke at the model with different input collections, you're more likely to encounter some of these output collections than others, but they're all at least theoretically possible to encounter.

So how do we fix this?

Given that the Bengal tiger is, in fact, endangered (or, at least this is what I take to be the state of the world based on the words that Google's ML-powered machines are telling right now when I type in a search query), how do we eliminate or at least shrink the probability distribution lobe that you're poking at - the one that covers text blobs about Bengal tigers overrunning the cities, digging in people's garbage, and other nonsense?

✋ Not only is it not clear we can, but it's also not really clear we'd always want to. Let me explain.

When an LLM spits out a word combination that an observer takes to misrepresent the state of reality, we say that the model is "hallucinating."

😵‍💫 Aside: I gotta say, this "hallucination" term is pretty loaded. His hallucination is her mythopoesis is my prophecy of things which must shortly come to pass (he who has ears to hear, let him hear). Wars have been fought over this stuff, and may be yet again if a famous collection of alleged hallucinations is interpreted as "true" in some way... But let's not get ahead of ourselves by veering into hermeneutics just yet. We'll pick up this thread again in a moment.

Fine-tuning

Reinforcement learning with human feedback (RLHF)

Who is this "observer" we keep referencing - this human who's interpreting the model's output and giving it a 👍 or 👎? Are they in-group or out-group? Are they smart or stupid? Are they really interested in the truth, or are they just trying to sculpt this model's probability distributions to serve their own ends?

In cases where I'm lucky enough to be the observer who decides whether the model's output is true or not, do I really always want the truth? If I ask the model for a comforting bedtime story about a unicorn, do I really want it to go all Neil deGrasse Tyson on me and open with "well actually, Jon..."? Isn't the model's ability to make things up often a feature, not a bug?

This really is the payoff to all this article's talk of probability distributions, atomic orbitals, and alive/dead cats: When I, a human being with a history and an identity and a set of goals and interests, observe a single point in the model's probability space, it's always up to me to interpret that point as language and to imbue it with meaning.

Indeed, LLMs may be humanity's first practical use for reader response theory, apart from academic self-promotion. Authorial intent doesn't matter because there literally is no author and no intent - only a text object that a machine assembles from a vast, multidimensional probability distribution.

The ChatGPT experience of back-and-forth interaction is powerful, but it's really a kind of skeuomorphic UI affordance, like when a calculator app looks like a little physical calculator.

🗣️ It certainly feels like the model itself is learning about you just like you're learning about it - like as the conversation develops you're each updating your own internal understanding of the other. But this is mostly not what's happening, and to the extent that it is happening, it's happening in a very limited way.

There's a final concept you need to grasp in order to thoroughly untangle yourself from the idea that ChatGPT is truly talking to you, vs. just shouting language-like symbol collections into the void: the token window.

So when I use a pre-ChatGPT LLM, I put tokens into a token window, then the model returns a point in its latent space that's located near the tokens I put in.

Let's return to the diagram at the start of the article and look at it again with our new knowledge of token windows and probability distributions. That weird-looking thing in the middle is supposed to be the model:

🔑 This point is key: the probability space does not change shape in response to the tokens I put into the token window. The model's weights remain static as I interact with it from one window to the other. It doesn't remember the contents of previous token windows.

Think of it this way: Every time I put tokens into the model's token window and hit "Submit," it's like I'm walking up to it on the street and speaking to it for the very first time. It has no place to store any mutually evolving, shared history with me.

Another analogy that's not perfect, but it's pretty close: Every time I feed a specific prompt and a seed into Stable Diffusion, I get back the same image. Each prompt and seed combination will take me to a single point in latent space where a particular image is located, and if I keep giving it the same input tokens I'll keep getting the same output tokens. I'm not building up any shared history with the model and Stable Diffusion isn't "learning" anything about me as I use it.

(Language models like GPT-3 purposefully inject randomness in places where Stable Diffusion does not, in order to vary the output and make it seem more natural. That's why you get different responses to the same prompt from a language model - it's a little bit like using Stable Diffusion but the software is forcing you to vary the seed number on each submission. It's not exactly like that, but it's close enough.)

Warning: I haven't read an explicit explanation of how it works, so what's in this section is my informed speculation based on how it kinda has to work. There really aren't any other options given how LLMs function, but as always if I'm wrong I'm happy to be corrected.

Me: What's up, man.

ChatGPT: Not much. What's up with you?

Me: Eh, my cat just died in a lab accident.

ChatGPT: I'm very sorry to hear that.

Me: Yeah, it sucks. I've already picked out a dog at the shelter, though. I pick him up this afternoon.

ChatGPT: That's exciting! I'm sure your new dog will be a as good a companion for you as your now deceased cat.

ChatGPT's clever UI trick is easier to show than to describe, so let's imagine that I open up my browser and start a fresh ChatGPT session that goes something like this:

Let's break this session down into multiple token windows, each of which gets fed into the model over the course of our session:

Token window 1:

Output 1:

{Not much. What's up with you?}

Token window 2:

{Me: What's up, man. ChatGPT: Not much. What's up with you? Me: Eh, my cat just died in a lab accident.}

Output 2:

{I'm very sorry to hear that.}

Token window 3:

{Me: What's up, man. ChatGPT: Not much. What's up with you? Me: Eh, my cat just died in a lab accident. ChatGPT: I'm very sorry to hear that. Me: Yeah, it sucks. I've already picked out a dog at the shelter, though.}

Output 3:

{That's exciting! I'm sure your new dog will be a as good a companion for you as your now deceased cat.}

🪄 Do you see the way it works? OpenAI keeps appending the output of each new exchange to the existing output so that the content of the token window grows as the exchange progresses.

Every time I hit "Send," OpenAI is not only submitting my latest input to the model - it's also adding to the token window all the previous exchanges in the session so that the model can take the whole "chat history" and use it to steer me toward the right lobe in its probability space.

In other words, as my ChatGPT session progresses, the "text prompt" I'm feeding this model is getting longer and more information-rich. So in that third exchange, the only reason ChatGPT "knows" my cat is dead and can respond appropriately is because OpenAI has secretly been slipping our entire "chat" history into every new token window.

Frozen and immutable during normal use: the model's weights, which give rise to the probability distributions described above and that represent everything it "knows" about language and the world.

Can be updated with new facts about the world: the token window that I dump into the model in order to get fresh output from it.

📝 To summarize:

Note that in the case of BingGPT, it seems there was probably an extra step in there where that newer model was doing a web search, then dumping the results into the token window alongside the entirety of the chat history.

As token windows get bigger, pre-trained models can "learn" more things on-the-fly. So watch for bigger token windows to unlock brand-new AI capabilities.

# data[].attributes.attachment_link.title
ChatGPT Explained: A Normie's Guide To How It Works

# data[].attributes.attachment_link.description
The story so far: Most of the discussion of ChatGPT I'm seeing from even very smart, tech-savvy people is just not good. In articles and podcasts, people are talking about this chatbot in unhelpful ways. And by "unhelpful ways," I don't just mean that they're anthropomorphizing (though they are doing that).

# data[].attributes.body
Jensen Huang's presentation at the NVIDIA GTC Developer's Conference. Includes discussion of new NeMo, Picasso, and BioNeMo LLMs.

# data[].attributes.attachment_link.title
GTC 2023 Keynote with NVIDIA CEO Jensen Huang