# data[].attributes.body
Academic Integrity through Assessment Design with Mark Glynn | Think UDL

# data[].attributes.attachment_link.readable_text
Academic Integrity through Assessment Design with Mark Glynn

Welcome to Episode 75 of the Think UDL podcast: Academic Integrity through Assessment Design with Mark Glynn. Mark Glynn is the Head of the Teaching Enhancement Unit at Dublin City University. I have been wanting to talk to Mark for a long time about what he is doing with UDL in Ireland and beyond. There are so many things that he and his team are doing to help spread the word and train faculty in UDL, that it was difficult to focus on just one topic. Recently, he has been working on the intersection of UDL, academic integrity and assessment design which has really piqued my interest. And so today's conversation centers on how to make assessments more "cheat-proof" and authentic, so as to work on the prevention of academic dishonesty rather than "catching" students cheating after the fact, so to speak. And the UDL principles are what guides this design! Mark has also graciously provided many resources that he mentions during our discussion and you can find all of those on our ThinkUDL.org website for this episode number 75. Thank you for joining us as we talk about authentic assessments, academic integrity and UDL!

Resources
Teaching Enhancing Unit (TEU) Presentation slides linking AI to UDL UDL Progression checklist Great examples of how assessments have changed captured by Sally Brown Cressey Fraud triangle Dr Sarah Eaton presentation video - Building a Shared Culture of Academic Integrity that we use to promote that "one small change" drive that we have in DCU webpage on assessment design
This blog post for Ouriginal

Transcript

Lillian Nave 00:00

Welcome to think UDL, the universal design for learning podcast where we hear from the people who are designing and implementing strategies with learner variability in mind. I'm your host, Lillian nave. And I'm interested in not just what you're teaching, learning, guiding and facilitating, but how you design and implement it and why it even matters. Welcome to Episode 75 of the think UDL podcast academic integrity through assessment design, with Mark Glenn. Mark Lynn is the head of the teaching enhancement unit at Dublin City University in Dublin, Ireland, and I have been wanting to talk to Mark for a long time about what he's doing with UDL in Ireland and beyond. And there are so many things that he and his team are doing to help spread the word and train faculty in UDL that it was difficult to focus on just one topic. But recently, he has been working on the intersection of UDL, academic integrity and assessment design, which has really piqued my interest. And so today's conversation centers on how to make assessments more cheap, proof and authentic, so as to work on the prevention of academic dishonesty rather than catching students cheating after the fact so to speak. And the UDL principles are what guides this design. Mark has also graciously provided many resources that he mentions during our discussion. And you can find all of those on our think udl.org website for this episode number 75. Thank you for joining us as we talk about authentic assessments, academic integrity and UDL. Thank you so much, Mark Glenn, for joining me today to talk about what you have been doing at Dublin City University. I've been following you for quite a while. So thank you so much for taking the time to talk with me today.

Mark Glynn 02:18

Thanks, Lillian. And thanks for the invitation.

Lillian Nave 02:22

I'm very excited about this. So I'm going to go right into my first question, which is, as you know, what makes you a different kind of learner?

Mark Glynn 02:31

I've heard this question so many times at this stage. And all the time as I'm using in the kitchen, how am I going to answer that question? For me, very similar to a lot of your previous speakers, I don't see myself as two different there's little bits and pieces that would make me slightly unique, but not completely different. In so far as my, the people around me, influenced me an awful lot. And I would definitely, if I was looking at a pedagogy type, I'd fit into the social constructivist, you know, but I learned by doing I learned by stuff resonating with me seeing examples. I'm not really big into reading reams upon reams of text. So I much prefer something that's explained in ordinary language. I actually sometimes think it's a skill that as academics, we miss out on we sometimes try to overcomplicate something. And listen, if you can explain it to your granny, you can explain it to anyone. That's the way I'd look at it. And for me, that's the way I like learning and also the way I like teaching.

Lillian Nave 03:43

Right. So making it understandable to your audience. Yeah.

Mark Glynn 03:47

And for me, that was a big difference between college and my school experience. I wasn't a big fan of my school experience. But when I went into into college, they they made it understandable. They made it releasable. And they essentially treated me like an adult as those awkward teenagers who taught he was an adult long before I actually was,

Lillian Nave 04:10

you know, yeah, a wise soul. Right? Or cheeky

Mark Glynn 04:15

soul depends on who you

Lillian Nave 04:19

one or the other, right? Maybe both at once. Yeah, that knowing your audience is so important. I'm, I'm learning that my audience constantly changes to and as you say, you know, wanting to speak to your students as adults. It kind of kept them where they are, where it's authentic, where it makes sense to them. Is I've noticed it's changing a little bit too with my students. I'm getting first year students who have spent the last year and a half during a pandemic during their high school years. So then they come to college and they are not like the students I've had for You know, 18 years before the last two. And so it's it's trying to figure out what my authentic way of teaching is for a students who are, are slightly different, and how am I going to match their, what they're coming into college with. So I think that is really important. And we're gonna be talking about these sort of authentic things coming up to that's what I'm really excited about.

Mark Glynn 05:24

In recent years, my students have actually been different because they're not students or staff, because the main part only role is to support academic staff. So that in itself brings a different challenge when it comes to teaching.

Lillian Nave 05:39

Yes, absolutely. We have to know our audience and, and be able to speak to them, not below them, not above them, but but kind of explain right to where they're going to grasp and kind of start running with the material. And it's, it's a delicate balance. I've, I've definitely struggled with that sometimes. So, okay, so in in my teaching career, you know, I've worked with a lot of different groups of learners. Sometimes students also like you, sometimes staff, I've had first generation students, international students, we also have, I've worked with the T fellows, which are teaching excellence, fellows that come from all over the world, to learn at Appalachian, about how to be better K 12. Teachers. I've taught at different institutions. And I've found that some students don't understand some of the things that we maybe take for granted in academia, including academic integrity, and what academic integrity means. So how would you define academic integrity in your context?

Mark Glynn 06:43

For me, academic integrity definitions are there for everyone to see, you can have it and Google a local, very sort of different places, horrible for me, I want students to see academic integrity as part of their studies, rather than rules to have to follow. You know that that's the way you would look at academic integrity, it should be something that's that's integral to what we do, and something that we aspire to all the time. But that's the way I look at academic integrity. And, like, as I say, you can you can look up the definition anywhere and talk about honesty and morality, morality and commitment to commitment to behaving appropriately. But for me, I just want to see it as part of what we do. It's part of everything we do and strive to be so.

Lillian Nave 07:40

Yeah, yeah, I can, you're making that connection with the personal integrity to like, it's just part of how you learn, right? You don't want to cheat, you want to be a learner that is that has integrity. And that can carry that into the rest of the world, not just in college studies, or, you know, in the training or something like that, that that's part of who the student is. And we often I feel like in our institutions, that academic integrity is, like you said, a set of rules. And it's about, okay, well, you have to cite things this way. And if it's like this, you cite it this way. Or you have to make sure you follow these particular rules, and you note these particular things. And if you don't, then you're cheating, or you are, you are failing, you are, you know, not doing what you're supposed to do. And it it comes down to a lot of rules, we often have to put on our syllabus.

Mark Glynn 08:39

And that's where I see there's different unpeople that only introduce or only view academic integrity at a college level, I think miss out on that, because for me, it's something that should come in from the very, very young level, not just citation and referencing, but it's the ethical approach of doing things, right. Because it's properly proper to do them. Right. And it's just like UDL really, to be honest, it's something we all should just do.

Lillian Nave 09:07

Yes, yes. And and I think we, I wanted to have this conversation, like when we talked preliminarily, I was really excited that this would be our conversation, because we could talk you and I could talk about a whole bunch of things about Universal Design for Learning. And, and that could be a series of episodes, but for this particular, you know, chance, I wanted to talk about that academic integrity, because of, first of all, a lot of changes recently and, and what that means for academic integrity, but also because I don't think we talk about it enough in relationship to Universal Design for Learning. And we haven't really defined it well or maybe understand it well. And it might be part of that hidden curriculum that we don't because we don't explain it to our students well and what it means in our context then students are tripped up, it becomes an artificial barrier, for example, different cultures are going to view academic integrity differently. So some students may come to a university, and they think they have to do everything by themselves, they can't talk to a friend, they can't seek a tutor. That would somehow be cheating, you know, this is kind of an extreme case, but that they, they have to do everything as an individual. Otherwise, they're not learning or they're not smart enough, or they maybe they don't belong there. And I don't agree with that, but, but there are others who might come from a place where we do all this together, we have, you know, we work together, we talk to each other, when we're writing the papers, we have a peer review, we critique each other, where there's a lot of kind of group learning social constructivism, right, where people are really helping each other learn. And yes, you might need to note, hey, I spoke with these two classmates, when I was formulating this paper, you know, I want to recognize their work in my thought process. But that's not cheating, right? That's not against the rules. And we just we haven't really specified I think a lot of times we haven't specified what is academic integrity, and what it might mean, even in your particular class, or your institution, or your educational, you know, experience, you know what it means. And that's, I really have liked what you've been doing around that lately. And that's, I wanted to ask you, yeah,

Mark Glynn 11:37

so for me, we have such similar views on this. Like it is about unethical behavior. First and foremost, today's about whether it's collusion or collaboration, you know, the it's that which one is it? Which one does it fall into? And as you rightly said, there are cultural differences that come into it, whether it's, we have what some international students think it is, it's actually disrespectful if I don't give the views of the experts in the field. And they literally copy and pasted the views of the experts in the field. Whereas older cultures, and older people would say, well, actually, it's more appropriate to interpret their views and reflect on their views and give your own interpretation of it. Or some people, like you said, could have the support to get an extra tutor extra tuition. Maybe it's not putting them at a disadvantage, or sorry, an advantage over other people who don't have the money to get the extra tuition, to get that extra support beyond the people who are well off enough to, to have that or be in a position to do it. Are they cheating? Well, no, they're just doing what everybody does and start helping themselves helping the children get to that next level. But where's that? Where's that? borderline between cheating and actually using all the resources at your disposal?

Lillian Nave 13:00

Right, right. And that's what I love getting out into the open and and making sure that, that instructors, people who are in charge of this, you know, environment, educational environment, are making that known to to their students. So we've got we just have a clear understanding, and everybody's on the same page. So it's true

Mark Glynn 13:22

to exactly that. It's to stop people thinking, this is how to get me this is how to punish me. It's actually how can we do this properly? And and when you have the conversation, and the transparency of the conversation around to academic integrity, it becomes the norm. And that's what you want. It's not that we're trying to cut you out, or trying to help you learn.

Lillian Nave 13:44

Absolutely, yes. Alright. So these, one of the biggest parts then of academic integrity are assessments. And I'm interested in your work and how your work with UDL has influenced your views on assessments.

Mark Glynn 14:03

I'll preface this that I said in the introduction I learned from others. I am literally standing on the shoulders of giants because I'm I'm a lot of the work here that we'll be chatting about today. It's definitely being a team based work so that the people that I work with have collated. I have a wonderful team within the teaching enhancement unit. So reporting on behalf of those I think is the best way forward. But I in the teaching enhancement, you know, we wanted to tackle assessment and academic integrity with a proactive approach. Rather than trying to catch them after they've cheated. We wanted to catch them before they did. And we wanted to do that true assessment design. So working on the basis that prevention is better than cure. We wanted to look see how can we design assessments How can we as an institution behave or how can individual lecturers behave, what approaches they can take to reduce the opportunity for, for plagiarism, to reduce the rationalization for plagiarism and reduce the pressure on plagiarism. Those three points are actually from a thesis theory called the Cressy fraud triangle. And it comes from from the world of banking, but I would apply it to the world of, of education, because fraud, and plagiarism to me are one of the same thing, fraud and cheating. So, what we've decided to do is to look at the literature and see what's out there in terms of proactively eliminating plagiarism. And we found seven different principles when we collated all of these different papers that we pulled on assessment design, we found seven different principles, different ways that we can go about proactively reducing plagiarism. And we took those principles and then circulated it to my counterparts across Ireland, and indeed, well, anywhere grad that I had networks I was pushing it out to. And I said, we found seven, what are we missing. And we got a variety of different responses back from actually several 10s, if not even 20 responses back from different institutions. And we were able to combine all of them together, and condense them down to 12 principles of academic integrity assessment design principles, and I'll circulate them to you for the podcast. But firstly, the principles again, these 12 can be broken down into three distinct areas. So what an institution can do, what a lecture can do. And then the student partnership or student ownership. And again, if I remind you of the Cressi, fraud triangle, each one of them pressure, opportunity and rationalization, we want to look at how those 12 principles can be used to tackle each one of those points of the triangle. And essentially, the smaller we can make that triangle, less chance people have of of committing plagiarism.

Lillian Nave 17:16

Wow, it is fantastic. Can you tell me more about those institutions in what the institution can do, what lecturers can do and what students ownership brings to it? And yes, I will have these in the resources on our webpage for our listeners, Okay,

Mark Glynn 17:32

super. So that what the institution can do is, well, obviously, you can set high standards, you can set the values, you have an academic integrity policy, and you make sure that it's consistent across all the different programs that you don't have one approach in the School of Law and a completely different approach. In the School of Chemistry. You want to make sure that the students has all that detailed information from day one, about breaches of academic integrity, and how they might avoid them, and all that sort of stuff. And essentially, like academic integrity is what we build our reputation on. So an institution cannot risk not having academic integrity embedded in in what we do. So we must take steps to make sure, for example, that we update and edit our assessments and our our program assessment strategies on a regular basis. Because if for example, and I genuinely think this is is unethical, of a lecture to make an assignment and keep that for the next 1020 years, just not edited at all, a lady called Sarah Eaton, who's from your neck of the woods, gave a fascinating presentation. And she was the person actually who gave me that phrase on an ethical behavior from lectures and doing that. So there's different processes that we can put in place as a university regular assessment reviews, putting the academic policies in place, putting the supports in place to enhance academic integrity and having that transparency about academic integrity and having that positive, proactive approach rather than a reactive punitive approach. And from a lecturers point of view, there's so many things they could do. Ensure obviously I talked about regularly updating your your assessments will ensure they're authentic, ensure the current under relevant, give students scaffolding, and again, this is all relevant to UDL. If you scaffold an assessment, maybe for I can think it's a it's an end of semester essay they must do. Well give them key points and milestones throughout the semester as to what they must reach. Rather than leaving all the pressure to the last minute. Give them a rubric to show them what They must do an audit where the marking criteria is, is spread out and what's getting more marks versus less marks, what you expect of your students. And all these little things can be done. And they are just little things. And this is where I love the approach of Tom Tobin, the One Small Change, there's loads of these little things that we can change in each one of our assessments. And it will evolve over time. And that's all we ask people to do is to look at their assessments, look at these 12 principles and say, right, how many of those Am I able to, for want of a better expression tick the box against or how many of those I'm able to address this year, and what will I work on for doing next year. And that would be from a lecture point of view, and then looking at from a student ownership point of view. For me, I would see it as trying to reduce and this is where student ownership comes in is trying to reduce the rationalization element of the Crafty fraud triangle, the rationalization like show them tell them how to cite properly, tell them how to reference properly, gives them the option of choices within the assignment, again, UDL connotations in there with the choices, but give them the element of choice because they can then personalize the assessment. And they can say it wasn't relevant to me. But also, it's harder for them to use the likes of contract cheating services, because you're asking them to do something personal, and maybe related to other course material are related to their own experience outside. So all of these things, all of these elements of the these 12 principles do have direct overlap with universal design, as well as improving academic integrity. So going back to the tick box analogy, you're taking two boxes at once, by being good and ethical behavior.

Lillian Nave 21:52

Right, right. And those so many things that you have said, make my eyes light up with UDL principles. And one of the continuous I think ideas that are in each of these, the institution, the lectures and the students, is this transparency, and that the clarity that we need to give our students from the institutional level, and from our just our classes, like from the lectures or instructors, and even the students being able to understand what is is needed, what they have to do and things like a rubric, or I love providing annotated examples, like here's a great essay, right? And look at what happened in the first paragraph, it laid out the points well, and here's the citation that was used. So they have really good examples. And that's not cheating, either. Right? Some, some instructors, I have heard will say But wait, you know, we want them to figure it out. But providing an example is, is helping them to figure it out. And they still have to do all the work. We're just, I think taking away a barrier of how am I supposed to start this? I've never done this before. I don't even know what it looks like how would you know how would a presentation to accompany look like? I would have no clue because I've never done it before?

Mark Glynn 23:21

And don't exactly you're taking away the fear that students may have? Yeah, the rationalization that they'd have where, oh, well, I've never done it before. So of course, I'm going to copy off Lillian, because she's going to load your time. Or the pressure that you're under, when I have this assignment, I know nothing about it, I need help from somebody else. So I'm going to ask Lillian to help me with this. And, and again, like we need to get into into my head and into your head as students and just sort of think, well, it's done. Okay, how much is it collaboration versus collusion? The point that you brought up earlier on. So for me, if we are transparent if we give them sample answers, and as you say annotations to previous answers, you've shown them what you're expecting from them, you could also redesign the assessment, where there's an element of peer review in this beforehand. So before it even gets to you as a lecture, some of the peers have looked at it, and you could look and say, Oh, I see what Johnny's done with his I'm gonna actually do that with mine. I really liked the way he structured that. Both your grading criteria will change accordingly then as well, because you're you know that they've seen these examples beforehand. So there's a whole load of different things where we can help people designed or assessments better to accommodate more learners and better accommodate more learners.

Lillian Nave 24:50

Yeah, I had a really fantastic result with an assignment I worked on a couple years ago, in which there was a peer review within that session. assignment was an annotated bibliography, which is using citations and very much one of those kind of academic integrity assignments. And I found that students were not, weren't doing well, well, the first time I signed it. And so I realized that I needed to clarify what I was asking because I was not getting, you know, the result that we needed to get right that they were supposed to be finding peer reviewed scholarly journals, and, and then analyzing if it would be a good source or not. And I thought, Okay, well, I need to revamp this assignment and included a day where they brought in their assignment. And then we worked as a workshop. And they passed around their annotated bibliography to three other people. And we had like a rubric in front of us about what is what does it mean to be timed, timely? Is it authoritative, you know, what about the source? And they were able to, you know, to see by looking at one other students, they'll say, Oh, I didn't even think about this. And if they see another student, right, oh, I didn't do that in mind. And I said, That's okay. If by the end of the of the class, you get yours back and you say what you would change and what you would do right the next time, then you've learned something, right? You're much better.

Mark Glynn 26:23

Let's flip that on its head to where we're thinking of the student who has prepared a good annotated bibliography. He or she then can sort of puff out their chest, and they're thinking, Yeah, I did, well, I'm seeing this. And so it gives them confidence as well in their own learning. So it's a win win situation, as far as I'm concerned, and it's, it's not rocket science, because if you cast your your mind back to primary, or what we what you would call Elementary School, Teacher, we did that we passed her homework sheets around the classroom as we were kids, so all we're doing is just the high school that that that college to university version of that, we're just taking it upper level.

Lillian Nave 27:03

Yeah, and we're learning from our peers, and we're learning, you know, learning more ourselves too, as we do it. You know, anytime you have to teach a lesson, you learn so much more than if you're just, you know, getting that lesson from somebody else. So anytime you got to say, I looks like this part, I would, you know, change or it needs some some help here, I couldn't find this, then it's a, it's a different level of learning I found and students really have agents,

Mark Glynn 27:33

yeah. And that's just agencies to keywords, you're giving them agency into their work. So that is going back to the Cressi fraud triangle reducing the rationalization because they're, they're proud of our work, they know how to do what they need to do. So they're going to put the time and effort into it. And if they see you, and going back to your example of within the class, having that workshop to flesh it, I mean, you're spending time and effort on their assignment help promote, so they will feel I'm gonna put time and effort into this assignment. Whereas if you're the lecture that just gives out the same assignment time and time again, and gives out scantly feedback or no feedback at all. Students won't care. They're just sort of, I'll move on to the next one.

Lillian Nave 28:15

Yeah, right. And, and thinking about assignments, if if you've got a an instructor lecturer who's given the same assignment to the same class of 50, or 100, for the last five years, you better believe there's examples online, somewhere, right, 100%.

Mark Glynn 28:30

And now, now, more than ever, it's easier to find stuff online. And for me, one of the challenges that we have and don't get me wrong, I'm possibly one of the biggest kids should know even though we barely know one another, I love the internet and all the different technologies that are available. And when I go on Google and I search for something, all of a sudden, the next ads that I'm looking at, and the next website is relevant to my previous search. Brilliant, of course, yes. However, put yourself in the students shoes, they're getting ads, based on their profile based on their searches. They're getting ads from contract cheating sites. They're getting ads from all these services that we used to be able to detect years ago because they'd have to come in and stick a notice on the student board. And we'd rip it down straightaway. As soon as we'd see us. Well, now, that's gone underground. And we're not seeing that at all. Which comes back to my point about wanting to have the academic integrity conversation out in the open. And let us have it and do everything we can to proactively reduce the amount of plagiarism that goes on.

Lillian Nave 29:33

Yeah, and you know, I think about that to carrying forth into their lives, like wanting to be people with integrity, and then you know, you go into your whatever your job or your life, whatever it takes you that you have that integrity as well. And I think about the the flip side, like what if students are saying, well, I'm never going to use this right? I don't it doesn't matter if I cheat because I'm you know, I'm never gonna do this and that's where that authentic part comes in, right? That needs to be authentic, because I would be more likely to kind of put low effort into something, if I think well, I just need to hand this in to pass this class. And I'm never going to do this, you know, I'll never need to know this term exactly,

Mark Glynn 30:15

I show where there's authenticity in their worth or personalization in there. Because remember, and if you take part time, student, for example, there, they have an awful lot of pressure outside, they have their day job going on, or they have family they need to look after, and then they'd come to study. So if you give them a useless assignment, that means nothing to them in their day job, or in their day life, there'll be like, Oh, just let me get it and get it over and done. Whereas if you give them something that's relevant to their day job, they'll actually do a really good job of it, and then want to improve further their job as well as for college. And you've built that intrinsic motivation, you've tapped into that motivation for students to do better. And whereas the passing a true delight of the text matching services, that's the extrinsic motivation, you know, you wanted to get better results, go for intrinsic every time, make it authentic, make it something they can have a passion about. And you'll get brilliant assignment products for want of a better term, which again, you could learn from as an educator.

Lillian Nave 31:27

Yeah, and you've just described to that whole first column, the green column about engagement and motivation. And the Universal Design for Learning principles is, is creating that, you know, bringing in interest and motivating students sustaining them throughout the process of that assignment, you know, how is it worthwhile? How is it valuable, and helping them to see that value to complete it? With Integrity? It's all I see it very linked, as you do as a link to our Universal Design for Learning principles.

Mark Glynn 31:59

I can I look at the principles and I can nearly draw direct lines between them and under 12 principles that we have, by no means are we saying that the 12 principles are, are to the standards of the cost guidelines. But we can definitely see the relevance. And it's it's taking the two boxes at once. It's helping us design a better learning experience for for students. Again, Transparency is key. Personal choice is key. giving them information upfront is key, all of this UDL and academic integrity, and we even view it on our language is is so important to the language that we use when we're saying improving academic integrity, not catching plagiarism. You know, it's not not catching cheating. Yeah. And it's how to make it, how to make it better how to make the learning experience better. And that's what UDL does, as well. So,

Lillian Nave 32:55

yeah, yes, absolutely. And that word transparency we've been using a lot, I think, is key, if it makes me think of, I'll add this to our resources, the tilt, transparency, and learning and teaching by Marian Winckelmann, I need to get her on the podcast, too. But that that goes along with the idea of the purpose. There's three parts that purpose, task and criteria. So letting your students know what the purpose of this assignment is, like you're doing this so that you can be a better nurse one day, or you can solve a problem in your economics job, right? And then the task lays out exactly what they what a student needs to do to be successful. And the criteria are the things that we've just talked about a rubric, or an annotated example of a student's work, you know, or something, something like that. And that that transparency is, I think, absolutely essential, and something somehow we haven't been doing forever. Like, why didn't we do

Mark Glynn 33:56

this? Yeah, it just went when you think of it, it actually makes common sense. You know, and even when you think about making documents, but making them accessible, since I've learned how to do that, and it only took somebody to show me really a difference that will make but since I've learned to do that, I've actually became better at my job. I've made better documents, I've given better presentations, because I've made stuff. I've used the accessible tools within within the documents. And what I want to do is integrate that into the assessments. So when the students are actually compiling the Word document or compiling the PowerPoint presentation, or whatever it is they're doing, as part of it, that we embed UDL into the assessment as well, that we showed them that they must have all texts and why they should have we showed them that they should use Heading One, Heading Two, Heading Three. And again, not just for screen readers, but it'll generate your table of contents very easily, you know, simple stuff. It just it makes sense. That's what gets me, you know?

Lillian Nave 35:03

Absolutely it does all of this makes perfect sense to me. Now, I don't know why I didn't know it, you know, before I had been introduced to Universal Design for Learning. But now it makes perfect sense.

Mark Glynn 35:16

And it's the same with the academic integrity principles, there is nothing majorly scientific or heavy about the 12 principles, you'd nearly say it's the 12 principles that are bloody obvious. You know, when you're when you're looking down truest. It works, you know. And what we want to be able to do is to not put where we talk, from a UDL point, not put barriers in front of students, we actually don't want to put barriers in front of lectures when it comes to assessment design. Yeah, which is why we made it as simple as we did, and drilled it down to, to the 12, the 12 principles. One of the other things that you mentioned transparency, but one of the other things is connection, is what I think is really useful. And it's the connection between assessments. So why do I point that out, it allows people to connect between modules, because far too often, the chemistry teacher teaches chemistry and doesn't care about biology. And the biology teacher teaches biology and doesn't care about what's going on in the physics class, and so on. But the student has the overall learning experience of the science course. So we need to connect those modules. And connecting those modules helps people not just people with learning disabilities, but helps all students connect, start learning. And even if we do it a simple by connecting our assessments, that the assessment from module one feeds into the assessment for module two, maybe, for example, you mentioned doing an annotated bibliography. So that could be the first assessment. And then the second assessment is taking the topic that you did the annotated bibliography on, write a paper on or do a presentation on. And then your your third assessment, maybe in a different module is about something else that maybe you have to now write a proposal or a solution to a challenge that has came up from the area that you're researching. You could you mentioned rubrics earlier on, you could have rubrics that within the rubric for the second assignment, it says, What have you done with the feedback from the first assignment to make this second assignment better? So simple little things like that structure, a scaffold, and there's your UDL and your assessment design, put into one

Lillian Nave 37:32

Exactly. Right, right. And, and we often can see those connections as lectures or instructors, we know that the assignment for Module One leads into the assignment for Module Two, but we might not have made that really clear to our learners,

Mark Glynn 37:49

same percentage of time we didn't make it, we assume a connection is made on a course the students see it, but in actual fact, they don't.

Lillian Nave 37:58

Yeah, they're so flooded with this new information. And there they are. Really, amateur learners, you know, new in that field, oftentimes, and don't see that, oh, this is building towards this regular product that we always see in this field, the big because they're just not used to the regularity, or they don't know what's coming. And I often think about a.to.id type of drawing, that we look at it and it looks like a sea of dots that don't make sense. But until you follow and connect those dots with all of the numbers, you realize, oh, it's a cat or it's a giraffe, right? Or something like that. It makes sense at the end, but our students are just going from one to the other, not completely confused, not knowing what's going on. So if we can situate it like, here's our first assessment, it's the first step in a large research process that you will be doing all semester, it fits in here, you need to do it. So you can figure out who is a prominent voice in this area, you're going to need to find, you know, a philosopher that is going to be able to speak on the topic that you're going to get to you know, and they don't realize that you have it all planned. But we have to again, make that transparent and make that connection transparent to our students. And I am guilty of that too.

Mark Glynn 39:18

I'm me too. Absolutely. I'm not pretending to be perfect by any stretch of the imagination. But I used a funny you used to dot dot connected dots have never, never made excuse me that for a poem but never made that connection myself. I've, I've used the the example of where when, you know, look at those optical illusions. You know, when you look at the image, you see a lady and when I look at the image, I see a doc and now we see completely different things. But then once you point out where the lady is, then I don't see the duck anymore. All I see is the lady so that's where what I was trying to explain the same point two lectures I said, You think everything is perfect and you think everything is clear. What's your students don't see that you're stupid. See something else, and trying to put yourself into the shoes of the student. And what I think is a really good way to do that. And going back to scaffolding and being transparent with it, is get a student from last year to talk about the assessment from last year. So talk about, that's a great idea. And from this year, they're hearing it in the students voice, though, like because when I went to one the chemist by training way back when I couldn't even tell you what color white lab coat is. Now, let me tell you both. So but when I was doing my, my masters and did various different certificates, teaching a learning reflection is key to what you do. When I was a chemist reflection was looking in the mirror, you know that that's what I taught a section was. So for me, if I when I got or if a chemistry former chemistry graduate, came to me and said, Oh, well, I knew that was our I thought it was looking a mirror. But now this is what it is now, well, I could I can resonate with that person. That would make my learning that little bit easier. So if I got a student from the previous year to talk me through their assessment, and it could be something as simple as recorded video that a lecturer puts up, yeah. And displays to the students in the same way, you would get the lecture to put up sample answers. Get them to put up a couple of just students chatting about the assessment, how they get on last year, what they talked about at first, and what they think about it now, again, trying to make it more authentic for them, get get alumni members get graduates to say, God, I remember Macklin waffling here, but now I realized it was the best subjects because I'm out in the world. And that's what I do all the time. You know?

Lillian Nave 41:45

Yeah. And also our students are going to know their process much better than we would I had students, for example, I had students come back to talk to my friend, first year students about the class and the things that they said for advice was not going to be the things I would say, Well, you know, yeah, things like, You got to make sure you get to the, you know, get the library portion done early, because you don't have enough time because it comes at this point in the semester, when you've got all these other things that of course, I didn't know were happening. But the students were were the experts, at least and in that, and I was always surprised, like, oh, I never would have thought of that. During the semester. Give me

Mark Glynn 42:30

a give me a funny example, when I was teaching chemistry I was, I got the students to find a YouTube video. And I said, Find any YouTube video that is related to some content on our course, on the syllabus. And the deal was that every student had to post the video on the discussion forum, within our VLE. And you couldn't put up a video that somebody else has put up already. Right. So they had the flexibility of finding this video at their leisure. So there was a UDL element there, they could personalize it a video that meant something to them that want to relate against easy experiment and the easy assessment. The students were looking for a catch, you know, it was that easy. Yeah. But what happened was, I said, and give me a couple of sentences about why you think this video is good. And I got there is over 80 students, so I got over 80 different videos come back to me, of which 50 or more actually really good videos. And that's probably being a bit optimistic about but I asked them the students to rate everybody else's video, I said, just just give me your top five, give it a score out of five. And the videos that I taught were absolute childish tripe, the students loved, I did a video on I still remember, this is 20 years ago, and the students found a video about how to name organic chemistry compounds. And so I thought, right, this is very relevant to our syllabus. But it was like a Sesame Street song or the you know, the kids TV problem. But yeah, it's like Sesame Street song on how to do the names for us. And I'm not gonna sing it now. So don't worry about it. But I, if I found that trying to research for my class, I would have ignored it. And so that's a way to challenge yeah, that got the top rating in the class. Yeah, because they taught it to help me remember how to do it. So that I learned a very valuable lesson there that is always try bring the students and and in this case, it was a students as partners now what's known as I didn't know what it was called at the time, but students with partners and assessment rather than doing stuff to the students do stuff with the students. And that opened my eyes, you know?

Lillian Nave 44:48

Yeah. And that's it. That is a very authentic type of collaboration to Yeah, and brings ownership to the

Mark Glynn 44:57

unfolding because the students who replied because obviously they had all of YouTube to search from whereas the students that went second had all the YouTube minus to one video. And it goes student that went at it had to sift through all the others. What happens to things really interesting happened? One, the students that did it the first couple, when they saw other videos posted by the colonics. They said, Oh, can I do it again? So like, How many students do you know what else to do? And assessment a second time? You know, I'm just wow. The second thing was on the students said to me years later, he said, I was one of the last students to submit the video. But what happened was, every time I went onto YouTube, because I'd watch 70 Odd other chemistry videos, they kept on getting recommendations for chemistry videos. So you know, yes, I was teaching them chemistry subversively true the power of Google, you know?

Lillian Nave 45:50

Yes, right. Oh, what a? What a great addition. I love it. So well, we're in a strange time now. And I'm actually going to ask the next two questions kind of combined into one about how you think COVID-19 And the pandemic has changed the assessment landscape and kind of in additionally, how do UDL and academic integrity figure into this change in the assessment lens,

Mark Glynn 46:15

so So for me, and not for one second to go on to paint COVID in a positive light, it's been a horrible pandemic for everybody worldwide. However, there's a silver lining to it. And I think the silver lining, in this case, his assessment, and how it's changed assessment, when we had the height of the pandemic, and we had a lockdown. And for far too many months than I care to remember. But when we had the lockdown, all of our assessments had to change all of our traditional exam hall assessments, which are two hours and we cram however many 100 students into this sweaty exam hall to try do these exams. That to me is on authentic assessments, just that's called a call it what it is. But that had to stop, we had to come up with a different way of assessing, we had to think about the traditional assessments that we've done for aeons of putting people into these examples, under stressful situations, to okay, how am I going to assess it? How am I going to change it? And as much as there were people that had concerns about academic integrity, and they tried to introduce the software that would proctor them remotely, which I think yes, nightmare and totally unaccessible won't even go down that road. But as much as a good portion of people try to do that and try to essentially mimic what they were doing in an exam hall in an online environment. Loads of people took the better approach of actually redesigning their assessment. And what I'd like to think and I don't have to figures, the exact figures off the top my head. But I'd like to think that the large majority of those that made the changes through assessment have kept their changes. Yes. And that to me, is progress. That to me is improving it and yes, you will have your detractors, and you will have the people saying, Oh, well, how do you know it's them? That's doing this. Like, to me, you're always going to get that no matter what assessments you do. And and if you're taking that approach, when he wouldn't have a continuous assessments at all, everything would be in an exam hall scenario. what frustrated me was where the professional bodies are coming to us, telling us how to assess I know you must do it in an exam hall, because if not, we want to credit your program. They were as we should have been driven by the pedagogy, and what's best for the student learning experience. And, and without a shadow of a doubt. What's best is authentic assessment. And for the large portion of modules. Authentic assessment is not a two hour written exam. I can't remember the last time I wrote for two hours, ever. Yeah, no.

Lillian Nave 49:05

You mean it's not an everyday occurrence in your adult life,

Mark Glynn 49:07

but like my daughter would come back now. She's long since graduated. Now when she came back, she in her hands were sore from writing.

Lillian Nave 49:15

Yeah. Oh, I remember that. Yeah, links used

Mark Glynn 49:17

to type when she'd be typing probably quicker and better than she'd be be writing that literally. And it was more more difficult for a lecturer to interpret her writing. Whereas if it's typed, easy to do, spellcheck can be done. Yeah, formatting can be done. It's just a nicer experience for the student and the staff member. So yeah, again, it's a no brainer. I hope those those assessment practices continued. That's, that's that would be my silver lining to the cloud studies. COVID-19.

Lillian Nave 49:49

Yeah, I cannot remember where I heard this, but I heard the term COVID dividend, and there are certain COVID dividends that came like spending a little more time with my family, right? Because we were all at home maybe too much time. But that was a COVID dividend. And I think in academia, this assessment, rebirth really, of authentic assessments and multiplicity of different kinds of assessments, um, has been a dividend that has come from the the COVID 19 pandemic.

Mark Glynn 50:22

And I think what happens with assessment is we get comfortable, and we assess the way we were assessed, and there are electrodes were assessed the way they were assessed, and they think it is the right thing to do. And we don't think of the long term learning attends to be surface learning that happens then, because students cram and students just learn off what needs to be done. Where are they going to learn off? Where they're going to learn after model answers, where are they getting the model answers to get them from somewhere else? So that's academic integrity problems, then that we have. It's not they're tall. So what they're doing, they're regurgitating somebody else's content, even though they're doing it in an exam hall scenario, when we know it's them. It's just you're testing whether they have good memory or not. You're not testing the content.

Lillian Nave 51:08

Right? And will they remember it? Five years later? Actually, probably not.

Mark Glynn 51:12

There's a gentleman called Phil race over in the UK. And I don't use that term lightly. He is an absolute gentleman. And he has many an article about the authentic assessment and the how the exam hall is the wrong approach. And we need to be better, we need to do better.

Lillian Nave 51:30

And I'll have to include that in our resources as well.

Mark Glynn 51:33

Yeah. All little all the stuff fill ratios is brilliant. Okay,

Lillian Nave 51:38

great. So, um, we, I think you have laid out a very good case about why we should be creating authentic assessments and why it is an ethical imperative to, and how universal design for learning figures into it. So if someone is listening now, and is inspired to change around their assessments, what advice do you have for someone who is interested in creating an authentic UDL guided assessment in their higher ed courses, or wherever they're needing to assess learners? And they just don't know where to start?

Mark Glynn 52:19

Where to answer that question. For me, oh, one small change. Again, going back to the Tom Tobin thing, don't be afraid to change. Like an actual fact, by standing still, you're moving backwards with that expression. I can't remember who said that. But for me, you need to change and you need to evolve. So just to one one small bit, maybe give the rubric to students. Maybe and here's a, here's a what could be a game changer. give the students the similarity report, a text matching service report, give it to them before the deadline, allow them to see us before the deadline, because that want to know them learn from us and not be punished by us. You know, I don't know, allow them to submit drafts, or allow them. Again, a lovely one, from Sarah Eaton. Again, the lady I mentioned earlier on, she said, be kind. And and she said there was a policy expression she had, it was a get out of jail card. Right. And I loved that expression to get into jail cards for the assessments where if a student is under pressure and isn't going to meet the deadline, allow them one get out of jail card allowed them to be late, say no questions asked. Just 11th I thought I was a lovely example. And each one of those changes. So give them the rubric in advance, give them the text matching in advance, give them a get out of jail card. That's all in the command of each individual lecture to do 100% Absolutely no doubt about it my mount. So they are three quick examples of one small change.

Lillian Nave 54:05

Yeah, you're, you're making me think I need to apply that Cressy fraud triangle to my assessments and find out if I am pressuring students into rationalization and kind of look at what my parameters are. If I only give them two days to do this really long assignment. I I'm almost pushing them into doing a poor job, right? If it takes a lot longer. And in thinking about how I can rethink from my students perspective, what that might look like to them. And I hadn't thought about the ways that we sometimes as instructors are pressuring our students to cheat because we're not either providing the time or the scaffolding or, or the clarity, I would say for what Students, we

Mark Glynn 55:00

need to look at it for far too often we look at our module, right and have the expression that's my module, but not exist yet program. And the student has several different modules. So where you may think in your module of three assessments, that's exactly what's needed. Actually, no, I put a fourth in because they learned so much better when I give them this extra assessment. Well, if every lecture did that, and all of a sudden the student has 24 assessments or 25 assessments to do in a 12 week time period. Yeah. And more than likely, it's the same deadlines. We have this thing called week seven syndrome, where week seven within the semester, all the assessments start. And you're thinking if lecturers start to talk to one another, and say, Well, God forbid, where if they start talking to one another, and plan their assessments? And going back to what I said earlier on connecting thorough assessments, that that would really, really help.

Lillian Nave 55:57

Yeah, absolutely. The This has given us a lot to think about. And I know our resource page is going to be full of the resources you've told me about. And those are the 12 principles on assessment design. That's going to be really helpful. I'm really excited to put all this together

Mark Glynn 56:17

delighted to share it with you. And as he said, it's really it was a team effort as my colleagues, I won't name every single one of them said, but they I literally have stood on the shoulders of giants in this respect. So be delighted to share all of the content that we have.

Lillian Nave 56:33

Well look at you modeling academic integrity, right. Yeah. Just fantastic.

Mark Glynn 56:38

I just know, I've been slated after they listen to it, if I don't.

Lillian Nave 56:42

That's right. That's right. Oh, that's perfect. Well, thank you so much. I really appreciate the time you've given me and all of this really great information that helps us to be better instructors and lectures. And I just can't thank you enough for the work you've been doing for a long time, and I'm really glad I got a chance to talk to you.

Mark Glynn 57:01

It was absolutely my pleasure. Thank you very much pleasure and privilege.

Lillian Nave 57:16

You can follow the think UDL podcast on Facebook, Twitter and Instagram to find out when new episodes will be released, and also see transcripts and additional materials at the think udl.org website. The think UDL podcast is made possible by college star. The star stands for supporting transition, access and retention in post secondary settings, and the website provides free resources and instructional aides based on UDL principles. If you'd like to know more, go to the college star.org website. Additional support for the podcast is made possible by Appalachian State University where if you call it Appalachian, I'll throw in Appalachia. The music on the podcast was performed by the Odyssey quartet comprised of Rex Shepherd, David Pate, Bill Folwell and Jose Cochez, our sound engineer is Tanner Jones and I am your host, Lillian Nave. Thank you for joining us on The think UDL podcast.

# data[].attributes.attachment_link.title
Academic Integrity through Assessment Design with Mark Glynn | Think UDL

# data[].attributes.attachment_link.description
Welcome to Episode 75 of the Think UDL podcast: Academic Integrity through Assessment Design with Mark Glynn. Mark Glynn is the Head of the Teaching Enhancement Unit at Dublin City University. I have been wanting to talk to Mark for a long time about what he is doing with UDL in Ireland and beyond.

# data[].attributes.body
How to Write an UnGoogleable Exam Question – Part 1 – David's adventures in the classroom.

# data[].attributes.body
Multiple-Choice Questioning as a Valuable Learning Opportunity - The Effortful Educator

# data[].attributes.attachment_link.readable_text
Multiple-choice quizzes are somewhat polarizing in education. Most teachers either love them or hate them. Some see these questions as a way to assess student learning while also providing options. Others believe multiple-choice questions are not very valid or reliable because students are able to guess and may be given credit for information they really did not know. And, to be honest, I see both sides of the argument. In my classroom, I want to provide efficient and effective instruction. By that, I mean, I want to use methods that most clearly present the information to the students for understanding. After instruction, I want to provide many opportunities (at varying time delays) for students to access and retrieve that information during assessment. At its simplest, class time is spent either trying to get 'information in' or 'information out'.

I want the 'information out' stage of class to be as honest as possible for my students and myself. No one is assisted by an assessment that does not accurately evaluate student knowledge. So, I understand why some don't appreciate the multiple-choice question. I'm sure we've all been in a situation where we haven't known the answer to a question, so we just guess...and, perhaps, 25% of the time we get it right without actually knowing the answer. That is why I created the template below in an attempt to curtail this sort of profit by guessing. This requires students choose the correct answer, like a traditional multiple-choice question, but students are then tasked with cognitively interacting with all distractors.

But maybe my worries about multiple-choice questions has been unfounded. A paper by Little, Bjork, and Bjork (1) put some of the critiques to the test.
"Critics have argued that multiple-choice tests are less effective as learning events than cued-recall or short-answer tests are because they present the correct answer among the alternatives, thereby bypassing the need for retrieval and allowing test takers to rely on recognizing the correct answer."
But is this actually the case?

Little, Bjork, and Bjork experimented with a goal of discovering whether multiple-choice questions, when written with proper incorrect alternatives, can actually task students with using retrieval that increases more than just the later recall of correct answers and their associated information. Can these questions amplify the recall of initially incorrect alternatives when they are the correct answer on later recall tests?

In other words, if 'B' is the correct answer, but students have also considered alternatives 'A', 'C', and 'D' before choosing 'B', is that better for remembering all answer choices (A-D) on a later assessment? This is in opposition to the thought that recall questions (fill-in-the-blank, short answer, essay, et cetera) are better for retention of material because these style questions provide no help. Multiple-choice questions may be better for study because they stimulate more thought about more material, with the caveat that all alternatives must be competitive and realistic options...and, believe me, I understand that this is much easier said than done.

So, what did the researchers find? Their evidence not only supports the hypothesis that multiple-choice questions with competitive alternatives can require retrieval (rather than just recognition), but also substantiate the idea that these questions improve retrieval of material associated with the incorrect alternatives. These results contradict what many believe about the legitimacy of multiple-choice questions as study aids. It would seem that these questions, with more alternatives, actually can promote more cognition of material.

While this evidence is quite interesting to me as a classroom teacher and my 'wheels start turning' about how to use this in my classroom with my students, I think it appropriate to revisit a necessary component of high quality multiple-choice questions: all of the alternatives must be competitive. This can be quite difficult to produce for many reasons and can be very time consuming. Those opposed to the use of multiple-choice questions often state that the time spent writing competitive alternatives could be better spent creating other types of questions. Also, what is competitive to one student may not be to another. I believe these to be valid points. Writing a good multiple-choice question takes practice, but I think, as this evidence indicates, the benefits can be great.

What jumps out at me, that is also important for multiple-choice questions (or really any type of question) to be successful is student cognition with the material. If the students genuinely consider and retrieve information about all alternatives, then multiple-choice questions can be very advantageous. If, however, students read the stem and just look for the correct answer, mostly ignoring all other alternatives, then thinking about all of the different aspects of the material will suffer.**

There needs to be a conversation with students about the results from this research and how students should use multiple-choice questions to study/practice. I believe it would be a rare occurrence for students to look at a multiple-choice question as anything other than a task to find the correct answer and move on. Explicitly modeling and discussing how students can use the questions in a more beneficial manner is a must. That is why I developed the template shown above. Students must think about and consider all aspects of the question. So, instead of just looking for the answer, they are cognitively involved with all alternatives; comparing and contrasting information, explaining, describing, et cetera.

Another strategy to encourage students to use all aspects of the multiple-choice question is to have them rank the answer choices from most correct to least correct. With this approach, students again are tasked with considering differing aspects of each answer to rank the alternatives, oftentimes with an explanation of why they ranked a certain choice in a particular order. In reality, the order isn't all that important, as long as students can use relevant information to support their ranking. The cognition involved while working with the material is what I'm interested in.

The long-term goal of using these strategies is to create healthier study habits with students; that they can see these questions as powerful learning opportunities. This plays into the larger goal of having students seeing effortful learning as more effective learning. As the results from Little, Bjork, and Bjork indicate, multiple-choice questions, when used appropriately, can be at least as potent for learning as questions requiring the recall of information.

**There is some upside to this way of thinking, though. If students can read the stem and feel confident they know the answer before even looking at the alternatives, they can be relatively sure they know the material...assuming they answered the question correctly. Essentially, this takes the recognition of a multiple-choice question and turns it into a cued-recall question.

Little, Jeri & Bjork, Elizabeth & Bjork, Robert & Angello, Genna. (2012). Multiple-Choice Tests Exonerated, at Least of Some Charges: Fostering Test-Induced Learning and Avoiding Test-Induced Forgetting. Psychological science. 23. 10.1177/0956797612443370.

# data[].attributes.attachment_link.title
Multiple-Choice Questioning as a Valuable Learning Opportunity - The Effortful Educator

# data[].attributes.attachment_link.description
Multiple-choice quizzes are somewhat polarizing in education. Most teachers either love them or hate them. Some see these questions as a way to assess student learning while also providing options. Others believe multiple-choice questions are not very valid or reliable because students are able to guess and may be given credit for information they really did not know.

# data[].attributes.attachment_link.title
Wood & Gibbs 2020 - Invigilated exams to take-home papers.pdf

# data[].attributes.body
Preparing assessments in the context of pandemic – lydiaarnold.net

# data[].attributes.body
Phil Race’s guide to assessment forms and their respective pros and cons 

# data[].attributes.body
Learning Module: Designing Online Assessments | Taylor Institute for Teaching and Learning | University of Calgary

# data[].attributes.attachment_link.provider_name
Taylor Institute for Teaching and Learning

# data[].attributes.attachment_link.readable_text
The design of assessments within a course can have a significant impact on student learning and engagement. Assessments are primarily developed to demonstrate achievement of course learning outcomes and serve as a motivational factor in driving student decision-making. Creating meaningful online assessments poses distinct challenges for instructors accustomed to teaching in face-to-face contexts. It is possible to develop effective, rigorous online assessments but special attention needs to be given to alignment with learning outcomes, technological limitations and academic integrity.

# data[].attributes.attachment_link.title
Learning Module: Designing Online Assessments

# data[].attributes.attachment_link.description
The design of assessments within a course can have a significant impact on student learning and engagement. Assessments are primarily developed to demonstrate achievement of course learning outcomes and serve as a motivational factor in driving student decision-making. Creating meaningful online assessments poses distinct challenges for instructors accustomed to teaching in face-to-face contexts.

# data[].attributes.body
5 Reasons to Stop Doing Timed Online Exams During COVID-19 | Learning Innovation

# data[].attributes.attachment_link.readable_text
Like many of you, my home is now an office a school and a residence hall. Living with four remote learning college students (2 seniors, a junior, and a sophomore - none enrolled at my institution), provides me with an up-close view of the COVID-19 postsecondary learning experience.

More accurately, I am seeing the learning experience of the subset of more privileged students. The college kids living under my roof benefit from fast internet, modern laptops, and a full fridge. Their only jobs are to finish out the semester. And despite their many privileges, I still see these remote learning college students struggle at times.

Across higher ed, we have patted ourselves on our collective backs for making the rapid transition from residential-to-remote learning. I'm not immune from a bit of self-congratulation. But the real people that we should be giving props to is our students. They are the ones who have had to simultaneously move both house (for residential students) and learning mode.

Among the biggest struggles that I'm witnessing with the remote learning college students at my home is in their timed online exams.

Requiring our remote learners to take timed online tests seems like a higher ed self-inflicted wound.

There are at least five reasons why no remote learner exam during the time of pandemic should be timed.
Timed Online Exams: #1 - Create Unnecessary Stress:
During a pandemic, among the absolute top priorities of every college and university should be to avoid contributing to any unnecessary stress for our students. I'd argue that this is true even in non-global pandemic times, but we save that debate for later.

Timed exams create stress. Not for every student and every exam. But enough students in enough exams.

Timed exams created stress, even if everything else in the test-taking environment is normal and supportive. Nothing in our students'students' test-taking environment is normal and supportive nowadays.

A stressed-out brain does not learn. Even with all the evidence from learning science, we in higher ed have not been able to let go of the myth that a little bit of stress is good for learning. It is not.

Stress and anxiety can push up short-term performance (as measured in test scores) but is horrible for long-term retention.
#2 - Are Vulnerable to Bandwidth Problems:
How is your home bandwidth? With six laptops competing for bandwidth in my house, not to mention every neighbor also on the internet all day, my bandwidth sometimes sucks. And we the top Comcast tier.

How are you surviving with remote work and schooling if you are on DSL, cellular, or satellite?

Combine iffy home bandwidth with the performance challenges that Canvas, Blackboard, D2L, and other learning management platforms have been experiencing as a result of massively heightened usage. The LMS providers seem to be doing a decent job of scaling their services to meet demand, but reports of platform sluggishness are widespread.

With timed online exams, particularly exams that only show a single question on a page, bandwidth and other performance issues are deadly for the test-taker. Can you imagine your exam time ticking down as you wait for the questions to load?
# 3 - Are Poorly Correlated with Understanding:
Timed exams measure a student's ability to answer questions quickly. A timed exam is assessing speed, not understanding.

There is no correlation between recall speed and understanding. Students who can come up with answers quickly are not smarter than those who need to think before answering.

The ability to finish an exam quickly is not a measure of preparation, diligence, or mastery.
#4 - Do Not Protect Against Cheating:
Sometimes you hear that instructors assigned timed exams to guard against cheating. The idea is that students will not have time to go and find the answers from other sources.

The idea that timed exams protect against cheating is another one of those myths that higher ed can't seem to let go. The reality is that if students want to cheat, then they will find a way to do so.

In our current universal remote learning reality, I suppose that we could institute an online proctoring system involving webcams, keystroke logging, and maybe drones. I'm dubious if this will work, and I'm sure that the benefits would not be worth the effort.

Much better to have an honor code that your entire community believes in. Put that honor code at the top of every exam. Talk about the reasons behind the honor code in your synchronous online sessions. Record a video about the honor code that your students can watch asynchronously. Require each student to agree to the honor code through a quiz.

But stop believing that a timed exam will do much to prevent cheating. It won't.
#5 - Place Undue Burden on Students Who Require Learning Accommodations:
I do not doubt that most schools are totally on top of the steps necessary to ensure that students with identified learning support requirements are receiving extra time on timed exams. If a student was approved for additional time during residential classes, that approval should carry forward to remote learning.

Even in the best of circumstances, however, requiring that our students navigate the systems in place to allow extra exam time places an additional requirement on these learners. The task of getting extra time for exams has to be more difficult in a remote learning environment.

We should also be willing to admit that there is a gap between those students who are allowed to take extra time on an exam, and those students who would benefit from additional time.

Students (and their parents) who are savvy navigators of the education system have jumped through the necessary hoops to secure extra exam time. Those with fewer resources may not be on anyone's list of students entitled to accommodations.

So no more online timed exams during COVID-19.

If you have set your exams with a time limit, please consider rethinking this decision.
Anyone care to make a case for sticking with online timed exams?

# data[].attributes.attachment_link.title
5 Reasons to Stop Doing Timed Online Exams During COVID-19 | Learning Innovation

# data[].attributes.attachment_link.description
Like many of you, my home is now an office a school and a residence hall. Living with four remote learning college students (2 seniors, a junior, and a sophomore - none enrolled at my institution), provides me with an up-close view of the COVID-19 postsecondary learning experience.

# data[].attributes.body
Student Agency in Uncertain Times | University of Venus

# data[].attributes.attachment_link.readable_text
As we've moved largely to online delivery for our courses, there has been an increased concern around academic integrity and cheating -- students, the narrative goes, will take advantage of this move online to cheat when they might otherwise not. We must, common wisdom tells us, be ever-vigilant in the name of rigor. I am grateful to Thomas J. Tobin for his insight and perspective on this issue, originally shared on the POD Listserv and allowing it to be reprinted here. It deserves a larger audience. -- Lee Skallerup Bessette

There is a fear among many newly remote instructors about the possibility of students cheating in an environment that may be unfamiliar to instructors and learners alike. What's on the rise is not academic dishonesty, but suddenly online instructors' fear of losing control over the conditions under which assessments are handled.

Students are not out there cheating any more than they usually do, although one thing that could increase cheating is putting students into a bind where they get tempted -- you know, like moving everyone to a hastily constructed online environment with new rules and social norms. Something like that.

Put plainly, students are most tempted to act dishonestly when they feel anxiety and pressure. Most don't act on those temptations regularly, but we all have our tipping points. The current shift to remote instruction can create both anxiety and pressure. The best thing that we can all do, regardless of the subjects we teach or support, is to offer students options, voices and choices.

Lower time pressure: don't put time limits on your online activities, including exams.

Lower due-date pressure: allow late submissions, even if you lower the grade for such things (I don't, but do like you do).

Lower grade anxiety: spread out the points across activities and avoid having one big thing carry enough points to possibly wreck a final grade. Or, consider minimalist grading (hi, Peter Elbow), specifications grading (hi, Linda Nilson) or ungrading (hi, Jesse Stommel).

Lower communication anxiety: Explicitly tell students that you are open to hearing about the challenges they face, and give them multiple avenues for making contact if things don't go as planned. Actually say that you are willing to accept late work, offer extensions, think of accepting revised work -- whatever you are willing to do, say it, so that students do not feel that you are unapproachable and they have no choice but to "make the grade" any way they can.

This can be a hard conversation to have with faculty colleagues. They know the statistic that more than two-thirds of students have cheated on an activity at some point during their studies. But they don't know that this ratio measures whether everyone has ever acted dishonestly. Don't tell my alma mater, but I used a not-allowed answer key for a test when I was a sophomore back in the Mesozoic Era. I ran out of time for studying, and the exam was worth 40 percent of the final grade. Heck yeah, I cheated on that one.

The real number to think about is that in your specific course, in this specific semester, we're really talking about 1 to 2 percent of interactions seeing dishonest conduct. The vast majority of students, when we tell them what honest work and study look like, will do as we ask them to do.

That's not to say that we shouldn't enact academic integrity penalties and policies, but saying no has to be accompanied by what yes looks like, too. Here is a white paper to help with your approach during COVID-19 times (PDF, estimated reading time, 15 minutes); it was written in 2017, and the approach in it remains a useful path for me, my students and my colleagues.
Thomas J. Tobin is the program area director for distance teaching and learning on the Learning Design, Development & Innovation (LDDI) team at the University of Wisconsin at Madison, as well as an internationally recognized speaker and author on topics related to quality in technology-enhanced education, especially copyright, evaluation of teaching practice, academic integrity and accessibility/universal design for learning.

# data[].attributes.attachment_link.title
Student Agency in Uncertain Times | University of Venus

# data[].attributes.attachment_link.description
As we've moved largely to online delivery for our courses, there has been an increased concern around academic integrity and cheating -- students, the narrative goes, will take advantage of this move online to cheat when they might otherwise not. We must, common wisdom tells us, be ever-vigilant in the name of rigor.

# data[].attributes.attachment_link.description
In 2014, the Bloomsbury Learning Exchange (BLE) Consortium initiated a wide-ranging, two-year-long research and dissemination project focusing on the use of technology in assessment and feedback. Our...

# data[].attributes.headline
Ensuring academic integrity and assessment security with redesigned online assessment

# data[].attributes.subject
Ensuring academic integrity and assessment security with redesigned online assessment

# data[].attributes.body
This is an excellent guide prepared by the team at Deakin to help work through the considerations in transferring assessment online.

# data[].attributes.attachment_link.readable_text
Some assessments can easily move to online delivery or submission some assessment will need to be redesigned to ensure that quality and rigour are maintained.

1. Academic integrity, assessment security and digital assessment - outlines the latest research on how to keep digital assessment tasks secure.

2. Three key questions to guide redesigning exams for online delivery - provides prompts to help unit chairs work through design decisions

3. Redesigning exams: decision helper - provides a decision tree to assist unit chairs move exams to new formats

4. Examples of converting single answer correct/MCQ questions to a unique answer format

5. Tips for moving practical exams or assessments to online equivalents

6. Tips for moving complex unique response exams (eg essay style) to online equivalents

The CRADLE team have conducted a range of projects on assessment security and academic integrity, including work on detecting contract cheating (including a CRADLE Suggests resource), the quality of contract cheated work, and the security of online examinations.

# data[].attributes.attachment_link.title
Ensuring academic integrity and assessment security with redesigned online delivery | DTeach

# data[].attributes.attachment_link.description
Some assessments can easily move to online delivery or submission some assessment will need to be redesigned to ensure that quality and rigour are maintained. Academic integrity, assessment security and digital assessment - outlines the latest research on how to keep digital assessment tasks secure.

# data[].attributes.body
Start with the Student in the pivot online – rattus scholasticus

# data[].attributes.attachment_link.description
Flowchart to help instructors when an exam needs to move online

# data[].attributes.body
5 Quick ideas for supporting assessment at a distance – lydiaarnold.net

# data[].attributes.body
Guide to writing an effective open book exam

# data[].attributes.body
Assessment | Centre for Innovation in Learning and Teaching

# data[].attributes.attachment_link.description
This section of the Key Issues contains answers questions related to assessment in blended or online learning, such as how to combat plagiarism, how to proctor online tests and examinations, and how to use Learning Management Systems for formative assessment.

# data[].attributes.body
Bibliography | Types of Assessment Task | UTA | Manchester Metropolitan University

# data[].attributes.attachment_link.readable_text
Take-home Exam

What is it?

A take-home examination is a time-constrained assessment task, presented to students at some time in advance of the assessment and completed at home. They are sometimes used to reduce student anxiety about exams and to eliminate disadvantage for some students.

The examination paper is handed out and then the completed scripts are returned within a specified period of time. This could be anything from a day to two weeks.

The traditional format of the examination could be changed to release questions in the form of short tasks (to test a range of knowledge, as an exam would), to be completed over short fixed periods, rather than releasing all of the questions at once for a single submission. (eg one question every two days, with submission followed by release of the next question).
Recommended levels:
3- 7
Suitable for groups?
No

Things to think about

Students will of course be able to discuss the question in advance of the exam, and this may lead to some similarity of answers, just as you might find for a coursework assessment. You could have a pool of questions which are allocated randomly to different students, if you want to reduce this possibility.

Students who need help with typing or transcribing their work will need to ensure that they have appropriate support available during the time period set for the exam, or you could consider accepting submissions from them in a different format (eg audio).

Assignment length

The usual rule of thumb for an unseen exam question is that students should be able to sit and write the answer in half the time available for the question (the other half of the time being for thinking and planning). In a two-hour exam, students might be able to write 1500 - 2000 words in total but this is very variable depending on discipline and level. With a take-home exam, you might expect students to spend more than two hours on it, so you might increase your expectations of the length. This would clearly have an effect on the time needed to mark the exams, which needs to be factored in.

Group work

Not applicable.

Plagiarism Issues

Submissions could be made through Turnitin to reduce this possibility.
" list of assessment tasks

# data[].attributes.attachment_link.description
A take-home examination is a time-constrained assessment task, presented to students at some time in advance of the assessment and completed at home. They are sometimes used to reduce student anxiety about exams and to eliminate disadvantage for some students. The examination paper is handed out and then the completed scripts are returned within a specified period of time.

# data[].attributes.attachment_link.provider_name
Teaching perspectives from the Business School

# data[].attributes.attachment_link.readable_text
By Susan Smith
Associate Dean (Education and Students)

The spread of the Covid-19 virus across the globe and the imminent move to online delivery of teaching and learning is presenting us with the additional challenge of moving our assessments online at short notice.

This is something that we had not anticipated or designed into our course and module delivery and which requires careful consideration. In addition, we must be mindful that changes to the anticipated assessment modes are likely to increase student anxiety particularly where students are unfamiliar with the format or rubric against which they will be marked. We must be clear and consistent in our module level communications regarding any changes to the assessments.

Some modes of assessment, which typically require a physical presence, have a more obvious adaptation to online assessment e.g. presentations, whilst others can require more creativity to meet the learning outcomes e.g. unseen exams.

What are the considerations?

Meeting the learning outcomes of the module, albeit in a different way.

Whether a switch on an accredited course or module carrying professional exemptions will be accepted by the professional body (positive confirmation should be sought prior to communicating proposed changes).

The alternative assessment should require an equivalent amount of effort from the student. The School's assessment equivalences document may help guide this decision.

Design the assessment to mitigate the risks. Some pros and cons of various assessment modes are highlighted on the Summative Modes of Assessment presentation on the School's Teaching and Learning web pages.

e.g. a take away paper with a very limited window may replicate near exam conditions but we should be mindful of variations in students' quality of connection to the internet, time zone differences and accommodating reasonable adjustments. The desire to create time constrained situations to reduce academic integrity concerns should be balanced against redesigning the nature of the tasks to focus on application and extending the time window. This is an already stressful time for our students.

Moderation of revised assessments.

Some students may have challenges engaging with online assessment. We will put in place alternative arrangements as required.

Communicating changes

Communication to students should be clear and transparent regarding the:

Assessment mode e.g. you will now be assessed by a report rather than an unseen exam.

Rubric or marking scheme that you intend to use.

Please ensure that alternative assessments are approved, and set up by the CAO team prior to communicating to students. All assessments must be submitted via Turnitin or Canvas and not directly to faculty emails. In the event that an electronic submission point cannot be created, submission will be arranged via a dedicated CAO email box.

Offer students the opportunity to practice the assessment mode if they have not come across it previously so that they understand what is required and how you will apply the rubric or marking guide.

Show them exemplars of what you're looking for, preferably at different levels of achievement.

Resources

The Business School teaching and learning web pages feature information on the various modes of assessment along with an assessment equivalence guide as well as other information around assessment and feedback.

A useful document to help evaluate the various alternatives to in-person assessment by Sally Brown and Kay Sambell is available as a download from Sally's website.

Ideas around writing effective take-home exams.

If you wish to share useful resources or top tips, this Padlet holds a collection of resources sourced by Business School faculty.

# data[].attributes.attachment_link.description
By Susan Smith Associate Dean (Education and Students) The spread of the Covid-19 virus across the globe and the imminent move to online delivery of teaching and learning is presenting us with the additional challenge of moving our assessments online at short notice.

# data[].attributes.attachment_link.provider_name
National Forum for the Enhancement of Teaching and Learning in Higher Education

# data[].attributes.attachment_link.title
10 Points to Consider in Choosing Alternative Assessment Methods for the Online Environment - National Forum for the Enhancement of Teaching and Learning in Higher Education

# data[].attributes.attachment_link.description
For COVID-19 Coronavirus Contingency Planning Students can learn through many different but equitable assessment methods. An alternative online assessment is not a lesser form of assessment, but a different assessment to achieve the same aim.

# data[].attributes.body
Make it something relatable and to do with the students experience if possible

# data[].attributes.attachment_link.readable_text
Does anyone really know the definition of a "good" assessment? Does such a thing even exist?

The second question has no clear answer (which means it's not likely to show up on an assessment any time soon). As for the first, the jury's probably still out, but Dianne Conrad and Jason Openo are closer than most. Openo serves as director of the Centre for Innovation and Teaching Excellence at Medicine Hat College in Alberta, Canada, and Dianne Conrad teaches in Athabasca University's Center for Distance Education. Conrad has long wanted to write on this topic, and in early 2016 she convinced Openo, then a doctoral student with similar perspectives on assessment, to join her. The final product is Assessment Strategies for Online Learning (Athabasca University Press), which came out this summer.

"Inside Digital Learning" asked Openo and Conrad to share their thoughts on the state of assessment and the potential for its evolution. Their comments have been lightly edited for length and clarity.
Q: Did you find in the process of researching the book any particularly notable or innovative examples of online assessment?
Conrad: In short, no, although [other academics quoted] in the Appendix shared ideas and techniques that we both use and support. In fact, in Chapter 7 on authentic assessment, we searched out examples of assessments from online courses and demonstrated how they could be better structured to reflect authenticity and use, to [take] advantage [of] the affordances of online. Online offers opportunities to introduce exciting assessment potential, and that is often not taken up by instructors who are not well versed in how to manage online teaching.

Openo: I disagree with Dianne a little bit here, but the crux of the answer rests on how you define notable or innovative. I think our reflections from the field give a good indication of conscious alignment between instructional beliefs and instructional practice. Ellen Rose talks about her challenges with online discussions and her introduction of self-assessment into the process. Terry Anderson talks about using voice marking in PDFs to provide more personalized and humane feedback to students. This can also be achieved with the Marco Polo Walkie Talkie app that I am planning to use in my next course. Archie Zariski [at Athabasca] is using a technological tool, Audacity, to support oral assignments, whereas Beth Perry [at Athabasca] is using a pedagogical strategy, negotiated assessment, to work one on one with her students.

Getting students involved in the creation of open educational resources, eportfolios and using social networks for educational purposes are all notable and innovative. But the most innovative part is the intentional, creative way that instructors are using technology to support their pedagogical beliefs. And what I hope I see in this collection of reflections from the field, and the conversations I have had since with faculty members, is a true paradigm shift in assessment practice to more authentic assessments that ask students to bring their whole selves to work on relevant problems.
Q: What are the most pressing concerns you hear from faculty members regarding assessment?
Openo: The most pressing concern I hear from faculty is the need for professional development in the area of assessment. Behind this is a pressing concern for quality, equity and fairness. Assessment has consistently been one of the most important needs identified in several faculty development surveys, and despite the complexity involved in designing and conducting research, faculty development has a cumulative impact on teaching, including improved assignments. Dianne suggests that if an instructor has a good grasp of teaching techniques, assessment shouldn't be difficult. But technological, pedagogical content knowledge represents a pretty phenomenal and underappreciated skill set.

Here it's necessary to expand this discussion to include contingent faculty. For many, online education represents a cost-cutting tool and a way to undercut existing faculty. As Adrianna Kezar has written, faculty development can go a long way to eliminating the worst aspects of the growing adjunct situation, but it cannot resolve the long-term structural issues, and this needs to be important to higher education administrators truly concerned about online educational quality. In addition to the human costs involved when working in a state of precarity, there are costs to educational quality as well.

In a large study at Washington State of students who took online courses, all student groups showed learning gaps, and when Mueller, Mandernach and Sanderson in 2013 looked at different persistence rates between students learning online taught by full-time and part-time instructors, they found that students learning from full-time instructors were more likely to complete the course. They also suggest that adjunct faculty may grade more leniently due to perceptions of job insecurity. Part-time faculty not engaging in learner-centered assessment practices or grading more leniently out of concern for their jobs is supported by other researchers, and the issues faced by contingent faculty may be exacerbated by online instructors. If part-time adjuncts are indispensable but invisible, then online contingent faculty are doubly invisible because they work away from the brick-and-mortar institution.

Faculty development appears to be a key component in successful online education; good assessment practice has to be a big part of that faculty development curriculum, and providing those faculty development opportunities must account for the unique working conditions online faculty may face. A lot more research is needed in this area.

Conrad: In my current role, I don't hear much about this from faculty. As a matter of fact, I have not heard faculty ever talk much about assessment in all my years of teaching!
Q. What technology tools could be most helpful for online faculty members looking for a different approach to assessment?
Conrad: Assuming a web platform, such as Moodle, the field is wide-open to various technologies, but what must come first is a sense of how the technology will enhance the pedagogy being used. So, pedagogy first; technology second. In our field bells and whistles dominated in the early years. We understand now, better than before, that indeed technologies are tools to be applied mindfully and appropriately. That said, teachers looking for ways to integrate technology via social media into assignments and assessment have almost unlimited choices: Facebook, Twitter, wikis and blogs can support group meetings among learners and text. There are so many new animation programs that learners can draw on (literally): GoAnimate, Powtoon, Prezi ... I learn more every time I assign a media-based project. The issue here for teachers is to get away from the notion of assessment as a Word document. Introduce project-based assessment that requires media. Let the learners choose their medium (or provide a range of choices). My experience, at the graduate level, is that they love this. The creativity manifested is astounding. But don't lose sight of the academic nature of the assignment.

Openo: There is a saying that the best tool is the one that people will use. Don't let the learning management system be a walled garden that constrains you. Assemble a range of sites, apps and networks that are interoperable or compatible with the LMS to achieve specific pedagogical purposes that encourage agency and expression. In online contexts, engagement is absolutely essential, and these tools need to create deeper and more meaningful human interaction. One of the interesting uses of Twitter, exemplified by Real Tweets From WWII, is how to use Twitter to teach history and creative writing. There are a panoply of applications, tools and communities that can extend the digital learning environment, and all of these can become evidence for assessment. Dianne's point about creativity is crucial. Student products are the greatest proof of educational quality, and the level of creativity allowed by technology is something instructors consistently talk about.
Q: What conclusions did you draw about the most effective techniques for assessing students' learning?
Openo: Most of my favorite lines from the book are ones that Dianne wrote, including the line that positivist approaches to education that objectified learning are giving way to constructivist views. We did not draw conclusions about the "most effective" techniques. We have advocated for a greater embrace for pedagogies of engagement, and we have covered a lot of territory for how to think about that and accomplish it.

Here's my big conclusion -- assessment sucks. This was well expressed in another of my favorite rants on the subject of assessment. I don't actually think it's much of a secret anymore that most instructors would describe assessing their students as a necessary evil. There has to be a better way, and that was the motivation for writing the book. How can we make this process more meaningful to our students and to us as instructors? How can we fully achieve the potential online learning represents? In online learning, assessment plays a key role in moving from low-end elearning to high-end elearning (Duus, 2009). Low-end elearning is characterized by content transfer, where internet-based communications technology is used for the transfer of knowledge -- or mode of delivery, as in pizza delivery by drone. High-end elearning requires the creation of new knowledge through the creative use of technology -- or delivery as liberation. Internet-based communications technology are best employed when they maximize student engagement in knowledge creation and liberate them from time, distance and assessment constraints. Maybe not most effective, but hopefully more effective (or at least more engaging).
Q: What are the end goals of better assessments? How can faculty members harness the results from more effective assessments to improve their teaching?
Conrad: The end goal of better assessment is better learning. Using the assessment tool as a learning tool, rather than a "jump-through-the-hoop" activity or a measurement exercise, can enhance the learning experience. Growth and learning can occur through the assessment activity -- rather than it just serving as a regurgitation or rehash of already learned material. In my master's program at a major research university, I had only one exam to write. The class objected strenuously, but the prof was set in his ways. So we sat down to write a three-hour totally regurgitative exam. I was so exhausted from writing booklets and booklets of material that I eventually gave up and left. Apparently, I had "thrown up" enough on the pages, because I did well in the course. This activity was not productive.

Openo: The end goal of better assessments is better learning, and better learning about teaching in online contexts. Faculty members can harness the results from experimental assessments through discipline-based education research, the Scholarship of Teaching and Learning (SoTL) and the Scholarship of Technology-Enhanced Learning (SoTEL). Rightly or wrongly, there is increased emphasis on evidence-based teaching practice. By turning toward these forms of scholarship, one turns their teaching into "community property," as Shulman once said. There is so much we still don't know about the complexities of teaching and learning with technology, optimal course designs, and it's a rapidly evolving field.

What I find most interesting about the impact of teaching online is that we believe teaching online is causing a "pedagogical renaissance" (we won't say revolution). The proof of this is that seven in 10 faculty members who have taught an online course say the experience helped them improve their teaching both online and in the classroom. Teaching online is deepening our understanding of teaching in general, and we can harness this through any number of research approaches, like action research, design-based research and course-based analytics.
Q: Why do you think assessment is one of the most challenging aspects of the academic experiences to navigate in an online context?
Conrad: Assessment is always one of the most challenging aspects of academia regardless of delivery mode. It's often the "tack-on" in the course design, especially if the course design is being executed by a content expert with no educational training (which often happens). Once we move into the online context, what we often see is a holus-bolus import of the face-to-face version of the course to the online medium. This, as a basic premise, is a bad idea for many reasons, but the assessment piece will also suffer because (regardless of its initial quality) it has not taken advantage of the opportunity to become something better.

Online assessment, assuming the teacher's firm grasp of good online teaching techniques, should not be difficult at all; online media offer so many opportunities to create interactive, authentic assessments that engage learners in the continuation of their course's learning journey rather than presenting the assignment(s) as barriers or hoops to moving forward in the course or completing the course.

The book offers suggestions as to the creation of authentic online assessments, not in a "how-to, here's-a-list" manner but in a solid pedagogical manner: "here's what learning is, here is how authentic assessment fits in."

Openo: I largely agree with Dianne, but I'd like to add a couple observations. If we transport all of the traditional challenges involved in assessment to an online setting, it is only going to make chronic challenges more difficult. We see this with online testing and lock-down browsers, and the International Center for Academic Integrity's day against contract cheating [was] on Oct. 17. It's a technological arms race to stop cheating for the most common forms of assessment, exams and term papers. It's going to be a cat and mouse game unless we change the game.

The biggest challenge, for me, is disaggregating assessment as learning and assessment for learning from assessment of learning. As long as grades exist, students will bend towards focusing more on the letter than the learning, and these structural limitations are intrinsically problematic. I enjoyed Dan Houck's recent piece on his personal refusal to be part of this hopelessly reductive process, which is why we spend so much time in the opening chapters differentiating between assessment and marking. Houck expresses well the philosophical opposition to grading, and it's hard but necessary to separate grading from that process. Within those structural limitations, however, we still think there is some improvement and progress that can be made.

As Dianne has said, assessment is frequently the last thing instructors want to think about. It's subjective and inconsistent, and some of the most important higher-order skills, such as critical thinking, are hard to assess. It's an exhausting process involving superhuman amounts of time, energy and effort. Just look at most faculty members at the end of a semester: assessment is the hardest work of teaching. Giving meaningful feedback to help students learn is hard and time-consuming, and students will always want and could always benefit from more feedback, and figuring out how to do this without exhausting yourself is a challenge.

When the online dimension is added on top of all this, it can make everything more difficult. That is why we think moving online is time for a reset and asking fundamental questions. What are we assessing? Why? How can student work provide the best evidence that learning has taken place? What tools are available that offer new alternatives? I love what Zawacki-Richter and Anderson (2014) say about teaching online: online instructors "bring many of the fears, inhibitions, and bewilderment of students when first exposed to the very different context of teaching in mediated and networked contexts." Teaching online often puts experts in an uncomfortable place, and when that happens, they may not be in a place to be creative.
Q: Is there anything else from the book or your research that you want to emphasize?
Conrad: We are pleased with the book and its progression from basic beliefs through to application of authentic and engaging types of assessment. We wrote it that way to emphasize the importance of a sound philosophical and pedagogical foundation for one's teaching practice. Once we understand what we believe in as far as what our purpose as teachers is, we can search for constructive ways to communicate that end goal to learners. Assessments can form a part of that strategy, when constructed to encourage authentic learning. The notion of "authenticity" is also very important, and the book dwells on that extensively. Authentic assessment, in part, prevents the learners from spitting out rehashed materials as it encourages a practical understanding and appreciation of real-life tasks or problems that learners can relate to. Authentic assessment invites (forces?) learner engagement with the task or question at hand. These principles are based on adult education principles -- and these foundational ideas are also outlined in the book.

Openo: One of the things we openly wonder about is how long we would write a book about assessment in "online learning contexts"? Could there ever be a second edition with this title, recognizing the great convergence between online, blended and face-to-face? Online learning is rapidly expanding. It's not new anymore. It's getting better, and as Hicks observes, we have reached a point in time where all faculty need to have a level of competence with online learning technologies. It's not optional, and it's very dynamic. So how will the terminology, which has always been problematic, continue to shift, merge and emerge?

Bates suggests that the terminology struggles to keep up with what's happening, and that's true. How long will these problematic terms remain with us, and will the language change before practices change? For a lot of personal reasons, I am reluctant to use the word "disrupt," and we avoided hype in the book, but when/will/how new forms of credentialing and badging disrupt the current environment and bring more transparent evidence of student learning? It's cliché, but writing the book raised more questions than answers, and for those who have been unhappy and dissatisfied with the way assessments have been done in postsecondary education, I think inventing new approaches is a very exciting area to be working in.

# data[].attributes.attachment_link.title
Q&A: Strategies for better assessments in online learning

# data[].attributes.attachment_link.description
Does anyone really know the definition of a "good" assessment? Does such a thing even exist? The second question has no clear answer (which means it's not likely to show up on an assessment any time soon). As for the first, the jury's probably still out, but Dianne Conrad and Jason Openo are closer than most.

# data[].attributes.headline
JISC guide to ensuring continuity of service

# data[].attributes.subject
JISC guide to ensuring continuity of service

# data[].attributes.attachment_link.readable_text
In an increasingly uncertain world, it pays for universities and colleges to give serious consideration to preparing for circumstances where either staff or learners (or both) are unable to physically spend time on campus. This extends to those providers of transnational education, where restrictions on mobility or other concerns may encourage the use of blended and online learning.

Wherever you are on your journey towards digital transformation, there will still be some practical, 'common sense' steps that you can take to ensure you are able to provide support and a meaningful level of teaching and learning, regardless of where your staff or students may find themselves at little or no notice.

This guidance is for the planning of UK delivery of operations; further country specific guidance pertaining to transnational education delivery is currently being developed. Please contact esther.wilkinson@jisc.ac.uk or transnational@jisc.ac.uk in relation to this aspect of the guidance.

It is important that this guidance is seen as an evolving document; we would welcome your feedback and further discussion. Please contact steve.bailey@jisc.ac.uk.

Summary of key considerations

How online resources can ensure continuity

Ask all teaching staff to ensure they can access and know the basics of how to use the

Ensure all teaching staff have access to support for using the

Ensure the accessibility of all content you provide to learners online

Make the most of your Jisc digital content subscriptions

Lecture capture

Assess and promote existing platforms for broadcasting and/or capturing lectures

Ensure your policies allow you to extend the scope of your lecture capture activity or take necessary extraordinary measures to give you the ability to do so

Consider what hardware staff might need to effectively capture or broadcast lecture content

Assessment and feedback

Ensure you offer clear and supportive messages to students, reassuring them that mitigating circumstances will be taken into account and that it is highly unlikely that they will be penalised for circumstances outside of their control when it comes to assessment

Ensure your is enabled to manage online assessment

Support teaching staff to create robust, meaningful and accessible online assessments

Induction

Depending on the time of year, you may need to start planning how you can replicate key aspects of your regular summer induction programme remotely

Student wellbeing

Think about the platforms and communication channels you already have and how they can be used to broadcast information to either all or specific groups of students

Don't rely on a single communication channel

Think about how messages are going to be approved and sent, and by whom, to ensure a consistency of information

Pay particular attention to vulnerable students and what additional contact needs they may have

Consider creating a helpdesk to pull information together and provide a means for students to speak to someone

Keeping in contact with staff

Don't forget that your staff will also have their own needs and anxieties, which may overlap with those faced by your students, but which will require separate consideration and treatment

Consider using platforms such as Yammer to encourage informal communication between staff to create a continued sense of community and connectivity

Effective remote working for staff

Encourage staff to make their home environment as conducive as possible to effective remote working. Depending on the duration of the remote working it may be necessary to conduct a DSE assessment on their working environment

Start with any existing disaster recovery or business continuity plans to help identify core systems and vital information and records

Use the collaboration features on platforms such as Office365 or Google G-Suite to enable teams and individuals to plan and work effectively while working remotely

Encourage regular use of video alongside audio and chat collaboration to enhance presence and support team spirit

Ensure you provide appropriate support and guidance for such platforms so as not to exclude less confident users

Encourage a management culture which is built around trust, flexibility and outputs, rather than one which leaves staff feeling the need to account for every minute of their working day

Schedule regular team meetings and one-to-ones to ensure the wellbeing of your staff, and that priorities are clear

Post-incident closure and review

Consider the criteria for ending the incident and who will make that decision

Consider a phased return to normal operations, rather than assuming an immediate and blanket reversion

Review your response to the incident, looking for lessons that can be learnt to improve future planning and incident response

Review any 'emergency' measures undertaken during the incident, looking to ensure a managed return to normal operations

# data[].attributes.attachment_link.title
Ensuring continuity of learning during enforced absence

# data[].attributes.attachment_link.description
Practical tips to maintain teaching, learning and business operations during circumstances where staff or learners are unable to physically spend time on campus.