# data[].attributes.headline
See the maturity model section for knowledge development as an example

# data[].attributes.subject
See the maturity model section for knowledge development as an example

# data[].attributes.body
And the associated action plan https://docs.google.com/document/d/1uYdHaRovMxNxonSiUiUZNV2iwzXmQ6dx4w9ll6FPeJk/edit?usp=sharing

# data[].attributes.attachment_link.description
Create a new document and edit with others at the same time -- from your computer, phone or tablet. Get stuff done with or without an internet connection. Use Docs to edit Word files. Free from Google.

# data[].attributes.headline
Principles of good assessment and feedback

# data[].attributes.subject
Principles of good assessment and feedback

# data[].attributes.body
This guide has excellent case studies and links to podcasts from universities who are implementing innovative practice in assessment and feedback

# data[].attributes.attachment_link.readable_text
Why review assessment practice?

Introducing the seven principles

Principle one - help learners understand what good looks like

Principle two - support the personalised needs of learners

Principle three - foster active learning

Principle four - develop autonomous learners

Principle five - manage staff and learner workload effectively

Principle six - foster a motivated learning community

Principle seven - promote learner employability

Why review assessment practice?

The pandemic accelerated digital transformation of assessment for some institutions and caused others to question not only how they do things but whether they are doing the right things?

Assessment and feedback forms a significant element of staff and student workload and many studies have shown that students, in higher education, are less satisfied with assessment and feedback than with any other aspect of the experience.

This guide will help you determine what good assessment and feedback looks like in your context so you can benchmark existing practice and plan improvement.

Assessment and feedback landscape

Through consultation with universities, a survey and a review of the current literature we gained a picture of the UK assessment and feedback landscape in higher education. View Jisc's assessment and feedback higher education landscape review.

About the examples in this guide

Each of the principles is accompanied by short case studies of implementation.

Where possible we have chosen examples where digital technology has been used to support and enhance good practice.

Previous research found good practice was often difficult to scale up because it required manual intervention or tools that were not interoperable. Our examples show innovative practice delivered at scale and using open standards to facilitate seamless integration with existing tools and administrative systems.

Assessment and feedback: direction of travel

For a number of years assessment and feedback practice has been on a trajectory away from assessment of learning to what is termed assessment for learning.

Key to this has been helping students monitor and regulate their own learning and trying to ensure that any feedback activity feeds forward leading to future improvement.

Current assessment practice increasingly includes activities that could be termed assessment as learning. The very act of undertaking assessment and feedback activities is an essential part of the learning process.

All three aspects of assessment still need to happen but we are thinking differently about the relationship between them.

Assessment of learning describes the institutional quality assurance processes that lead to students acquiring some form of verified credential.

Assessment for learning is the overall learning design, ensuring we are assessing the right things at the right time with plenty of formative opportunities to feed forward. This is the cog wheel making everything revolve.

Assessment as learning is a learning experience where the formative and summative elements work well together. Tasks appear relevant, students can see what they have gained by undertaking the activity, they feel involved in a dialogue about standards and evidence and the continuous development approach helps with issues of stress and workload for staff and students.

Text description for assessment with purpose graphic

Purple centre - the inner circle represents assessment for learning the institutional quality-assured processes that lead to a qualification.

The middle cog represents assessment for learning learning design emphasising formative opportunities that feed forward to future improvement.

The outer circle represents assessment as learning the lived experience of students and staff when active learners contribute to decision-making and are able to monitor and regulate their own learning.

Beyond the Technology podcast

Listen to our series on rethinking assessment and feedback from the Beyond the Technology podcast:

Introducing the seven principles

Educational principles are a way of summarising your shared educational values as a college or university. They serve to guide the design of learning teaching and assessment.

A well-thought-out set of principles:

Describes a shared set of values and a vision

Summarises and simplifies a lot of research evidence on good pedagogic practice

Provides a benchmark for monitoring progress

Serves as a driver for change

Principles offer a robust way of gaining ownership and buy-in and they need to be written in a way that requires action rather than passive acceptance.

Our seven principles

1. Help learners understand what good looks like by engaging learners with the requirements and performance criteria for each task

2. Support the personalised needs of learners by being accessible, inclusive and compassionate

3. Foster active learning by recognising that engagement with learning resources, peers and tutors can all offer opportunities for formative development

4. Develop autonomous learners by encouraging self-generated feedback, self-regulation, reflection, dialogue and peer review

5. Manage staff and learner workload effectively by having the right assessment, at the right time, supported by efficient business processes

6. Foster a motivated learning community by involving students in decision-making and supporting staff to critique and develop their own practice

7. Promote learner employability by assessing authentic tasks and promoting ethical conduct

Applying the principles

The principles offer an actionable way to improve learning teaching and assessment and can be applied to any aspect of learning design.

There is no one size fits all approach. You need to decide what the principle means in your context and how best to apply it to the learning experience in your organisation. Our self-assessment template (.docx) allows you to adapt the principles to your own context. You can also download the self-assessment template as a pdf.

Do principles change over time?

Principles need to evolve over time if they are to reflect current education research and remain aspirational and a call to action.

The usefulness of educational principles drew widespread attention when Professor David Nicol published a set of principles for good assessment feedback practice as part of the Re-Engineering Assessment Practices (REAP) project in 2007.

Many education providers saw the value and adopted the idea so there are various sets of principles in the public domain ( see ESCAPE assessment principles pdf). Almost all of them are based on the basic REAP principles.

Our 2021 principles reflect the greater prominence of issues such as accessibility and inclusivity in current thinking. Where we continue to champion examples of good practice that were recognised some time ago, it is with a new perspective on why and how certain approaches are more effective than others.

It is too early to judge how long this set of principles will last. At the moment they represent a call to action and a manifesto for change. We very much hope to find that in five years time they are no longer aspirational and require another refresh.

Related resources

Designing learning and assessment in a digital age - A primer created for university leaders exploring the current and future opportunities for digital assessment.

QAA Hallmarks of Success Playbook: Assessment in digital and blended pedagogy - A useful guide to develop a shared understanding of assessment and feedback practices that are relevant to and support teaching and learning. It examines what success looks like in a digital and blended environment, barriers that may be encountered and how they can be overcome. The playbook can be used to drive discussions about assessment as learning, feedback and feedforward practices, as well as considering quality enhancement, moderation and academic integrity.

Assessment ideas for an AI-enabled world (.pptx) - Codesigned with the sector, we've produced a set of postcards to provoke discussion and reflection around assessment approaches in higher education. Each idea postcard outlines the type of assessment task, key characteristics and appropriate formats.

Principle one - help learners understand what good looks like

Why is this important?

Students undertaking an assessment need to understand what is required of them and the criteria against which their performance will be judged.

This sounds self-evident but many learners find it difficult to derive this information from the available sources such as course and module handbooks. Even where criteria and grade descriptors are provided, they may be couched in academic language that requires some skill to interpret.

It helps to have a common template for assignment briefs, so that the essential information is presented, in plain English, in a consistent way for each assignment.

Feedback for learning: a comprehensive guide, an e-book by FeedbackFruits, gives useful guidance on designing rubrics that support active learning. It explains how to create a rubric that is relevant, unambiguous and serves as a bridge to future performance by avoiding traps that can result in a rubric becoming a task-focused checklist.

By implementing this principle you will, however, take this a step further and engage students in activities that allow them to create meaning from the criteria and engage in discussions about quality. You may ask them to rewrite criteria in their own words and compare their understanding with that of others. Later they may even define the criteria by which they think their work should be judged.

The role of feedback

The capacity to recognise and interpret feedback, and use this to lead to further improvement, is key to developing the ability to recognise what good looks like

Feedback provides information about where a learner is in relation to their learning goals so that they can evaluate progress, identify gaps or misconceptions and take action that results in enhanced performance in future.

Feedback should be constructive, specific, honest and supportive.

Effective feedback shouldn't only focus on current performance or be used to justify a grade. It should also feed forward in an actionable way so the learner understands what they need to focus on in order to improve.

Feedback is conventionally thought of as a dialogue between student and tutor but it can come from a variety of sources. Self and peer feedback can be equally valuable and we discuss this further in relation to principle number 4 develop autonomous learners.

Ipsative assessment

Ipsative assessment is the idea that, rather than evaluating a learner's performance against an external benchmark, we simply look at whether they have improved on their own previous performance. This is after all the most effective measure of learning gain.

At University College London this approach is proving useful with master's level students and those studying MOOCs.

To grade or not to grade?

Defining what good looks like does not have to mean assigning a grade.

There is a school of thought that believes grading is counter-productive and diminishes students' broader curiosity about a subject. Read more about upgrading in Jesse Stommel's article.

Diverse assessment formats and different customs and practices across disciplines may distort marks. Even without these complications, experts question whether it is possible to distinguish the quality of work with the precision implied by percentage marks.

Higher education does indeed operate without grading in some areas. We recognise the value of a doctorate without questioning whether the holder achieved a few percentage points more or less than one of their counterparts.

The absence of a grade can oblige learners to focus on their feedback and encourage deeper reflection.

Technology can help

Some of the ways technology can help:

Making information about the assessed task and criteria available in digital format helps ensure it can be accessed readily on a range of devices. It also helps with applying a consistent template

Having the information in digital format allows cross-referencing to a wide range of examples

Use of digital tools makes it easier to give 'in-line feedback' so students can see to which section of their work a feedback comment refers

Digital tools can support better dialogue around feedback

Storing feedback in digital format increases the likelihood that students will refer back to the feedback in future

Digital tools that support self, peer and group evaluation are all means of actively engaging students with criteria and the process of making academic judgement. You can find examples of this in the section about putting each of our principles into practice throughout this guide

Putting the principle into practice

Guidance and templates

Sheffield Hallam University makes all of its guidance and templates available via its assessment essentials website.

Manchester Metropolitan University (MMU) has developed guidance on assessment grading, criteria and marking (pdf). This helps ensure consistency across the organisation. MMU uses an app to make this information available to students via any mobile device.

The University of Reading's A-Z of assessment methods (pdf) can help you choose an appropriate type of assessment

Adaptive comparative judgement

Assessing a piece of work objectively against a rubric is not easy even for an experienced evaluator.

Comparative judgement works on the principle that people make better judgements if they compare two items, and decide which is better, than if they try to evaluate something in isolation.

Repeated comparison of pairs ultimately allows the items to be rank ordered. This usually takes nine to 12 rounds of comparison.

Adaptive comparative judgement (ACJ) tools automate the process of presenting a group of assessors with pairs to compare. The items are initially selected randomly. Comparison between very good and very bad is obvious very quickly. The computer algorithm can then start to select the pairs that will most improve the reliability of the ranking so more effort goes into assessing those that are closely matched.

Research has been undertaken using this technique to compare the evaluative judgement of staff and students. It has also been applied to using artificial intelligence (AI) to partly automate marking of complex items such as essay scripts.

The most common application is, however, in engaging students with assessment criteria and 'learning by evaluating' to identify what good looks like.

Using ACJ tools to provide peer evaluation and feedback can provide learners with a rich body of evidence to help them improve their work.

ACJ is being used at many universities internationally. In the UK examples include use at Goldsmiths, University of London and the University of Manchester.

The technique is explained in this presentation and session recording, which includes a case study from the University of Manchester.

Developing research skills at the University of Waterloo

Students undertaking an education degree at the University of Waterloo have the opportunity to gain a research credential in their specialist area during their work experience placement.

To gain the credential, the students have to assess their own research against a research skill development framework and demonstrate, during a capstone interview, the evidence why their experience merits special recognition.

The process is supported by a structured workbook in the e-portfolio that allows students to engage with the assessment framework before, during, and after their individual research experiences. The workbook acts as the basis for receiving personal guidance and feedback to support independent study.

"Engaging in this work reinforced how influential even a small degree of reflection and feedback can be in supporting student success. Implementing these changes required minimal additional effort but had huge impact on the students' ability to relate their experiences to the assessment framework."
Jeremy Steffler, University of Waterloo

Read more about this University of Waterloo case study.

Principle two - support the personalised needs of learners

There is no such thing as a standard learner. Disability, neurodiversity, cultural and religious background, work and family commitments and personal preference all play a part in shaping our experience of learning.

Moreover, these factors combine to make the learning experience individual. Disabled students are no more a homogenous group than learners that have been bundled together under the unhelpful label BAME (black and minority ethnic). In March 2021, the Commission on Race and Ethnic Disparities recommended that the UK government stop using the term BAME. The government is currently considering its response to the Commission's 24 recommendations.

Accessible and inclusive

Good assessment and feedback practice should allow a learner to apply their own individuality to demonstrating their knowledge and competencies.

We are increasingly aware of the need to make assessment accessible ie to ensure that people with disabilities do not face barriers because of the format or tools used. We need to promote similar awareness of the benefits of making it fully inclusive ie fair and equal to all students regardless of their diverse backgrounds.

Offering different means to demonstrate achievement of a learning outcome is an important way to comply with disability discrimination legislation. Applying this as a design principle goes further and enhances the overall assessment process to benefit all students.

Assessment designers need to be familiar with the law relating to accessibility and related guidelines such as WCAG (Web Content Accessibility Guidelines). For example, offering a digital paper as an alternative to hardcopy examination is only useful and legally compliant, if the digital copy meets accessibility compliance regulations. Software and resources used for digital assessment are legally required to meet the compliance standards. This needs to be considered in purchasing decisions and in-house development.

The emphasis on inclusivity relates to a broad agenda to ensure that our curricula are not skewed - for example to a particularly 'white' view of the world. There is much good work being undertaken on decolonising curricula and we need to ensure that assessment design reflects a global, multicultural perspective on the subject matter.

Compassionate pedagogy

Since the start of the pandemic there have been growing calls for us to develop more 'compassionate pedagogy'.

As online classes have taken us into one another's homes, teachers have gained a different kind of understanding of the conditions under which students study and the competing demands of work and family they have to juggle.

Digital poverty is only one aspect of this, time pressures and access to a suitable place to study are equally relevant and need to be factored into how we design learning and assessment.

Inclusivity means that no one's socio-economic background should be a barrier to learning.

Promoting mental health and wellbeing

Promoting mental wellbeing is high on the agenda amid survey evidence from higher education that both student and staff mental health has declined in recent years. Assessment points are often the trigger for stress and anxiety related illness.

An approach that allows for individual needs, and is founded on compassion for the learner, can help.

This is a topic that cuts across a number of our principles. Implementing principles number five: manage staff and learner workload effectively and number six: foster a motivated learning community will also have a positive impact.

Some of the ways technology can help

Using digital tools makes it easier to implement universal design for learning (UDL) guidelines by providing multiple channels and options for engagement with learning and assessment activities.

Digital technology offers a vast range of assistive tools that allow learners to respond to the same activity or content in a way that is adapted to their personal needs. Examples include screen readers and braille keyboards for blind students, video and audio captioning for learners with hearing problems, the ability to change font size and colour to support dyslexic students or certain types of visual impairment. Some locked down virtual desktops and third-party tools used for summative assessment do not permit the use of the assistive technologies required by some learners. This is an issue that should be fully investigated before using such tools.

Aside from these adaptations, taking a summative assessment in familiar surroundings using familiar digital tools, may prove less stressful than a traditional example setting for many learners. In some cases, formal examinations may be one of the rare occasions where a student uses a pen.

Artificial intelligence (AI) is enabling the development of 'adaptive' learning tools that respond dynamically to student responses in order to personalise learning pathways. Their main use is in formative testing. For example, a student can be presented with progressively more difficult questions based on what they answer correctly.

We are increasingly used to technology providing us with friendly reminders when we need to do something in many aspects of our lives. Digital tools provide various ways to give learners 'nudges' and reminders about upcoming deadlines and important due dates via email, text, mobile app or reminders in the VLE, helping to provide clarity and alleviate stress.

Putting the principle into practice

Making large classes feel small

A decade ago, the University of Sydney was unable to find a tool that could help solve the problem of engaging students with feedback and giving them personal care when cohorts could consist of thousands of students and learners felt lost in the crowd.

They developed a student relationship engagement system (SRES) which integrates with any learning management system using LTI and is now freely available to other universities. SRES is described as a 'Swiss army knife for instructors'.

Today over 3,000 teachers are using SRES across more than a thousand courses at the University of Sydney. There were over a million student interactions with the portal in 2021 and 92.4% rated it as helpful. This is enhancing the learning experience of almost all 75,000 students at the University of Sydney with many more users worldwide.

Text description for elements of the student relationship engagement system (SRES) approach diagram

Top row title is evaluative judgement with self assessment, peer feedback, teacher feedback and rubrics underneath

Second row title is developing feedback literacy with appreciating feedback, making judgements, managing affect and taking action underneath

Beneath these two rows is listed positive relationships, which results from the factors in the other boxes being implemented

This diagram shows the elements that are important in contemporary approaches to helping students get the most out of assessment and feedback. Most importantly, they all exist within a wrapper of positive relationships with the tutor and others in the learning community.

SRES allows tutors to curated data from various sources in tailored dashboards that tell them what they need to know in order to best support their particular learners.

Data about logins and grades can be accompanied by other information that helps the instructor get to know the students. This could involve asking students about their preferences, what helps them learn and even their dreams and ambitions. Another example is that students can record an audio clip of how to pronounce their name.

These questions are embedded in the learning management system (using LTI) so to the learner they appear seamlessly alongside an introductory course in module information.

Highly personalised feedback can be created within the context of data such as whether the student has submitted on time or how they say they are feeling.

SRES also helps turn feedback into a dialogic process. Before submitting an assignment, students can say what they would most like feedback on, so they are primed to be more likely to engage with that feedback. Tutors can also close the feedback loop by asking students to respond and say what they thought about the feedback and what they will do differently next time. It is also possible to hide the grade until the student has responded to the feedback.

SRES also supports a wide range of peer-to-peer feedback activities.

Another use is to facilitate learning from both self and peer feedback. The portal can present results of student self-assessment alongside the evaluation given by their peers encouraging them to reflect on how this makes them feel and how they plan to improve on the next task as a result.

Closing the attainment gap at Brunel University

Brunel University implemented digital open book assessments as a response to the pandemic and discovered that this helped close the attainment gap for certain students. Students who entered the University with BTEC qualifications, particularly black students with a BTEC, did significantly better in terms of degree outcome than in previous years.

This improvement can be attributed to the change in assessment practice with the open book format being similar to their prior experience. Closed book examinations tend to benefit students with A-level qualifications who are used to being assessed in this way.

"Instead of sitting in a sports hall for three hours and having to rely on memory, students were able to use any resources available to complete the task - something that is probably more akin to what those with BTECs would have done previously."
Mariann Rand-Weaver, Brunel University

Read the full case study in Jisc's rethinking assessment report.

Supporting remote placement students

At Murdoch University hundreds of nursing students each year undertake a rural or remote clinical placement often up to 3000 km from home.

This placement may be the first time that the student is away from family, friends and normal support structures for an extended period. The staff member monitoring the student's progress is rarely in the same location as the student and internet connectivity cannot always be relied upon.

The University uses its PebblePad e-portfolio tool to help connect with, and support, the learners.

As a result of codesign discussions with students, compulsory reflective assessments about their clinical experiences, have moved from written format to video blogs. Students can record the vlog anywhere and upload them to PebblePad later to be shared with on-site and remote assessors.

These 'vlogs' allow for individual creativity and personal connection as the tutors are now using the vlog format to provide feedback to the learners.

Beyond the Technology podcast

In our podcast on rethinking assessment and feedback, Danny Liu and Benjamin Miller speak about how they are adapting their assessment and feedback practices at the University of Sydney and how their use of the student relationship engagement system (SRES) has helped to solve the problem of engaging a large number of students with personalised feedback.

Principle three - foster active learning

Active learning has been a key tenet of curriculum design and learning space design for several years. Approaches to teaching have evolved in response to evidence that techniques such as lecturing, in contexts where students are passive recipients of information, are highly ineffective.

Building in formative assessment opportunities greatly enhances the effectiveness of learning activities.

The use of quizzes delivered via mobile devices during lectures, is a common example.

Active engagement with learning resources

We tend to think of active learning as something that involves interaction with a teacher or with peers but engagement with learning resources also offers opportunity for learners to evaluate their own understanding.

Interactive questions embedded in online textbooks is an emerging area that has considerable potential for learning enhancement.

Even in the absence of sophisticated digital tools, teachers can include discussion points for example by linking the prompts to a discussion thread in material on the learning platform.

Research by Professor David Nicol at the University of Glasgow suggests we have previously focused too much on feedback comments, and the dialogue around them, as the main mechanism by which students evaluate the quality of their own work. He believes that comparison with learning resources can be equally valuable in building student capacity to plan, evaluate, develop and regulate their own learning.

Time to revise

An area that receives little attention in learning design is how students prepare themselves for summative assessment. What revision strategies do they employ and what resources do they use?

The Open University, UK recognises this and carries out an annual survey on student experience of assessment, feedback and revision (SEFAR). It concludes that revision and examination represents a distinct phase of learning but it remains challenging to determine effective analytics for this phase that can help inform better learning design.

The most conclusive finding was that the use of sample papers for practice appears to improve exam question literacy. Conversely, lack of engagement with these resources could be a trigger that a student requires some targeted support.

Some of the ways technology can help

The use of quizzes delivered via mobile devices during lectures, is a well-known example of formative testing. It can be used to check whether most students have grasped key concepts and allow the lecturer to adapt the session if further explanation is required.

Interactive questions embedded in online textbooks is a more recent development. Research shows this type of practice question can significantly enhance learning compared to straightforward reading of text

Video resources can be very engaging but difficult to navigate. Digital tools that help students bookmark and add notes to particular sections, foster active learning and aid revision.

Putting the principle into practice
The Doer Effect
Research from Carnegie Mellon University suggests a causal relationship between students doing practice questions while reading and enhanced learning outcomes. This phenomenon, known as the ' doer effect ' has been replicated in very large-scale studies in other US universities. The studies show that formative practice enhances the learning effect by a factor of six compared to simple reading of the text.

The research findings are very clear. The main factor inhibiting embedding such practice questions in every online textbook is the work needed to generate the questions. A textbook for a semester long module may require hundreds or thousands of questions requiring both subject matter and question item authoring expertise.

Artificial intelligence (AI) is a promising solution to this problem. A research study (pdf) based on well over three quarters of a million student-question interactions showed no significant difference in the difficulty of, or student engagement with, questions written by subject matter experts and those generated using AI.
Interactive presentation
Universities using Feedback Fruits software have access to a tool known as interactive presentation. The tutor can upload video content, such as a pre-recorded lecture, and lock specific moments in the timeline with practice questions.

Students have to answer these questions before they can continue to watch the rest of the video.

The teacher has access to an analytics dashboard to see which questions students struggle with.
Synote
Synote is an award-winning, open source application developed at the University of Southampton that makes video resources easier to access, search, manage and exploit.

Imagine trying to use a textbook that has no contents page, index or page numbers. Lengthy video recordings, such as recorded lectures, can be equally difficult for students to navigate.

Synote allows students to bookmark particular sections of a recording and associate their notes with the correct section. Students can also take live notes during lectures, using Twitter, on any mobile device then upload them into Synote so they can be synchronised with a recording of the lecture.

Principle four - develop autonomous learners

It is often said that one reason students tend to express a relatively high level of satisfaction with lectures, is the fact that they don't need to do anything during a lecture.

The remark is not entirely tongue-in-cheek. Tutors frequently complain that students see themselves as passive recipients of learning content. Similarly, in relation to assessment, some learners view it as the tutor's role to deliver feedback to them.

Developing students' ability to self-regulate and manage their own learning is a key goal of effective learning and assessment design. We also touch on this in relation to principle number one help learners understand what good looks like and principle number three foster active learning.

Activities that involve reflection and self, peer and group evaluation all work towards this goal.

The power of comparison

Many studies have observed that students appear to learn more from generating feedback for their peers than they do from engaging with peer feedback comments provided for them.

Research by Professor David Nicol, at the University of Glasgow, suggests this is because when students review their peers' work, after producing their own, they make comparisons of the peer's work with their own and this activates powerful internal feedback. Such comparisons can generate valuable learning whether the work of the peer is stronger or weaker than the reviewer's own.

A study comparing peer and tutor feedback (Nicol and McCallum 2021) found that students of all abilities were able to identify all areas for improvement that the tutor identified as well as areas that the tutor did not mention. However, to match the tutor feedback, the students had to make multiple sequential comparisons (ie compare more than one peer work with their own).

This research suggests that well-structured peer review activities can reduce teacher workload and generate more and better feedback for learners.

Self-paced learning

Most educators recognise the importance of developing learners' capacity to self-regulate and the role that engagement with feedback plays in this. Our efforts in this area are however, sometimes at odds with a fixed regime of content delivery via lectures and a rigid assessment schedule. Inevitably, some learners struggle to keep up whilst others are insufficiently challenged.

Higher education could do more to encourage self-pacing within bounded limits, such as an individual module. Students could be allowed to test their knowledge when they feel ready and resubmit until they have mastered a topic.

There may be lessons to be learned in this area from the school sector. The modern classroom project suggests the following classification of learning and assessment activities:

Must do: non-negotiable tasks covering core concepts and essential skills

Should do: valuable opportunities to develop skills that will not prevent the student transitioning to the next stage of learning if there is a good reason for them to be excused

Aspire to do: extensions for students who have already mastered the normal scope of the topic

Analytics to track progress help ensure that the group as a whole is on track. Sharing of aspects of goal setting and tracking can help students identify others to collaborate with or who can provide help.

Policies such as 'ask three (peers) before me' can encourage peer learning.

Similarly, assignment briefs can be split in two with half of the group addressing each aspect of the topic then teaching and learning from peers who did the opposite assignment.

Some of the ways technology can help

Early attempts to implement peer review found it could be time consuming to administer. Digital tools make it possible to implement peer review activities at scale.

Features such as allocation of reviews, linking to assessment criteria, managing anonymity and tracking which students have completed the work, are all easier in the digital environment.

Use of the open standard LTI means that tools supporting self, peer and group review can be seamlessly integrated into the learning management system.

Putting the principle into practice

Peer assessment at VU Amsterdam

Students in pharmaceutical sciences at VU Amsterdam (Vrije Universiteit) undertake peer assessment of one another's reports as a mandatory pass/fail part of their course2.

The exercise is structured so that the students are undertaking the peer assessment individually but the report they are evaluating is the work of a group of three students.

The students are required to address each of the assessment criteria so the feedback is complete. Use of technology enables direct linking to the assessment criteria and enforcing the requirement to address each criterion.

Learners are also required to rate the quality of the feedback they receive from other peer reviewers.

This activity engages learners with the assessment criteria and also encourages them to reflect on the process of giving good quality feedback.

Feedback given by students matches well with instructor feedback. This means staff time can be saved by monitoring a random sample to ensure quality is being maintained.

This approach, whereby engagement with the feedback process feedback is an individual responsibility, works much better than previous initiatives where groups gave feedback on other groups.

"Unlike in previous years, there were no longer any complaints about the feedback process."
Jort Harmsen, VU Ansterdam

Two stage examinations

It is not unusual for a learner to walk out of a formal examination and immediately think of something they should have done differently. Normally it's too late but what if you had the chance to put it right?

Research by Professor David Nicol and colleagues at the University of Glasgow has taken this idea step further and researched the impact of a two-stage examination structure3.

In their model a student takes an exam and then completes reflective questions to surface their internal feedback about their performance. They are asked to identify any weaknesses they are aware of and any aspects of their work on which they would like to have expert feedback.

The students then take the same exam again but this time working in groups.

At the end of this stage, they answer another set of reflective questions, for example, about how the group answer differed from their own, whether the group discussion made them aware of strengths and weaknesses of their own answer that they hadn't identified and which answer they thought was better.

The purpose of this study was to find out more about how students generate inner feedback through comparisons. The finding was that this process is very powerful.

The fact that students undertook the work individually and reflected on their performance before engaging in dialogue, served to start them generating inner feedback so they gained greater value from the group discussion. The key to harnessing inner feedback is to make the process explicit using reflective questions to which students must respond in writing.

The researchers believe that the findings have broad applicability and that providing a rubric or high-quality exemplars as comparators could equally help students better evaluate the quality of their own work.

Professional learning passport for teachers

The Education Workforce Council (EWC) is the independent regulator for the education workforce in Wales. It has developed a professional learning passport (PLP) to enable newly qualified teachers to capture their learning and development and receive support from mentors.

EWC developed the passport using its e-portfolio tool. Teachers can capture any learning or experience that shows they are developing competence against the professional standards for teaching and leadership. Teachers reflect on experience, and plan learning in areas that need development, in an organic way that feels quite natural and unlike many formal assessment practices.

This approach prompts new teachers to consider all the learning experiences that occur during their working days from moments in the classroom, to discussion with colleagues, to formal training opportunities - and reflect on how these impact on their development as professionals and their teaching practice. The completed PLP enables the teacher to present their learning and development journey.

Principle five - manage staff and learner workload effectively

We need to ensure we are assessing the things that matter and doing so in a way that allows the student learning from each assignment to feed forward into future tasks.

Common problems to avoid are over-assessment and assessment bunching.

A programme of study is likely to be broken down into smaller modules. Each module will have its own learning outcomes and there may be overlap between them. If each module team designs its own assessment without reference to the overall picture, you may find that some learning outcomes are assessed multiple times and that assignments are 'bunched' at particular times of year.

Learners are not able to produce their best work if they have multiple assignments to complete within a short timeframe. A group of student union officers known as the academic integrity collective points to overassessment and bunching as factors that may panic students into cheating.

This type of bunching also puts stress on staff marking and grading the assignments.

We also need the right balance between formative and summative assessment. Applying our principles means designing in formative opportunities where students can generate feedback from a variety of sources and act upon it to feed forward into the future learning.

Students and staff should spend most of their working time on tasks related to enhancing learning. This means business processes need to allow them to access the information they need, submit assignments, enter marks and feedback and view information from different sources, easily and quickly.

Academic staff working with badly designed business processes often describe themselves as being on a treadmill with no time to stop and reflect how things might be done better.

Staff also need to be supported to use the available tools. For example, assignments submitted electronically by students may be printed out by the staff marking and grading them. There could be many advantages to marking online if staff were given the necessary tools and shown how to adjust the view to suit themselves and take advantage of features such as 'quick mark' options.

Some of the ways technology can help

Understanding the curriculum

A description of the curriculum in digital format can allow you to get an overview using only basic analysis, such as number and type of assignments per module, to see where some learners may be being over assessed.

A similar overview of assessment dates will reveal assignment bunching.

Personalising information

Many institutions now use apps that provide students with a personalised overview of administrative information such as timetable, assignment due dates etc that can help them manage their workload better.

For staff, it is increasingly common to have a personalised dashboard view showing the status of assignment tasks and their priority.

Workload management

Online workflows make it easier to manage team marking across large cohorts.

Digital feedback tools offer the possibility to reuse/adapt frequent comments to save tutors time.

Digital tools offer possibilities to generate and mark questions for both formative and summative development. The type of questions that can be created and automatically marked using the QTI (question test interoperability) open standard go far beyond basic multiple-choice and can include evaluation of what higher-order skills learners are using to come up with their answer.

Putting the principle into practice

Transforming the Experience of Students through Assessment (TESTA)

TESTA has produced guidance on revised assessment patterns that work.

The University of Greenwich map my assessment tool helps with planning, identifying clashes and modelling the consequences of change. The tool is available for others to use.

One-stop-shop EMA (electronic management of assessment) solution at the University of Wolverhampton

The University of Wolverhampton used the Jisc assessment and feedback life-cycle to help its analysis and planning to create an end to end solution for EMA (electronic management of assessment).

This 'one-stop shop' is built around the core tools of the Canvas learning management system and Turnitin academic integrity solution. The Learning Tools Interoperability (LTI) open standard is used to facilitate seamless interoperability throughout the workflow.

An assessment centre provides staff with an overview of all assessment tasks in their courses.

Staff can select and mark work from the assessment centre and are provided with information about approaching marking deadlines. They can also see whether marks and feedback have been published to the student record system and the students.

The project experienced few technical difficulties and focused much of its time on defining policy and addressing some inconsistencies in marking practice.

View a set of presentation slides and watch the session recording on this case study.

Timely feedback for large classes

The University of Utrecht department of law faces a 'massiveness' problem.

The department has 800 new students each year and the current learning design requires them to complete an essay each week. Feedback and grades must be returned within three days in order to feed into the next assignment.

This has to be managed with a team of two professors and 25 'correctors' who provide feedback. The correctors are a combination of assistant professors, teaching assistants and third-year students.

The University has implemented Revise.ly feedback software to help solve this problem. The correctors use a shared comment set to ensure that the approach to feedback is consistent no matter who does the marking. Although the role translates as 'corrector' in English, all comments use a 'feed forward' approach to help with future improvement.

The tool is integrated into the learning management system using the LTI open standard. It can be used on any device and fits easily into the existing workflow, including incorporating it into workflows involving plagiarism-checking or peer-review.

Learners find the approach very helpful because the feedback is readily accessible and the structured approach means they can search what type of comments they want to see (eg focus on structure) and review this type of comment across different assignments. The search function was added at the request of the students.

Enhancing the effectiveness of personal tutoring at the University of Cumbria

Personal academic tutoring is an important feature in helping students adjust to university life and develop their academic potential.

The University of Cumbria supports 10,000 students a year in this way and turned to technology to address inconsistencies across different parts of the University.

Each learner now has a workbook in the e-portfolio system that is used to provide clarity and structure to the personal tutoring process and a more supported experience for the students.

An introductory 'About You' page allows students to provide background information about themselves. Each of the 30 minute meetings has a separate page and, in advance of each meeting, students are prompted to fill in questions that give useful pointers for discussion.

During the meeting, or immediately afterwards, the personal tutor adds notes summarising the discussion and providing signposting for the student to further help and support where appropriate.

Find out more about the University of Cumbria's experience of enhancing personal tutoring.

Principle six - foster a motivated learning community

This ties in with principle number two support the personalised needs of learners. Involving students in discussions around assessment practice helps make it more inclusive and allows students who may face barriers, for example, due to a disability, to voice their concerns.

Assessment literacy

Institutions commonly focus on developing learners' study skills, graduate attributes and digital literacies but none of these fully addresses student understanding of, and engagement in, the assessment process.

Study skills resources tend to help students develop their assessment 'technique' through essay writing, presentation and preparing for exams rather than understanding the nature and purpose of assessment and feedback practice.

Engaging students as active participants in making academic judgement develops their assessment literacy and gives assessment a sense of purpose which is motivating.

Participating in discussions about assessment approaches, criteria and standards can help develop students' academic judgement as well as developing skills important for employability. Self and peer review activities are the most active way of allowing students to practice making evaluative judgements.

Developing academic practice

It is just as important for staff to reflect on their assessment and feedback practice as it is for students to strive to improve their own performance.

New findings from education research and advances in technology may offer improved ways of implementing good practice even if the underlying pedagogy is still sound.

Course and module teams may spend a lot of time discussing approaches at the curriculum design stage only for assessment and feedback to be left to personal choice.

Feedback is the area that most often takes place in a 'black box'. There may be little or no team discussion leading to inconsistency of approach and variation in the quality and quantity of tutor feedback.

Tutors who haven't had the opportunity to discuss feedback practice with colleagues may produce feedback that is skewed towards either praise or correction or short-term and focused on the task in hand rather than developmental.

Despite rigorous quality processes, marking and grading may still be subjective. In the absence of whole team approaches, tutors may develop tacit and personalised standards that lead to inconsistency and inequality for learners.

Listening to the student voice can reveal how the lived experience of assessment and feedback practice matches the expectation of course designers.

Some of the ways technology can help

Our digital experience insights surveys can play a part by showing how your students and staff are using the technology you offer, what is making a difference to their learning and working experiences and where improvements can be made.

Digital tools supporting self, peer and group evaluation can help develop student assessment literacy.

Digital storage of marks and feedback can simplify analysis to identify anomalies. Analytics about the types, quantity and timeliness of tutor feedback can stimulate discussion about what is most appropriate in each context and the equity of the learning experience across an institution.

Online communities of practice permit sharing of experience across a wider network than a single course or institution. Sometimes it can be easier to share ideas and get an objective view on a hard-to-solve problems with colleagues who are removed from the working practices and day-to-day politics of your own institution.

Putting the principle into practice

Find out more about our change agent network supporting student staff partnerships across the UK.

University College London (UCL) is one of the active participants in the network. The University has established a central team of digital assessment advisors and runs 'assessment mythbusting' town hall events and assessment 'hackathons'. UCL also has appointed a group of student assessment design advisors to help develop and pilot new forms of assessment. Find out more about UCL's partnership approach to assessment transformation.

Streamlining global teacher training

The University of Sunderland runs an international teacher training course which attracts students from 60 countries. Assessment used to include a lesson observation carried out face-to-face in the classroom.

Improvement in the reliability of videoconferencing tools enabled the course team to develop the VEDA (video enhanced dialogic assessment) approach which won a university teaching award. This allows the lesson observation to be either recorded or live streamed.

Use of the open standard LTI Deep Linking allows the session to be recorded using the Panopto lecture capture tool and uploaded straight to the Canvas VLE (virtual learning environment). Data protection compliance is important as the videos contain images of children.

Course leader, Elizabeth Hidson found not only did this online viva-voce style assessment save on time and travel costs, it also facilitated improved professional judgement. Assessors spent more time and energy in the online space than was possible at the back of the classroom and the dialogue stimulated better reflective practice.

"Our ideas of 'evidence' have become more sophisticated and holistic based on better quality dialogue and professional judgements."
Elizabeth Hidson, University of Sunderland

The initiative was introduced as a response to the pandemic but, as the candidate usually bears the cost of the assesser's visit, there is interest in offering this option in future.

Assessing the process of thinking

We tend to focus on immediate peers when we talk about learning communities but academic development is also taking place at scale. Across France open standards and data science are being applied to improve assessment practice.

The French ministry of education uses digital technology to develop more authentic ways to measure traditional competencies and 21st-century skills.

These assessments don't only provide information about whether the answer is correct. They capture a rich set of data that reveal the students' thought processes.

Solving problems in mathematics and science requires students to use cross-curricular skills such as calculating, modelling, and scientific reasoning. Until recently, it has been difficult to measure these skills because traditional math and science assessments contain test items that are scored, based on a student's final answer.

Using an extension of the open standard QTI (question and test interoperability), the ministry is developing PCIs (portable custom interactions) to deploy authentic assessments that measure skills such as creativity, problem solving, collaboration and critical reasoning. The questions cover a wide range of types including game-like situations and interaction with chat bots to measure creativity.

Researchers make sense of a vast amount of data derived from these digital tests by defining patterns based on what knowledge and skills are involved in answering each question and common types of error.

In simple terms, there is a difference between the activity pattern of a student who has conceptual understanding and knows how to apply it in context and another who achieves the same answer via trial and error or guesswork.

Large-scale research has the potential to deliver valuable insights to help learning designers. Read about large-scale mathematics assessment in France (pdf).

More immediately, the examples of authentic question types openly shared, can provide inspiration and a rich reference source for others to use.

If you thought that item banks and automated marking could only be used with very basic multiple-choice questions (MCQs) think again.

View a set of presentation slides and watch the session recording on this case study.

Our thinking is influenced by Dr Yong Zhao. He suggests education, rather than trying to fix perceived 'deficits', by measuring against prescribed standards, should be cultivating individual strengths.

Employability, viewed from this perspective, can be seen as the ability to translate your uniqueness into value for others.

If higher education is to remain relevant in a changing world, it needs to demonstrate that our learning and assessment practices prepare people for the world of work and participation in democratic society.

Authentic assessment

There are frequent calls for assessment to become more 'authentic'. By this we mean that the tasks which are assessed should reflect things the learner may have to do in real life.

Traditional assessment formats, such as essays or exams, don't really mirror any other real-world situations.

Authentic assessment doesn't only apply in subjects we traditionally think of as 'vocational'. The skills we can test using formats such as closed book exams, which rely heavily on recall, are an equally poor fit with the working practices of academics or researchers of the future.

Authentic tasks are more likely to focus on deeper learning and the learner's ability to apply their knowledge and skills. Give the learner some agency in deciding the topic and medium to be assessed, and you are moving towards the ideal of creating value from individual uniqueness.

Promoting ethical conduct

Academic misconduct features high amongst institutional concerns about assessment practice.

There is a perception that cheating and collusion are easier in the digital environment. However, such practices are generally only possible where the assessed task lends itself to finding an answer ready prepared. This is especially true now that the practice of purchasing an assignment written to order by an 'essay mill' has been outlawed.

Not only is it harder to cheat at authentic tasks, they provide a sound foundation for a discussion of ethical practice. Academic writing and referencing can appear arcane whereas giving credit to the work of others in the world of business is a more familiar concept to introduce an understanding of ethics.

The University of Exeter has created guidance on designing work integrated assessment.

This is a topic that cuts across a number of our principles:

Principle number one, help learners understand what good looks like is relevant. Student engagement with the performance criteria can help develop an understanding of the importance of acknowledging source material and intellectual property rights

Principle number five, manage staff and learner workload effectively is important as pressure of over assessment can be a reason why students cheat

Putting the principle into practice

Our employability toolkit is a good starting point to find guidance and examples.

Policing a virtual shopping centre

The University of Northampton is using online scenarios to replace in-person training exercises for apprentice police officers. Videos of alleged robbery, shoplifting and suspicious activity in a shopping centre are accessed via the virtual learning environment (VLE).

Student responses to the scenarios are discussed in group webinar sessions which means that, for the first time, the whole class can learn from the experience.

The University has also created a mock courtroom scenario. Police apprentices present statements and are cross examined by two criminal barristers representing the defence and the prosecution.

Group member evaluation

In the world of work, overall team performance is as much a part of recognition and reward strategy as individual performance. Assessing a learner's contribution to group work is thus a very authentic scenario but fraught with difficulty.

At Maastricht University in the Netherlands, students in the department of data science and knowledge engineering experience authentic learning and assessment from the start of their course. The department has a philosophy of problem-based and project centred learning. Students work in groups of six to seven to solve real-world problems.

The issues that arise will be familiar to anyone who has tried group learning. Students complain that some of them work harder than others with some students failing to complete tasks or handing work in late.

It can be hard for tutors to get to the bottom of the issues as some of the complaints are contradictory. Often, they don't hear about the issues until a deadline is approaching by which point it is too late to intervene and improve the group dynamic.

The solution to the problem was to implement group member peer evaluation at a point when the group has had time to settle down but there is still time for the initiative to result in improvement.

Group member evaluation is done anonymously using a tool designed by FeedbackFruits in collaboration with a group of universities. The tool is integrated into the learning management system via LTI.

Learners have to evaluate themselves before they assess other group members. They then look at how their self-evaluation matches that of their peers and have the opportunity to discuss this with their tutor.

A rubric was developed to assist the evaluative process. The rubric has six categories: attending internal team meetings; responsibility; communication; interaction and collaboration; initiative and timely submission of assigned work. There is a descriptor equating to: acceptable; satisfactory or exemplary for each category.

Students were at first encouraged, but not obliged, to provide additional feedback in the form of a comment. It is now felt to be good practice to require further comment on a low score. The reason behind a low score on attendance may be self-evident but a poor score for 'initiative' can be less so.

Most students value the initiative and take it seriously although some take longer to build up trust in the practice. Some timid learners have had their confidence boosted to find that peers appreciate their ideas and think they should speak up more.

View a presentation and watch the session recording about this case study.

The economics of panic buying

At Stirling University an economics exam involving 'doing the calculations by hand, from memory, in a cold sterile environment' has been replaced by real-world problems requiring flexibility and creativity to address the issues.

First year students in 2020 were required to design strategic interactions to address the pandemic. Topics ranged from panic buying, hoarding and price gouging to vaccine uptake and sharing. Students were free to use their own choice of medium including blog posts, digital posters and videos

The boss wants an answer by the end of the day

The environmental science department at Brunel University London replaced a three-hour exam with a more authentic task. Students were confronted with what it feels like to turn up for work one morning and find your boss needs a report on a complex matter by the end of the day.

The assessed task was a complex question drawing on a wide range of information encountered during the course. The students had to demonstrate their working and provide references with highlighted annotations. They had seven hours to tackle the problem.

# data[].attributes.attachment_link.title
Principles of good assessment and feedback - Jisc

# data[].attributes.attachment_link.description
How good learning, teaching and assessment can be applied to improving assessment and feedback practice.

# data[].attributes.headline
Assessment ideas for an AI enabled world - recently published by AI Centre

# data[].attributes.subject
Assessment ideas for an AI enabled world - recently published by AI Centre

# data[].attributes.body
- On tables colleagues share the one action or learning point from each themed discussion that they are taking away
- What further support is required to fully utilise these resources?

# data[].attributes.headline
Questions to consider and discuss in the world cafe session facilitated by Sarah Knight (with assessment and feedback)

# data[].attributes.subject
Questions to consider and discuss in the world cafe session facilitated by Sarah Knight (with assessment and feedback)

# data[].attributes.body
How might you utilise the Beyond Blended report in discussion with your members?

What further support do you need to gain a better understanding of these resources?

What are the key pain points you are hearing from members in relation to learning, teaching and assessment?

# data[].attributes.headline
Questions to consider and discuss in the world cafe session facilitated by Elizabeth Newall:

# data[].attributes.subject
Questions to consider and discuss in the world cafe session facilitated by Elizabeth Newall:

# data[].attributes.body
- What institutional agendas do you think this research speaks to?

- Which institutional contacts do you have who would benefit from seeing the findings of this research?

- How do we measure the impact of this research on the sector?

# data[].attributes.headline
Questions to consider and discuss in the world cafe session facilitated by Simon Birkett

# data[].attributes.subject
Questions to consider and discuss in the world cafe session facilitated by Simon Birkett

# data[].attributes.body
Who would be the key contacts/roles you would share this with?

How might you use these resources to support strategic conversations with your members?

Would any of your members be interested in being a pilot?

# data[].attributes.headline
Making digital transformation achievable - critical foundations for success - presentation from ALT-C 

# data[].attributes.subject
Making digital transformation achievable - critical foundations for success - presentation from ALT-C 

# data[].attributes.headline
Assessment tools and the learning ecosystem

# data[].attributes.subject
Assessment tools and the learning ecosystem

# data[].attributes.body
Read Gill Ferrell's recent blog post about how we are speaking to assessment platform vendors to see how their systems support the Jisc principles of assessment and feedback:

# data[].attributes.attachment_link.readable_text
We've been talking with stakeholders across the sector about tools to support assessment and feedback for some months now. There's no shortage of cool stuff out there (much of it increasingly powered by AI) and I'm nervous that there is an expectation in some quarters that our research will produce a 'top 10' list to pick from. Spoiler: it won't.

If anything, these conversations are broadening the scope of the project and making it clear that any discussions about assessment practice, and the tools to support it, need to be situated in a much broader context of future directions for learning and the kind of ecosystem we need to support them. These are just a few personal reflections on what we are seeing so far [ deliberately provocative and sufficiently badly presented so you can be sure I created them].
Ecosystems, lifecycles and learner journeys
The models, frameworks, diagrams and process maps we used to talk about this area are almost universally organisationally focused [ please someone prove me wrong and share alternatives].

It's too easy with these models to lose sight of this as a learner journey and forget what that learner is already carrying with them on the journey. Too often we treat the learner as a blank sheet of paper for learning to be written on. Distinctions such as traditional/non-traditional learners aren't much more helpful.

Each learner is bringing with them their lived experience to date and the richness of that diversity is something we should value and build on in the design of learning and assessment activities.

Authentic assessment shouldn't be only about the context to which learning is applied; it is equally about how individual experience contextualises the learning.

How do we create a representation of the assessment life-cycle and its place in the ecosystem that allows for learner individuality?
How many academics, procurement officers, IT staff and change management consultants does it take to change a lightbulb?
It's not easy to take an ecosystem approach in an organisational context where it's really difficult to change anything.

Delivering a rich and adaptable plug and play learning ecosystem is technically possible but there are a host of organisational and cultural reasons why the education sector finds it difficult to take advantage of the possibilities.

We end up in a situation where core information systems become increasingly monolithic with duplicate systems needed to meet some important requirements and other things going on entirely under the radar posing all manner of risk.

We know what interoperability means in principle yet we fail to demand it of our edtech suppliers. We allow suppliers to claim that they meet open standards whilst still charging for bespoke interfaces that deliver a poor user experience.

As a sector, our level of IT procurement maturity is low. We tend to try to minimise risk by specifying functionality requirements to the nth degree thus locking ourselves into ways of doing things that will be outdated by the time our implementation is complete and radically constraining which of the available market options we can consider.
Curriculum design and analytics
I struggle to remember how long it is since we first talked about the possibility that a continuous flow of information about learning activity would make 'stop and test' a thing of the past. Over assessment remains a significant issue for both staff and students.

A range of excellent learning design tools have helped build curriculum design capacity across the sector but assessment design remains a weakness. Staff lack the skills and confidence to design differently and issues of assessment bunching and over, or under, assessing certain learning outcomes within a programme of study remain.

Whilst we have used the tracking capabilities of digital tools to measure individual learner activity and engagement for predictive purposes, we are not currently using data to improve curriculum design. This is an area where Jisc is currently exploring possibilities. What if, rather than looking for patterns in a sea of data, we could define upfront what data will help us understand if our learning designs are working as intended? What if curriculum analytics could help us design learning experiences that are relevant, engaging and proven to accelerate learning?
Generative Artificial Intelligence (AI)
I really think we are sweating the small stuff here!

It is good to see that the dialogue is progressing beyond initial knee-jerk reactions and people are coming up with lots of well-reasoned arguments about why we need to prepare our learners for a world where this technology is ubiquitous how it could be used to help redesign assessment in ways that are authentic, creative and can support the kind of critical reasoning that constitutes information literacy in this new scenario.

There are a host of ideas on how generative AI can be used to save staff time on repetitive tasks, create better and more extensive question banks etc. [ I confess to a certain amount of scepticism about some of the time-saving on things like lesson and assessment planning - generating and comparing a lot of options could give you a better end product but is it really quicker?]

Something we've barely scraped the surface of yet is how AI could be used to turn academic descriptions of curriculum into descriptions of skills and competencies that are meaningful in wider contexts. The idea that this could give an impetus to the idea of ongoing recognition of achievement/micro-credentials is exciting and obvious benefit to learners.

However, the debate around ethics has shifted too quickly to focus on academic misconduct/acknowledgement of sources etc. This has to be missing the point when we don't actually know what the underlying sources are.

We are worried about what our students might do with the technology yet we are not asking the really big questions of the edtech suppliers. What about the carbon footprint of AI, what about the low-paid human labour that goes into training AI, what about the data sources and algorithms? The big edtech suppliers need to be prepared to give us some answers to these questions. What is equally worrying is the proliferation of start-ups who are focused on getting their neat idea to market faster than anyone else and who may through naivety and/or lack of resources cut corners on due diligence
Recognising achievement
Rethinking assessment practice is a vital part of making higher education fit for the modern age and part of that concerns how and when we recognise achievement.

Programmes of study should deliver a value that is more than simply the sum of the component parts but that doesn't mean those component parts are not worthy of recognition. It is increasingly difficult for learners to devote years at a stretch to gaining a qualification. Learners often have to interrupt their studies and too often that is equated with failure. Offering recognition for what has been achieved can motivate the learner (and those supporting them) to future completion.

We can even question why assessment has to take place to a rigid timetable in such a significant majority of cases. What if there was greater flexibility around learning at your own pace and being assessed when you feel ready?

How far are we from a scenario where learners have a rich record of the skills and competencies they are gaining all along their learning journey in a form that is digital verifiable, transferable and stackable? Join the discussion

This work is raising some big topics and all of them are being addressed in various ways by Jisc directly and in collaboration with a range of stakeholder groups.

I pulled out this particular group of topics because they are all things that we will be following up in a forthcoming event with colleagues from across Europe.

1EdTech Europe: Evolving the Digital Learning Ecosystem Across Borders will take place at the University of Nottingham on 14-15 September.

Jisc will be leading interactive sessions on Assessment and Feedback and Curriculum Analytics, joining a panel discussion on the learner journey in the age of AI and engaging in sessions on all of the other topics raised in this blog post.

If you have good practice related to the above topics to share, we may be able to squeeze in a few more contributions so get in touch: gill@aspire-edu.org

Find out more and join us:
https://web.cvent.com/event/ca43d697-f0cf-47d6-aafd-9e6219970f13/summary

# data[].attributes.attachment_link.title
Assessment tools and the learning ecosystem

# data[].attributes.attachment_link.description
We've been talking with stakeholders across the sector about tools to support assessment and feedback for some months now. There's no shortage of cool stuff out there (much of it increasingly powered by AI) and I'm nervous that there is an expectation in some quarters that our research will produce a 'top 10' list to pick from.

# data[].attributes.headline
Future trends for assessment and feedback

# data[].attributes.subject
Future trends for assessment and feedback

# data[].attributes.body
Read Simon Walker's blog post:

# data[].attributes.attachment_link.readable_text
We are working with Professor Simon Walker on exploring the impact of technology on the future of assessment and feedback. Simon has shared this post about his research to date.
As the world continues to shift towards a digital future, with generative artificial intelligence playing an important role, the way we assess education and professionals is also evolving. In this blog post, we will explore some trends and innovations in assessment, along with the challenges and opportunities they present.

Advancements in technology have allowed for the integration of digital assessment design, delivery, submission, marking, and moderation platforms in many universities. While this has the potential to improve the student and staff experience and reduce staff workload, there is still much technical work to be done in terms of providing LTI standards that allows data to flow across the digital ecosystem and provide integrated plug-and-play tools with platforms. Institutions must take a coordinated approach that involves strong strategic leadership, excellent communication, resources and staff to support assessment (re)design and training, partnerships with students for assessment co-design, and a willingness to work with platform providers to ensure data validity, integration, and security. They also have to bear in mind the need to update regulations and policies as they pivot to more digital assessment as well as identifying educational initiatives to create robust ethical practice. That's a lot of getting ducks in a line.

Assessment practices are shifting from knowledge-based, traditional, standardized tests towards more diverse, authentic, and personalized evaluation methods that better align with students' aspirations and careers. Teachers are designing assessments using digital technologies, including synchronous formative testing, randomised question banks, project-based assessments, and portfolios.

Maintaining and setting standards is critical, and the important role of Professional, Statutory, Regulatory bodies (PSRBs) for programmes that are accredited can play an important role. PSRBs work with universities to accredit their programmes, which are entry points to the profession or denote professional identity. All stakeholders must be confident in the controls around digital assessment that ensure the authenticity of each individual's learning outcomes.

The modern workplace demands a set of interpersonal skills that allow people to work effectively in teams, communicate well, and adapt to change. With the growing importance of these skills, new methods of evaluating them are constantly being developed. The challenge with assessing soft skills lies in how to quantify something that is subjective and difficult to measure. Traditional methods such as oral assessment, combined with innovative digital strategies, including gamification and simulations, are becoming popular ways to evaluate these types of competencies.

New forms of assessment such as Virtual Reality (VR) open up a world of possibilities for immersive evaluation of real-world scenarios. Digital portfolios allow students to collect, organise, and showcase their work in a digital format, providing a more holistic view of their abilities. Cohort size is important, as larger cohorts may have to rely more heavily on automated high stakes assessment with interventions such as investigatory interviews to provide controls against academic misconduct. Digital portfolios allow students to collect, organise, and showcase their work in a digital format. This can provide a more holistic view of a student's abilities, including various forms of evidence, such as videos, images, and written work. Some disciplines can take to digital assessment more easily than others. For example architecture uses an educational approach that relies on staff closely monitoring students' hand drawn sketchbooks, discussing their ideas, and questioning them about their decisions. Digital outputs closely align to formative work, and can be easily authenticated as that students' work.

The emerging interest in post-humanist approaches to curriculum conception and design have an impact on assessment. Assessment for social value or social good refers to its ability to stimulate meaningful effort that can contribute to positive social outcomes or benefits beyond financial returns. The focus is on analysing the environment, an organization, project, or policy to produce viable and imaginative solutions to local or global challenges. Digital assessment provides a natural home for this shift in pedagogical approach.

So organisational transformation to digital assessment requires a coordinated cross-institutional approach that involves strong leadership, excellent communication, resources and staff to support assessment (re)design and training, partnerships with students for assessment co-design, and a willingness of professional services, academics, and technical services to work with platform providers to ensure data validity, integration, and security. By embracing these trends and innovations, we can transform assessment practices to better align with students' aspirations and careers, maintain and set standards, and contribute to positive social outcomes.

Please get in touch with us by emailing Sarah.knight@jisc.ac.uk if you would like to join our Jisc working group to contribute to and review the research we are undertaking around assessment and feedback and the role technology can play.

Also see our Jisc guidance on Principles of good assessment and feedback and our supporting podcasts.

# data[].attributes.attachment_link.title
Future assessment scenarios: trends and innovations

# data[].attributes.attachment_link.description
We are working with Professor Simon Walker on exploring the impact of technology on the future of assessment and feedback. Simon has shared this post about his research to date. As the world continues to shift towards a digital future, with generative artificial intelligence playing an important role, the way we assess education and professionals is also evolving.

# data[].attributes.body
View Simon Walker's recent presentation summarising the findings from our survey and interviews

# data[].attributes.headline
Listen to Elizabeth interviewing Helen Beetham and Sheila MacNeill discussing beyond blended

# data[].attributes.subject
Listen to Elizabeth interviewing Helen Beetham and Sheila MacNeill discussing beyond blended

# data[].attributes.attachment_link.title
Alt-C Curriculum Learning Design workshop Sept2023

# data[].attributes.headline
Beyond blended: Post-pandemic curriculum and learning design: lessons from the higher education (HE) sector

# data[].attributes.subject
Beyond blended: Post-pandemic curriculum and learning design: lessons from the higher education (HE) sector

# data[].attributes.body
Available to download from:
https://ji.sc/beyond-blended

Sign up to receive advance notice of the Beyond blended web guide:
https://ji.sc/register-beyond-blended

# data[].attributes.headline
Approaches to curriculum and learning design across UK higher education (phase 1 report)

# data[].attributes.subject
Approaches to curriculum and learning design across UK higher education (phase 1 report)

# data[].attributes.attachment_link.readable_text
This report contains:

A review of current literature and research into blended learning and models of practice

A survey of the higher education (HE) sector on current practice/challenges around developing degree courses

A series of interviews with a range of educational practitioners

Recommendations identifying areas where Jisc could develop resources, guidance and effective partnership approaches

Key findings

Teaching and learning strategies are the main drivers for the curriculum design process in organisations

Since the pandemic, there is evidence of major curriculum reviews and new interest in modes of delivery

Learning design resources may be a route to encouraging open educational practice

There are clear links between quality assurance, quality enhancement and staff engagement with learning design

A strategic approach is needed for consistency in curriculum design and support

Staff workload allocation, staff reward structures and investment in academic staff development were key concerns

Engaging students as co-creators in curriculum and learning design is more of an aspiration than a realisation for many

Examples of practice from the sector

The University of Hull launched a new competency-based curriculum framework replacing learning outcomes with programme-level, discipline-specific competencies

As part of its digital transformation strategy, Teesside University has been rolling out a series of design bootcamps to produce an action plan

At the University of Northampton, a learning designer is on review panels, and the university has incorporated quality standards into its workshops/support

Based on the pandemic experience, UCL is keeping assessments online with a new team of digital assessment advisors to support academics and develop the assessment system

The Open University is exploring ways to use more qualitative data representing the student voice, alongside their quantitative data

Challenges for universities moving forward

To meet the challenges of flexibility around space, place and modes of delivery, universities should consider providing updated workload models. These need to recognise (and reward) the changing needs for staff to engage with design activities, evaluate their practice and provide support for innovation to engage learners.

There is a need for a shared vocabulary and understanding around modes of participation. Evidence of different student experiences and outcomes could be shared across the sector, along with examples of different ways of organising time, space, and independent and collaborative working.

"Managing any support initiatives will be tough. With limited resources 'teaching to the middle' is common. The gap between eager/advanced online tutors, those in the middle and those who are lagging behind has grown and that will have to be addressed." - Survey participant

Hear from the authors

Helen Beetham and Sheila MacNeill joined our Beyond the Technology podcast to discuss the findings from their review of learning and curriculum design in higher education. Listen to the podcast episode.

Next steps

Find out how Jisc can support you with digital tools and mechanisms to enrich the learning and teaching experience for your students on campus and in the work place.

# data[].attributes.attachment_link.title
Approaches to curriculum and learning design across UK higher education - Jisc - Jisc

# data[].attributes.attachment_link.description
Looking beyond the emergency responses to lockdowns and restrictions - are we delivering learning in flexible and accessible ways?

# data[].attributes.body
View our recent presentation from ALT-C conference which gives an overview of key findings -

# data[].attributes.attachment_link.title
ALTC23 Newall and Catana Equity, diversity and inclusion_ understanding international students digital shock

# data[].attributes.attachment_link.readable_text
Never has it been so important for us to have a better understanding of the digital experiences and expectations of international students at UK universities. Full-time international students represent 17% of the undergraduate and 67% of the postgraduate taught student population in the UK. They contribute approximately £26 billion per annum to the economy and provide an important and valued contribution to cultural and intellectual diversity and talent.

This report provides an opportunity for readers to view the international student experience through a digital lens, highlighting alternative perspectives and considering issues that might impact the digital experience of international students.

Some academic studies have described the shift from one country's cultural and educational use of digital to another as a 'digital shock' for students, and it can often take months for them to become familiar with systems and processes. Students may arrive with different or unsupported software and hardware, which can hinder their adjustment to their new teaching and learning environment. They might also struggle with online assessment and engaging with digital learning, or grapple with the principles of plagiarism.

Key findings

Many international students are likely to experience digital systems and tools - and the pedagogy underpinning their use - very differently from domestic students

As the diversity of UK international student population changes, it is important to consider the impact this may have on the nature and extent of student support required in order that HE courses maximise equitable learning outcomes

When considering the experience of international students, there is a clear need for institutional leaders to integrate international and digital (transformation) strategies, as well as with other areas, including equality, diversity and inclusion (EDI), curriculum and assessment design and technology-enhanced learning (TEL) support

Quality international student digital experience may be best delivered via a more holistic and integrative strategic approach that works across and between areas of UK HE and organisational strategy

We would like to acknowledge the contributions to this report from Advance HE, AGCAS, ALT-ELESIG, BUILA, the British Council, the Department for Education, HESA, HeLF, UCAS, UKCISA and UUKi.

Stay informed

Our research is split into four phases, with this report resulting from the first. Register your interest to join our mailing lis t so that we can keep you up to date with our progress and share any subsequent reports.

# data[].attributes.attachment_link.title
International students' digital experience phase one: a review of policy, academic literature and views from UK higher education - Jisc

# data[].attributes.attachment_link.description
Never has it been so important for us to have a better understanding of the digital experiences and expectations of international students at UK universities. Full-time international students represent 17% of the undergraduate and 67% of the postgraduate taught student population in the UK.

# data[].attributes.body
We will be selecting 10 - 12 universities to pilot the digital transformation toolkit with their use of the model for digital maturity, developing action plans and their institutional roadmap

# data[].attributes.headline
Demonstrating digital transformation member stories and podcasts

# data[].attributes.subject
Demonstrating digital transformation member stories and podcasts

# data[].attributes.body
See our recent member stories from universities who hosted the Demonstrating digital transformation events with Staffordshire University, UCL, Greenwich and Northampton universities sharing their digital developments:

# data[].attributes.attachment_link.description
Focused on improving their cyber security posture, the IT team at Coleg Y Cymoedd, together with Jisc, completed a cyber security assessment. By evaluating the effectiveness of their current measures, the process enabled them to identify what steps they could take to strengthen their security further.

# data[].attributes.headline
Digital transformation in HE guide - to be published late September

# data[].attributes.subject
Digital transformation in HE guide - to be published late September

# data[].attributes.body
Read our recent blog post which describes the guide and how it could support higher education providers:

# data[].attributes.attachment_link.readable_text
Some higher education providers (HEPs) have well-developed plans, but for those still feeling their way, Jisc has a fresh set of resources. It comprises three elements: a framework, a maturity model and a guide.

This work has been developed with support, input and feedback from sector organisations such as Universities UK, Advance HE, UCISA, the Association of University Administrators, the Association for Learning Technology, plus a 200-strong sector-led working group.

Collated over the past year and with considerable engagement and buy-in from institutions, their key staff and professional bodies - we're confident the guidance is of high quality and value.

And not just for the UK; we're now being asked to share our advice on digital transformation in education with international colleagues.

A sector first

The first pieces of work were published earlier this year with the sector's first framework for digital transformation in higher education.

The framework recognises that, while each institution's digital journey will differ, there is a defined set of elements that must be considered as part of an organisation-wide digital transformation plan. These include:

Effective digital leadership

Appropriate investment

Robust and secure infrastructure

Stakeholder engagement

Digitally capable staff and students

The framework is accompanied by materials to help assess digital maturity across all business activities, and to signpost support from UK sector bodies.

It aims to inform the development of strategic vision and planning, foster innovation, streamline business processes and operations, and develop partnerships for collaboration.

It also highlights how policies and processes might align to promote cross-team approaches that reduce complexity and fragmentation.

Later in September we'll be publishing the final two sections of our digital transformation resource - a comprehensive guide and a maturity model.

Assessing digital maturity

The model provides a structure to help HE providers assess their digital maturity across all aspects of their business.

It will help universities benchmark themselves against the sector and set their own baseline from which to move forward.

Defining digital maturity is an important step in producing a digital transformation strategy and creating actionable plans.

The model is supported by an action plan template and a set of guidelines. It describes how there are three levels of digital maturity, which can be assessed against each aspect of activity identified in the framework:

Emerging to established - reactive stage

Lack of strategic digital leadership

Short-term investment

Unintegrated systems and operations

Lack of innovation

Lack of stakeholder understanding and engagement

Dispersed, project-based digital activities

Established to enhanced - proactive stage

Developing a proactive strategic approach

Enabling effective digital leadership

Making efforts to integrate systems and operations

Engaging and upskilling all stakeholders

Moving towards a service delivery model

Enhanced to mature - integrated stage

Comprehensive and integrated strategic approaches

Effective digital leadership

Long-term, adaptable planning and investment

Integrated and efficient systems and operations

Engaged, informed and appropriately skilled stakeholders

Partnership approaches to innovation

Longer-term delivery model approaches (less project based)

As the sector becomes more familiar with using the model, we hope it will provide a common language for HE providers to discuss digital transformation and share best practices.

We are particularly interested to find out how providers might use the toolkit to support digital transformation across their organisation, or in one area of work, no matter what their digital plan looks like, or at what stage of development.

Organisations just starting on the digital transformation path could find the maturity model and framework a useful starting point, while others that have already agreed a plan could use our resources to convert that vision into action.

The materials could also help HE providers with highly developed digital maturity to conduct a review with a view to continual improvement.

Further information and how to get involved

To hear more about how Jisc can support HE providers with digital strategies, join our session at the Association of Learning Technology (ALT) annual conference (6 Sept) called 'Making digital transformation achievable - critical foundations for success', delivered by Jisc's Sarah Knight, Simon Birkett and Elizabeth Newall, with Lou McGill, a consultant who authored the guide and model.

HE providers interested in using the maturity model to identify their level of maturity are invited to express an interest to work with Jisc by getting in touch with sarah.knight@jisc.ac.uk or simon.birkett@jisc.ac.uk

Submit your details to notified of the release of the new guide.

# data[].attributes.attachment_link.title
How is Jisc supporting universities to make digital transformation achievable? - Jisc

# data[].attributes.attachment_link.description
While it's true that all universities are transforming digitally, the process is far from uniform.

# data[].attributes.headline
Framework for digital transformation in higher education 

# data[].attributes.subject
Framework for digital transformation in higher education 

# data[].attributes.body
Explore the framework which we published in March this year:

# data[].attributes.attachment_link.readable_text
The framework offers a structure to guide the development of strategic vision and planning, fostering innovation, streamlining business processes and operations, and developing partnerships for collaboration. It highlights how policies and processes might align to promote cross-team approaches to reduce complexity and fragmented processes.

The need for digital transformation

UK universities are facing significant disruption due to the economic crisis and limited resources, along with the impact of COVID-19 on staff and student wellbeing. One of the ways the sector is responding to challenges and disruption is by revisiting their strategies for digital.

Successful digital transformation requires effective digital leadership, appropriate investment, robust secure infrastructure, stakeholder engagement, and digitally capable staff and students. The complexities and scale of higher education institutions present challenges to achieving ambitious digital strategies.

This digital transformation framework has been developed for senior leaders, managers and change agents with responsibility for supporting digital transformation - across a range of different roles and locations within their organisational structures.

Aims of the framework

To support the development of a shared understanding of digital transformation across the sector and within HE organisations

To encourage a holistic approach - identifying patterns and connections across traditional boundaries

To encourage collaborative approaches and build on the collective wisdom of the sector and organisations

To help reduce complexity and fragmented processes

To support HE organisations to articulate a strategic vision for digital transformation and develop actionable plans to achieve this

To inform decision making and prioritise investment

To focus on people and practices, not just processes and technology

To highlight leadership and digital capability as critical success factor

Developing the framework

This framework has been developed in consultation with a range of UK professional bodies: Advance HE, Association for Learning Technology (ALT), Association of Higher Education Professionals (AHEP), Association of University Directors of Estates (AUDE), Quality Assurance Agency for Higher Education (QAA), Society of College, National and University, Libraries (SCONUL), Universities and Colleges Information Systems Association (UCISA), Universities UK (UUK) and Vitae.

This ensures the framework reflects sector priorities and links to key UK HE models and frameworks already in use.

The framework will be supported by an evolving range of materials to help organisations assess their digital maturity across all business activities. These materials will signpost through to support available from UK higher education (HE) sector bodies, as well as Jisc resources, products and services that can enable digital transformation.

Get involved

To get started, read our digital transformation in higher education guide, which contains a digital maturity model, action plan, and roadmap.

If you are a senior leader working in higher education and you're interested in joining our working group for digital transformation, please register your interest.

# data[].attributes.attachment_link.title
Framework for digital transformation in higher education - Jisc

# data[].attributes.attachment_link.description
A comprehensive perspective on how the digital environment can support positive work, research and learning experiences, and promote a sense of belonging and wellbeing.