# data[].attributes.body
Share your reflections, feedback and ideas here for priorities to take forward to support sector with this area

# data[].attributes.headline
Improved Text-to-Speech Capabilities Available for All Digital Assessments

# data[].attributes.subject
Improved Text-to-Speech Capabilities Available for All Digital Assessments

# data[].attributes.body
The 1EdTech Support for Speech Synthesis Markup Language (SSML) Using the 'data-ssml' Property Version 1.0 specification, or Data-SSML for short, is designed to improve pronunciation capabilities in digital learning resources and assessments. Data-SSML provides a standardized way to add instructions to control the pronunciation and presentation of content by text-to-speech synthesizers. This specification helps with read-aloud technology for all students and also assists visually impaired students using screen readers. Students using the accessibility feature will better understand what they’re being asked, making for a more equitable learning environment.

“Your GPS mispronouncing a street name is one thing, but if text-to-speech programs mispronounce words during an assessment, it can cause confusion and impact a person’s academic performance,” said Beatriz Arnillas, 1EdTech. “This specification reduces that risk, so every learner has the opportunity to show what they really know.”

1EdTech’s Data-SSML specification leverages the W3C’s SSML standard to include pronunciation cues in HTML and websites, expanding the options educators can confidently use to assess learner knowledge.

More info: https://www.1edtech.org/standards/data-ssml

# data[].attributes.attachment_link.title
1EdTech Support for Speech Synthesis Markup Language (SSML) using the ‘data-ssml’ Property | 1EdTech

# data[].attributes.body
As education is becoming more flexible/personalised, I'm expecting more programmatic assessment/360 degree feedback, etc. moving away from high-stake exams at the end of a module.

# data[].attributes.body
We moved fully to competence education framework  over the last 3-4 years. Has seen a real shift in our assessments to more performance/portfolio style assessments (and huge reduction in exams!)

# data[].attributes.attachment_link.readable_text
I have taken some of the ideas from the Assessment Futures workshop and written an assessment future scenario. Many thanks to the workshop participants for their ideas and apologies if I have misinterpreted them. The future scenario is meant to provoke and challenge what we do now, and help us to identify what would be needed to make it happen. So I'd welcome critical feedback.

Background

In 2024, universities are facing increasing financial pressures and a need to improve institutional efficiencies. The move to more remote assessments has been derailed by academic integrity concerns around the use of generative AI, increased students collusion using readily available social media apps and wide availability of essay mills.

Student numbers have continued to increase with more mature, and international students applying. The impact of global warming is only just beginning, but increasing temperatures, extensive flooding and new environmental sustainability policies are about to put even more pressures on universities, both through increased risk to the physical campus estate, and increased climate migration.

In-person exams, both digital and analogue became increasingly complex and difficult to scale. A lack of suitable rooms, intermittent power supplies, wifi and student timetabling challenges. Increasing numbers of students are being identified as neurodivergent and requiring reasonable adjustments.

Employers increasingly demand better employability skills from graduates. A growth in higher level apprenticeships and employer led qualifications starts to threaten the university market.

The future - 2035

A trend towards block teaching and large credit modules had already started in some universities in 2024, in part to reduce the complexity of degree level courses and the growing administrative burden on institutions. Legacy student management systems were moving to the cloud and improved data integration was opening up new opportunities.

By 2035 stackable degrees and micro-credentials have become a reality across the education sector. Students can transfer seamlessly between courses and institutions.

A big enabler was a change in assessment in university courses. Large modules allowed a reduction in the number of high stakes assessments. Although some institutions had moved away from exams completely it proved more difficult for large courses especially where they were regulated by professional bodies.

At the University of Banford (a fictional institution Bryant, Lanclos, Phipps, 2023) a new assessment approach has emerged characterised by the assessment of the process alongside product to demonstrate learning, skills and address issues of academic integrity.

The approach uses personalised and adaptive learning and individual final assessments created for each learner. This has been made possible due to the availability of learning activity and performance data, gathered from students as they undertake tasks throughout their large modules. The in-module learning is assessed using a range of approaches from automated AI marking, peer assessment and some by tutors. There is no formal checking of student work for academic integrity for the tasks undertaken during the module although they can get feedback using automated tools. They could in theory collude, use generative AI or even buy an essay to meet the formative tasks throughout their module - their choice.

However at the end of the module students are required to demonstrate what they have learned and show they are able to apply their learning. Every student receives a personalised AI generated adaptive learning assessment that checks their learning and level based on the work they have submitted during the course. A sort of viva but delivered digitally. The idea is if the work they have submitted is genuinely their own, and the exam is used to verify the level and skills from course work. If they have taken "short-cuts" during the year whether using AI to automatically generate answers, purchased ready made answers or products from others then they will struggle to pass the personalised assessment. Students can be credited with micro-credentials for the learning they can demonstrate. They are also able to re-take the assessment after reviewing the gaps in their existing learning. The personalised assessment will provide a level of attainment and student may undertake further student to "raise" their level of attainment if they so desire.

Some subjects at Bamford are using a continuous assessment approach with an e-portfolio and individual tasks are verified at each stage. This removes the stress of high stakes exams and spreads the assessment administrative burden across the year.

Another department creates authentic assessments the student needs to take at the end of the year. These are scenarios or roleplays generated using AI, where the students are assessed on their ability to respond to situations, solve problems or undertake tasks within immersive or real environment.

Discussion

So let's imagine this future of assessment scenario was possible in the next few years. The idea was generated in response to a challenge to personalise assessment and address issues such as generative AI and academic integrity.

What would the student experience look like? There seems to be a lot of assessment still but this could be framed as assessment for learning, where undertaking tasks and getting feedback was part of the learning experience.

Could it really reduce the administrative burden? There seems to be a tendency with digital that as we automate tasks we just end up doing more. Being able to generate assessments and automate marking could enable scaling of assessment for more students. We should be using the technology to find ways to reduce the number of assessments to a minimum.

It would require good data around the learning criteria and the student performance as well as software that was capable of creating personalised assessments and delivering them to large numbers of learners. Products exist today that mean it is technically already possible to deliver assessment in this way.

However the cultural change required by leaders and staff could be the greatest barrier to any transformation in assessment in the future.

We'd welcome further thoughts and feedback, so please contact innovation@jisc.ac.uk.

# data[].attributes.attachment_link.title
Assessment Future Scenario - Personalised assessments - Co-design

# data[].attributes.attachment_link.description
I have taken some of the ideas from the Assessment Futures workshop and written an assessment future scenario. Many thanks to the workshop participants for their ideas and apologies if I have misinterpreted them. The future scenario is meant to provoke and challenge what we do now, and help us to identify what would be [...]

# data[].attributes.body
Optionality in Assessment -

# data[].attributes.attachment_link.description
This QAA Collaborative Enhancement Project (CEP) is led by the University of Manchester in partnership with Imperial College London, the University of York and University College London. CEPs offer funding for small groups of member institutions to work together on projects to enhance the quality of the participating institutions’ student learning experience.

# data[].attributes.headline
Beyond Blended: Rethinking curriculum and learning design

# data[].attributes.subject
Beyond Blended: Rethinking curriculum and learning design

# data[].attributes.attachment_link.readable_text
The need for new approaches

Four aspects of beyond blended learning

Four modes of participation in learning and session types

Six pillars for designing 'beyond blended' learning

Six pillars: place

Six pillars: platform

Six pillars: pace

Six pillars: blend

Six pillars: flex

Six pillars: support

Using the beyond blended materials

Strategic approach to beyond blended

Strategic lenses

Curriculum review and design

Curriculum and learning design process

Curriculum lenses

Other established curriculum frameworks

The need for new approaches

In 2022, following the disruptions of the pandemic, we felt it was timely to consolidate our understanding of the different benefits and challenges of online learning, and the value of in place learning (where learners and educators are physically present in the same space). We wanted to align this with existing thinking and research on how best to integrate technology into the curriculum through design, content creation and teaching methodologies.

Higher education (HE) sector experiences during the pandemic offered an opportunity to consider the whole curriculum, the notion of students being in place at university and the value of both online and in place learning, teaching and assessment (LTA). They also presented a chance to highlight what we learned during our wholly online experiences and acknowledge that universities and curriculum teams already have a lot of the expertise they need to navigate and adapt to various challenges.

Our 2022 report on approaches to curriculum and learning design across UK higher education offered a review of learning and curriculum design across the HE sector. It highlighted a pre-pandemic focus on activities and resources, particularly around enhancing and encouraging use of the virtual learning environment (VLE), with learning design focusing on tasks and sequencing. Support for media production (eg lecture recording, video and audio content) was a clear focus, as well as how students and learners were using digital devices to access resources, to produce notes and maybe develop their skills in some generic software.

Post-pandemic, we are looking at a more holistic, whole curriculum approach to design. There's been a clear focus on different modes of participation and on trying to be as flexible as we can for both students and staff.

Going beyond blended learning

Moving forward from previous notions of blended learning, we are now talking about major alternatives that are available to curriculum designers and to students as choices about how to participate in a particular activity or module or part of the curriculum.

Our 2023 beyond blended report offered lessons from the HE sector on post-pandemic curriculum and learning design. It highlighted a need for a shared vocabulary between curriculum design teams. There were a lot of words in that space such as 'hybrid', 'hyflex', 'flexible', 'blended' that were not always being used in a clear enough way to allow the conversation to be student-centred, or to allow that conversation to really centre the curriculum.

Our research identified some increasingly important issues for curriculum design:

In a beyond blended approach to curriculum design we consider the two blends of time (asynchronous and synchronous) and place (in place and/or using an online platform). In addition to the more traditional learning design issues like activity, task, technology and materials. We suggest stepping back and thinking:

The beyond blended report described:

Four aspects of learning

Four different modes of participation in learning

Six pillars or precepts for curriculum design in the four modes

This guide offers a simple vocabulary for describing modes of learning and considers some key questions around how these contribute to student learning and engagement. It is equally important to take these conversations forward from the curriculum space into the institutional space to consider strategic aspects such as platform procurement considerations, what kinds of physical spaces are needed, and what kinds of flexibility should be offered. These wider organisational aspects are critical as organisations adopt more strategic approaches to digital transformation.

Definitions

Curriculum design: reviewing, planning and developing a course of study. This is usually a formal departmental and institutional process, mapping to graduate outcomes, benchmarks and professional standards. It produces formal documentation for the organisation.

Learning design: defining how learning will be supported within each course, module or unit. This is a professional activity that involves defining tasks, tools and technologies, core content, sessions and class groups, assignments and assessments, and opportunities for interaction and feedback. It produces a learning plan and associated materials to support teaching staff and students.

The terms are often used interchangeably in UK HE, and the two design processes in any case overlap or iterate. For example, feedback from learning in practice (how did learners experience these activities or meet the learning outcomes?) should inform the next iteration of development. So we use the term curriculum and learning design to mean all the processes of reviewing, planning and designing a programme of study and how students will learn within it.

Applying the beyond blended resources

If you are already familiar with our beyond blended research and materials you may like to go straight to the section on using the beyond blended materials: alternative approaches for different perspectives. This section offers a series of different lenses on the six pillars, which provide useful prompts around a range of strategic and curriculum approaches.

Let us know how you are using the beyond blended resources at your institution. Fill in our beyond blended resources feedback form.

Four aspects of beyond blended learning

We have many options when designing the curriculum or learning activities, and different contextual aspects affect the choices we make about what, when, where and how we blend these options. These include: subject discipline requirements, teaching frameworks, assessment criteria, teaching approaches, practical considerations, culture and traditions, resources available and student needs and expectations.

We need to draw on both established research and new thinking to understand why a particular session might be delivered live and online rather than self-directed and in place, or how these two sessions might complement each other. It is particularly important to understand how different sessions can be designed to engage students actively in their learning.

We have used the term 'in place' to describe when learners and educators are co-located (usually, but not always, on campus). In contrast, 'online' learning describes when participants come together in the same digital platforms without the need to be present at the same location.

In place and online learning environments come with a range of facilities, functions and materials. These can be encountered synchronously (meaning live, at the same time, in shared or timetabled time) or asynchronously (meaning at different times, usually within a time window such as a teaching week or an assignment timeframe).

Digital modes of learning can blur what were once clear boundaries, creating intermediate settings. Digital media can bend time boundaries as well as offering new experiences of space. This means students and educators must negotiate new choices and trade-offs, as well as situations where expectations may be unclear, and make choices on how to blend these various options. Some can be planned ahead while others may be made as a programme of study progresses or as student circumstances and needs change. We offer a range of practical materials to help consider factors affecting these choices around time, pace, place and platform.

The four aspects of learning

These four aspects of learning can be blended in many ways:

1. Time, pace and timing: synchronous (live, shared time) and asynchronous (independent, own time)

2. Space: place and platform

3. Learning materials: tools, facilities, learning media and other resources (digital, print-based, other materials)

4. Groups, roles and relationships: teacher-led and peer learning, varieties of group learning

In relation to learning content, materials were traditionally print-based (text or graphical). Both live classes and own-time study now involve the use of digital resources far more than print - e-books, e-journals, presentation slides, videos, digital maps and timelines, digital notes and annotations.

Digital environments can change the meaning and the learning potential of interactions in many ways. These can be impacted by many variables such as learner preferences and needs, experience and competence in communicating and/or collaborating digitally, and different curriculum and teaching approaches.

Other aspects can be considered and may even be 'blended': for example student-led and teacher-led learning.

Tips and ideas

Think beyond traditional session types. Rather than trying to 'translate' existing sessions into an online mode, look at how new activities and session types are emerging, and how digital technologies can expand notions of time and space.

Think of time and place as resources that link curriculum needs with organisational processes (room allocation, timetabling...).

Focus on high value space/time combinations for learner engagement - explain the value clearly, in terms that will make sense to students.

Remember to get informal and formal feedback from your students around the space/time combinations that work for them.

Different modes and blends provide settings for different activities and interactions - the modes approach complements, but does not replace, a focus on learning activity.

Four modes of participation in learning and session types

When considering the four aspects of learning the last two, learning materials and groups, roles and relationships can often be decided flexibly and collaboratively as learning unfolds. The first two, time and pace and space and time, shape the overall design of the curriculum and require some forward planning.

The practical resources in this guide illustrate some of the educational reasoning to help you plan a particular blend of modes across diverse sessions and activities.

We have identified four possible modes of participation in learning that shape every session, activity and interaction. They are:

Tips and ideas

Understand the four modes as pedagogic resources when planning activities - what works for your students, your subject area and your setting? Can you share them with students to help them understand your expectations around their participation?

Session types

Each participation mode supports a different set of learning sessions with specific opportunities for learning. Different subject areas will make use of different session types, sequences and blends.

Curriculum design workshop cards

Our PowerPoint slideshow offers more detail for sessions in each of the modes describing features, pedagogical benefits, platform/place, facilities/functions and typical activities and interventions.

These session type 'cards' can be downloaded and used in a curriculum design workshop or course meeting to:

Beyond blended posters

We developed a series of posters to illustrate modes of student participation across different spaces, places and times. They provide a simple, graphical overview of how the four aspects of the beyond blended approach (place, time and pace, learning media resources, groups roles and interactions) work for different types of activities across the four different modes of participation (sync in place, sync online, async in place, async online). The posters offer a range of examples:

The posters have been designed as flexible resources that can be integrated with existing design frameworks. You can use them as starting points for discussions with a range of key stakeholders including students, programme/course design teams, learning space working groups, library, estates, IT and other professional services. They can help to describe, share and develop common understanding of the current context as well as help to develop the 'what will be' in relation to student engagement and the development of beyond blended approaches.

They could be used as starting points for discussions around questions such as:

While they can be used in one-off sessions with groups of students, a more consistent and wide-reaching approach could be achieved if the organisation incorporates them into a programme of activities to support student digital learning capabilities.

Suggestions and contexts for using the beyond blended posters:

Downloadable resources:

We also provide blank versions of the posters that can be adapted and/or expanded on depending on your own context.

Six pillars for designing 'beyond blended' learning

Working closely with our advisory group and workshop participants we have developed a simple set of precepts or 'pillars' for curriculum design in the four modes of participation in learning. These are not intended to replace other principles and frameworks, but to be used alongside them.

The pillars can be referenced early in the curriculum design process, when considering the overall shape of the curriculum, the number and type of sessions and the learners' journey through place and platform, live time and independent study. You can also use them near the end of curriculum review to check back that the questions we pose have been asked, even if the answers are still emerging. And, finally, they can be used in dialogue with other strategic processes, to ensure alignment.

The six pillars are:

1. Place: learners and educators are always physically somewhere

2. Platform: learners and educators can always be (virtually) somewhere else

3. Pace: learners experience time and pace differently (synchronous/asynchronous or responsive/reflective)

4. Support: learners and educators need support to engage in diverse modes

5. Flex: learners and educators expect choice and flexibility in mode(s) of learning

6. Blend: most learning has in place and online, synchronous and asynchronous elements

Downloadable resource: Beyond blended guide (pdf)

Our beyond blended guide features the six pillars on one side and on the other side is a wheel describing the different aspects of beyond blended: definitions, session types, process model, modes of participation, strategy and planning. This prints in A3 format.

Six pillars: place
Learners and educators are always physically somewhere.
Learning and teaching are activities grounded in physical spaces. Regardless of the form that education takes, whether it is traditional classroom settings, online learning environments or informal educational contexts, learners and educators both occupy a specific physical location while learning and teaching. This could be a university campus, a home office, a library, a coffee shop or even various outdoor settings.

The physical environment plays a crucial role in the educational experience, influencing factors such as accessibility, comfort, available resources and how conducive the overall atmosphere is to learning and teaching. The physical presence of learners and educators in a shared space can also foster interaction, collaboration and a sense of community.

Some curriculum design questions to consider about place:

Six pillars: platform
Learners and educators can always be (virtually) somewhere else.
Virtual platforms enable learners and educators to transcend physical boundaries and engage with educational content, resources and communities from almost anywhere. The dynamic and flexible nature of virtual presence allows for a diverse range of learning experiences. Learners and educators can connect, interact and share knowledge through formal courses, online events and collaborate with peers globally through digital tools. Educators can reach a broader audience, incorporate a variety of multimedia and interactive resources into their curriculum, experiment with innovative teaching methods and collaborate with peers and experts globally.

Virtual environments can allow learners to explore subjects at their own pace and according to their interests, and they offer potential to enhance the educational experience. This virtual mobility in education fosters a more inclusive and accessible learning environment, where knowledge and learning opportunities are not limited by a person's physical location.

Some curriculum design questions to consider about platform:

Six pillars: pace
Learners experience time and pace differently (synchronous/asynchronous or responsive/reflective).
Learners can engage with and process educational content in a variety of ways. In synchronous learning learners and educators interact in real time, such as in traditional classroom settings or live online classes. This environment is often responsive, requiring immediate engagement and interaction. Learners must keep pace with the instructor and their peers, which fosters a sense of community and immediacy but potentially limits individual reflection and processing time.

In contrast, asynchronous learning allows learners to engage with material at their own pace, in their own time. It offers learners the opportunity to digest information, reflect, think critically and apply knowledge without the pressure of immediate response. Asynchronous learning can be particularly beneficial for those who need more time to understand concepts or who juggle their education with other responsibilities. Both synchronous and asynchronous learning approaches offer distinct advantages and cater to different needs, emphasising the importance of flexibility and choice for learners.

Some curriculum design questions to consider about pace:

Six pillars: blend
Most learning has in place and online, synchronous and asynchronous elements.
The integration of both in place (physical) and online components, along with a mix of synchronous and asynchronous elements, creates a multi-faceted educational environment that can support a wide range of learning preferences and needs. In this approach learners can engage in real-time, in-person interactions during physical classes or lectures, benefiting from immediacy and communal aspects of learning. They can also be provided with online resources and digital platforms where they can access course materials, participate in discussions and complete assignments at their own convenience.

This combination of in place and virtual learning experiences allows for a dynamic and adaptable educational experience. Incorporating both synchronous and asynchronous approaches ensures that learners can benefit from structured, real-time engagement as well as thoughtful, individual exploration and reflection. It is a comprehensive approach to blending that recognises the diverse needs of learners and offers a more inclusive and versatile learning environment.

Some curriculum design questions to consider about blend:

Six pillars: flex
Learners and educators expect choice and flexibility in mode(s) of learning.
Learners increasingly seek options that allow them to tailor their educational experiences to their personal and professional lives, preferring a mix of in-person, online, synchronous and asynchronous learning opportunities. This flexibility enables students to balance education with other commitments, such as work or family, and to engage with material in ways that suit their learning preferences and pace.

Educators need to adapt to changing expectations by employing a variety of teaching strategies and technologies that support different modes of learning. They recognise the value of offering choices to enhance engagement, motivation and success across their diverse student cohorts. HE organisations that provide this level of choice and flexibility are better positioned to meet the evolving needs of their learners, fostering a more inclusive and effective educational environment.

Some curriculum design questions to consider about flex:

Six pillars: support
Learners and educators need support to engage in diverse modes.
Both learners and educators need support to navigate diverse modes of participation in learning effectively. Learners, with varying backgrounds, abilities and requirements, require guidance and resources to adapt and thrive in these varied educational settings. This support can include access to digital tools for online learning, personalised assistance for those with special needs, or strategies to develop self-directed learning skills. They need appropriate support at various stages of their course to build on their digital capabilities to become effective digital learners.

Educators may need professional development and support before they can facilitate diverse learning experiences effectively, incorporating both digital capability and pedagogical innovation. They may also require support in creating inclusive and engaging learning environments that cater to the diverse needs of their students.

Some curriculum design questions to consider about support:
Downloadable resources:
This table lists curriculum and strategic prompts for place, platform, pace, blend, flex and support.

Using the beyond blended materials

Alternative approaches for different perspectives

All the beyond blended approaches and materials can be used in a range of ways as appropriate for different contexts. We have highlighted which materials might be most useful for each stage. The materials can support strategic organisational level conversations and they are useful for different stages of curriculum review and design processes.

Strategic approach to beyond blended

Designing 'beyond blended' learning requires a strategic approach that considers a range of factors and supports informed decision making. The prompts in our strategic lenses align with Jisc's framework for digital transformation in higher education, which offers a structure to support a shared understanding of digital transformation across a higher education organisation. The framework identifies strategic goals for a wide range of activities, as well as examples of how organisations might work towards those aims. The framework is part of a wider digital transformation toolkit that includes a comprehensive maturity model for each area as well as roadmap and action planning templates.

Framework for digital transformation: knowledge creation and innovation, knowledge development, organisational digital culture, knowledge management and use, knowledge exchange and partnerships, digital and physical infrastructure

The knowledge development area of the framework maps directly to curriculum and learning design as it includes four aspects:

Other areas of the framework relate to some of the wider strategic aspects of beyond blended design that need to be considered:

Organisational digital culture includes digital culture and mindset (including digital capability), organisational identity, organisational wellbeing and organisational change

Digital and physical infrastructure includes digital infrastructure, connectivity, cybersecurity, digital support and estates management

Knowledge management and use includes information and data literacy, learning analytics and library services

You may find the digital transformation toolkit useful to help articulate any problems you are trying to address or to identify ways to support developments or investment that might be needed to achieve curriculum and learning design goals. The toolkit may also be useful to help you consider the prompts included in the strategic and curriculum lenses.

Strategic lenses

We have produced a series of lenses to facilitate strategic level or curriculum design discussions involving stakeholders across the organisation. You may choose different lenses depending on your role or the stage of curriculum review and design that you are involved with.

Each lens offers a set of related prompts or questions for each of the six 'beyond blended pillars'. These are not definitive and can be edited to suit your organisational needs. There is some repetition between the lenses as there are some issues that are relevant across all of them. A blank template is also included so you can design a complete lens to suit your project or organisation.

The strategic lenses have been created for those leading curriculum change and those who are designing and implementing it. They highlight specific areas that require a strategic approach to address some of the challenges faced by the HE sector. Each lens provides a series of questions for each of the six pillars.

You can record your own specific strategic organisational questions in the blank template (eg the impact of a new build or refurbishment or expanding course types and student reach).
Downloadable resources:
Each lens starts from our 'beyond blended pillars' and offers a set of related prompts or questions. These are not definitive and can be edited to suit your organisational needs. There is some repetition between the lenses as there are some issues that are relevant across all of the lenses. A blank template is also included so you can design a complete lens to suit your project or organisation. These lenses have been designed to provide a starting point for strategic discussions around blended learning, such as planning and using campus spaces, timetabling and other related agendas.

Curriculum review and design

Curriculum review is an ongoing process that typically occurs on a cyclical basis, but it can also be triggered by wider organisational factors (eg new builds or refurbishments, new platform procurement and implementation, new strategic goals, expanding into new types of courses) as well as external factors (eg disruptive events like the pandemic, National Student Survey, disruptive technologies such as AI, government interventions, changing student demographics).

Curriculum review can take place at different levels and could involve a range of different stakeholders representing teams with different perspectives. Review processes could take place over a period of time, could involve asynchronous activities as well as in person or online meetings or workshops. A review at strategic level could also feed into reviews at programme or module level, or even trigger them. The materials in this guide can support the different curriculum review processes at both a strategic and a more operational level.

Learning and teaching strategy review (organisational level)

Strategic review might take place in a range of contexts and be influenced by organisation-wide activities:

Materials to support strategic conversations and planning:

Programme design review (faculty/school level)

Programme design review might take place in different contexts and may be influenced by strategic review outcomes:

Regular or one-off (formal) programme review meetings involving teaching teams, educational developers, IT team/s, learning designers, estates team, students, quality assurance and compliance teams

Informal curriculum team or teaching team meetings (to inform more formal programme review)

Internal curriculum design workshops or events mediated by specialists (eg educational developers)

Module review (school/department level)

Module review may be needed as a result of strategic or programme level review, but may also happen as teaching teams continue to develop and refine their teaching sessions. Changes resulting from module review will ultimately feed back into programme review and some may require consideration at a higher level (eg changes to assessment):

Teaching team meetings (formal and informal)

Wider meetings with invited specialists (eg library, educational developers, e-learning team, learning designers)

Meetings with students (co-design)

Curriculum and learning design process

Curriculum and learning design involve planning and decision making, based on intentional design principles in the context of available information. It leads to outcomes that can be used to communicate and coordinate the curriculum, and shapes learning, teaching and assessment. There are some common processes used across the UK sector in relation to curriculum and learning design. Our research identified that some issues related to curriculum review and design have become more prominent and are prompting the sector to rethink the curriculum design process. These include:

Ensuring coherence and development pathways in programme-level design

The rate of change demands more regular review and more in-built flexibility

A greater variety of courses on offer in terms of time and place creates more complex curriculum designs and portfolio offerings (eg micro, distance)

The need to represent complex curricula to all involved (eg learner journeys, course maps, flexible models, choice points) and the role of data in this

Deciding how to engage students in design (complex, time-dependent)

The impact of organisational agendas on curriculum design (eg EDI, wellbeing, sustainability [use of place and space])

The need to identify gaps between organisational policies/frameworks and design practices

A lack of recognition (time, resource, expertise) for curriculum design

Emerging interest in curriculum analytics - what data in practice is available and is useful, respecting student privacy?

Curriculum and learning design process model

The diagram below illustrates different elements of the design process and how they relate to each other.

Process model diagram showing different elements of the process and how they relate to each other: key information for design, design process, deliverables

Key information for design (purple section on the process model diagram)

Design is informed by a range of internal organisational factors. It can be constrained by operational aspects such as costing models, staff resources, timetabling, allocation of rooms and other available facilities. Design should also reflect organisational values and principles outlined in strategic documents and consider EDI policies. In addition, designers need to understand features of their typical student cohort, their experiences and expectations around the value of the course (through stakeholder insights, alumni surveys) as well as internal quality frameworks and requirements.

External information should inform the overall rationale articulated in the business case for a course. This can include labour market and competitor intelligence or futures-thinking to identify potential demand, including graduate and employer needs. Design must also meet government standards and benchmarks as well as specific requirements from professional and standards bodies and other wider agendas shaping learning, teaching and assessment.

Key information feeds forward into the design process and is fed back as design is realised in practice as data from systems (eg student achievements and destinations), and qualitative feedback from teaching staff and students.

Key information can shape design in several ways:

Design process (orange section on the process model diagram)

Curriculum and learning design may take place as a formal meeting, workshop-style session, a design sprint, or over a longer time frame, coordinated by shared documents or a shared design space. It may be carefully structured and facilitated, for example using established external or internal frameworks, or more open-ended. It is important that protected time/space for collaborative working and co-production is provided.

Design involves making decisions about specific aspects of a course, in accordance with (internal and external) principles or standards, and considering the available information. The principles may be broken down and/or made actionable in the form of prompts.

Aspects

The outcome of the design process is usually shaped by the required documentation or a design prototype (eg course map, timeline, pro forma, VLE instance). The prototype dictates the aspects of the course to be determined in the design process. Overall aims and outcomes for the course will be broken down into specific sessions, activities, experiences or learning periods, which shape the requirements for learning platforms and spaces, media and materials.

Principles and prompts

As previously described, the principles to be applied in curriculum design will be informed by internal or externally imposed factors, as well as the specific pedagogic interests and commitments of the course team. Consensus is needed to ensure that these are being realised through the course design.

The following prompts can help to support design across modes of participation:

How can different modes of participation (in place and online, live and asynchronous) support students to engage?

What modes best support the subject-specialist practices students undertake?

Where will students be studying? How will they work together and alone?

How much time will educators and students spend on different activities?

What choice and flexibility is really helpful:

To allow educators to respond to students' emerging interests and needs?

To keep each student engaged while keeping the cohort 'on track' together?

To provide pedagogically meaningful alternatives?

Curriculum design workshop or design sprint

Key considerations when running a workshop include:

Deliverables (green section on the process model diagram)

Decisions made in the design workshop are usually shaped by the way deliverables are specified, such as conforming to a particular style of documentation or a design prototype. These deliverables are used to communicate the design, to realise it in practice, to coordinate resources and to provide students with a coherent experience of their course overall.

Key considerations for realising a design in practice include:

How will the design and associated expectations be communicated?

How will the design support educators and learners to navigate their roles, tasks and interactions?

What choices and agency will educators and learners have?

What opportunities will there be to give and receive feedback?

Realising a design in practice (example):

Course handbook - decisions made in the design process are clearly communicated with the teaching staff and students who will realise them in practice

Timetable - synchronous and asynchronous session types are planned, allocating staff/student time and providing for different kinds of interaction

Spaces and places - online and real world spaces are made available, affording different interactions (teachers-learners-materials) and activities

Materials - different media and tools with different functions are provided within sessions and spaces to support learning activities and interactions

Design review (represented by the arrows in the process model diagram)

Part of any curriculum and learning design process is the review and refining stage (indicated by the arrows in the process diagram). It is critical to identify how feedback will be gathered from all stakeholders and how you will respond to this. Capturing the student voice is as important as gathering and analysing data around student outcomes. It is also useful to consider how you might respond to changes in teaching teams or cohort parameters. Many improvements can be made to course design without major revalidation, allowing you to respond to staff or student feedback in a timely way.

Other considerations include the need to choose an effective evaluation framework to enable improvement and how you can respond to changes to learning environments and resources. New online platforms, tools and learning spaces offer potential to offer different modes of participation and possibly changes in teaching and assessment practices.

Downloadable resources:

Our editable slide deck provides options to adapt and use the process model and associated resources in workshops or strategic review settings.

Process review

If your organisation is reviewing its curriculum and learning design process the following questions can prompt some useful discussions:

How well does the 'process model' reflect your experiences and practices?

Are there things you could be doing differently? Where does resource (time, attention, expertise) need to be added to make things work better?

If you have an existing framework or design principles, how are they realised in your current design process? How do they work alongside the process model and six pillars?

Our curriculum lenses aimed at practitioners can inform a review of curriculum and learning design.

Curriculum lenses

Our curriculum lenses are aimed at people who are supporting teams as they are redesigning or involved in managing the institution's learning design support. These are mapped to the six pillars and offer a series of prompts to consider. The prompts can be added to, or edited to suit course issues and organisational approaches to design.
Downloadable resources:
Each lens responds to our six 'pillars' with a set of prompts or questions for design teams. The prompts can be added to or edited to suit course issues and organisational approaches to design. A blank template is included to allow curriculum teams, departments or organisations to develop their own questions, meeting the requirements of their own curriculum principles and the needs of their own students. These lenses have been developed to support curriculum design for diverse and flexible modes of learning (that we call 'beyond blended').

Other established curriculum frameworks

We reviewed models and frameworks in use (over 30 models from across UK higher education) during phase one of this project. The approaches in this web guide are not intended to replace established frameworks for curriculum design, or to suggest that organisations should change approaches that are working well. To that end, we describe here how 'beyond blended' links to other approaches and can support them in practice.

Design as process

'Beyond blended' is a set of considerations that can be included in a wide range of design processes. It can be used in the sequencing phase of a typical design process to decide what sessions will be delivered and how students will participate in them. In the resourcing phase of a design process beyond blended can support planning for the physical spaces required and any specialist facilities and materials, the platforms to be used and why these are appropriate.

Examples: Co:Lab (2022), Carpe Diem (2020), UCEM module (2020) development process, ABC Learning Design (2015) workshop (process, not content).

Learning activity design

The four aspects of beyond blended learning (place/platform, time/pace, materials and group interactions) are included in most learning activity models. For example, an activity design model developed in previous Jisc projects, as described in our guide to designing learning and assessment in a digital age, has 'time' as an axis running through the middle of the model, from learners' prior state at the left to their achieved outcomes at the right. This time axis can be distributed into a range of sessions and activities within them. At the top of the model is the learning environment (place or platform). The roles that other people play in the learning activity (at the bottom of the model) are partly determined by the constraints of time and place.

Examples: Laurillard's Conversational Framework (2002), ABC version (2015) of Laurillard's activity types, Open University (OU-LDI) (2012) activity design cards.

Central circle: Activity - Interaction of learner(s) with environment leading to planned outcomes supported by other people in specific roles

Top right quadrant: Impact of learning environment on learning outcomes

Reciprocal arrow from central circle between quadrants: Intended learning outcomes - Acquisition of new knowledge, skills and capabilities; evidence of these

Bottom right quadrant: Roles others play in facilitating learning outcomes

Reciprocal arrow from central circle between quadrants: Other people - Peers, tutors, facilitators, mentors, instructors, employers

Bottom left quadrant: Interaction between learners and others involved in the activity

Reciprocal arrow from central circle between quadrants: Learners - Preferences, needs, motivation, skills, knowledge, capabilities; modes of participating

Top left quadrant: Interaction between learners and aspects of the learning environment

Reciprocal arrow from central circle between quadrants: Learning environment - Networks. devices, resources, applications, physical learning space and virtual environment

Constructive alignment

As well as aligning activities with outcomes (for example using Bloom's taxonomy [2001] [revised], the SOLO taxonomy [2019]), activities can be aligned with an appropriate mode of participation in learning. The resources in this guide offer a wide range of activities mapped to different sessions and modes.

Learning cycles

These models describe a preferred order of activities to support and structure learning. Different modes of participation in learning might be more or less appropriate at different phases of the cycle. For example, a flipped classroom approach might typically start with assimilation activities (asynchronous and online) followed by a consolidation session (synchronous, in place) where students can ask and answer questions, share ideas, practice application and get formative feedback, followed by a review phase (online again). A Kolb learning cycle might schedule on-site activities during the concrete experience and active experimentation phases. These activities could be live or asynchronous (eg students collecting data within an allocated time window, with access to a lab or field site).

Examples: Kolb's experiential learning cycle (1984), TREC (trigger, review, expectations/evidence, consolidation) (2021), Merrill's instructional design principles (2002), flipped learning models (2006).

Timeline and storyboarding approaches

Example: ABC Learning Design (2015) (storyboard aspects). The different sessions outlined in our downloadable pdf session types in the four modes of participation in learning can easily be incorporated into a timeline or storyboard approach.

Universal design for learning

One of the benefits of exploring modes of participation in learning from an educational perspective is that it reveals differences in how students respond to place and platform, the different paces of learning students prefer (reflective/responsive) and whether they prefer tasks to be live or undertaken in their own time, structured or relatively open-ended. Programme teams can consciously use different modes to ensure students all have some learning experiences that suit them, or that challenge them to try new approaches. Within modes, teaching teams can provide multiple ways for students to interact by exploiting different configurations and functions.

This is highly compatible with the first UDL principle of offering students multiple ways to engage and the third principle of offering multiple means of acting and expressing ideas.

Example: UDL Guidelines (2018).

Co-production/co-creation

Attention to modes of participation in learning does not create any new opportunities to involve students in curriculum planning. But it does help to engage students meaningfully if there is a clear vocabulary that describes different sessions and kinds of learning and also puts student learning centre stage. Talking about responsive and reflective time, or the difficulties of learning in transit, or the way certain resources can cross time and place, all allow for new and productive conversations with students. The same is true of communication among professionals involved in the curriculum.

Principles-based models

Many universities use a set of principles to guide curriculum design. Designing for different modes of participation in learning is compatible with a wide range of curriculum principles such as equity, diversity and inclusion, sustainability, digital capability, citizenship, student-centred learning and more. Our strategic lenses (described above) have mapped our six pillars to some of these agendas and the same could be done for other principles identified at HE provider level or by a programme team.

Examples include these from the University of Liverpool (2023) and University of Portsmouth (2018).

Resources feedback

Are you finding the resources in this guide useful? Fill in our form to let us know how you are using the resources at your institution. Your feedback is really important to us and helps us develop more guidance.

Fill in our beyond blended resources feedback form.

# data[].attributes.attachment_link.title
Beyond blended: rethinking curriculum and learning design - Jisc

# data[].attributes.attachment_link.description
The need for new approaches Four aspects of beyond blended learning Four modes of participation in learning and session types Six pillars for designing 'beyond blended' learning Six pillars: place Six pillars: platform Six pillars: pace Six pillars: blend Six pillars: flex Six pillars: support Using the beyond blended materials

# data[].attributes.headline
Standards and interoperability still an issue 

# data[].attributes.subject
Standards and interoperability still an issue 

# data[].attributes.body
Main blocker to synoptic assessment for us is our Student Information System!!

# data[].attributes.attachment_link.title
Learning Impact Europe Conference | 9-11 October 2024 | Barcelona, Spain

# data[].attributes.attachment_link.description
Learning Impact Europe is a fantastic opportunity for education providers, public bodies, and edtech suppliers to come together and tackle the most pressing issues in education. We’re working to create a trusted digital ecosystem built on open standards and collaboration.

# data[].attributes.headline
Hybrid, media rich, personalised assessments

# data[].attributes.subject
Hybrid, media rich, personalised assessments

# data[].attributes.body
I see process oriented assessments as a future trend. I also see / would like to see better technical support for more complex (in the context of our current tool sets) assessments - where students will still be asked to present some form of written 'narrative' (which can be subject to the same and emerging academic integrity checks) - but alongside this can submit linked media rich artefacts that demonstrate process, performance, personal involvement etc. Robust support for this seems limited (e.g., large file submissions that are date stamped, locked and accessible for quality feedback etc.). The 'personalised' element here is that students should have the ability to select the format of their artefact - to express their capabilities - in their way.

# data[].attributes.body
I wrote a blog after the session which may be slightly more coherent than what I said in the September meeting. Or maybe not.

# data[].attributes.body
It would be great to find a way to facilitate more cross-sector sharing of experience and learnings related to assessment delivery. As I pointed out on the call, some sectors, for example the work-based and professional learning sectors have been using in-person digital assessments for many years. So, we don't need to reinvent the wheel all the time and indeed there may be opportunity not only to improve student experiences, but also open-up new commercial opportunites for universities in quieter periods of the year.

# data[].attributes.attachment_link.readable_text
Why review assessment practice?

Introducing the seven principles

Principle one - help learners understand what good looks like

Principle two - support the personalised needs of learners

Principle three - foster active learning

Principle four - develop autonomous learners

Principle five - manage staff and learner workload effectively

Principle six - foster a motivated learning community

Principle seven - promote learner employability

Why review assessment practice?

The pandemic accelerated digital transformation of assessment for some institutions and caused others to question not only how they do things but whether they are doing the right things?

Assessment and feedback forms a significant element of staff and student workload and many studies have shown that students, in higher education, are less satisfied with assessment and feedback than with any other aspect of the experience.

This guide will help you determine what good assessment and feedback looks like in your context so you can benchmark existing practice and plan improvement.

Assessment and feedback landscape

Through consultation with universities, a survey and a review of the current literature we gained a picture of the UK assessment and feedback landscape in higher education. View Jisc's assessment and feedback higher education landscape review.

About the examples in this guide

Each of the principles is accompanied by short case studies of implementation.

Where possible we have chosen examples where digital technology has been used to support and enhance good practice.

Previous research found good practice was often difficult to scale up because it required manual intervention or tools that were not interoperable. Our examples show innovative practice delivered at scale and using open standards to facilitate seamless integration with existing tools and administrative systems.

Assessment and feedback: direction of travel

For a number of years assessment and feedback practice has been on a trajectory away from assessment of learning to what is termed assessment for learning.

Key to this has been helping students monitor and regulate their own learning and trying to ensure that any feedback activity feeds forward leading to future improvement.

Current assessment practice increasingly includes activities that could be termed assessment as learning. The very act of undertaking assessment and feedback activities is an essential part of the learning process.

All three aspects of assessment still need to happen but we are thinking differently about the relationship between them.

Assessment of learning describes the institutional quality assurance processes that lead to students acquiring some form of verified credential.

Assessment for learning is the overall learning design, ensuring we are assessing the right things at the right time with plenty of formative opportunities to feed forward. This is the cog wheel making everything revolve.

Assessment as learning is a learning experience where the formative and summative elements work well together. Tasks appear relevant, students can see what they have gained by undertaking the activity, they feel involved in a dialogue about standards and evidence and the continuous development approach helps with issues of stress and workload for staff and students.

Text description for assessment with purpose graphic

Purple centre - the inner circle represents assessment for learning the institutional quality-assured processes that lead to a qualification.

The middle cog represents assessment for learning learning design emphasising formative opportunities that feed forward to future improvement.

The outer circle represents assessment as learning the lived experience of students and staff when active learners contribute to decision-making and are able to monitor and regulate their own learning.

Beyond the Technology podcast

Listen to our series on rethinking assessment and feedback from the Beyond the Technology podcast:

Introducing the seven principles

Educational principles are a way of summarising your shared educational values as a college or university. They serve to guide the design of learning teaching and assessment.

A well-thought-out set of principles:

Describes a shared set of values and a vision

Summarises and simplifies a lot of research evidence on good pedagogic practice

Provides a benchmark for monitoring progress

Serves as a driver for change

Principles offer a robust way of gaining ownership and buy-in and they need to be written in a way that requires action rather than passive acceptance.

Our seven principles

1. Help learners understand what good looks like by engaging learners with the requirements and performance criteria for each task

2. Support the personalised needs of learners by being accessible, inclusive and compassionate

3. Foster active learning by recognising that engagement with learning resources, peers and tutors can all offer opportunities for formative development

4. Develop autonomous learners by encouraging self-generated feedback, self-regulation, reflection, dialogue and peer review

5. Manage staff and learner workload effectively by having the right assessment, at the right time, supported by efficient business processes

6. Foster a motivated learning community by involving students in decision-making and supporting staff to critique and develop their own practice

7. Promote learner employability by assessing authentic tasks and promoting ethical conduct

Applying the principles

The principles offer an actionable way to improve learning teaching and assessment and can be applied to any aspect of learning design.

There is no one size fits all approach. You need to decide what the principle means in your context and how best to apply it to the learning experience in your organisation. Our self-assessment template (.docx) allows you to adapt the principles to your own context. You can also download the self-assessment template as a pdf.

Do principles change over time?

Principles need to evolve over time if they are to reflect current education research and remain aspirational and a call to action.

The usefulness of educational principles drew widespread attention when Professor David Nicol published a set of principles for good assessment feedback practice as part of the Re-Engineering Assessment Practices (REAP) project in 2007.

Many education providers saw the value and adopted the idea so there are various sets of principles in the public domain ( see ESCAPE assessment principles pdf). Almost all of them are based on the basic REAP principles.

Our 2021 principles reflect the greater prominence of issues such as accessibility and inclusivity in current thinking. Where we continue to champion examples of good practice that were recognised some time ago, it is with a new perspective on why and how certain approaches are more effective than others.

It is too early to judge how long this set of principles will last. At the moment they represent a call to action and a manifesto for change. We very much hope to find that in five years time they are no longer aspirational and require another refresh.

Related resources

Designing learning and assessment in a digital age - A primer created for university leaders exploring the current and future opportunities for digital assessment.

QAA Hallmarks of Success Playbook: Assessment in digital and blended pedagogy - A useful guide to develop a shared understanding of assessment and feedback practices that are relevant to and support teaching and learning. It examines what success looks like in a digital and blended environment, barriers that may be encountered and how they can be overcome. The playbook can be used to drive discussions about assessment as learning, feedback and feedforward practices, as well as considering quality enhancement, moderation and academic integrity.

Assessment ideas for an AI-enabled world (.pptx) - Codesigned with the sector, we've produced a set of postcards to provoke discussion and reflection around assessment approaches in higher education. Each idea postcard outlines the type of assessment task, key characteristics and appropriate formats.

Principle one - help learners understand what good looks like

Why is this important?

Students undertaking an assessment need to understand what is required of them and the criteria against which their performance will be judged.

This sounds self-evident but many learners find it difficult to derive this information from the available sources such as course and module handbooks. Even where criteria and grade descriptors are provided, they may be couched in academic language that requires some skill to interpret.

It helps to have a common template for assignment briefs, so that the essential information is presented, in plain English, in a consistent way for each assignment.

Feedback for learning: a comprehensive guide, an e-book by FeedbackFruits, gives useful guidance on designing rubrics that support active learning. It explains how to create a rubric that is relevant, unambiguous and serves as a bridge to future performance by avoiding traps that can result in a rubric becoming a task-focused checklist.

By implementing this principle you will, however, take this a step further and engage students in activities that allow them to create meaning from the criteria and engage in discussions about quality. You may ask them to rewrite criteria in their own words and compare their understanding with that of others. Later they may even define the criteria by which they think their work should be judged.

The role of feedback

The capacity to recognise and interpret feedback, and use this to lead to further improvement, is key to developing the ability to recognise what good looks like

Feedback provides information about where a learner is in relation to their learning goals so that they can evaluate progress, identify gaps or misconceptions and take action that results in enhanced performance in future.

Feedback should be constructive, specific, honest and supportive.

Effective feedback shouldn't only focus on current performance or be used to justify a grade. It should also feed forward in an actionable way so the learner understands what they need to focus on in order to improve.

Feedback is conventionally thought of as a dialogue between student and tutor but it can come from a variety of sources. Self and peer feedback can be equally valuable and we discuss this further in relation to principle number 4 develop autonomous learners.

Ipsative assessment

Ipsative assessment is the idea that, rather than evaluating a learner's performance against an external benchmark, we simply look at whether they have improved on their own previous performance. This is after all the most effective measure of learning gain.

At University College London this approach is proving useful with master's level students and those studying MOOCs.

To grade or not to grade?

Defining what good looks like does not have to mean assigning a grade.

There is a school of thought that believes grading is counter-productive and diminishes students' broader curiosity about a subject. Read more about upgrading in Jesse Stommel's article.

Diverse assessment formats and different customs and practices across disciplines may distort marks. Even without these complications, experts question whether it is possible to distinguish the quality of work with the precision implied by percentage marks.

Higher education does indeed operate without grading in some areas. We recognise the value of a doctorate without questioning whether the holder achieved a few percentage points more or less than one of their counterparts.

The absence of a grade can oblige learners to focus on their feedback and encourage deeper reflection.

Technology can help

Some of the ways technology can help:

Making information about the assessed task and criteria available in digital format helps ensure it can be accessed readily on a range of devices. It also helps with applying a consistent template

Having the information in digital format allows cross-referencing to a wide range of examples

Use of digital tools makes it easier to give 'in-line feedback' so students can see to which section of their work a feedback comment refers

Digital tools can support better dialogue around feedback

Storing feedback in digital format increases the likelihood that students will refer back to the feedback in future

Digital tools that support self, peer and group evaluation are all means of actively engaging students with criteria and the process of making academic judgement. You can find examples of this in the section about putting each of our principles into practice throughout this guide

Putting the principle into practice

Guidance and templates

Sheffield Hallam University makes all of its guidance and templates available via its assessment essentials website.

Manchester Metropolitan University (MMU) has developed guidance on assessment grading, criteria and marking (pdf). This helps ensure consistency across the organisation. MMU uses an app to make this information available to students via any mobile device.

The University of Reading's A-Z of assessment methods (pdf) can help you choose an appropriate type of assessment

Adaptive comparative judgement

Assessing a piece of work objectively against a rubric is not easy even for an experienced evaluator.

Comparative judgement works on the principle that people make better judgements if they compare two items, and decide which is better, than if they try to evaluate something in isolation.

Repeated comparison of pairs ultimately allows the items to be rank ordered. This usually takes nine to 12 rounds of comparison.

Adaptive comparative judgement (ACJ) tools automate the process of presenting a group of assessors with pairs to compare. The items are initially selected randomly. Comparison between very good and very bad is obvious very quickly. The computer algorithm can then start to select the pairs that will most improve the reliability of the ranking so more effort goes into assessing those that are closely matched.

Research has been undertaken using this technique to compare the evaluative judgement of staff and students. It has also been applied to using artificial intelligence (AI) to partly automate marking of complex items such as essay scripts.

The most common application is, however, in engaging students with assessment criteria and 'learning by evaluating' to identify what good looks like.

Using ACJ tools to provide peer evaluation and feedback can provide learners with a rich body of evidence to help them improve their work.

ACJ is being used at many universities internationally. In the UK examples include use at Goldsmiths, University of London and the University of Manchester.

The technique is explained in this presentation and session recording, which includes a case study from the University of Manchester.

Developing research skills at the University of Waterloo

Students undertaking an education degree at the University of Waterloo have the opportunity to gain a research credential in their specialist area during their work experience placement.

To gain the credential, the students have to assess their own research against a research skill development framework and demonstrate, during a capstone interview, the evidence why their experience merits special recognition.

The process is supported by a structured workbook in the e-portfolio that allows students to engage with the assessment framework before, during, and after their individual research experiences. The workbook acts as the basis for receiving personal guidance and feedback to support independent study.

"Engaging in this work reinforced how influential even a small degree of reflection and feedback can be in supporting student success. Implementing these changes required minimal additional effort but had huge impact on the students' ability to relate their experiences to the assessment framework."
Jeremy Steffler, University of Waterloo

Read more about this University of Waterloo case study.

Principle two - support the personalised needs of learners

There is no such thing as a standard learner. Disability, neurodiversity, cultural and religious background, work and family commitments and personal preference all play a part in shaping our experience of learning.

Moreover, these factors combine to make the learning experience individual. Disabled students are no more a homogenous group than learners that have been bundled together under the unhelpful label BAME (black and minority ethnic). In March 2021, the Commission on Race and Ethnic Disparities recommended that the UK government stop using the term BAME. The government is currently considering its response to the Commission's 24 recommendations.

Accessible and inclusive

Good assessment and feedback practice should allow a learner to apply their own individuality to demonstrating their knowledge and competencies.

We are increasingly aware of the need to make assessment accessible ie to ensure that people with disabilities do not face barriers because of the format or tools used. We need to promote similar awareness of the benefits of making it fully inclusive ie fair and equal to all students regardless of their diverse backgrounds.

Offering different means to demonstrate achievement of a learning outcome is an important way to comply with disability discrimination legislation. Applying this as a design principle goes further and enhances the overall assessment process to benefit all students.

Assessment designers need to be familiar with the law relating to accessibility and related guidelines such as WCAG (Web Content Accessibility Guidelines). For example, offering a digital paper as an alternative to hardcopy examination is only useful and legally compliant, if the digital copy meets accessibility compliance regulations. Software and resources used for digital assessment are legally required to meet the compliance standards. This needs to be considered in purchasing decisions and in-house development.

The emphasis on inclusivity relates to a broad agenda to ensure that our curricula are not skewed - for example to a particularly 'white' view of the world. There is much good work being undertaken on decolonising curricula and we need to ensure that assessment design reflects a global, multicultural perspective on the subject matter.

Compassionate pedagogy

Since the start of the pandemic there have been growing calls for us to develop more 'compassionate pedagogy'.

As online classes have taken us into one another's homes, teachers have gained a different kind of understanding of the conditions under which students study and the competing demands of work and family they have to juggle.

Digital poverty is only one aspect of this, time pressures and access to a suitable place to study are equally relevant and need to be factored into how we design learning and assessment.

Inclusivity means that no one's socio-economic background should be a barrier to learning.

Promoting mental health and wellbeing

Promoting mental wellbeing is high on the agenda amid survey evidence from higher education that both student and staff mental health has declined in recent years. Assessment points are often the trigger for stress and anxiety related illness.

An approach that allows for individual needs, and is founded on compassion for the learner, can help.

This is a topic that cuts across a number of our principles. Implementing principles number five: manage staff and learner workload effectively and number six: foster a motivated learning community will also have a positive impact.

Some of the ways technology can help

Using digital tools makes it easier to implement universal design for learning (UDL) guidelines by providing multiple channels and options for engagement with learning and assessment activities.

Digital technology offers a vast range of assistive tools that allow learners to respond to the same activity or content in a way that is adapted to their personal needs. Examples include screen readers and braille keyboards for blind students, video and audio captioning for learners with hearing problems, the ability to change font size and colour to support dyslexic students or certain types of visual impairment. Some locked down virtual desktops and third-party tools used for summative assessment do not permit the use of the assistive technologies required by some learners. This is an issue that should be fully investigated before using such tools.

Aside from these adaptations, taking a summative assessment in familiar surroundings using familiar digital tools, may prove less stressful than a traditional example setting for many learners. In some cases, formal examinations may be one of the rare occasions where a student uses a pen.

Artificial intelligence (AI) is enabling the development of 'adaptive' learning tools that respond dynamically to student responses in order to personalise learning pathways. Their main use is in formative testing. For example, a student can be presented with progressively more difficult questions based on what they answer correctly.

We are increasingly used to technology providing us with friendly reminders when we need to do something in many aspects of our lives. Digital tools provide various ways to give learners 'nudges' and reminders about upcoming deadlines and important due dates via email, text, mobile app or reminders in the VLE, helping to provide clarity and alleviate stress.

Putting the principle into practice

Making large classes feel small

A decade ago, the University of Sydney was unable to find a tool that could help solve the problem of engaging students with feedback and giving them personal care when cohorts could consist of thousands of students and learners felt lost in the crowd.

They developed a student relationship engagement system (SRES) which integrates with any learning management system using LTI and is now freely available to other universities. SRES is described as a 'Swiss army knife for instructors'.

Today over 3,000 teachers are using SRES across more than a thousand courses at the University of Sydney. There were over a million student interactions with the portal in 2021 and 92.4% rated it as helpful. This is enhancing the learning experience of almost all 75,000 students at the University of Sydney with many more users worldwide.

Text description for elements of the student relationship engagement system (SRES) approach diagram

Top row title is evaluative judgement with self assessment, peer feedback, teacher feedback and rubrics underneath

Second row title is developing feedback literacy with appreciating feedback, making judgements, managing affect and taking action underneath

Beneath these two rows is listed positive relationships, which results from the factors in the other boxes being implemented

This diagram shows the elements that are important in contemporary approaches to helping students get the most out of assessment and feedback. Most importantly, they all exist within a wrapper of positive relationships with the tutor and others in the learning community.

SRES allows tutors to curated data from various sources in tailored dashboards that tell them what they need to know in order to best support their particular learners.

Data about logins and grades can be accompanied by other information that helps the instructor get to know the students. This could involve asking students about their preferences, what helps them learn and even their dreams and ambitions. Another example is that students can record an audio clip of how to pronounce their name.

These questions are embedded in the learning management system (using LTI) so to the learner they appear seamlessly alongside an introductory course in module information.

Highly personalised feedback can be created within the context of data such as whether the student has submitted on time or how they say they are feeling.

SRES also helps turn feedback into a dialogic process. Before submitting an assignment, students can say what they would most like feedback on, so they are primed to be more likely to engage with that feedback. Tutors can also close the feedback loop by asking students to respond and say what they thought about the feedback and what they will do differently next time. It is also possible to hide the grade until the student has responded to the feedback.

SRES also supports a wide range of peer-to-peer feedback activities.

Another use is to facilitate learning from both self and peer feedback. The portal can present results of student self-assessment alongside the evaluation given by their peers encouraging them to reflect on how this makes them feel and how they plan to improve on the next task as a result.

Closing the attainment gap at Brunel University

Brunel University implemented digital open book assessments as a response to the pandemic and discovered that this helped close the attainment gap for certain students. Students who entered the University with BTEC qualifications, particularly black students with a BTEC, did significantly better in terms of degree outcome than in previous years.

This improvement can be attributed to the change in assessment practice with the open book format being similar to their prior experience. Closed book examinations tend to benefit students with A-level qualifications who are used to being assessed in this way.

"Instead of sitting in a sports hall for three hours and having to rely on memory, students were able to use any resources available to complete the task - something that is probably more akin to what those with BTECs would have done previously."
Mariann Rand-Weaver, Brunel University

Read the full case study in Jisc's rethinking assessment report.

Supporting remote placement students

At Murdoch University hundreds of nursing students each year undertake a rural or remote clinical placement often up to 3000 km from home.

This placement may be the first time that the student is away from family, friends and normal support structures for an extended period. The staff member monitoring the student's progress is rarely in the same location as the student and internet connectivity cannot always be relied upon.

The University uses its PebblePad e-portfolio tool to help connect with, and support, the learners.

As a result of codesign discussions with students, compulsory reflective assessments about their clinical experiences, have moved from written format to video blogs. Students can record the vlog anywhere and upload them to PebblePad later to be shared with on-site and remote assessors.

These 'vlogs' allow for individual creativity and personal connection as the tutors are now using the vlog format to provide feedback to the learners.

Beyond the Technology podcast

In our podcast on rethinking assessment and feedback, Danny Liu and Benjamin Miller speak about how they are adapting their assessment and feedback practices at the University of Sydney and how their use of the student relationship engagement system (SRES) has helped to solve the problem of engaging a large number of students with personalised feedback.

Principle three - foster active learning

Active learning has been a key tenet of curriculum design and learning space design for several years. Approaches to teaching have evolved in response to evidence that techniques such as lecturing, in contexts where students are passive recipients of information, are highly ineffective.

Building in formative assessment opportunities greatly enhances the effectiveness of learning activities.

The use of quizzes delivered via mobile devices during lectures, is a common example.

Active engagement with learning resources

We tend to think of active learning as something that involves interaction with a teacher or with peers but engagement with learning resources also offers opportunity for learners to evaluate their own understanding.

Interactive questions embedded in online textbooks is an emerging area that has considerable potential for learning enhancement.

Even in the absence of sophisticated digital tools, teachers can include discussion points for example by linking the prompts to a discussion thread in material on the learning platform.

Research by Professor David Nicol at the University of Glasgow suggests we have previously focused too much on feedback comments, and the dialogue around them, as the main mechanism by which students evaluate the quality of their own work. He believes that comparison with learning resources can be equally valuable in building student capacity to plan, evaluate, develop and regulate their own learning.

Time to revise

An area that receives little attention in learning design is how students prepare themselves for summative assessment. What revision strategies do they employ and what resources do they use?

The Open University, UK recognises this and carries out an annual survey on student experience of assessment, feedback and revision (SEFAR). It concludes that revision and examination represents a distinct phase of learning but it remains challenging to determine effective analytics for this phase that can help inform better learning design.

The most conclusive finding was that the use of sample papers for practice appears to improve exam question literacy. Conversely, lack of engagement with these resources could be a trigger that a student requires some targeted support.

Some of the ways technology can help

The use of quizzes delivered via mobile devices during lectures, is a well-known example of formative testing. It can be used to check whether most students have grasped key concepts and allow the lecturer to adapt the session if further explanation is required.

Interactive questions embedded in online textbooks is a more recent development. Research shows this type of practice question can significantly enhance learning compared to straightforward reading of text

Video resources can be very engaging but difficult to navigate. Digital tools that help students bookmark and add notes to particular sections, foster active learning and aid revision.

Putting the principle into practice
The Doer Effect
Research from Carnegie Mellon University suggests a causal relationship between students doing practice questions while reading and enhanced learning outcomes. This phenomenon, known as the ' doer effect ' has been replicated in very large-scale studies in other US universities. The studies show that formative practice enhances the learning effect by a factor of six compared to simple reading of the text.

The research findings are very clear. The main factor inhibiting embedding such practice questions in every online textbook is the work needed to generate the questions. A textbook for a semester long module may require hundreds or thousands of questions requiring both subject matter and question item authoring expertise.

Artificial intelligence (AI) is a promising solution to this problem. A research study (pdf) based on well over three quarters of a million student-question interactions showed no significant difference in the difficulty of, or student engagement with, questions written by subject matter experts and those generated using AI.
Interactive presentation
Universities using Feedback Fruits software have access to a tool known as interactive presentation. The tutor can upload video content, such as a pre-recorded lecture, and lock specific moments in the timeline with practice questions.

Students have to answer these questions before they can continue to watch the rest of the video.

The teacher has access to an analytics dashboard to see which questions students struggle with.
Synote
Synote is an award-winning, open source application developed at the University of Southampton that makes video resources easier to access, search, manage and exploit.

Imagine trying to use a textbook that has no contents page, index or page numbers. Lengthy video recordings, such as recorded lectures, can be equally difficult for students to navigate.

Synote allows students to bookmark particular sections of a recording and associate their notes with the correct section. Students can also take live notes during lectures, using Twitter, on any mobile device then upload them into Synote so they can be synchronised with a recording of the lecture.

Principle four - develop autonomous learners

It is often said that one reason students tend to express a relatively high level of satisfaction with lectures, is the fact that they don't need to do anything during a lecture.

The remark is not entirely tongue-in-cheek. Tutors frequently complain that students see themselves as passive recipients of learning content. Similarly, in relation to assessment, some learners view it as the tutor's role to deliver feedback to them.

Developing students' ability to self-regulate and manage their own learning is a key goal of effective learning and assessment design. We also touch on this in relation to principle number one help learners understand what good looks like and principle number three foster active learning.

Activities that involve reflection and self, peer and group evaluation all work towards this goal.

The power of comparison

Many studies have observed that students appear to learn more from generating feedback for their peers than they do from engaging with peer feedback comments provided for them.

Research by Professor David Nicol, at the University of Glasgow, suggests this is because when students review their peers' work, after producing their own, they make comparisons of the peer's work with their own and this activates powerful internal feedback. Such comparisons can generate valuable learning whether the work of the peer is stronger or weaker than the reviewer's own.

A study comparing peer and tutor feedback (Nicol and McCallum 2021) found that students of all abilities were able to identify all areas for improvement that the tutor identified as well as areas that the tutor did not mention. However, to match the tutor feedback, the students had to make multiple sequential comparisons (ie compare more than one peer work with their own).

This research suggests that well-structured peer review activities can reduce teacher workload and generate more and better feedback for learners.

Self-paced learning

Most educators recognise the importance of developing learners' capacity to self-regulate and the role that engagement with feedback plays in this. Our efforts in this area are however, sometimes at odds with a fixed regime of content delivery via lectures and a rigid assessment schedule. Inevitably, some learners struggle to keep up whilst others are insufficiently challenged.

Higher education could do more to encourage self-pacing within bounded limits, such as an individual module. Students could be allowed to test their knowledge when they feel ready and resubmit until they have mastered a topic.

There may be lessons to be learned in this area from the school sector. The modern classroom project suggests the following classification of learning and assessment activities:

Must do: non-negotiable tasks covering core concepts and essential skills

Should do: valuable opportunities to develop skills that will not prevent the student transitioning to the next stage of learning if there is a good reason for them to be excused

Aspire to do: extensions for students who have already mastered the normal scope of the topic

Analytics to track progress help ensure that the group as a whole is on track. Sharing of aspects of goal setting and tracking can help students identify others to collaborate with or who can provide help.

Policies such as 'ask three (peers) before me' can encourage peer learning.

Similarly, assignment briefs can be split in two with half of the group addressing each aspect of the topic then teaching and learning from peers who did the opposite assignment.

Some of the ways technology can help

Early attempts to implement peer review found it could be time consuming to administer. Digital tools make it possible to implement peer review activities at scale.

Features such as allocation of reviews, linking to assessment criteria, managing anonymity and tracking which students have completed the work, are all easier in the digital environment.

Use of the open standard LTI means that tools supporting self, peer and group review can be seamlessly integrated into the learning management system.

Putting the principle into practice

Peer assessment at VU Amsterdam

Students in pharmaceutical sciences at VU Amsterdam (Vrije Universiteit) undertake peer assessment of one another's reports as a mandatory pass/fail part of their course2.

The exercise is structured so that the students are undertaking the peer assessment individually but the report they are evaluating is the work of a group of three students.

The students are required to address each of the assessment criteria so the feedback is complete. Use of technology enables direct linking to the assessment criteria and enforcing the requirement to address each criterion.

Learners are also required to rate the quality of the feedback they receive from other peer reviewers.

This activity engages learners with the assessment criteria and also encourages them to reflect on the process of giving good quality feedback.

Feedback given by students matches well with instructor feedback. This means staff time can be saved by monitoring a random sample to ensure quality is being maintained.

This approach, whereby engagement with the feedback process feedback is an individual responsibility, works much better than previous initiatives where groups gave feedback on other groups.

"Unlike in previous years, there were no longer any complaints about the feedback process."
Jort Harmsen, VU Ansterdam

Two stage examinations

It is not unusual for a learner to walk out of a formal examination and immediately think of something they should have done differently. Normally it's too late but what if you had the chance to put it right?

Research by Professor David Nicol and colleagues at the University of Glasgow has taken this idea step further and researched the impact of a two-stage examination structure3.

In their model a student takes an exam and then completes reflective questions to surface their internal feedback about their performance. They are asked to identify any weaknesses they are aware of and any aspects of their work on which they would like to have expert feedback.

The students then take the same exam again but this time working in groups.

At the end of this stage, they answer another set of reflective questions, for example, about how the group answer differed from their own, whether the group discussion made them aware of strengths and weaknesses of their own answer that they hadn't identified and which answer they thought was better.

The purpose of this study was to find out more about how students generate inner feedback through comparisons. The finding was that this process is very powerful.

The fact that students undertook the work individually and reflected on their performance before engaging in dialogue, served to start them generating inner feedback so they gained greater value from the group discussion. The key to harnessing inner feedback is to make the process explicit using reflective questions to which students must respond in writing.

The researchers believe that the findings have broad applicability and that providing a rubric or high-quality exemplars as comparators could equally help students better evaluate the quality of their own work.

Professional learning passport for teachers

The Education Workforce Council (EWC) is the independent regulator for the education workforce in Wales. It has developed a professional learning passport (PLP) to enable newly qualified teachers to capture their learning and development and receive support from mentors.

EWC developed the passport using its e-portfolio tool. Teachers can capture any learning or experience that shows they are developing competence against the professional standards for teaching and leadership. Teachers reflect on experience, and plan learning in areas that need development, in an organic way that feels quite natural and unlike many formal assessment practices.

This approach prompts new teachers to consider all the learning experiences that occur during their working days from moments in the classroom, to discussion with colleagues, to formal training opportunities - and reflect on how these impact on their development as professionals and their teaching practice. The completed PLP enables the teacher to present their learning and development journey.

Principle five - manage staff and learner workload effectively

We need to ensure we are assessing the things that matter and doing so in a way that allows the student learning from each assignment to feed forward into future tasks.

Common problems to avoid are over-assessment and assessment bunching.

A programme of study is likely to be broken down into smaller modules. Each module will have its own learning outcomes and there may be overlap between them. If each module team designs its own assessment without reference to the overall picture, you may find that some learning outcomes are assessed multiple times and that assignments are 'bunched' at particular times of year.

Learners are not able to produce their best work if they have multiple assignments to complete within a short timeframe. A group of student union officers known as the academic integrity collective points to overassessment and bunching as factors that may panic students into cheating.

This type of bunching also puts stress on staff marking and grading the assignments.

We also need the right balance between formative and summative assessment. Applying our principles means designing in formative opportunities where students can generate feedback from a variety of sources and act upon it to feed forward into the future learning.

Students and staff should spend most of their working time on tasks related to enhancing learning. This means business processes need to allow them to access the information they need, submit assignments, enter marks and feedback and view information from different sources, easily and quickly.

Academic staff working with badly designed business processes often describe themselves as being on a treadmill with no time to stop and reflect how things might be done better.

Staff also need to be supported to use the available tools. For example, assignments submitted electronically by students may be printed out by the staff marking and grading them. There could be many advantages to marking online if staff were given the necessary tools and shown how to adjust the view to suit themselves and take advantage of features such as 'quick mark' options.

Some of the ways technology can help

Understanding the curriculum

A description of the curriculum in digital format can allow you to get an overview using only basic analysis, such as number and type of assignments per module, to see where some learners may be being over assessed.

A similar overview of assessment dates will reveal assignment bunching.

Personalising information

Many institutions now use apps that provide students with a personalised overview of administrative information such as timetable, assignment due dates etc that can help them manage their workload better.

For staff, it is increasingly common to have a personalised dashboard view showing the status of assignment tasks and their priority.

Workload management

Online workflows make it easier to manage team marking across large cohorts.

Digital feedback tools offer the possibility to reuse/adapt frequent comments to save tutors time.

Digital tools offer possibilities to generate and mark questions for both formative and summative development. The type of questions that can be created and automatically marked using the QTI (question test interoperability) open standard go far beyond basic multiple-choice and can include evaluation of what higher-order skills learners are using to come up with their answer.

Putting the principle into practice

Transforming the Experience of Students through Assessment (TESTA)

TESTA has produced guidance on revised assessment patterns that work.

The University of Greenwich map my assessment tool helps with planning, identifying clashes and modelling the consequences of change. The tool is available for others to use.

One-stop-shop EMA (electronic management of assessment) solution at the University of Wolverhampton

The University of Wolverhampton used the Jisc assessment and feedback life-cycle to help its analysis and planning to create an end to end solution for EMA (electronic management of assessment).

This 'one-stop shop' is built around the core tools of the Canvas learning management system and Turnitin academic integrity solution. The Learning Tools Interoperability (LTI) open standard is used to facilitate seamless interoperability throughout the workflow.

An assessment centre provides staff with an overview of all assessment tasks in their courses.

Staff can select and mark work from the assessment centre and are provided with information about approaching marking deadlines. They can also see whether marks and feedback have been published to the student record system and the students.

The project experienced few technical difficulties and focused much of its time on defining policy and addressing some inconsistencies in marking practice.

View a set of presentation slides and watch the session recording on this case study.

Timely feedback for large classes

The University of Utrecht department of law faces a 'massiveness' problem.

The department has 800 new students each year and the current learning design requires them to complete an essay each week. Feedback and grades must be returned within three days in order to feed into the next assignment.

This has to be managed with a team of two professors and 25 'correctors' who provide feedback. The correctors are a combination of assistant professors, teaching assistants and third-year students.

The University has implemented Revise.ly feedback software to help solve this problem. The correctors use a shared comment set to ensure that the approach to feedback is consistent no matter who does the marking. Although the role translates as 'corrector' in English, all comments use a 'feed forward' approach to help with future improvement.

The tool is integrated into the learning management system using the LTI open standard. It can be used on any device and fits easily into the existing workflow, including incorporating it into workflows involving plagiarism-checking or peer-review.

Learners find the approach very helpful because the feedback is readily accessible and the structured approach means they can search what type of comments they want to see (eg focus on structure) and review this type of comment across different assignments. The search function was added at the request of the students.

Enhancing the effectiveness of personal tutoring at the University of Cumbria

Personal academic tutoring is an important feature in helping students adjust to university life and develop their academic potential.

The University of Cumbria supports 10,000 students a year in this way and turned to technology to address inconsistencies across different parts of the University.

Each learner now has a workbook in the e-portfolio system that is used to provide clarity and structure to the personal tutoring process and a more supported experience for the students.

An introductory 'About You' page allows students to provide background information about themselves. Each of the 30 minute meetings has a separate page and, in advance of each meeting, students are prompted to fill in questions that give useful pointers for discussion.

During the meeting, or immediately afterwards, the personal tutor adds notes summarising the discussion and providing signposting for the student to further help and support where appropriate.

Find out more about the University of Cumbria's experience of enhancing personal tutoring.

Principle six - foster a motivated learning community

This ties in with principle number two support the personalised needs of learners. Involving students in discussions around assessment practice helps make it more inclusive and allows students who may face barriers, for example, due to a disability, to voice their concerns.

Assessment literacy

Institutions commonly focus on developing learners' study skills, graduate attributes and digital literacies but none of these fully addresses student understanding of, and engagement in, the assessment process.

Study skills resources tend to help students develop their assessment 'technique' through essay writing, presentation and preparing for exams rather than understanding the nature and purpose of assessment and feedback practice.

Engaging students as active participants in making academic judgement develops their assessment literacy and gives assessment a sense of purpose which is motivating.

Participating in discussions about assessment approaches, criteria and standards can help develop students' academic judgement as well as developing skills important for employability. Self and peer review activities are the most active way of allowing students to practice making evaluative judgements.

Developing academic practice

It is just as important for staff to reflect on their assessment and feedback practice as it is for students to strive to improve their own performance.

New findings from education research and advances in technology may offer improved ways of implementing good practice even if the underlying pedagogy is still sound.

Course and module teams may spend a lot of time discussing approaches at the curriculum design stage only for assessment and feedback to be left to personal choice.

Feedback is the area that most often takes place in a 'black box'. There may be little or no team discussion leading to inconsistency of approach and variation in the quality and quantity of tutor feedback.

Tutors who haven't had the opportunity to discuss feedback practice with colleagues may produce feedback that is skewed towards either praise or correction or short-term and focused on the task in hand rather than developmental.

Despite rigorous quality processes, marking and grading may still be subjective. In the absence of whole team approaches, tutors may develop tacit and personalised standards that lead to inconsistency and inequality for learners.

Listening to the student voice can reveal how the lived experience of assessment and feedback practice matches the expectation of course designers.

Some of the ways technology can help

Our digital experience insights surveys can play a part by showing how your students and staff are using the technology you offer, what is making a difference to their learning and working experiences and where improvements can be made.

Digital tools supporting self, peer and group evaluation can help develop student assessment literacy.

Digital storage of marks and feedback can simplify analysis to identify anomalies. Analytics about the types, quantity and timeliness of tutor feedback can stimulate discussion about what is most appropriate in each context and the equity of the learning experience across an institution.

Online communities of practice permit sharing of experience across a wider network than a single course or institution. Sometimes it can be easier to share ideas and get an objective view on a hard-to-solve problems with colleagues who are removed from the working practices and day-to-day politics of your own institution.

Putting the principle into practice

Find out more about our change agent network supporting student staff partnerships across the UK.

University College London (UCL) is one of the active participants in the network. The University has established a central team of digital assessment advisors and runs 'assessment mythbusting' town hall events and assessment 'hackathons'. UCL also has appointed a group of student assessment design advisors to help develop and pilot new forms of assessment. Find out more about UCL's partnership approach to assessment transformation.

Streamlining global teacher training

The University of Sunderland runs an international teacher training course which attracts students from 60 countries. Assessment used to include a lesson observation carried out face-to-face in the classroom.

Improvement in the reliability of videoconferencing tools enabled the course team to develop the VEDA (video enhanced dialogic assessment) approach which won a university teaching award. This allows the lesson observation to be either recorded or live streamed.

Use of the open standard LTI Deep Linking allows the session to be recorded using the Panopto lecture capture tool and uploaded straight to the Canvas VLE (virtual learning environment). Data protection compliance is important as the videos contain images of children.

Course leader, Elizabeth Hidson found not only did this online viva-voce style assessment save on time and travel costs, it also facilitated improved professional judgement. Assessors spent more time and energy in the online space than was possible at the back of the classroom and the dialogue stimulated better reflective practice.

"Our ideas of 'evidence' have become more sophisticated and holistic based on better quality dialogue and professional judgements."
Elizabeth Hidson, University of Sunderland

The initiative was introduced as a response to the pandemic but, as the candidate usually bears the cost of the assesser's visit, there is interest in offering this option in future.

Assessing the process of thinking

We tend to focus on immediate peers when we talk about learning communities but academic development is also taking place at scale. Across France open standards and data science are being applied to improve assessment practice.

The French ministry of education uses digital technology to develop more authentic ways to measure traditional competencies and 21st-century skills.

These assessments don't only provide information about whether the answer is correct. They capture a rich set of data that reveal the students' thought processes.

Solving problems in mathematics and science requires students to use cross-curricular skills such as calculating, modelling, and scientific reasoning. Until recently, it has been difficult to measure these skills because traditional math and science assessments contain test items that are scored, based on a student's final answer.

Using an extension of the open standard QTI (question and test interoperability), the ministry is developing PCIs (portable custom interactions) to deploy authentic assessments that measure skills such as creativity, problem solving, collaboration and critical reasoning. The questions cover a wide range of types including game-like situations and interaction with chat bots to measure creativity.

Researchers make sense of a vast amount of data derived from these digital tests by defining patterns based on what knowledge and skills are involved in answering each question and common types of error.

In simple terms, there is a difference between the activity pattern of a student who has conceptual understanding and knows how to apply it in context and another who achieves the same answer via trial and error or guesswork.

Large-scale research has the potential to deliver valuable insights to help learning designers. Read about large-scale mathematics assessment in France (pdf).

More immediately, the examples of authentic question types openly shared, can provide inspiration and a rich reference source for others to use.

If you thought that item banks and automated marking could only be used with very basic multiple-choice questions (MCQs) think again.

View a set of presentation slides and watch the session recording on this case study.

Our thinking is influenced by Dr Yong Zhao. He suggests education, rather than trying to fix perceived 'deficits', by measuring against prescribed standards, should be cultivating individual strengths.

Employability, viewed from this perspective, can be seen as the ability to translate your uniqueness into value for others.

If higher education is to remain relevant in a changing world, it needs to demonstrate that our learning and assessment practices prepare people for the world of work and participation in democratic society.

Authentic assessment

There are frequent calls for assessment to become more 'authentic'. By this we mean that the tasks which are assessed should reflect things the learner may have to do in real life.

Traditional assessment formats, such as essays or exams, don't really mirror any other real-world situations.

Authentic assessment doesn't only apply in subjects we traditionally think of as 'vocational'. The skills we can test using formats such as closed book exams, which rely heavily on recall, are an equally poor fit with the working practices of academics or researchers of the future.

Authentic tasks are more likely to focus on deeper learning and the learner's ability to apply their knowledge and skills. Give the learner some agency in deciding the topic and medium to be assessed, and you are moving towards the ideal of creating value from individual uniqueness.

Promoting ethical conduct

Academic misconduct features high amongst institutional concerns about assessment practice.

There is a perception that cheating and collusion are easier in the digital environment. However, such practices are generally only possible where the assessed task lends itself to finding an answer ready prepared. This is especially true now that the practice of purchasing an assignment written to order by an 'essay mill' has been outlawed.

Not only is it harder to cheat at authentic tasks, they provide a sound foundation for a discussion of ethical practice. Academic writing and referencing can appear arcane whereas giving credit to the work of others in the world of business is a more familiar concept to introduce an understanding of ethics.

The University of Exeter has created guidance on designing work integrated assessment.

This is a topic that cuts across a number of our principles:

Principle number one, help learners understand what good looks like is relevant. Student engagement with the performance criteria can help develop an understanding of the importance of acknowledging source material and intellectual property rights

Principle number five, manage staff and learner workload effectively is important as pressure of over assessment can be a reason why students cheat

Putting the principle into practice

Our employability toolkit is a good starting point to find guidance and examples.

Policing a virtual shopping centre

The University of Northampton is using online scenarios to replace in-person training exercises for apprentice police officers. Videos of alleged robbery, shoplifting and suspicious activity in a shopping centre are accessed via the virtual learning environment (VLE).

Student responses to the scenarios are discussed in group webinar sessions which means that, for the first time, the whole class can learn from the experience.

The University has also created a mock courtroom scenario. Police apprentices present statements and are cross examined by two criminal barristers representing the defence and the prosecution.

Group member evaluation

In the world of work, overall team performance is as much a part of recognition and reward strategy as individual performance. Assessing a learner's contribution to group work is thus a very authentic scenario but fraught with difficulty.

At Maastricht University in the Netherlands, students in the department of data science and knowledge engineering experience authentic learning and assessment from the start of their course. The department has a philosophy of problem-based and project centred learning. Students work in groups of six to seven to solve real-world problems.

The issues that arise will be familiar to anyone who has tried group learning. Students complain that some of them work harder than others with some students failing to complete tasks or handing work in late.

It can be hard for tutors to get to the bottom of the issues as some of the complaints are contradictory. Often, they don't hear about the issues until a deadline is approaching by which point it is too late to intervene and improve the group dynamic.

The solution to the problem was to implement group member peer evaluation at a point when the group has had time to settle down but there is still time for the initiative to result in improvement.

Group member evaluation is done anonymously using a tool designed by FeedbackFruits in collaboration with a group of universities. The tool is integrated into the learning management system via LTI.

Learners have to evaluate themselves before they assess other group members. They then look at how their self-evaluation matches that of their peers and have the opportunity to discuss this with their tutor.

A rubric was developed to assist the evaluative process. The rubric has six categories: attending internal team meetings; responsibility; communication; interaction and collaboration; initiative and timely submission of assigned work. There is a descriptor equating to: acceptable; satisfactory or exemplary for each category.

Students were at first encouraged, but not obliged, to provide additional feedback in the form of a comment. It is now felt to be good practice to require further comment on a low score. The reason behind a low score on attendance may be self-evident but a poor score for 'initiative' can be less so.

Most students value the initiative and take it seriously although some take longer to build up trust in the practice. Some timid learners have had their confidence boosted to find that peers appreciate their ideas and think they should speak up more.

View a presentation and watch the session recording about this case study.

The economics of panic buying

At Stirling University an economics exam involving 'doing the calculations by hand, from memory, in a cold sterile environment' has been replaced by real-world problems requiring flexibility and creativity to address the issues.

First year students in 2020 were required to design strategic interactions to address the pandemic. Topics ranged from panic buying, hoarding and price gouging to vaccine uptake and sharing. Students were free to use their own choice of medium including blog posts, digital posters and videos

The boss wants an answer by the end of the day

The environmental science department at Brunel University London replaced a three-hour exam with a more authentic task. Students were confronted with what it feels like to turn up for work one morning and find your boss needs a report on a complex matter by the end of the day.

The assessed task was a complex question drawing on a wide range of information encountered during the course. The students had to demonstrate their working and provide references with highlighted annotations. They had seven hours to tackle the problem.

# data[].attributes.attachment_link.title
Principles of good assessment and feedback - Jisc

# data[].attributes.attachment_link.description
How good learning, teaching and assessment can be applied to improving assessment and feedback practice.