# data[].attributes.body
Some aspects of these mindsets may be reflected through the three types of higher education curriculum: included (what is taught), excluded (what is not taught) and hidden (covert). Each different type of curriculum has an impact on institutional cultures and educational outcomes.

encourage initiative that is transparent and inclusive.

# data[].attributes.attachment_link.title
Creating inclusive curricula in higher education

# data[].attributes.body
- the classroom reflects the unjustice and students have to be given equal access opportunity
- respect and value voices from the margins in our textbooks, teaching our classrooms, managing our educational institutions and representing us in media and in government.

Prioritizing listening to the more marginalized voices and ensuring everyone has an opportunity to express themselves.

In the same way that VC invited participants to select conferences and guests for sessions, students can be asked (e.g. via surveys or in dialogue)

Creating spaces where participants, rather than facilitators, can choose and change the direction of the conversation, embracing serendipity.

Ensuring participants have agency to participate in the ways that feel most comfortable for them, via audio or chat, with cameras on or off.

In recorded conversations, having time before and after a session that is less formal and unrecorded, where some people may feel more comfortable speaking and interacting.

Extend community beyond sychronous space

gives time for every voice to participate and for a workshop to feel more interactive.

It involves protecting participants from others who may hijack the experience for their own agenda, temporarily equalizing participants despite hierarchies outside the space, and helping participants connect with one another (Parker, 2018).

The Studio Visit inherently challenges the hierarchical power dynamic of experts “presenting to” students as authority figures.

it goes beyond the “contractual” equity to ensure it goes beyond words and documents and becomes the lived experience within a social space.

# data[].attributes.attachment_link.readable_text
Maha Bali and Mia Zamora

Intentionally Equitable Hospitality (IEH) is a facilitation praxis that was first developed by the co-directors and members of the grass-roots movement called Virtually Connecting (Bali, Caines, Hogue, DeWaard & Friedrich, 2019). Virtually Connecting (VC) has "challenged academic gatekeeping via rendering private hallway conversations that build social capital at face-to-face conferences into public hybrid conversations in which people who cannot attend conferences are able to participate" (Bali et al, 2019, para. 7). As such, IEH was initially focused on hybrid professional development intended to promote equitable access to conversations, with multiple volunteers as facilitators. This paper recontextualizes the concepts and spirit behind IEH to work in a formal educational context, replacing facilitators of VC conversations with teachers in formal contexts, and replacing participants at conferences with students in classes.

IEH begins with the notion that the teacher or workshop facilitator is a "host" of a space, responsible for hospitality, and welcoming others into that space. IEH requires intentionality about who is involved in the design of that space, noticing for whom the space is hospitable and for whom it is not. IEH is iterative design, planning, and facilitation in the moment. It also includes the interactions outside of formal gatherings that influence formal, synchronous interactions.

As Priya Parker (2018) has suggested, the way we gather matters. This observation holds for educational contexts. A class is often a unique entity, with its own chemistry or "personality". It holds particular memories. A class occurs at a particular time in one's life, and it is experienced in a particular place. Learning together holds the potential for unique growth moments, and can be truly transformational if it is tied to a sense of belonging. If a student gains the experience of being included and heard, it makes a critical difference in what kind of learning is possible for all.

But this aspiration is often at odds with institutional mandates that hem teachers in with an emphasis on content and prescribed learning outcomes. How can teachers foster an authentic and collective sense of belonging when designing for impactful learning? How can they create an equitable environment that is hospitable to diverse students? Since all gatherings are essentially collective endeavors, learning design for equity-in-community is a critical component of IEH facilitation. IEH is a values-based approach that promotes co-learning among students, who might be different in innumerable ways, by prioritizing the needs and wants of the most marginalized among them.

As a critical pedagogical approach, IEH centers values rather than measurable predetermined outcomes. Intentionally equitable hospitality is not neutral. Rather, it prioritizes the values of social justice while fostering learner/participant agency within the learning space, while never forgetting the ways in which power and oppression work outside of that learning space, and how they influence it. For educators, the "intentionality" at the core of an IEH approach is a crucial first step. When we wish to practice IEH, we need to continually renew our intentions to notice oppression and injustice and seek to redress them, to iteratively modify and adapt our practices according to the responses and reactions of participants/learners, particularly those who bring marginalized perspectives.

When we say "we", we don't mean "we" educators only, but a practice of spreading an IEH mindset and praxis for everyone participating. The work requires a constant renewal of the daily effort to pay attention, to interrogate one's own positionality, to imagine and extend one's own critical engagement with "the other", and to reckon with the limits of one's own understanding of other people's lives-to model this and explicitly discuss this so that co-learners can begin to practice IEH with one another. As such, IEH works on multiple levels and in different directions, not all of which will work in harmony in every context-a behavior considered welcoming by one group may exclude another. Ultimately, the educator is host, but never the gatekeeper. The students are essential co-creators of meaning in the learning community. This seems a simple paradigm shift, but takes a significant amount of critical self-reflection.

IEH work is always in process, taking the form of an aspirational journey, but never an arrival. Educators must embrace iterative modification, emergence, and revision thinking. This article aims to set a framework for how a teacher, instructional designer, or faculty developer can incorporate IEH into their learning design, and it shares real stories from our practice.

Learning environments, whether virtual, hybrid, or in-person, are inherently inequitable, for two reasons. First, because they mirror the outside world, encompassing the range of oppressions including white supremacy, heteropatriarchy, capitalism, colonialism, ableism, xenophobia-and all of the intersections within participant identities in the learning space. Second, because learning spaces introduce their own power dynamics, related to institutional cultures and how learners are expected to interact with authority figures, and related to hidden curricula that encourage or hinder certain behaviors, including, for example, host controls over muting/unmuting in video conference platforms, seating arrangements in in-person classrooms, and angling of cameras and location of microphones in Hyflex/Dual Delivery classrooms. Individuals within these learning spaces have an unequal opportunity to learn, to contribute to everyone else's learning, to grow. If we cannot control oppression and social injustice outside of the learning spaces we design, at the very least we can resist them within spaces we are able to control and influence, recognizing that our work will always be a journey, it will always be partial and that we continually strive for more, because:

Marginality can be visible and invisible... Those at the centers can never see what it looks like to be on the margins, because the world looks different from the margins...

This unique perspective, especially on suffering and in-betweenness, is why we need to have voices from the margins in our textbooks, teaching our classrooms, managing our educational institutions and representing us in media and in government. This is why Kamala Harris matters so much to girls, black women, Asians and immigrants in America. Because cognitively understanding systemic inequalities does not prepare you to truly understand the experiences of being marginal/ized (Bali, 2021, para. 9-10).

Representational justice matters in the classroom. We have a responsibility to elevate voices historically oppressed by privilege. IEH includes the ongoing work of representational justice-to understand that representation matters and that we all possess the agency to manifest narratives of both our neighbors and ourselves that embody humanity, truth, and respect. Attending to representation in a classroom can be corrective-undoing past misrepresentation-or reflective-embracing the full breadth of complex identities existing around us. When we create equitably hospitable spaces, we help nurture the growth of everyone present, so that each co-learner may gain more insight and more power beyond the boundaries of the spaces we traditionally inhabit.

The work of "opening up" conversation to diverse lived experience in a classroom context is an essential component of the social justice values underpinning IEH. When students can discover stories "that feel like home" in their curriculum, or even better, if they find the courage to share stories that represent their own perspective, we see the beginnings of trust-in-community. Learners honor the stories of others-whether empathizing with those stories comes naturally due to shared experience or uncomfortably-via epistemic listening to stories from perspectives one has never encountered fully before. This intimate and deliberate design work allows for narrative emergence in classroom spaces and aspires to create the conditions for co-learning in compassion. But this kind of trust can only be built with careful and equitable forms of engagement. IEH is a map for this nuanced and ongoing effort.

One of the main criticisms of traditional instructional design is its over-emphasis on design ahead of when actual teaching takes place. It assumes that if you design something thoughtfully enough, with some sort of empathetic imagination as to who students are and what they may want, then when you enact this design in practice as a teacher, everything will fall into place. However, what actually happens in practice may differ from the plan (see Alhadad, Bali, Gachago, Pallitt & Zamora, personal communication; Bali, M., Gachago, D., and Pallitt, N., 2022; )

IEH does not dismiss the importance of planning and design, but it recognizes that making a space hospitable happens at various times in the process:

Pre-design: Whom do we consult in the process of imagining our design? Do we contact past students? Do we discuss with colleagues from diverse, especially marginalized, backgrounds? Do we have access to our current students and how far can we involve them in the design?

Design: In what ways can our design anticipate power inequities, and in what ways can these be redressed a priori? In what ways can we leverage flexibility and openness in design and diverse pathways and "Universal Design for Learning" in order to make room for participant/student voices to modify the design, a design that is inherently participatory and emergent with learner agency baked into it by design?

In the facilitation/teaching moment: What kind of practices promote psychological safety for diverse participants to participate fully and also speak up when something is not comfortable or welcome? Can we move away from safe to brave spaces? How do facilitators respond when something goes wrong, when one participant's behavior may create discomfort, offend, or actively oppress? How do participants themselves respond?

Beyond the moment (sustaining community): How do we iterate, beyond a particular teaching moment to reflect on our own, and with our students/participants in order to help us notice anything we may have missed, and imagine how to do better next time?

Some of the practices of IEH as done in VC that transfer directly to classroom teaching include:

Prioritizing listening to the more marginalized voices and ensuring everyone has an opportunity to express themselves.

In the same way that VC invited participants to select conferences and guests for sessions, students can be asked (e.g. via surveys or in dialogue) to choose topics for the day, for example, ahead of class or during class.

Creating spaces where participants, rather than facilitators, can choose and change the direction of the conversation, embracing serendipity.

Ensuring participants have agency to participate in the ways that feel most comfortable for them, via audio or chat, with cameras on or off.

In recorded conversations, having time before and after a session that is less formal and unrecorded, where some people may feel more comfortable speaking and interacting.

Using semi-synchronous spaces to sustain community between synchronous sessions.

Social injustices are multidimensional. One typology of social justice is Nancy Fraser's (2005), which differentiates between:

Economic injustice, such as access to digital devices and infrastructure, which can be redressed via redistribution or resources;

Cultural injustice, which involves the erasure or misappropriation of certain cultures, such as the absence of Indigenous and non-white cultural perspectives from many Western curricula, and can be redressed via reappropriation; and

Political injustice, which involves the inability to participate in democratic decision-making about one's own fate and circumstances, and can be redressed via participatory parity.

These three dimensions are, of course, not mutually exclusive; people's identities are intersectional. For example, many populations suffer from a combination of these, such as African Americans. In some contexts, a group may suffer one kind of oppression but not in others. For example, Christians are the majority in many countries but are religious and cultural minorities in Egypt, where many of them are economically prosperous but overall their culture is not well-represented in curricula. In this context, they may therefore suffer ideological and interpersonal oppression. Economic inequality becomes particularly important in online learning if sessions are synchronous and students do not have access to dedicated devices, stable internet bandwidth, or private spaces to learn in their homes.

Oppression exists on many levels, from ideological oppression in society (racism, sexism, etc.), to institutionalized oppression (policies and laws that are unjust, such as redlining), to interpersonal oppression (such as microaggression) and internalized oppression (such as impostor syndrome). Recognizing these many levels and naming them, and how they manifest within our learning spaces can help us attempt to redress them. When we, those with power to control and design a learning space, recognize these oppressions, it enables us to promote "parity of participation" (Fraser 2005) for the learners in the design of the learning process.

Inviting everyone to participate in creating "community participation guidelines" as a democratic process does not remove the power dynamics amongst the participants or the internalized oppression of some participants. This may leave them silent or unable to object if other, louder voices, suggest items they do not agree with. Indeed, internalized oppression may mean that those unused to having agency may make decisions harmful to themselves, for lack of imagination of their own potential or of alternative options (Walker & Unterhalter, 2007). For example, students unused to having agency might, when given the choice of presenting their work as a regular paper or using any multimedia of their choice, fall back on what they have internalized as considered "good academic work" traditionally. How do we create environments that dismantle oppression, while raising critical consciousness of all involved, in order to achieve "parity of participation" so that the most marginalized in our group feel empowered as co-owners of the space?

"Feminist education for critical consciousness is rooted in the assumption that knowledge and critical thought done in the classroom should inform our habits of being and ways of living outside the classroom" (hooks, 1994, p. 94).

We have critiqued instructional design that assumes a good design done ahead of time will produce particular results. In reality, much of the work of creating socially just learning spaces requires facilitation in the moment, whether this is done via negotiation amongst learners or by the facilitator listening and responding to learners. Centering equity in the design process helps reduce equity emergencies (e.g. one person dominating a conversation) but does not eliminate them. Similarly, a well-facilitated session centering equity is incomplete without pre-work that promotes equity (e.g. who is involved in designing the session, how is the time agreed upon, is there community building done ahead of time?) and post-work that builds on the session to take equity work forward.

The examples we offer of IEH applied in educational settings may not apply IEH in all the ways VC has done so, but show its spirit in various ways.

Virtually Connecting

IEH originated from how the VC community strove towards equitable hospitality before, during, and beyond conversations. Before a conversation took place, work was done to ensure those furthest from justice could choose which events to do VC at and select guests at conferences with whom they would like to speak. This work involved inviting guests and ensuring representation of women and marginalized groups in the guest lineup where possible. It also involved announcing sessions and personally reaching out to virtual participants to join sessions when they showed interest but didn't register. During the session, the presence of both onsite and virtual buddies ensured there were hosts welcoming people in and checking that needs of both sets of participants were met, and participants could focus on the conversation while technology was taken care of. Beyond the session, VC built community such that IEH was a central value and praxis we developed, and we often reflected asynchronously or synchronously after events on ways we could improve the equity in our hospitality. Beyond learning how to manage the technology to run a VC session, new volunteer buddies shadowed more experienced buddies to learn from the ways different people modeled equitable hospitality to fit their personalities. Occasionally, VC would solicit feedback from participants formally, but informal feedback was collected all the time.

What hyflex classrooms can learn from Virtually Connecting

A model of hybrid synchronous teaching called Hyflex often involves a teacher in a classroom, teaching some students in-person while others join virtually via video conferencing. This is a model that is both technically expensive and pedagogically complex to apply well, but some educational institutions attempted versions of it during the COVID-19 pandemic, as a way to ensure distancing and flexibility within classrooms without being fully online. This model has so much in common with the Virtually Connecting format, of having a group of people together in one space connecting with a virtual group. (Bali, 2020)

If IEH is not applied in a Hyflex setting, virtual students are likely to feel like second-class citizens. One way to counter that, to ensure virtual students' voices are heard and needs are met, is to have someone like the VC "onsite buddy," someone sitting in class who is responsible for checking in with virtual folks to check if they have questions or concerns. In cases where teachers do not have TAs, rotational student volunteers can do this role. There can also be something like a VC "virtual buddy" responsible for virtual students, helping with things like breakout rooms, etc., and perhaps communicating with the onsite buddy, or if they are a TA, occasionally facilitating smaller virtual conversations.

Without this kind of support, the teacher's attention will either be divided between the two groups, giving neither of them sufficient attention, or they will end up focusing on one group more than the other. Occasionally, equity-minded teachers end up focusing on the virtual students, which is still problematic for the in-person students.

As with VC, some degree of digital literacies and competence helps promote a more equitable experience. For example, when teachers want to solicit responses from all students, they can choose tools that both onsite and virtual students can use at the same time, using external polling tools rather than the ones within the web conferencing tool, or they can use collaboratively edited documents instead of writing on paper or the in-class whiteboard.

In all of this, listening to student feedback continually can ensure teachers redress any inequities that occur, and perhaps sometimes students themselves can suggest solutions to increase equity in a Hyflex classroom.

Telling small stories to build trust

How can an educator lay the groundwork for a learning community built on sincere connection and trust? How can this be achieved when a group of different people comes together for a class? How can an educator convene a group of strangers and effectively foster both compassion and critical acumen?

You cannot insist upon trust. It has to be something that emerges from moments-moments that build upon each other over time. A formal class affords this time in ways one-off PD events do not. There are many quick warm-ups and introductory activities to engage students in IEH (see Bali, Caines & Zamora, undated). These moments make a big difference.

Getting to know one's peers is not necessarily an outcome that bears much academic significance in a traditional class. But to know your co-learners, little by little over time, is a key reason why most people feel motivated and connected in a course. Authentic and voluntary participation is also key to the success of peer learning modalities. I, Mia, incorporate many warm-up and reflective activities leading to short but sincere moments of sharing during class time. They are never a waste of time. Rather, these activities become a critical building block of IEH.

Small stories make all the difference in building a sense of community. There is secret power in a story. Stories can be a small gift given with purposeful intention. They can provide insight into the life of the storyteller. They are like a seed that is germinated (or a sachet of tea that slowly infuses in boiled water)-pervading gently in one consciousness, growing an understanding of something new. A small story can be a bridge between different people's perspectives.

In lieu of the typical introductory protocol of going around the room (or Zoom grid) to introduce oneself by name and major (a ritual that is at best boring, and induces dread and stress for many students), strategies like the IEH "Image Gallery" warm-up can be used (adapted from "Four Ideas for Checking In"). Here, everyone looks at a grid of randomly selected abstract images. Examples of pictures in the gallery might include a cozy fireplace, a bright electric cityscape, a rugged mountain terrain, or a labyrinth garden. Each co-learner is asked to choose an image from the gallery and explain why they chose it. How does it relate to how they might feel at the moment? What does that image mean to them and why? The result is numerous small stories that each individual chooses to share. The critical element of IEH design here is that each participant has the agency to choose whatever insight they want to share. There is no expectation or prescription of what one must tell others about themselves. It is the interpretive openness of the prompt-"What do YOU see?"-that generates something unique from each participant. The insights yielded are the glue of growing trust and understanding between co-learners.

Student remix as IEH co-creation

As a teacher-facilitator, I, Mia, am always exhilarated when I see signs of IEH values taking hold. IEH is evident when students intuitively suggest ideas for new forms of co-creation. It happens when students come up with their own (unsolicited) ideas for how to actively engage with each other. By imagining new ways to remix activities, they build upon IEH foundations and make evident new levels of trust.

A simple example of IEH student re-mix was when a thoughtful student asked to return to the Image Gallery protocol towards the close of a course. The special twist on the original protocol was that instead of each student selecting an image and then sharing a personal insight, this time each student would select an image inspired by thinking about another student in the course. What image in the gallery resonates when thinking about a peer in the course?

In this case, I immediately knew that it would only work as an equitable protocol if each and every student received a "shout out" from another student. In the class of 20 students, I designated 5 random groups of 4, and asked each group to self-assign who they would select an image for within that small group, so that it was ensured that every person was accounted for. Since this remix was conducted in Zoom, I also invited them to send "shout outs" to other students through use of the chat if they wanted to. These would be "extra" insights in addition to the one "shout out" they shared with the overall group. I knew that many wanted to share their positive insights with more than just one other student in the class. The result was an exhibition of generosity, insight, and complex forms of trust. Students shared things they noticed in each other, and reminded everyone of knowledge they gained from certain individuals along the way. They paid particular forms of tribute to each other, and they made evident how each person has their own unique way of contributing to a community's overall learning.

I, Maha, am a faculty developer, and when the COVID-19 pandemic forced us to move everything online, I tried to apply IEH in online workshops, to model it for faculty, so they could use it with their students. I quickly realized that VC-style conversations without topics and informal dialogue, are difficult to facilitate equitably in larger groups, say, larger than 12 or 15 (something Autumm Caines, 2015, often calls "The Interpersonal Multitudes Barrier").

When we offered "morning coffee" and "ask us anything" sessions that were announced as open dialogue, centering faculty members' interests and needs over faculty developer agendas, this usually worked out because numbers were small enough for everyone to participate, like a VC session. However, with larger, more formal workshops, we needed something different.

At first, we used polling tools, the chat, and Google Docs, but this still centered the workshop facilitator's agenda and privileged text-based over oral interaction. It seemed better to use breakout rooms, have people discuss in smaller groups, and then report back. This potentially gives time for every voice to participate and for a workshop to feel more interactive. However, poorly designed breakout sessions could be disastrous: you had to balance the number of participants per room, the timing, and the breakout settings. It was sometimes necessary to have a facilitator in each breakout room, which was not always logistically feasible, and we also learned that breakout groups needed very clear instructions to follow or they felt lost or awkward. This is where virtual Liberating Structures (LS) were a saving grace.

Liberating Structures are "are easy-to-learn microstructures that enhance relational coordination and trust. They quickly foster lively participation in groups of any size, making it possible to truly include and unleash everyone... [They] can replace more controlling or constraining approaches." Among the ten principles underlying the design of LS, the two most relevant for IEH are:

Include and Unleash Everyone

Practice Deep Respect for People and Local Solutions

When we used LS in our workshops, faculty members' key takeaway was often to plan to use more breakout rooms in their classes, and their key PD need was to learn how to use breakout rooms smoothly. In response, we offered hands-on workshops on creating breakout rooms and workshops explicitly teaching LS. Students since then generally gave positive feedback on the use of breakout rooms in classes.

LS are designed to promote equitable, respectful collaboration, through the sequence of steps, distribution of timing and group size. Originally designed for in-person collaboration, most LS work well with virtual breakout rooms.

Three particular structures demonstrate IEH by respecting all voices equitably:

1-2-4-all (https://www.liberatingstructures.com/1-1-2-4-all/) is a dialogue version of "think-pair-share". In 1-2-4-all, a facilitator shares a prompt. Each participant thinks about it on their own first. This gives people who are more reflective, non-native speakers and marginalized groups a chance to think quietly without interruption and formulate their thoughts before sharing in the second stage of sharing with pairs. The next stage is discussion in a group of four before sharing with the larger group. When people test out their ideas in small groups, they all contribute and gain confidence while building on each others' ideas before sharing with the larger group. It ensures everyone participates, in contrast to most large group discussions, where certain individuals tend to dominate the discussion.

Conversation Café (https://www.liberatingstructures.com/17-conversation-cafe/) places participants in groups of 4-5 to respond to a prompt, giving micro-timing for individual participation within rounds (and thus it helps if someone volunteers to be the "timer"). In the first round, each person takes one minute to respond to the prompt, and in the second round, each person responds to other people's responses in one minute. Afterward, there is some open conversation time before wrapping up with key takeaways that a "volunteer note taker" writes on a shared document. This approach approximates equity by giving equal time, but of course, some people need time to think before speaking (and would benefit from 1-2-4-all). Others might need different amounts of time to express their ideas, either because they are non-native speakers, they take time to express complex ideas, or they just speak more slowly. IEH might ask people who speak more quickly to give up their time for others, give people time to write quietly in a shared space before sharing orally, explicitly give more time to marginalized groups (e.g. students in a student-faculty conversation), or let marginalized groups speak first.

Troika Consulting (https://www.liberatingstructures.com/8-troika-consulting/) is another LS that uses microtiming and introduces reciprocity well. Each group of 3 works together in 3 rounds, switching roles each round. Every person has the opportunity to act as a "client," asking for consultation from others, and the other two perform a quick consultation using a specific format; all of this occurs within 10 minutes. This structure is surprising in how quickly, in the space of 10 minutes, one can receive useful ideas for a challenge one is facing, from people who may not be experts. The reciprocity of asking for help once and giving help twice in the same structure tends to also be satisfying for participants. This kind of structure done within faculty development workshops empowers faculty to set the agenda for what they want to discuss, to seek help from peers rather than faculty development experts, and when used in class, encourages students to seek help from colleagues not just look to teachers for help. It may help people to be given time to prepare for their "challenge" ahead of time before they go into groups since the time is usually limited and the process quick. It is important for facilitators to create a relaxed atmosphere before sending people into their groups so as not to create anxiety over the time limits.

For any LS, whether participants are strangers or not, it helps to dedicate time for people to introduce themselves or say hello before the activity. This alleviates the stress of working on the task, which could be difficult for some people to do without a warm-up.

A limitation of LS is that they require participants to "buy into" the participatory approach. If someone with lots of power decides not to follow the process, they may not end up making room for others to speak. Moreover, following a process that listens and builds upon ideas of all voices in a one or two hour workshop does not mean this will automatically transfer into changing the culture in a work environment unless there is further intentionality.

Some ways of addressing this is to use the facilitator's power with what Priya Parker calls "generous authority". Parker writes: "A gathering run on generous authority is run with a strong, confident hand, but it is run selflessly, for the sake of others. Generous authority is imposing in a way that serves your guests. (Parker 2018, p. 81). Replace "gathering" with "learning space" and "guests" with "learners/participants". Generous authority" is using power to achieve outcomes that are generous, that are for others" (Parker 2018, p. 82). It involves protecting participants from others who may hijack the experience for their own agenda, temporarily equalizing participants despite hierarchies outside the space, and helping participants connect with one another (Parker, 2018).

In practice, when using LS, the facilitator or teacher needs to explain the process to everyone and perhaps make explicit ground rules around sharing space and having all voices heard. If the groups are extremely unfamiliar with such approaches, there may need to be some co-facilitators present with small groups or moving between groups to check in and help out. The facilitator is likely to succeed better if they use inclusive warm-up activities that allow participants to get to know one another and get into the mood with low-stakes activities and discussions that do not build on anyone's authority. Some participants will resist unless they know the purpose of every step. It is sometimes worthwhile to stop and make time to discuss purpose before or after an activity, to help more participants stay on board.

The facilitator's generous authority involves reminding people of timing-its importance for equitable and productive dialogue-and designing who ends up in which group with thoughtfulness. For example, if the facilitator knows the audience well, they may avoid placing extremely dominant participants with extremely shy ones, unless they can have a co-facilitator to support equity in that group. The facilitator may decide to ensure marginalized groups are not tokenized in groups, e.g. never have one student among five faculty, but rather 2-3 students with two faculty. There also needs to be a safe way for a participant to report issues or seek help from the facilitator at any point. Finally, participants unwilling to try these ways of doing things may choose to "pass" but not be "counted" among participants, possibly observing, but indicating clearly so facilitators don't count them among active group members in something like Troika. It is important to recognize that people who resist these approaches may come from all walks of life: they may be very powerful people unwilling to give up on hierarchy, or they may be marginalized groups unwilling to make themselves vulnerable. It is important to remind ourselves of the purpose of a gathering, invite the right combination of people first, and design the groupings and the flow of the experience in ways that help us meet the purpose; it is also important to remind ourselves these decisions can be really complex (Parker, 2018).

In terms of follow-up beyond a session, it would depend on the facilitator's relationship with and power amongst the group of participants, but at the very least, a live session or meeting can end with agreements on future commitments or next steps based on the dialogue. Where there is a sustained relationship among facilitators and participants, there may be some degree of negotiation behind the scenes before and after sessions that can help make sessions more constructive and fruitful beyond the meetings themselves. Reflecting on higher education, the power dynamics when a faculty developer is offering a workshop to tenured professors in one department is different than if they were a learning community across disciplines that meets regularly and is very different from a class. And a class of 15 in a seminar-style classroom is different from a class of 40 learning online which is different from a class of 80 learning in a large lecture hall.

In August 2020, we (Maha, Mia) collaborated with Autumm Caines and approached an organization called OneHE (whose goal has been to enhance higher education globally) to suggest a collaboration with "Equity Unbound" (Equity Unbound is an emergent, collaborative network which aims to create equity-focused, open, connected, intercultural learning experiences across classes, countries, and contexts). The collaboration we proposed was to create and curate an Open Educational Resource that demonstrated to educators worldwide a variety of ways for building community online. We felt this was needed because people unfamiliar with online teaching needed ideas for building community online, especially in these traumatic times of physical distancing. Faculty developers and educators worldwide were overwhelmed, so we designed these resources with demonstrations and templates they could adapt.

These resources practiced IEH by:

Involving multiple people around the world from different contexts in their design, so people who experienced online differently or had different resources could notice and point out inequities or challenges with each suggested strategy for community building

Offering adaptations for elements that may not be accessible to some groups. E.g. synchronous and asynchronous options; breakout and no-breakout adaptations

Inviting feedback from the public and inviting viewers to contribute. We used critical and constructive feedback on the resources, to iterate and adapt them since.

For concrete, practical examples, see Bali & Zamora (forthcoming, in Learning Design Voices)

"Studio Visits" are a way to enrich both classrooms and professional development engagements with the practical experience of experts in the field. The visits are virtual by nature, and bring together diverse co-learners from different locales, and with different levels of training or professionalization. A studio visit is marked by its unscripted nature, and there is little to no formal agenda set for the time allotted, as people enter into fluid conversation around a shared interest. A Studio Visit encourages co-learners to have informal, open discussions, giving participants access to dig deeper into how someone's scholarship, research, art, or practice has developed. The Studio Visit inherently challenges the hierarchical power dynamic of experts "presenting to" students as authority figures. We often host Studio Visits on a range of topics on behalf of "Equity Unbound"-an equity-focused, open, connected, intercultural curriculum that builds critical digital literacies in a global context, highlighting issues of web representation, digital colonialism, and safety/security risks.

Sometimes Studio Visits are conducted within a closed class setting with participation limited to the professor, the guest speaker, and the students. In these cases, IEH facilitation is a matter of extending a pre-established classroom culture. At other times, Equity Unbound Studio Visits are open. Professors join students from varying countries and cultural contexts, and co-learners represent diverse institutions. In these cases, there is an explicit power difference between the participants. Educators engage with confidence and professionalism, often rendering some students unsure. Some may intuitively take a back seat to those who they perceive as more "qualified" to participate. Also, some faculty participants have established collegial connections with other faculty participants. Therefore, a Studio Visit can be an occasion to have a chance to virtually reconnect with colleagues. But acquaintanceship among a few may leave others feeling like inherent "outsiders." In these open contexts, we have discovered that IEH is crucial in pursuing equitable co-learning. We must be intentional in the way we welcome everyone. The design for "opening up" the Q & A time should prioritize students and unaffiliated co-learners to share insights and ask questions. We must also remind all participants to be mindful of the varying levels of professional training and to make space for everyone to feel included in the ensuing conversation. Without clearly stating these intentions, the conversations are likely to be "hijacked" by collegial senior faculty who unintentionally get carried away in their own participation, leaving little room for the less practiced.

I, Maha, recently co-facilitated a session where students, faculty, and staff worked together to brainstorm and prototype ways of integrating gender into the curriculum at my institution. Although we used participatory approaches from a combination of Design Thinking and Liberating Structures, and kept people in small, diverse groups to make time for every voice to speak, we discovered later that in some breakout groups, student voices were not heard because some of the faculty spent a lot of time talking and no one took on the responsibility to ensure all voices were heard. We had used another Liberating structure known as "TRIZ" (a Liberating Structure that involves solving complex problems in a playful way by going backwards, working towards an anti-goal) in that session, which does not micro-time the group time, versus something like Conversation Café, which does remind us to give everyone a turn and equal time.

What the above two experiences highlight for us is that for IEH to truly work, it is not enough for a facilitator to have these values and enact these practices. It is important for the community to have internalized the values or at least a few key people willing to co-facilitate in small groups or as backup in big groups. This is much harder to achieve in one-off meetings versus ones where the same group meets multiple times, like a class, and can learn from the facilitator's modeling. Beyond internalizing the values, participants willing to help facilitate sessions need to practice the ability to recognize oppression and microaggression as it happens, have strategies for countering it, and have a strong enough voice to speak up against it and challenge it, even if it is uncomfortable or against someone with more power than themselves. This is a big ask, and may not be "safe" to do in every context. The original Virtually Connecting context usually involved people with different degrees and types of power but who did not work together in the same institution and have those formal hierarchies of power between them. However, within the context of an institution, such power dynamics cannot be overlooked. Therefore, it seems that in an institutional context or in one-off meetings, it may be necessary to ensure that those with the most power are reminded orally of the importance of IEH, and those with the least power are aware of their rights as well, so they may advocate for themselves. Having some co-facilitators there, especially when people are unused to participatory approaches, can help, because, in virtual environments. the facilitator cannot see the dynamics of all the rooms at a glance the way they would in a physical setting.

IEH may not come naturally to all educators and instructional designers. It requires the practices of both equity and care to work in harmony, where equitable pre-planning and design are embodied with care in the facilitation of a learning space (Bali & Zamora, 2020 & 2022). It is also important to emphasize that IEH is not one practice or a list of practices, because "the notion that one model of care will work for everyone is absurd...humans vary in their abilities to give and receive care" (White & Tronto, 2004, p. 450). Nel Noddings (2012) asserts that a caring approach means we "do unto others as they [emphasis added] would have done unto them." This may mean anticipating others' wants and needs and designing to address them, or it may mean a more democratic and compassionate approach that is more participatory (see Bali et al., 2022; Alhadad, Bali, Gachago, Pallitt & Zamora, personal communication).

IEH is a praxis that strives towards socially just care. In a "caring-with" democracy, we can set a goal of structuring institutions and practices so that each person's individual preferences can be honored." (Tronto, 2015, p. 34). Bali & Zamora (2022) name "socially just care" as follows:

This is when social justice is realized and embodied in a caring manner by participants in a social space... We did not name it "democratic care" (after Tronto, 2015) because democratic processes do not necessarily lead to equitable outcomes. We did not call it "parity of participation" because such a term does not emphasize the importance of care in order to create the social justice end goal. Socially just care, rather, promotes social justice and parity of participation in its designs and planned processes, and is enacted with care such that it always iterates to nurture self-determination, agency and justice for all involved, in whatever manner meets their diverse care needs, and addresses the multiple dimensions of injustice individuals and groups may face. It distributes the care responsibility so that the care is not "partial," and it goes beyond the "contractual" equity to ensure it goes beyond words and documents and becomes the lived experience within a social space.

About. (2018, October 31). Equity Unbound. http://unboundeq.creativitycourse.org/about/

Bali, M. (2020, November 18). What hyflex livestream models can learn from @VConnecting. Reflecting Allowed. https://blog.mahabali.me/educational-technology-2/what-hyflex-livestream-models-can-learn-from-vconnecting/

Bali, M. (2021). Foreword. Voices of practice: Narrative scholarship from the margins. Hybrid Pedagogy. https://voicesofpractice.pressbooks.com/

Bali, M., Caines, A., DeWaard, H. , Hogue R. J., & Friedrich. C. (2019). Intentionally equitable hospitality in hybrid video dialogue: The context of Virtually Connecting. eLearn Magazine. https://elearnmag.acm.org/archive.cfm?aid=3331173

Bali, M., Caines, A., & Zamora, M. (undated). Equity Unbound & OneHE Community Building Resources. https://onehe.org/equity-unbound

Bali, M., Gachago, D., and Pallitt, N. (2022). Compassionate learning design as a critical approach to instructional design. In Burtis, M., Jhangiani, S., Quinn, J. (Eds.), Toward a critical instructional design. Hybrid Pedagogy. https://criticalinstructionaldesign.pressbooks.com/

Bali, M. & Zamora, M (2020). Four ideas for checking in. Equity Unbound & OneHE Community Building Resources. https://onehe.org/eu-activity/four-ideas-for-checking-in/

Bali, M. & Zamora, M. (2021, February 25). #OpenEd20 Day 2 Plenary: Maha Bali & Mia Zamora. Open Education Conference 2020. Virtual. https://www.youtube.com/watch?v=NEeZvM6_8UE

Bali, M., & Zamora, M. (2022). Equity/Care matrix: Theory and practice. Italian Journal of Educational Technology. https://doi.org/10.17471/2499-4324/1241

Bali, M. & Zamora, M. (forthcoming). Designing and Adapting for Community with Intentionally Equitable Hospitality. In T. Jaffar, S. Govender, & L. Czerniewicz (Eds). Learning design voices: Perspectives from the margins. https://learningdesignvoices.com/

Caines, A. (2015, December 24). A call for more #HumanMOOC discussion groups. Or. The very human problem of access with more thoughts on the Interpersonal Multitudes Barrier (IMB). Is a Liminal Space. http://autumm.edtech.fm/2015/12/13/a-call-for-more-humanmooc-discussion-groups-or-the-very-human-problem-of-access-with-more-thoughts-on-the-interpersonal-multitudes-barrier-imb/

Fraser, N. (2005, December 1). Reframing justice in a globalizing world. New Left Review. https://newleftreview.org/issues/ii36/articles/nancy-fraser-reframing-justice-in-a-globalizing-world

hooks, bell. (1994). Teaching to transgress: Education as the practice of freedom. Routledge.

McCandless, K. H. L. & Lipanowicz, H. (2022a). 1-2-4-All. Liberating Structures. https://www.liberatingstructures.com/1-1-2-4-all/

McCandless, K. H. L. & Lipanowicz, H. (2022b). Conversation café. Liberating Structures. https://www.liberatingstructures.com/17-conversation-cafe/

McCandless, K. H. L. & Lipanowicz, H. (2022). Introduction. Liberating Structures. https://www.liberatingstructures.com

McCandless, K. H. L. & Lipanowicz, H. (2022). TRIZ. Liberating Structures. https://www.liberatingstructures.com/6-making-space-with-triz

McCandless, K. H. L. & Lipanowicz, H. (2022b). Troika consulting. Liberating Structures. https://www.liberatingstructures.com/8-troika-consulting/

Noddings, N. (2012). The language of care ethics. Knowledge Quest, 40(5), 52-56.

OneHE (2022). OneHE: Develop your teaching your way. https://onehe.org

- (2020). Warm up activities. https://onehe.org/equity-unbound/warm-up-activities/

Parker, P. (2018). The art of gathering: How we meet and why it matters. Riverhead Books.

Tronto, J. C. (2016). Who cares?: How to reshape a democratic politics. Cornell Selects.

Virtually Connecting (2022). Virtually Connecting. https://virtuallyconnecting.org

Walker, M., Unterhalter, E. (2007). The capability approach: Its potential for work in education. In Walker, M., Unterhalter, E. (Eds.), Amartya Sen's Capability Approach and Social Justice in Education (pp. 1-18). Palgrave. https://doi.org/10.1057/9780230604810_1

White, J. A. & Tronto, J. C. (2004). Political practices of care: Needs and rights. Ratio Juris, 17 (4), 425-453.

# data[].attributes.attachment_link.description
Maha Bali and Mia Zamora Intentionally Equitable Hospitality (IEH) is a facilitation praxis that was first developed by the co-directors and members of the grass-roots movement called Virtually Connecting (Bali, Caines, Hogue, DeWaard & Friedrich, 2019).

# data[].attributes.body
Uncovering to what extent does offering a level of care affect student attainment and performance.

Gilbert, P. (2015)  The evolution and social dynamics of compassion.   Social and Personality Psychology Compass

‘Caring is a motivational system which is as deeply embedded ingrained hardwired into the mammalian brain as the motivation to reproduce or to eat’

A strong hold on care can lead to more engagement and successful outcomes from a psychologically enhanced desire

# data[].attributes.headline
Learning outcomes and assessment criteria in art and design. What’s the recurring problem? (Dudley Nganjo)

# data[].attributes.subject
Learning outcomes and assessment criteria in art and design. What’s the recurring problem? (Dudley Nganjo)

# data[].attributes.body
I liked this article for pointing out issues faced when teaching art and design and trying to determine what quantifies success in a students learning outcome

‘So, I suggest, in art and design whilst it is important that students know what they have to do on any course of study, it is not necessarily through published learning outcomes. Learning outcomes might be seen as necessary for administrative purposes but they are not sufficient in helping students develop an idea of what they will be learning and how they will go about it. Indeed, in a highly supportive context, learning outcomes might be so generalised as to only define the landscape and the boundaries of their intended learning. The knowing of what to do becomes developmental and personalised’

# data[].attributes.attachment_link.readable_text
This paper is a critical reflection on the development of learning outcomes in art and design. It builds on a Learning and Teaching Support Network (LTSN) - funded project from 2003 that sought to provide support for colleagues seeking to make sense of the then new quality agenda. It seeks to identify recurrent issues and make a number of recommendations for further development.

Keywords: learning outcomes, assessment criteria, constructive alignment, art and design

Abstract

This paper is a critical reflection on the development of learning outcomes (LOs) in art and design. It builds on a Learning and Teaching Support Network (LTSN) - funded project from 2003 that sought to provide support for colleagues seeking to make sense of the then new quality agenda. It seeks to identify recurrent issues and make a number of recommendations for further development.

Around 10 years ago, I completed a project, funded by the then Learning and Teaching Support Network (LTSN) Art, Design and Communication Subject Centre, intended to provide support for art and design teachers who were developing learning outcomes and assessment criteria for their students ( download the project report). My intention was to lay out the issues as clearly as possible and provide advice, particularly for new staff, on how outcomes and assessment might be related to the student experience. There was an underlying imperative to the project at the time which was the emergence of 'learning outcomes' as key features of learning in the new quality agenda established by the Quality Assurance Agency (QAA). Although outcome-led learning was not new to higher education, the utilisation of learning outcomes within the definition of quality learning was. This paper is my personal reflection on those issues based on work I have carried out subsequently both formally as research and informally in supporting institutions and departments in writing outcomes and assessment criteria.

There was emerging a good deal of literature surrounding outcome-led learning, particularly on how to write coherent outcomes within a skills context and, increasingly so, in a context in which the written form, such as essays and reports, was a predominant mode of evidencing achievement. There was less similar material on assessment, although the literature on assessment, particularly in secondary education, has a long history. There was, of course, even less literature on these topics within art and design.

I drew much of the material for the project from other sources. There was a broad consensus on the value of learning outcomes and how they might be articulated. There was an obvious concern to avoid ambiguity in the writing and presentation of learning outcomes with cautionary remarks and recommendations that I wove into the project, attempting, where possible, to highlight the challenges for art and design. Simultaneously, there emerged a most valuable critique of learning outcomes by Hussey and Smith (2002) that had the potential to derail the outcome juggernaut before it had developed any steam. Their comments, despite being equally relevant today, did not hamper the momentum that has led to learning outcomes being adopted in (almost) all HE institutions in the UK.

The design and use of assessment criteria has appeared less under the microscope largely because they have always been a requirement of the validation of courses in universities. Nevertheless, as the focus shifted towards student-centred learning, the literature began to pick up on broad issues such as the differences between norm-referenced and criterion-referenced assessment, the impact of assessment on learning, self- and peer assessment and so on.

John Biggs' book, Teaching for Quality Learning at University (2003) became highly influential during this period and has been used subsequently as a key reader on most courses for new teachers in universities throughout the UK. In my paper, I drew heavily on Biggs, particularly his SOLO taxonomy as it was quite evidently, like Bloom's taxonomy beforehand, a potential tool for developing and designing assessment criteria.

During this period, what became clear was that the developmental histories of learning outcomes and assessment criteria, for most institutions, were quite different.

Assessment criteria in most universities were derived from Bloom's taxonomy (1956). The convenience of the structure of the taxonomy, as it related to the undergraduate degree classification system, meant that the categories could quite easily form the basis of the overall generic assessment strategy of universities. It wasn't unusual, however, for departments to adapt the taxonomy to suit their disciplinary requirements or for universities themselves to insert extra levels to further differentiate the classes. Terminology, such as 'good', 'very good', 'excellent' and often 'outstanding', were, and still are, used to differentiate levels of competence although in themselves provided no insight into their meanings and how they related. Such terms are often used as qualifiers within level descriptors.

So, whilst the assessment policies, strategies and processes became embedded into the culture and social practices of universities and, hence, to be seen as a strong evidential basis for comparison of standards and quality between universities, learning outcomes, a much more recent phenomenon, had to find a way of fitting in.

Curiously, whilst the existence of learning outcomes and assessment criteria were deemed essential for a positive outcome during the QAA Subject Review process (1998-2000), no heed seemed to have been taken by almost half of the art and design subject community to ensure the clarity and use of assessment criteria.

Less than half (43 per cent) of providers' arrangements for the assessment of students are completely effective. Although students are assessed using a suitably wide range of methods, the majority of reports identify a number of sector-wide concerns with the assessment of students. These relate primarily to inconsistencies in the clarity and use of assessment criteria, and their relationship to published learning outcomes. The criteria for different assessment bands are not always sufficiently differentiated. Where generic criteria have been developed, these often fail to engage with subject-specific learning outcomes, and the reviewers also report inconsistencies in the grading of different levels of achievement. (QAA, 2000, Section 36, Teaching Learning and Assessment)

At the end of the report and by way of recommendation, the authors suggested 'there is a widespread need to improve the definition of assessment criteria, to relate them to published learning outcomes and to use them consistently' (QAA, 2000, Section 69, Conclusion).

At the time, this was not regarded as an issue peculiar to art and design. In 2000, the QAA published its 'Code of Practice for the Assurance of Academic Quality and Standards in HE' with Section 6 of the Code urging institutions to ensure that 'assessment tasks and associated criteria are effective in measuring student attainment of the intended learning outcomes' (Section 6: Assessment of Students).

In a workshop at the Centre for Learning and Teaching in Art and Design (CLTAD) at the University of the Arts in 2003, organised to investigate this concern, 16 experienced reviewers considered what they felt was behind this difficulty. The conclusion reached is interesting in as much as the problem, as manifest in individual institutions, was revealed to arise not so much from the interrogation of the evidence base submitted to the QAA as from their seeking of the actual student experience. If student groups reported, during the review, that they indeed did know what was required of them on their course and they were clear about how and why they were being assessed, despite the evident lack of clarity of the documentation given to them, then there was no issue to answer. In these instances, it was accepted that there existed strategies set up for students to receive this information other than those revealed through the formal course documentation. This is important insofar as the student voice about their experience was seen as paramount. It was only in those instances where the students reported that they didn't know what they had to do and how they were being assessed (despite published learning outcomes and assessment criteria) that reviewers could legitimately challenge the coherence of the outcomes and the criteria. All this gives rise to the following possibilities:

The ideal scenario is, of course, revealed in the top left sector. The bottom right sector represents the worst possible scenario. The other two sectors are interesting. The bottom left suggests that the course team has spent more effort developing outcomes than helping students to understand them. As Ferenc Marton said at the 'Improving Student Learning Conference' in Exeter (1995), 'If a student doesn't understand, then there is something we have taken for granted that we shouldn't have taken for granted'. This seems to me a case in point.

In my time, I have witnessed all four of the above scenarios but it is the top right sector that intrigues me most. Our obsession with establishing the accuracy/clarity of learning outcomes in the belief that this an essential prerequisite for quality learning to take place is undermined by those courses in which the written learning outcomes are largely unclear but the students are performing well. A lack of clarity in the learning outcomes, it seems, does not mean that students are not clear about what they have to do. Indeed, learning outcomes, ambiguous or otherwise, appear to be no substitute for established learner support systems and other frameworks that help students understand what they have to do in order to successfully complete a programme of work. Briefs and briefings are familiar in art and design along with tutorials, interim crits and feedback forums. It is during these supportive scenarios that art and design students formulate their intentions and actions and come to understand what 'imagination', 'creativity', 'risk-taking', etc, (the very terms regarded as potentially ambiguous) actually mean for them.

I would argue that the insistence that learning outcomes should be sufficiently clear 'to be measurable' has not helped those subject areas, such as the creative arts, in which articulating outcomes that involve the development of intuition, inventiveness, imagination, visualisation, risk-taking, etc, is challenging. In terms of meaningfulness, they equate to the notion of 'understanding', a cognitive term which is regarded as too complex and which should be substituted by other, more measurable, terms such as, 'explain', 'analyse', etc. Another drawback in the use for these terms, acknowledged by Biggs (2003), is that they are regarded as 'divergent' and as such do not invite one appropriate answer but a range of possibilities.

An example of this is art and design students' ability to visualise. This cognitive ability is a cornerstone of creative thinking. It requires the use of imagination and judgment and we expect all art and design students to develop it as they progress in their study. Fashion students develop their collections by visualising what comes next, furniture design students visualise their products individually, together and in a context, graphic design students visualise alternative layouts and so on.

Nevertheless, this concept is not easily captured in learning outcome form. It's not the kind of thing that can be measured easily. It is, in fact, developed within the whole complex process of the practice and over time.

Constructive alignment, again Biggs's notion (2003), attempts to tackle an important issue relating to the coherence of the programme of study and the student experience of it. There has to be a relationship between what a student is expected to do (the intended learning outcomes) on a programme, the programme content and delivery that supports the student's achievement and the process and judgments by which the quality of the learning is determined (assessment criteria). For me, Rowntree raised this issue forty years ago; 'to set the student off in pursuit of an un-named quarry may be merely wasteful, but to grade him (sic) on whether he catches it or not is positively mischievous' (1977).

It would be odd for any teacher, nowadays, to dissent from the sentiment that students need to know what they are expected to learn on any given course or module. However, the over-specification of outcomes for the purpose of measurement can be counter-productive in art and design and other creative disciplines. The more specific the learning outcome is the more specific the 'quarry' will be and the greater the constraint on whether the most appropriate quarry is caught or not. This is not just seen to be true of creative subjects;

Limits to the extent that standards can be articulated explicitly must be recognised since ever more detailed specificity and striving for reliability, all too frequently, diminish the learning experience and threaten its validity. There are important benefits of higher education which are not amenable either to the precise specification of standards or to objective assessment (Price et al, 2008)...

Apart from the constraints of the use of language, students who take a strategic approach to learning are not likely to depart from the published words and the concrete world. For them, literal interpretation is paramount. However, art and design students are often in pursuit of a 'quarry' of which they are given only partial knowledge. Indeed, this can be a deliberate learning strategy in the creative arts. For art and design students, formulating and finding their own quarry is an essential part of the discovery process. They do, nevertheless, need to know the 'landscape' and the 'boundaries' when they are in full pursuit. It might be that these are better articulated in the form of a discourse than in specific outcome form and more usefully manifested in project briefings, team meetings, etc.

Students also need to know how well they have performed in 'catching the quarry'. In the UK universities classification system, we require final judgments about student performance to be within one of five categories - 1 st, 2.1, 2.2, 3 rd and fail. This is often mirrored throughout courses by utilising grades rather than classes. Usually, assessment grades at module level reflect the system as a whole. Hence, grading systems use criteria to differentiate levels of performance. To satisfy both Rowntree and Biggs, therefore, we need to demonstrate the relationship (alignment) between the learning outcomes and the assessment criteria.

It is here that serious issues arise and often because of the dutifulness of teachers as they try to reconcile the language of learning outcomes with that of assessment criteria. Since the publication of the QAA Art and Design Overview Report in 2000, the difficulties identified in Section 36 continue to set challenges for teachers and administrators alike. Even more so given the observation and comments from the most recent QAA Institutional Audit Overview Report:

Most institutions have moved to a common assessment framework for the great majority of their degree programmes, and institutional as opposed to disciplinary or subject determination of assessment schemes is much more evident than hitherto. QAA (2011);

A finding that occurred more than once was that some programme specifications contained only generic learning outcomes and assessment criteria, so that subject-specific information was lacking and it was frequently not possible to determine from the programme specification any subject-specific assessment strategy. QAA (2011).

There is obvious uncertainty in the sector as to how generic and subject outcomes operate within the scheme of things. If they are overly generic they become less meaningful and it is unclear how the outcomes and assessment criteria relate to each other. If they are overly specific it becomes difficult for judgments about performance to be meaningful. It might be worth reflecting that the most accurate map of a region would have a one-to-one relationship with the terrain. That is, the map would be as big as the region and, hence, useless.

So, I suggest, in art and design whilst it is important that students know what they have to do on any course of study, it is not necessarily through published learning outcomes. Learning outcomes might be seen as necessary for administrative purposes but they are not sufficient in helping students develop an idea of what they will be learning and how they will go about it. Indeed, in a highly supportive context, learning outcomes might be so generalised as to only define the landscape and the boundaries of their intended learning. The knowing of what to do becomes developmental and personalised.

Observations

The requirement that all learning outcomes should use terms, particularly verbs, which are 'measurable' creates more challenges that it resolves. To insist on using terms such as 'identify', 'explain, 'analyse' and so on does not make the task of assessment any easier since explanations and analyses, etc, are discipline specific and are likely to be equally ambiguous for students who have not been yet been inducted into the language of the discipline.

The expectation that outcomes can be specified accurately and measurably at each level of the student's experience is fanciful. In art and design, teachers attempt to reconcile discipline benchmark statements, programme or course outcomes, stage-level outcomes, module outcomes and project outcomes, all within a coherent hierarchy using a limited set of verbs. Something has to give.

In art and design, outcomes are not achieved once and that is it. They are regularly returned to within and beyond levels. Therefore, assuming that outcomes, once addressed, are completed does not reflect the 'spiral' nature of the pedagogy of the discipline.

Designing and applying assessment criteria within an already overcrowded landscape of levels and outcomes can only add to student uncertainty.

In art and design, students complete 'projects' that are often driven by briefs that reflect practice within the professional world. Sketchbooks, journals and other supporting material are often submitted to demonstrate evidence of their achievement. In most cases, this can be seen as 'holistic' assessment in which the full collection of submitted material contributes to the final judgment. Holistic assessment practices do not easily lend themselves to 'analytical' assessment in which individual elements are marked and then the scores added up.

Given these observations, how do we move forward? How do we overcome the emergent technocratic ideology of learning and assessment but at the same time aspire to the broad interests of quality regimes in which students understand what is required of them, that the standards within the discipline are transparent to external agents and judgments about student achievement are seen to be valid and reliable? And, most importantly, how do we prevent the bureaucracy of outcomes and assessment overwhelming both teachers and students?

Unless a teacher has been party to the design and development of a programme, he or she will not necessarily understand how what they are expected to do fits into the whole. Programme design in such a complex landscape is often a negotiation of the language that embraces it. Only the course designers have a real understanding of how things fit together. New or part-time teachers, for instance, have to take the module outlines at face value and make sense of them in terms of their own professional experience. The greater the number of outcomes and the more elaborate the assessment scheme the more likely the whole thing will be sidestepped. Common sense often prevails in these circumstances. Therefore, there is a virtue in keeping the outcomes to a minimum even if this means a loss of specificity and apparent ambiguity. Rather than measurability, the focus should be on meaningfulness. It is better to provide a structure for discussions with the students to enable them to begin to engage in the discourses of the community in which they are joining than to assume they understand how they will perform against 'measurable' outcomes. For this purpose, students only need to understand whether they have addressed / met the outcome or not. It should be the stage level descriptors and the assessment criteria which provide the evidence of achievement.

Writing assessment criteria that is meaningful in relation to the outcomes and student performance is equally challenging. Constructively aligning outcomes and criteria makes sense in the abstract but it is the actual practice of relating them that is challenging. Biggs (2003) acknowledges the difficulty that arises in a predominantly 'divergent' set of disciplines such as the creative arts. It is not unusual to see assessment criteria expressed in the same language as the outcomes thus making it impossible to differentiate them. The nature of the assessment criteria needs to differentiate different levels of achievement without resort to the terms already used by the outcomes. Bloom's taxonomy is not helpful in this sense. Indeed, Bloom's taxonomy (1956) replicates most of the terms used in essay and report writing. The tension is only revealed when Bloom's taxonomy is used in a creative arts context where cognitive abilities other than knowledge and understanding are developed.

Biggs's SOLO taxonomy (2003) is a helpful guide in that it is derived from research and is deliberately hierarchical with each level embracing the previous one. So a student who takes a 'relational' approach to a task will also demonstrate capabilities in utilising 'multi-structural' and 'uni-structural' approaches. The taxonomy identifies students' concepts of, and approaches to, learning in an increasingly sophisticated way.

If we use the idea of a 'nested' hierarchy as the basis of the underlying quality structure for assessment in the way that Biggs does (and the way that Bloom's taxonomy is seen to do), but use concepts derived from the discipline rather than appropriate the terms uncritically from Bloom or elsewhere, then we might go some way to resolving some of the tensions revealed in recent QAA Audit reports.

Below is a worked example from Graphic Design, with explanatory commentary, to illustrate a possible approach to aligning learning outcomes with assessment criteria and avoiding the recurring pitfalls:

Example
Programme outcomes
The following programme outcomes are based on the threshold standards as expressed in the QAA Subject Benchmark Statement for Art and Design (2008). It is important to make reference to the Statement, in particular, to the threshold standard descriptors which set out what is required of a student to be eligible for an award in art and design. At the point of graduation (i.e. at Stage 3), students must demonstrate that they have the requisite skills and abilities, as outlined, in order to gain at least a pass degree. Apart from further descriptors alluding to a 'typical' pass, there are no indicators of further 'standards' relating to the classification system. It is expected that these will be articulated through the institution's / discipline's own assessment criteria.

At the end of the programme you will be able to:

A. Apply Graphic Design principles, research methodologies and analytical judgment in the generation of your research material, ideas, concepts and proposals

B. Select and utilise appropriate materials, techniques, methods and media in the realisation of your work

C. Identify and critically evaluate your practice within the contemporary cultural, historical, social and professional contexts through the use of a range of presentation skills

D. Work both independently and collaboratively (QAA, 2008).
Stage learning outcomes for Graphic Design
The following descriptors are deliberately succinct. They represent a 'nested' hierarchy of expectations of all students at each level of study so that it is taken that Stage 3 embraces Stages 1 and 2. They also represent the context in which the level standards are set. The key questions here are, 'What is it, in the broadest of terms, we want all students to be able to do at the end of the programme (Stage 3) and how do we get them there (Stages 1 and 2)?

Stage 1
Identify and utilise the skills and abilities of a graphic designer;
Articulate the focus of your work in relation to other specialisms;
Locate your work within a critical/theoretical and historical context.

Stage 2
Articulate your individual identity as a graphic designer;
Work independently and collaboratively;
Locate yourself and your work within broader issues and aspects of contemporary social practice.

Stage 3
Work effectively as a graphic designer.
Learning outcomes
The learning outcomes at module/unit level should seek to enable students to fulfill the Stage outcomes, again articulated as broadly as possible. Inevitably, unit briefs will combine creative expectations with more specific, often skill-based outcomes. The focus should be on meaningfulness rather than on measureability bearing in mind the support structures put in place to ensure that 'we haven't taken for granted what we shouldn't have taken for granted'.
Assessment criteria and grading/marking scale
The following scale is indicative and will vary depending on the nature and nuances of the discipline. Again, the categories are nested with each one incorporating the one below it. The hierarchy is based on students' abilities to integrate their thinking and progressively apply their thinking and abilities to more elaborate contexts. The scale can be applied at all three stages as the context and level of challenge is determined by the stage descriptor. Stage 3, for instance, is more challenging than Stages 1 and 2. As students progress they not only become more proficient in making and observing, say, they also get better at integrating these abilities and in more complex contexts. This is the essence of the 'spiral' curriculum.

Figure 2

The 'key words', which differentiate each level of achievement, in this instance, are based on the outcomes of an AHRB funded project (Davies and Reid, 2000 and Davies, 2006) which sought to identify discipline differences in art and design. What is important is that they are derived from the student experience of the subject. Other terms might be preferable but they should, ideally, be generated from observations of the structure of the learning outcomes of the discipline in context.

The act of assessment should be twofold. Firstly, determining whether the student has engaged with all the required learning outcomes (this is normally regarded as a pass). Then, using the assessment criteria, determining at what process level the student has been operating, given the full body of submitted work. This is not a question of adding up marks, moving from the specific to the general. It is more a question of determining an overall grade and justifying it with the available evidence against the criteria. Fine-tuning of the mark is negotiated afterwards.

As we have seen, the task of 'mapping' outcomes, curriculum and assessment can lead to a plethora of descriptors. In the search for accuracy and 'clarity', I would argue, the more we seek to map, paradoxically, the more the landscape becomes ambiguous if not for teachers then certainly for students. The use of matrices to generate assessment descriptors may appear methodical and structured but, in my experience, they produce far too many possibilities and lead to the analytical approach to assessment that Biggs, in particular, warns us off. They also attract the use of superlatives to differentiate the levels of achievement. At this point all is lost for the whole scheme becomes tautological.

Contact information
allandavies1@me.com
Biography

Allan Davies is a consultant in Higher Education Art and Design. Prior to this he was a Senior Adviser at the Higher Education Academy and before that he was Director of the Centre for Learning in Art and Design (CLTAD) at the University of the Arts. In the early days he was Head of Art and Design at what is now the University of Worcester and even further back he was programme leader for graphic information design at what was then Falmouth College of Art. During some of this he was a GLAD committee member. He is currently a QAA reviewer and was a member of the original QAA Subject Benchmark Statement for Art and Design panel.

References

Biggs, JB (2003) Teaching for Quality Learning at University, SRHE & OU Press.

Bloom, B.S. et al. (1956) Taxonomy of educational objectives: The classification of educational goals: Handbook I, Cognitive Domain, New York, David McKay.

Davies, A. and Reid, A. (2000) 'Uncovering Problematics in Design Education: Learning and the Design Entity', in Swann, C. & Young, E. (eds) Re-Inventing Design Education in the University: Proceedings of the International Conference, School of Design, Curtin University, Perth, pp.178- 184.

Davies, A (2006) '"Thinking like..." the enduring characteristics of the disciplines in art and design', in Davies, A (ed), Enhancing Curricula, 3rd CLTAD Conference Proceedings, Lisbon.

Hussey, T. and Smith, P. (2002) 'The trouble with learning outcomes', Active Learning in Higher Education, 3(3), pp. 220-233.

Price, M. O'Donovan, B. Rust, C. and Carroll, J. (2008) 'Assessment standards: a manifesto for change', Brookes eJournal of Learning and Teaching, Vol.2, No. 3.

QAA (2000) 'Art and Design Subject Overview Report 1998/2000'.

QAA (2008) 'Subject Benchmark Statement: Art and Design'.

QAA (2011) 'Outcomes from Institutional Audit (2007-09): Assessment and Feedback', 3 rd Series.

Rowntree, D. (1987), Assessing students: how shall we know them? London, Kogan Page (revised edition)

Figures 1 and 2 supplied by Allan Davies
Listing and header images: sections from photo by Paul Clark

# data[].attributes.attachment_link.title
Learning outcomes and assessment criteria in art and design. What's the recurring problem?

# data[].attributes.attachment_link.description
This paper is a critical reflection on the development of learning outcomes in art and design. It builds on a Learning and Teaching Support Network (LTSN) - funded project from 2003 that sought to provide support for colleagues seeking to make sense of the then new quality agenda.

# data[].attributes.headline
Seven Problems of Pointing to the Future of Education (with our hands tied behind our back) and Seven suggestions for addressing it (Dudley Nganjo)

# data[].attributes.subject
Seven Problems of Pointing to the Future of Education (with our hands tied behind our back) and Seven suggestions for addressing it (Dudley Nganjo)

# data[].attributes.body
‘Inevitably, the only way to cope is to behave strategically. "How do I pass?" becomes more important than "What do I understand?". Suppressing "What do I understand?" in some cases may lead to mental breakdown.’

In creative teaching a difficulty to overcome is students not focusing on the process but rather the end goal not realising the process makes the end goal closer a clearer in sight

# data[].attributes.attachment_link.readable_text
I've had some nice things happen recently, including publication of a paper with Loet Leydesdorff on musical communication and anticipatory systems in the journal Systems Research and Behavioural Science. The techniques I developed in this music paper have also been used in another paper which is getting published in the journal Interactive Learning Environments extending work on anticipatory systems into studying learning conversations and concept formation (work which I did in Russia - it would have been impossible to do in the UK). After having not published very much since my book last year (which I keep meaning to revisit but never get the time), it's quite nice to have two papers in good journals published at once.

My day job is not academic. Like many working in educational technology, we tend to spend our time chasing problems in systems that are designed by corporations, herding staff into new (sometimes uncomfortable) practices, dealing with the political can of worms that has become institutional IT, and now with the added intensity of COVID, there is no time to think. Which is a bad thing in a university.

It seems that in modern universities (particularly in the market-obsessed UK) thinking is regarded as inessential - something of a luxury. It's more important to "deliver" than to think. Good staff, under pressure to deliver, will know that delivery without thought is madness, and will spend much of their own time (unpaid) doing the thinking work, often in isolation. It will exhaust them as they then try to deliver something meaningful on top. It will exhaust them more that nobody around them actually sees the importance of thinking, or acknowledges that this is so critical and requires time and space.

I suppose my deep question here (which I ask when I do have time to think) is "Why have we become so stupid?" My answer to this is that Universities have convinced themselves that the environment they operate in is entirely cultural - it's a world of the "knowledge economy" and "markets", learning outcomes, institutional brands and certificates (ironically, this is what Loet Leydesdorff's work has concerned itself with!). COVID has shocked us all because it has reminded us of the fundamental importance of the natural environment - not just the world around us, but our own biology. As evolutionary biologist Simon Conway Morris wrote "First there were bacteria, now there is New York"

Stupidity comes from failing to understand one's environment.

We are biological organisms. We are not technical systems. Our biological constitution right down to the microbiome really matters. Many think the microbiome is deeply connected to our thinking and educational processes, but at the very least thought and consciousness sits on a biological substrate. We are at the beginning of discovering the extent of the connection between the microbiome, thought and consciousness as COVID transforms the natural environment of education.

The simple fact is that the epigenetic landscape of the campus - the biological markers which affect the ongoing development of each of us - is different from the epigenetic landscape of online learning. Ironically, this can help explain why and how universities have become somewhat thoughtless places. In many ways educational practices have become increasingly transactional over the last 20 years, whilst universities have spent their increasing income on new buildings - coffee bars, sports halls, students accommodation, learning spaces. It's as if the somewhat cold transactional processes of education for which the institution charged increasingly large fees needed to be compensated for with new spaces which were more biologically connected (and epigenetically rich) to make the transactional stuff bearable. Disneyland operates on a similar principle: generate enough oxytocin in the place, and nobody will complain about queuing for hours to buy a burger for $20.

Take the campus away - what happens? We're left with the cold transactional stuff administered by learning platforms, but no compensation. That's not going to work. The experience will be similar this account of a person who was living in a plush new flat in Battersea during lockdown: https://www.bbc.co.uk/news/business-53670199

The complex included restaurants, spas and bars. But when all those closed down because of the pandemic, the reality of her environment dawned on her.

"I spent seven weeks isolated there and realised that I absolutely hated it," she told the BBC. When you take away all the amenities that these developments advertise, then you realise you're just living in a glass box. It was a ghost town. It was just very soul-destroying living in this enormous development with no life going on."

That's the difference a change to the epigenetic environment makes. Education is meant to nourish the soul - but if the only soul nourishment took place in the campus bars and clubs (and the actual education was cold and transactional), then we're heading for trouble.

As if to underline the point and the folly of the educational establishment in believing they can ignore the biological environment, this rather shocking student-blaming video appeared from UniversitiesUK, promoting universities and exhorting students to cough-up, study online, and "stay strong":

The professoriate have, however, been reorganising themselves online. My "Important things group" (I ought to think of a better name) continues to meet (this is now week 13), discussing epigenetics (thanks to John Torday from UCLA) and physics (thanks to Peter Rowlands and his important nilpotent quantum mechanics). There are important contributions from Marc Pierson with his insights into systems theory, Richard Heiberger from Temple University with his brilliant insights into statistics, Andrew Crompton's (Liverpool) polymathery, Steve Watson's insights into education (Cambridge), Elizabeth Maitland's (Liverpool) work in management, Vinca Bigo's work on meditation (Kedge Management School, Marseille), Kerry Turner's knowledge of General Systems Theory and Sebastian Fiedler from Nurtingen and Geislingen University who brings insights into educational technology and psychology.

As biological organisms, we need to think. Universities were formed as places for thinking. The deep thinking is now happening online.

Does deep thought produce epigenetic markers which promote a confluence of ideas which is enjoyable and rewarding to participate in? My experience of comparing mindless managerial zoom meetings with this zoom meeting suggests to me that it does. Exploring the physiology, communication, environment, epigenetics, and technological activity seems to me to be an important research opportunity.

# data[].attributes.headline
How Emotion Matters in Four Key Relationships in Teaching and Learning in Higher Education Quinlan, K. (2016). (Dudley Nganjo)

# data[].attributes.subject
How Emotion Matters in Four Key Relationships in Teaching and Learning in Higher Education Quinlan, K. (2016). (Dudley Nganjo)

# data[].attributes.body
Relationship with - Subject, Self, Students & teachers and Peers

‘Most teachers are understandably reluctant to demand particular feelings as an assessed

outcome of higher education or to mandate a certain kind of formalized reflection.  However, hoping

that students gain confidence, feel passionate about a subject or an issue, develop sensitivities or

appreciation, or feel as if they belong to different communities as a result of our classes remain

worthy aspirations. ‘

Importance on creating a safe environment for students to feel belonging and influence increased desire and passion

# data[].attributes.headline
 Simplify flipped learning (Dudley Nganjo)

# data[].attributes.subject
 Simplify flipped learning (Dudley Nganjo)

# data[].attributes.body
Group space

Individual space

‘Changes the space in which direct instruction takes place’ you get the opportunity to directly help students with the ‘hard stuff’

Typically time is spent applying and understanding knowledge

Then go off for evaluation and analysis

‘Spend the bulk of class time in the middle of blooms Taxonomy’

InFlip learning would involve separating groups to to focus on individual space and other half for group space

# data[].attributes.headline
Teaching Art or Teaching to think like an Artist ? - Cindy Foley 2014 (Dudley Nganjo)

# data[].attributes.subject
Teaching Art or Teaching to think like an Artist ? - Cindy Foley 2014 (Dudley Nganjo)

# data[].attributes.body
“How can something so nebulous be so concrete ?”

‘Art education needs to focus on developing learners that think like Artists’

refers to the ‘The Lego Movie’ being a metaphor for suggesting the unassembled legos present creative ideas within to help us manifest ideas

Refers to the master builder being the educator forming great lesson plans

‘Visualise a classroom full of master builders’

‘Imagine if the future was not about discrete disciplines but rather was about disciplines like math, art and science being in service to ideas’

Create centres for creativity ?

# data[].attributes.attachment_link.title
Teaching art or teaching to think like an artist? | Cindy Foley | TEDxColumbus

# data[].attributes.headline
http://www.textjournal.com.au/speciss/issue16/Pace.pdf  Rethinking education as experience design engaging and memorable for their audience. (Dudley Nganjo)

# data[].attributes.subject
http://www.textjournal.com.au/speciss/issue16/Pace.pdf  Rethinking education as experience design engaging and memorable for their audience. (Dudley Nganjo)

# data[].attributes.body
- I particularly enjoyed this the most because it dives deeply into shaping your ability to teach from a different perspective and viewing yourself and the students differently as actor and audience but with the purpose to critique effectively in becoming a better tutor example below: - ‘Teachers, like actors, perform for an audience of students. Teachers prepare for their role, analyse past performances and constantly adjust the way they go about their craft, just as actors do. When teachers view their work as theatre, they gain benefits such as becoming more conscious of what makes a learning experience

# data[].attributes.headline
Intentionally Equitable Hospitality in Hybrid Video Dialogue: The context of Virtually Connecting

# data[].attributes.subject
Intentionally Equitable Hospitality in Hybrid Video Dialogue: The context of Virtually Connecting

# data[].attributes.body
This is the resource I like the most. It is about the ways to give remote students more power so they get more engaged. It is true that sometimes students in the remote space are neglected in Hyflex delivery. By allowing the remote participants to set agenda, making them facilitators, etc, we are going to engage them more. It also highlights the importance of creating a hospitable space which allows students to learn better. Social emotional well-being would bring a huge impact to their learning so making a space hospitable can improve students' learning performance.

# data[].attributes.attachment_link.readable_text
Being intentional, we, the Virtually Connecting (VConnecting) community, do not assume that saying "welcome" will mean people feel welcome. We are intentional about every step in our process. By equitable, we intend our practice to challenge power structures that contribute to unfair access and opportunities, in favor of emulating different power structures that promote greater equity. We constantly assess whether our processes and outcomes actually do this within our own community and others who use VConnnecting, including people who have critiqued us publicly and privately [ 1]. We agree "Equality is not a credential. Equality is a task. It is what we have to do, because we are not there yet" [ 2], yet we strive toward it. Equal access does not result in equal outcomes [ 3] and inviting people to participate is not equity without parity of participation [ 4].

VConnecting is a connectivist learning movement [ 5]. Using synchronous video technology, VConnecting hosts informal conversations between people who are attending academic conference onsite and people who are unable to attend. These conversations are livestreamed and recorded. We call these "hybrid conversations" since we seek to give equitable weight to onsite and virtual participation. VConnecting's purpose is to extend the onsite conversation, allowing the marginalized voices of those who are not present to participate (such as adjuncts, unaffiliated academics, graduate students, parents of young children, people with health issues, Global South scholars, or others who cannot regularly travel to conferences), while also allowing those who cannot attend an opportunity to network and learn from those who are present.

Online interaction in informal learning spaces, such as VConnecting, is reflected in communities of practice [ 6], affinity spaces [ 7], connected learning [ 8], and personal learning networks (PLNs), but these forms of community do not call out hospitality as a fundamental tenet. Online informal learning spaces (OILS) are often distributed and less hierarchical than formal learning spaces. This characteristic obscures power dynamics and lines of responsibility. In OILS it is not always obvious who invites whom and what are the entry points to these spaces. However, there is almost always implicit hospitality taking place. There are ways of inviting new members in, welcoming them and helping them become active members once they join, ways of recognizing and encouraging different modes and levels of participation, and considerations of whom to include and exclude-even when the space claims to be "open to all."

Among the Merriam-Webster definitions of hospitable, we find words like "generous and cordial reception," "generous and friendly welcome," "pleasant and sustaining environment," and "readily receptive; OPEN." In this article, we call VConnecting's dimension of community building "intentionally equitable hospitality " (IEhospitality) to differentiate it from hospitality that is superficially generous, friendly, or welcoming. Further, in order to be considered intentionally equitable hospitality it must question its own values and how this works in practice within power structures.

In the following sections, we describe (1) the benefits of intentionally equitable hospitality, (2) challenges with hospitality in hybrid video conversations, (3) VConnecting hospitality practices, and (4) conclusions and remaining challenges.

Benefits of Intentionally Equitable Hospitality

Hospitality is not the default in academia. Bonnie Stewart suggests "scholarship has never been particularly open to the public" it is "a carefully-managed ecosystem of gatekeeping measures" [ 9]. VConnecting is an intentionally inclusive space, an intercultural space meant to break down access barriers: "inclusive intercultural digital pedagogy is not a luxury, and can no longer be an afterthought. This time of walls and travel bans demands conviction and ingenuity from critical digital educators concerned with gestures of openness and hospitality" [ 10]. Being hosts of an informal, synchronous, and audiovisual conversation breaks down academic barriers as it allows interaction between established scholars and graduate students, for example, in ways not possible in most professional contexts [ 11, 12].

VConnecting challenges academic gatekeeping via rendering private hallway conversations that build social capital at face-to-face conferences into public hybrid conversations in which people who cannot attend conferences are able to participate. There remains a power differential between high-profile speakers and virtual participants, and the technical and location differences create a further power imbalance, all of which requires intentionally equitable hospitality in order to host a conversation that indeed challenges academic gatekeeping rather than reproduces inequalities.

For VConnecting, hospitality is a verb and a value which has complex underpinnings that are always impacted by context. Virtual participants are the center of why VConnecting exists, and ensuring they have a meaningful experience and opportunity to converse with onsite guests is key. Virtual participants are often people who are otherwise marginalized (global South scholars, graduate students, unaffiliated educators, and alt-academics are frequent Vconnecting participants and volunteers who would not normally be able to attend many conferences regularly), and intentionally equitable hospitality dictates that their needs and experiences are privileged. At the same time, onsite guests are taking time away from the precious onsite experience and we must also be hospitable to them and appreciative of their time and their choice to spend the time in conversation with virtual participants. Additionally, VConnecting considers the needs of conference organizers who may have concerns about the use of physical space, engagement with their paying participants, or the online attention that the conference may get because of a VConnecting session. Juggling all of the hospitality needs of multiple constituents is an ongoing, reflective process that is necessarily social justice focused.

Challenges with Hospitality in Hybrid Video Conversations

Through our experience, VConnecting has identified several challenges while conducting hybrid video conversations. These challenges are both logistical and related to intentionally equitable hospitality. We address each of these issues in VConnecting hospitality section of this paper.

Logistically, onsite audio is essential. Onsite participants must adequately hear virtual participants and vice versa. The challenge is caused in part by the chaos that often occurs at conferences when in public spaces. In addition, inadequate onsite audio, typically a personal laptop, make it difficult for clear communication.

Additionally, there is a visual imbalance. Some participants are physically together in a room, while others are individually joining from different locations, with their own microphone and camera. Images of the onsite and virtual participants differ in size and clarity, making it difficult to read body language, which leads to challenges in knowing when to speak, or even who wishes to speak.

Onsite there is a physical presence, especially if there is a group, which privileges feelings of togetherness among the onsite participants. This can cause onsite participants to lose sight of those who are virtual. Specifically, they see each other with equal space, make eye contact while speaking to each other, but they see all of the virtual people together as "one" person on a screen. When allotting time in a conversation, it is too easy for the onsite group to not allot equitable time for the virtual participants. This is an empathy issue, as the onsite participants have an easier time feeling connected to the conversation.

On a similar front, the virtual participants sometimes want to listen and not speak. Occasionally this is the result of power dynamics. The onsite guests may be prominent speakers and participants feel unable to interrupt the conversation in order to contribute. Silence has multiple interpretations and we must be sensitive to them.

VConnecting Hospitality Practices

"One of the fundamental principles of a pedagogy of hospitality: that however a gesture is conceived and offered, there are those who must receive it differently." -Kate Bowles [ 10]

Intentionally equitable hospitalit y requires a combination of facilitation skills, digital literacies, and intercultural sensitivities. The role of the facilitators as hosts is an important part of intentionally equitable hospitality that VConnecting strives toward. This role is called a "buddy." We use the term buddy as both a noun and a verb. To be viable, VConnecting sessions require, at minimum, one onsite buddy (at the conference) and one virtual buddy (not at the conference). Buddying a session entails more responsibilities than participants, including a wide array of logistics, only some of which are in this article. Specifically, we focus our discussion on the responsibilities around intentionally equitable hospitality. In the following sections, we have divided intentionally equitable hospitality practices of VConnecting into the following phases (1) pre-conference planning, (2) during-conference planning, (3) facilitating, and (4) sustaining community.

Pre-conference planning. The virtual planning phase takes place before an event begins. In this phase, we mobilize interested volunteers to lead the virtual and onsite activities (as virtual and onsite buddies).

The choice of which events we conduct VConnecting at are driven by what virtual participants are interested in rather than onsite participants or conference organizers. However, onsite participants and organizers can offer or initiate discussion about hosting a possible upcoming VConnecting event, and virtual participants may agree to engage. Ultimately, if no virtual person is interested, the event cannot run.

Inviting onsite participants takes place as part of the virtual planning phase and includes a focus on: (1) marginal onsite voices we would like to amplify, (2) high profile participants such as keynote speakers with which the virtual participants wish to converse, and (3) onsite participants who are familiar with VConnecting and wishing to hear voices from virtual participants. Combinations of these in one session helps everyone network and hopefully build community. Experienced VConnecting onsite participants and buddies help make new onsite participants comfortable.

Clear communication around planning and logistics is an important part of intentionally equitable hospitality for VConnecting. This includes scheduling across time-zones and cross communication between multiple channels. Though sometimes overlooked as a form of hospitality, clear communication during the planning process is vital to creating a hospitable environment.

Finally, we are intentional about inviting new voices into the conversation, as people often need the initial invitation to be personal not general. Marginalized people don't always see "anyone is welcome"to mean "you are welcome." Intentionally equitable hospitality implores us to extend personal invitations to marginalized persons. For example, if someone likes or shares our blog post, we intentionally reach out to invite them to participate. If we see someone on Twitter mention they are unable to go to a conference they wish they could go to, we invite them to participate.

During-conference planning.The planning phase during the conference occurs once the conference starts. Onsite, this involves finding an appropriate location, appropriate equipment, and notifying onsite participants of the location. IEhospitality considerations for onsite location include limiting background noise and visual interference (such as people walking behind the participants who don't wish to be on camera or windows that glare). Over time we have found a dedicated room is most effective. It also includes clear communications between onsite and virtual buddy, as well as between onsite buddy and onsite guests and virtual buddy and virtual participants.

Facilitating. The facilitation phase occurs during VConnecting sessions. The responsibility of intentionally equitable hospitality is held mostly by the virtual and onsite buddies but others pick up these responsibilities as they continue to participate with the community. These responsibilities are social, technical, and logistical.

Logistical: timely communications about the session including physical location and virtual details such as the link to join the session.

Technical: troubleshooting audio/visual issues that may present.

Social: greeting participants as they arrive in the virtual space (which also serves as a technical form of hospitality as this serves as an audio/visual check), being a good conversationalist to help amplify the voices of others, juggling the public/private tension, inviting participants to introduce themselves, etc., and using the text chat both to backchannel and to amplify the voices of those who are shy or have audio difficulties.

Intentionally equitable hospitality during these sessions is multicontextual, complex, and often paradoxical [ 13]. Though calling on someone to speak up might be empowering to some, it can be troubling to those who are shy and prefer to listen. Recording the session allows those who cannot make it at that time to listen afterward but may create an environment of high pressure for those who are participating live. These paradoxes abound, however, the perspective of intentionally equitable hospitality is not to resolve paradoxes but rather to embrace a multiplicity in an attempt to maximize affordances.

Sustaining community. Occurring all the time, sustaining community is standard ongoing operations of the community. One important point to add is that much of what makes VConnecting work is sustained interaction within the volunteer community semi synchronously, privately, and mostly textually via Slack. The community is sustained through other community specific activities such as larger planning projects, writing and scholarly opportunities, and modeling hospitality with new members, which is complicated when the entire team are volunteers. But it is also sustained through broader interactions that are outside of the community itself but which have cross-over membership.

For Whom Is VConnecting Equitably Hospitable?

Following the work of Hodgkinson-Williams and Trotter [ 14], which builds on the social justice model of Nancy Fraser, we would like to be explicit that equitable hospitality is contextual. Fraser's model looks at social justice in three dimensions: economic, cultural, and political. She also discusses how solutions to social justice can be transformative, but sometimes they are only ameliorative, and sometimes neutral, or even have a negative impact. Different individuals engaging in VConnecting (or not) experience these in different ways. For example, a graduate student has economic barriers to attending conferences and experiences exclusion from the culture and power of conferences, therefore, VConnecting can address all three dimensions of social justice for them. However, if the graduate student does not speak English, or does not have access to good internet infrastructure, or is too shy to participate in a session, VConnecting may have a negative effect, as it is one more space they are not allowed to participate in. VConnecting may also have a neutral or ameliorative effect if they are able to watch recordings (because this would still seem like broadcasting, or enhancing access, but not directly addressing cultural or political injustice as their voice does not get heard). However, VConnecting would have a transformative effect when it enables graduate students to have equitable conversations with high-profile conference participants, become heard, gain confidence, and develop reciprocal relationships with them over time, which empowers them as scholars [ 11, 12]. Another group of people who benefit from VConnecting are global South scholars, who benefit economically but also culturally and politically, particularly in terms of inserting their global South perspectives into conference experiences that often lack their presence, and if these global South scholars are involved in choices related to who is invited into a VConnecting session, for example. Again, as with graduate students, those who lack English, internet access, or willingness to participate in synchronous video will not be affected positively by VConnecting, and may indeed be affected negatively. In some countries, they are excluded because, for example, YouTube is blocked. If they speak English less fluently than other participants, or they have lower-quality internet connections, they may struggle to get heard in sessions. There may, indeed, be onsite guests who are not willing to listen to virtual participants, or who are disrespectful to others, and occasionally the virtual and onsite facilitators are unable to redress this inequality in the dialogue. But when global South scholars are able to participate fully, the impact can be transformative in redressing epistemic injustice at conferences, and addressing political issues (of representation) and influencing conversations across conferences sometimes.

There are potential participants with disabilities that would be completely excluded from VConnecting, such as those with hearing disabilities. However, there are those who lack mobility due to permanent or temporary health issues who have benefited from VConnecting to conferences they could not attend. In these cases, VConnecting ameliorates injustice by offering access but not challenging culture or power at conferences.

In terms of onsite buddies/volunteers, they benefit mainly from the ability to make connections and cultural capital at conferences, as many of them are not powerful academics in the traditional sense, even if economically, they are able to attend conferences.

It is also important to recognize VConnecting may lay outside of traditional academic hierarchies, but as Bonnie Stewart says of Twitter [ 1], develops new hierarchies of its own. And therefore, despite the intention to be equitably hospitable to new volunteers and participants, there are emergent power structures related to personalities, histories and social and cultural capital within the community.

Finally, there are many who participate in and benefit from VConnecting who are already well-connected and not particularly disadvantaged. For these people, VConnecting is beneficial as a community but does not serve a social justice purpose related to their own privilege. However, they become allies who care about redressing social justice issues of others in the community, and using whatever power and connections they have to support others, and in many instances, they have spoken about how VConnecting has made them more aware of the voices of those not regularly present at events.

In practice, VConnecting cannot and does not challenge academic gatekeeping in every instance, but strives to create an intentionally equitable environment of hybrid dialogue in which this can thrive. Not every volunteer practices intentionally equitable hospitality in every moment, and not every conversation redresses epistemic injustice, but there are opportunities to do so that did not exist without VConnecting.

Conclusions and Remaining Challenges

What we have learned about IEHospitality in hybrid spaces can be useful to other learning contexts. For example, those of us who teach online making aspects of equity and hospitality explicit within online group work, helps to prevent many of the common problems students face [ 15]. What volunteers learn and practice in VConnecting often transfers to our teaching and professional practice, a reminder how we can benefit greatly from using online learning techniques and technologies in our own learning first. In addition, buddies often gain social capital of their own through greater communication with each other and with guests who are often high-profile academics.

Cronin suggests openness is personal, contextual, cultural, and constantly negotiated [ 16]. Our notion is that VConnecting as individuals and as a community strive towards this understanding of openness as VConnecting's hospitality is growing in the same direction.

A group of us [ 17] recently asked the community what they expected from a radical hospitality of VConnecting. Kuhn [ 17] reminded us to consider those who stay in the margins, or who have no technical access, and how we might include them. Zamora [ 17] reminded us to continue "paying attention and not making assumptions about whether a person feels included or not" and "cultivating a keen awareness of the way power and privilege shape our inclusion."

"But a wall does not appear to those who do not come up against it," explained another [ 18]. As a hybrid space, VConnecting makes some of the walls of academic gatekeeping visible to everyone, and has been a bridge that aims to change the landscape of both the privileged conference-goers and the perpetually virtual participants (and everyone in between) via intentionally equitable hospitality.

References

[1] Bali, M., Beckingham, S., Zamora, M., Caines, A., Hogue, R. J., and Weller, M. Breaking the physical presence barrier: Virtually Connecting as an approach to open, inclusive conferences . Open Educational Resources (#OER17), London, UK. April 5, 2017.

[3] Czerniewicz, L. Considering inequality as higher education goes online [keynote address]. Association for Learning Technology Conference. Manchester, UK. September 9-10, 2015.

[5] Bali, M., Caines, A., DeWaard, H., and Hogue, R. Ethos and practice of a connected learning movement: Interpreting virtually connecting through alignment with theory and survey results. Online Learning 20, 4 (2016). DOI: https://doi.org/10.24059/olj.v20i4.965

[6] Wenger, E. Communities of practice and social learning systems: the career of a concept."In Blackmore, C. (Ed.) Social Learning Systems and communities of practice. Springer Verlag and the Open University, 2010.

[8] Ito, M., Gutiérrez, K., Livingstone, S., Penuel, B., Rhodes, J., Salen, K., and Watkins, C. Connected learning: An agenda for research and design [report]. Irvine: Digital Media and Learning Research Hub. 2013.

[15] Roberts, T.S. and McInnerney, J.M. Seven Problems of online group learning (and their solutions). Educational Technology & Society 10, 4 (2007), 257-268.

[17] Angell, N., DeWaard, H.J., Hogue, R. J. and Taleo, W. Help us define and give examples of "radical hospitality" #2. GITHUB issue. October 15, 2018.

Acknowledgment

The authors would like to acknowledge the feedback the VC community gave over the years that helped influence this paper, including most recently the team who worked with Mozilla Open Leaders program to develop community definitions of VC hospitality, among other things: thank you Nate Angell, Wendy Taleo, Helen J. DeWaard, and Rebecca J. Hogue. Thank you especially to VC community members who volunteer and embody this hospitality and extend it to others.

About the Authors

Maha Bali is Associate Professor of Practice at the Center for Learning and Teaching at the American University in Cairo. She is an editor at Hybrid Pedagogy journal, and editorial board member of: Teaching in Higher Education; Online Learning Journal; Journal of Pedagogic Development; Learning, Media and Technology; and Journal of Educational Technology in Higher Education. She has blogged for the Chronicle of Higher Education's Prof Hacker, DMLCentral blogs and Al-Fanar media. She is co-founder of virtuallyconnecting.org and co-facilitator of Equity Unbound. She is former International Director of Digital Pedagogy Lab. She is a learnaholic, writeaholic and passionate open and connected educator. She can be found on Twitter at @bali_maha and blogs at http://blog.mahabali.me

Autumm Caines is an instructional designer at the University of Michigan - Dearborn who has worked in the field of academic technology for 15 years. Her work focuses on online community, faculty development, educational technology, open education, and online security and privacy. Prior to her current position Autumm held appointments at St. Norbert College in De Pere, WI and Capital University in Columbus, Ohio, both in the USA. Autumm is a co-director of the online connected learning movement Virtually Connecting, which helps her to better understand presence and spontaneity in virtual synchronous conversations as well as equity and inclusion in online community.

Rebecca J. Hogue is an associate lecturer at University of Massachusetts - Boston. She is pursuing a Ph.D. in health professions education at the University of Ottawa, Canada and is co-founder of Virtually Connecting. Her professional background is in instructional design and software quality assurance. She holds a Master of Arts degree in distributed learning, and a Bachelor of Science degree in computer science. Rebecca teaches courses on instructional design foundations, the design and instruction of online courses, building educational multimedia, and designing online professional presence. She has more than 15 years of experience as an instructional designer in a variety of contexts (private sector, higher education, and medical education).

Helen DeWaard teaches digital and media literacy at the Faculty of Education, Lakehead University, Orillia. She has completed a master's of educational technology from the University of British Columbia, a master's of education from the University of Toronto, OISE and is currently pursuing a Ph.D. Her passions include teaching and learning with technology, digital storytelling, critical digital literacy and media literacy, connecting to global contexts, and digital badges. She volunteers with Virtually Connecting and the ISTE Inclusive Learning network, and has participated in the Mozilla Open Leaders project. Helen has served as an Open Education Fellow with eCampus Ontario and engages as a mentor through UNESCO to support others in open teaching and learning. She can be found at http://www.hjdewaard.com/. Helen actively tweets about education related topics @hj_dewaard and blogs at Five Flames for Learning

Christian Friedrich is the education and science advisor at Wikimedia Deutschland. As a member of the team "Education, Science & Culture," Christian is committed to work towards openness and freedom of education and science. Christian has led various higher education projects in the context of "open" and "digital and connected learning." He is also part of the vibrant Virtually Connecting community, which aims to provide virtual participation and representation at academic conferences. In 2018, Christian was named one of the Virtually Connecting Co-Directors. Christian works as a freelance consultant with organizations to foster organizational change and learning while considering meaningful use of technology. Together with colleagues and friends from academia and activism, Christian conceived Towards Openness, a knowledge base and platform for critical conversations around openness and education. Christian blogs erratically on his website christianfriedrich.org, and tweets as @friedelitis. He also hosts various German podcasts on technology, learning, and open education.

© ACM 2019. 1535-394X/19/06-3331173 $15.00

# data[].attributes.attachment_link.title
Intentionally Equitable Hospitality in Hybrid Video Dialogue: The context of Virtually Connecting

# data[].attributes.attachment_link.description
eLearn Magazine is a leading source of high-quality information on the uses of online learning and training strategies in a variety of contexts for K-12, higher education, and the corporate workforce.eLearn Magazine presents new technologies and approaches for creating, delivering, and supporting online instruction and workplace performance.

# data[].attributes.body
This article is about chat GPT. The writer illustrates the implications of Chat GPT and suggested more discussion on the ethical issues involved among students and academics. Whether Turnitin, for example, can check the plagarism by Chat GPT is uncertain. There will be continaully new technologies, but at the same time it is always good to reflect on how these technologies can better complement learning and teaching. It is just like LinkedIn Learning can help with self-directed learning but not replacing university.

# data[].attributes.attachment_link.readable_text
As I scrolled through my social media feeds over the Christmas break, "What do mean for the AI tools such as future of higher education?" pieces - I'm sure you saw them as well. Then, last weekend, I was at a barbecue dinner with a colleague, and our chat turned to "how's your class shaping up for the coming semester?" My colleague was aware of ChatGPT and isn't worried about it destroying the fabric of higher education or anything like that, but they were a bit unsure about if and how they could make use of it in their class.

First, if your class requires deliverable written work - essays, for example, but also lab reports, process blogs, compulsory forum posts and more - then some of your students are going to use ChatGPT to write their submissions. It's inevitable, and although the New York schools district is trying to bury its head in the sand and , this is both too difficult to enforce in practice

So, as you look over your course outline and assessment schedule, the first thing you need to ask yourself is whether you care if your students use ChatGPT. It's a question that goes to the heart of why we set deliverables at all, and includes reasons such as , but for most educators it is also about deeply held ideals about education as the pursuit of knowledge and truth, as well as what it means to give a good (or bad) grade.

Maybe you don't care, or at least you don't care enough to rejig your whole class to "defend" against students using ChatGPT to submit work that they don't understand. I think that's an OK position to take for now, although I'd encourage you to watch closely as other educators try things (and no doubt fail in interesting and unforseen ways).

We will learn from each others' experiences in how best to incorporate these tools in our classrooms. But in the meantime you should at least think hard on: "Why do I set that particular essay as the final assessment item in this course, and what would it mean if a student used ChatGPT to write it?"

Second, there are already some good ideas floating around on how to use ChatGPT productively in your teaching. And they aren't just useful if you're designing a new course on "applied AI language models"; there are ways to incorporate it into an existing course as well. The Centre for Learning and Teaching at Washington University has already got a helpful ChatGPT and AI composition tools page up on their website. And there is a bunch of folks experimenting with ChatGPT-designed syllabi

I think that people like Yoav Goldberg are on the right track when : "I think a 'write an essay with the help of chatGPT and discuss the process and the resulting prose' can be a super-effective assignment...in a humanities-centric, critical-ai program."

If you're looking for concrete opportunities to incorporate ChatGPT into your class this coming semester, look through your course outline for every time you ask your students to produce a written artefact in response to a question or prompt you give them. First, plug that prompt into ChatGPT and see what results you get. Try to get it to give you better (and worse) answers by tweaking the prompt. In what situations does it give a good and interesting answer, and in what situations is it wrong in subtle (or not-so-subtle) ways?

Then think: can I change the exercise so the student asks that question of ChatGPT and then has to critically reflect on the output they receive? This process could be scaffolded by first doing it in a facilitated class discussion setting and then later as an assessment task (and perhaps the students need to create their own rubric or criteria on which the output of the AI model should be evaluated). It's OK to start small; there's no need to rip up the whole course outline and replace everything with a ChatGPT exercise just yet.

There are heaps of variations on this basic idea, but the general idea is this: wherever in your current class you ask your students to write something, that's an opportunity to get the student to co-write (or evaluate) that same something with ChatGPT. On a superficial level, such "AI tool use" is the sort of process that knowledge workers will increasingly be incorporating into our daily workflows in the future. At a deeper level, it's an opportunity to reflect on where these tools work well and where they're useless.

One final point I made to my colleague in the context of incorporating ChatGPT into this semester's class is about availability. During the current initial research preview ChatGPT is free to use . But OpenAI, the creators of ChatGPT, haven't released any information about how long the free plan will be available. Meta's latest large language model , although it does look like ChatGPT is going to have a longer shelf life than that.

ChatGPT running costs are estimated at $3 million per month , and OpenAI is a private company, so they can't be counted on to keep it running free forever as a public good. If it's eventually priced similarly to GPT-3 (the AI language model on which it's based) then it'll be fairly cheap - probably only a few cents to write an article as long as this one - but you're still left with an equity issue unless you're willing to pay for a subscription for all your students. Even if your institution is willing, these tools don't yet have an (easy) way to sign up a whole class, ensure that students' activity stays under a given budget and send one easy bill. I'm sure that will come in time, and AI companies may well be willing to offer partnerships and scholarships to educational institutions doing this sort of thing - but be prepared for a small panic if ChatGPT's free trial gets turned off 24 hours before an assessment deadline.

I'm not trying to be the techno-optimist who downplays the potential downsides of ChatGPT in higher education. These tools may well , and higher education is (clearly) going to be affected if that happens. And I certainly don't want to heap yet greater burdens on our already overworked classroom educators - if you don't have the headspace to try this right now, then that's OK, too. My main point is that if you're careful you can start using AI writers in the classroom today - your students will be anyway, and this way you get to go on that journey with them.

Ben Swift is educational experiences lead and associate director (education) at the ANU School of Cybernetics. The ANU School of Cybernetics is activating cybernetics as an important tool for navigating major societal transformations through capability building, policy development and safe, sustainable and responsible approaches to new systems.

# data[].attributes.attachment_link.title
So, you want to use ChatGPT in the classroom this semester?

# data[].attributes.attachment_link.description
As I scrolled through my social media feeds over the Christmas break, "What do mean for the AI tools such as future of higher education?" pieces - I'm sure you saw them as well. Then, last weekend, I was at a barbecue dinner with a colleague, and our chat turned to "how's your class shaping up for the coming semester?"

# data[].attributes.body
"3. The curriculum is an outmoded means of coordinating conversation. It is a rigid inflexible object in a fast-changing, uncertain world. The means of coordinating conversation needs to become a much more flexible technology (indeed, this is where investment in technology should be placed, not in VLEs or e-Portfolio, which merely uphold the ailing curriculum)" - agree on this and have been considering if we can alter and transform our education as it happens and can it cohort or year specific ? Should it be ?

# data[].attributes.attachment_link.readable_text
I've had some nice things happen recently, including publication of a paper with Loet Leydesdorff on musical communication and anticipatory systems in the journal Systems Research and Behavioural Science. The techniques I developed in this music paper have also been used in another paper which is getting published in the journal Interactive Learning Environments extending work on anticipatory systems into studying learning conversations and concept formation (work which I did in Russia - it would have been impossible to do in the UK). After having not published very much since my book last year (which I keep meaning to revisit but never get the time), it's quite nice to have two papers in good journals published at once.

My day job is not academic. Like many working in educational technology, we tend to spend our time chasing problems in systems that are designed by corporations, herding staff into new (sometimes uncomfortable) practices, dealing with the political can of worms that has become institutional IT, and now with the added intensity of COVID, there is no time to think. Which is a bad thing in a university.

It seems that in modern universities (particularly in the market-obsessed UK) thinking is regarded as inessential - something of a luxury. It's more important to "deliver" than to think. Good staff, under pressure to deliver, will know that delivery without thought is madness, and will spend much of their own time (unpaid) doing the thinking work, often in isolation. It will exhaust them as they then try to deliver something meaningful on top. It will exhaust them more that nobody around them actually sees the importance of thinking, or acknowledges that this is so critical and requires time and space.

I suppose my deep question here (which I ask when I do have time to think) is "Why have we become so stupid?" My answer to this is that Universities have convinced themselves that the environment they operate in is entirely cultural - it's a world of the "knowledge economy" and "markets", learning outcomes, institutional brands and certificates (ironically, this is what Loet Leydesdorff's work has concerned itself with!). COVID has shocked us all because it has reminded us of the fundamental importance of the natural environment - not just the world around us, but our own biology. As evolutionary biologist Simon Conway Morris wrote "First there were bacteria, now there is New York"

Stupidity comes from failing to understand one's environment.

We are biological organisms. We are not technical systems. Our biological constitution right down to the microbiome really matters. Many think the microbiome is deeply connected to our thinking and educational processes, but at the very least thought and consciousness sits on a biological substrate. We are at the beginning of discovering the extent of the connection between the microbiome, thought and consciousness as COVID transforms the natural environment of education.

The simple fact is that the epigenetic landscape of the campus - the biological markers which affect the ongoing development of each of us - is different from the epigenetic landscape of online learning. Ironically, this can help explain why and how universities have become somewhat thoughtless places. In many ways educational practices have become increasingly transactional over the last 20 years, whilst universities have spent their increasing income on new buildings - coffee bars, sports halls, students accommodation, learning spaces. It's as if the somewhat cold transactional processes of education for which the institution charged increasingly large fees needed to be compensated for with new spaces which were more biologically connected (and epigenetically rich) to make the transactional stuff bearable. Disneyland operates on a similar principle: generate enough oxytocin in the place, and nobody will complain about queuing for hours to buy a burger for $20.

Take the campus away - what happens? We're left with the cold transactional stuff administered by learning platforms, but no compensation. That's not going to work. The experience will be similar this account of a person who was living in a plush new flat in Battersea during lockdown: https://www.bbc.co.uk/news/business-53670199

The complex included restaurants, spas and bars. But when all those closed down because of the pandemic, the reality of her environment dawned on her.

"I spent seven weeks isolated there and realised that I absolutely hated it," she told the BBC. When you take away all the amenities that these developments advertise, then you realise you're just living in a glass box. It was a ghost town. It was just very soul-destroying living in this enormous development with no life going on."

That's the difference a change to the epigenetic environment makes. Education is meant to nourish the soul - but if the only soul nourishment took place in the campus bars and clubs (and the actual education was cold and transactional), then we're heading for trouble.

As if to underline the point and the folly of the educational establishment in believing they can ignore the biological environment, this rather shocking student-blaming video appeared from UniversitiesUK, promoting universities and exhorting students to cough-up, study online, and "stay strong":

The professoriate have, however, been reorganising themselves online. My "Important things group" (I ought to think of a better name) continues to meet (this is now week 13), discussing epigenetics (thanks to John Torday from UCLA) and physics (thanks to Peter Rowlands and his important nilpotent quantum mechanics). There are important contributions from Marc Pierson with his insights into systems theory, Richard Heiberger from Temple University with his brilliant insights into statistics, Andrew Crompton's (Liverpool) polymathery, Steve Watson's insights into education (Cambridge), Elizabeth Maitland's (Liverpool) work in management, Vinca Bigo's work on meditation (Kedge Management School, Marseille), Kerry Turner's knowledge of General Systems Theory and Sebastian Fiedler from Nurtingen and Geislingen University who brings insights into educational technology and psychology.

As biological organisms, we need to think. Universities were formed as places for thinking. The deep thinking is now happening online.

Does deep thought produce epigenetic markers which promote a confluence of ideas which is enjoyable and rewarding to participate in? My experience of comparing mindless managerial zoom meetings with this zoom meeting suggests to me that it does. Exploring the physiology, communication, environment, epigenetics, and technological activity seems to me to be an important research opportunity.

# data[].attributes.body
"Apart from the constraints of the use of language, students who take a strategic approach to learning are not likely to depart from the published words and the concrete world. For them, literal interpretation is paramount. However, art and design students are often in pursuit of a ‘quarry’ of which they are given only partial knowledge. Indeed, this can be a deliberate learning strategy in the creative arts. For art and design students, formulating and finding their own quarry is an essential part of the discovery process. They do, nevertheless, need to know the ‘landscape’ and the ‘boundaries’ when they are in full pursuit. It might be that these are better articulated in the form of a discourse than in specific outcome form and more usefully manifested in project briefings, team meetings, etc" - Interesting way to articulate those "pitfalls" within arts and design assessing, the work is so subjective the students are in a microcosm of their own creation, rather than seeing the macro veiw of their work.

# data[].attributes.attachment_link.readable_text
This paper is a critical reflection on the development of learning outcomes in art and design. It builds on a Learning and Teaching Support Network (LTSN) - funded project from 2003 that sought to provide support for colleagues seeking to make sense of the then new quality agenda. It seeks to identify recurrent issues and make a number of recommendations for further development.

Keywords: learning outcomes, assessment criteria, constructive alignment, art and design

Abstract

This paper is a critical reflection on the development of learning outcomes (LOs) in art and design. It builds on a Learning and Teaching Support Network (LTSN) - funded project from 2003 that sought to provide support for colleagues seeking to make sense of the then new quality agenda. It seeks to identify recurrent issues and make a number of recommendations for further development.

Around 10 years ago, I completed a project, funded by the then Learning and Teaching Support Network (LTSN) Art, Design and Communication Subject Centre, intended to provide support for art and design teachers who were developing learning outcomes and assessment criteria for their students ( download the project report). My intention was to lay out the issues as clearly as possible and provide advice, particularly for new staff, on how outcomes and assessment might be related to the student experience. There was an underlying imperative to the project at the time which was the emergence of 'learning outcomes' as key features of learning in the new quality agenda established by the Quality Assurance Agency (QAA). Although outcome-led learning was not new to higher education, the utilisation of learning outcomes within the definition of quality learning was. This paper is my personal reflection on those issues based on work I have carried out subsequently both formally as research and informally in supporting institutions and departments in writing outcomes and assessment criteria.

There was emerging a good deal of literature surrounding outcome-led learning, particularly on how to write coherent outcomes within a skills context and, increasingly so, in a context in which the written form, such as essays and reports, was a predominant mode of evidencing achievement. There was less similar material on assessment, although the literature on assessment, particularly in secondary education, has a long history. There was, of course, even less literature on these topics within art and design.

I drew much of the material for the project from other sources. There was a broad consensus on the value of learning outcomes and how they might be articulated. There was an obvious concern to avoid ambiguity in the writing and presentation of learning outcomes with cautionary remarks and recommendations that I wove into the project, attempting, where possible, to highlight the challenges for art and design. Simultaneously, there emerged a most valuable critique of learning outcomes by Hussey and Smith (2002) that had the potential to derail the outcome juggernaut before it had developed any steam. Their comments, despite being equally relevant today, did not hamper the momentum that has led to learning outcomes being adopted in (almost) all HE institutions in the UK.

The design and use of assessment criteria has appeared less under the microscope largely because they have always been a requirement of the validation of courses in universities. Nevertheless, as the focus shifted towards student-centred learning, the literature began to pick up on broad issues such as the differences between norm-referenced and criterion-referenced assessment, the impact of assessment on learning, self- and peer assessment and so on.

John Biggs' book, Teaching for Quality Learning at University (2003) became highly influential during this period and has been used subsequently as a key reader on most courses for new teachers in universities throughout the UK. In my paper, I drew heavily on Biggs, particularly his SOLO taxonomy as it was quite evidently, like Bloom's taxonomy beforehand, a potential tool for developing and designing assessment criteria.

During this period, what became clear was that the developmental histories of learning outcomes and assessment criteria, for most institutions, were quite different.

Assessment criteria in most universities were derived from Bloom's taxonomy (1956). The convenience of the structure of the taxonomy, as it related to the undergraduate degree classification system, meant that the categories could quite easily form the basis of the overall generic assessment strategy of universities. It wasn't unusual, however, for departments to adapt the taxonomy to suit their disciplinary requirements or for universities themselves to insert extra levels to further differentiate the classes. Terminology, such as 'good', 'very good', 'excellent' and often 'outstanding', were, and still are, used to differentiate levels of competence although in themselves provided no insight into their meanings and how they related. Such terms are often used as qualifiers within level descriptors.

So, whilst the assessment policies, strategies and processes became embedded into the culture and social practices of universities and, hence, to be seen as a strong evidential basis for comparison of standards and quality between universities, learning outcomes, a much more recent phenomenon, had to find a way of fitting in.

Curiously, whilst the existence of learning outcomes and assessment criteria were deemed essential for a positive outcome during the QAA Subject Review process (1998-2000), no heed seemed to have been taken by almost half of the art and design subject community to ensure the clarity and use of assessment criteria.

Less than half (43 per cent) of providers' arrangements for the assessment of students are completely effective. Although students are assessed using a suitably wide range of methods, the majority of reports identify a number of sector-wide concerns with the assessment of students. These relate primarily to inconsistencies in the clarity and use of assessment criteria, and their relationship to published learning outcomes. The criteria for different assessment bands are not always sufficiently differentiated. Where generic criteria have been developed, these often fail to engage with subject-specific learning outcomes, and the reviewers also report inconsistencies in the grading of different levels of achievement. (QAA, 2000, Section 36, Teaching Learning and Assessment)

At the end of the report and by way of recommendation, the authors suggested 'there is a widespread need to improve the definition of assessment criteria, to relate them to published learning outcomes and to use them consistently' (QAA, 2000, Section 69, Conclusion).

At the time, this was not regarded as an issue peculiar to art and design. In 2000, the QAA published its 'Code of Practice for the Assurance of Academic Quality and Standards in HE' with Section 6 of the Code urging institutions to ensure that 'assessment tasks and associated criteria are effective in measuring student attainment of the intended learning outcomes' (Section 6: Assessment of Students).

In a workshop at the Centre for Learning and Teaching in Art and Design (CLTAD) at the University of the Arts in 2003, organised to investigate this concern, 16 experienced reviewers considered what they felt was behind this difficulty. The conclusion reached is interesting in as much as the problem, as manifest in individual institutions, was revealed to arise not so much from the interrogation of the evidence base submitted to the QAA as from their seeking of the actual student experience. If student groups reported, during the review, that they indeed did know what was required of them on their course and they were clear about how and why they were being assessed, despite the evident lack of clarity of the documentation given to them, then there was no issue to answer. In these instances, it was accepted that there existed strategies set up for students to receive this information other than those revealed through the formal course documentation. This is important insofar as the student voice about their experience was seen as paramount. It was only in those instances where the students reported that they didn't know what they had to do and how they were being assessed (despite published learning outcomes and assessment criteria) that reviewers could legitimately challenge the coherence of the outcomes and the criteria. All this gives rise to the following possibilities:

The ideal scenario is, of course, revealed in the top left sector. The bottom right sector represents the worst possible scenario. The other two sectors are interesting. The bottom left suggests that the course team has spent more effort developing outcomes than helping students to understand them. As Ferenc Marton said at the 'Improving Student Learning Conference' in Exeter (1995), 'If a student doesn't understand, then there is something we have taken for granted that we shouldn't have taken for granted'. This seems to me a case in point.

In my time, I have witnessed all four of the above scenarios but it is the top right sector that intrigues me most. Our obsession with establishing the accuracy/clarity of learning outcomes in the belief that this an essential prerequisite for quality learning to take place is undermined by those courses in which the written learning outcomes are largely unclear but the students are performing well. A lack of clarity in the learning outcomes, it seems, does not mean that students are not clear about what they have to do. Indeed, learning outcomes, ambiguous or otherwise, appear to be no substitute for established learner support systems and other frameworks that help students understand what they have to do in order to successfully complete a programme of work. Briefs and briefings are familiar in art and design along with tutorials, interim crits and feedback forums. It is during these supportive scenarios that art and design students formulate their intentions and actions and come to understand what 'imagination', 'creativity', 'risk-taking', etc, (the very terms regarded as potentially ambiguous) actually mean for them.

I would argue that the insistence that learning outcomes should be sufficiently clear 'to be measurable' has not helped those subject areas, such as the creative arts, in which articulating outcomes that involve the development of intuition, inventiveness, imagination, visualisation, risk-taking, etc, is challenging. In terms of meaningfulness, they equate to the notion of 'understanding', a cognitive term which is regarded as too complex and which should be substituted by other, more measurable, terms such as, 'explain', 'analyse', etc. Another drawback in the use for these terms, acknowledged by Biggs (2003), is that they are regarded as 'divergent' and as such do not invite one appropriate answer but a range of possibilities.

An example of this is art and design students' ability to visualise. This cognitive ability is a cornerstone of creative thinking. It requires the use of imagination and judgment and we expect all art and design students to develop it as they progress in their study. Fashion students develop their collections by visualising what comes next, furniture design students visualise their products individually, together and in a context, graphic design students visualise alternative layouts and so on.

Nevertheless, this concept is not easily captured in learning outcome form. It's not the kind of thing that can be measured easily. It is, in fact, developed within the whole complex process of the practice and over time.

Constructive alignment, again Biggs's notion (2003), attempts to tackle an important issue relating to the coherence of the programme of study and the student experience of it. There has to be a relationship between what a student is expected to do (the intended learning outcomes) on a programme, the programme content and delivery that supports the student's achievement and the process and judgments by which the quality of the learning is determined (assessment criteria). For me, Rowntree raised this issue forty years ago; 'to set the student off in pursuit of an un-named quarry may be merely wasteful, but to grade him (sic) on whether he catches it or not is positively mischievous' (1977).

It would be odd for any teacher, nowadays, to dissent from the sentiment that students need to know what they are expected to learn on any given course or module. However, the over-specification of outcomes for the purpose of measurement can be counter-productive in art and design and other creative disciplines. The more specific the learning outcome is the more specific the 'quarry' will be and the greater the constraint on whether the most appropriate quarry is caught or not. This is not just seen to be true of creative subjects;

Limits to the extent that standards can be articulated explicitly must be recognised since ever more detailed specificity and striving for reliability, all too frequently, diminish the learning experience and threaten its validity. There are important benefits of higher education which are not amenable either to the precise specification of standards or to objective assessment (Price et al, 2008)...

Apart from the constraints of the use of language, students who take a strategic approach to learning are not likely to depart from the published words and the concrete world. For them, literal interpretation is paramount. However, art and design students are often in pursuit of a 'quarry' of which they are given only partial knowledge. Indeed, this can be a deliberate learning strategy in the creative arts. For art and design students, formulating and finding their own quarry is an essential part of the discovery process. They do, nevertheless, need to know the 'landscape' and the 'boundaries' when they are in full pursuit. It might be that these are better articulated in the form of a discourse than in specific outcome form and more usefully manifested in project briefings, team meetings, etc.

Students also need to know how well they have performed in 'catching the quarry'. In the UK universities classification system, we require final judgments about student performance to be within one of five categories - 1 st, 2.1, 2.2, 3 rd and fail. This is often mirrored throughout courses by utilising grades rather than classes. Usually, assessment grades at module level reflect the system as a whole. Hence, grading systems use criteria to differentiate levels of performance. To satisfy both Rowntree and Biggs, therefore, we need to demonstrate the relationship (alignment) between the learning outcomes and the assessment criteria.

It is here that serious issues arise and often because of the dutifulness of teachers as they try to reconcile the language of learning outcomes with that of assessment criteria. Since the publication of the QAA Art and Design Overview Report in 2000, the difficulties identified in Section 36 continue to set challenges for teachers and administrators alike. Even more so given the observation and comments from the most recent QAA Institutional Audit Overview Report:

Most institutions have moved to a common assessment framework for the great majority of their degree programmes, and institutional as opposed to disciplinary or subject determination of assessment schemes is much more evident than hitherto. QAA (2011);

A finding that occurred more than once was that some programme specifications contained only generic learning outcomes and assessment criteria, so that subject-specific information was lacking and it was frequently not possible to determine from the programme specification any subject-specific assessment strategy. QAA (2011).

There is obvious uncertainty in the sector as to how generic and subject outcomes operate within the scheme of things. If they are overly generic they become less meaningful and it is unclear how the outcomes and assessment criteria relate to each other. If they are overly specific it becomes difficult for judgments about performance to be meaningful. It might be worth reflecting that the most accurate map of a region would have a one-to-one relationship with the terrain. That is, the map would be as big as the region and, hence, useless.

So, I suggest, in art and design whilst it is important that students know what they have to do on any course of study, it is not necessarily through published learning outcomes. Learning outcomes might be seen as necessary for administrative purposes but they are not sufficient in helping students develop an idea of what they will be learning and how they will go about it. Indeed, in a highly supportive context, learning outcomes might be so generalised as to only define the landscape and the boundaries of their intended learning. The knowing of what to do becomes developmental and personalised.

Observations

The requirement that all learning outcomes should use terms, particularly verbs, which are 'measurable' creates more challenges that it resolves. To insist on using terms such as 'identify', 'explain, 'analyse' and so on does not make the task of assessment any easier since explanations and analyses, etc, are discipline specific and are likely to be equally ambiguous for students who have not been yet been inducted into the language of the discipline.

The expectation that outcomes can be specified accurately and measurably at each level of the student's experience is fanciful. In art and design, teachers attempt to reconcile discipline benchmark statements, programme or course outcomes, stage-level outcomes, module outcomes and project outcomes, all within a coherent hierarchy using a limited set of verbs. Something has to give.

In art and design, outcomes are not achieved once and that is it. They are regularly returned to within and beyond levels. Therefore, assuming that outcomes, once addressed, are completed does not reflect the 'spiral' nature of the pedagogy of the discipline.

Designing and applying assessment criteria within an already overcrowded landscape of levels and outcomes can only add to student uncertainty.

In art and design, students complete 'projects' that are often driven by briefs that reflect practice within the professional world. Sketchbooks, journals and other supporting material are often submitted to demonstrate evidence of their achievement. In most cases, this can be seen as 'holistic' assessment in which the full collection of submitted material contributes to the final judgment. Holistic assessment practices do not easily lend themselves to 'analytical' assessment in which individual elements are marked and then the scores added up.

Given these observations, how do we move forward? How do we overcome the emergent technocratic ideology of learning and assessment but at the same time aspire to the broad interests of quality regimes in which students understand what is required of them, that the standards within the discipline are transparent to external agents and judgments about student achievement are seen to be valid and reliable? And, most importantly, how do we prevent the bureaucracy of outcomes and assessment overwhelming both teachers and students?

Unless a teacher has been party to the design and development of a programme, he or she will not necessarily understand how what they are expected to do fits into the whole. Programme design in such a complex landscape is often a negotiation of the language that embraces it. Only the course designers have a real understanding of how things fit together. New or part-time teachers, for instance, have to take the module outlines at face value and make sense of them in terms of their own professional experience. The greater the number of outcomes and the more elaborate the assessment scheme the more likely the whole thing will be sidestepped. Common sense often prevails in these circumstances. Therefore, there is a virtue in keeping the outcomes to a minimum even if this means a loss of specificity and apparent ambiguity. Rather than measurability, the focus should be on meaningfulness. It is better to provide a structure for discussions with the students to enable them to begin to engage in the discourses of the community in which they are joining than to assume they understand how they will perform against 'measurable' outcomes. For this purpose, students only need to understand whether they have addressed / met the outcome or not. It should be the stage level descriptors and the assessment criteria which provide the evidence of achievement.

Writing assessment criteria that is meaningful in relation to the outcomes and student performance is equally challenging. Constructively aligning outcomes and criteria makes sense in the abstract but it is the actual practice of relating them that is challenging. Biggs (2003) acknowledges the difficulty that arises in a predominantly 'divergent' set of disciplines such as the creative arts. It is not unusual to see assessment criteria expressed in the same language as the outcomes thus making it impossible to differentiate them. The nature of the assessment criteria needs to differentiate different levels of achievement without resort to the terms already used by the outcomes. Bloom's taxonomy is not helpful in this sense. Indeed, Bloom's taxonomy (1956) replicates most of the terms used in essay and report writing. The tension is only revealed when Bloom's taxonomy is used in a creative arts context where cognitive abilities other than knowledge and understanding are developed.

Biggs's SOLO taxonomy (2003) is a helpful guide in that it is derived from research and is deliberately hierarchical with each level embracing the previous one. So a student who takes a 'relational' approach to a task will also demonstrate capabilities in utilising 'multi-structural' and 'uni-structural' approaches. The taxonomy identifies students' concepts of, and approaches to, learning in an increasingly sophisticated way.

If we use the idea of a 'nested' hierarchy as the basis of the underlying quality structure for assessment in the way that Biggs does (and the way that Bloom's taxonomy is seen to do), but use concepts derived from the discipline rather than appropriate the terms uncritically from Bloom or elsewhere, then we might go some way to resolving some of the tensions revealed in recent QAA Audit reports.

Below is a worked example from Graphic Design, with explanatory commentary, to illustrate a possible approach to aligning learning outcomes with assessment criteria and avoiding the recurring pitfalls:

Example
Programme outcomes
The following programme outcomes are based on the threshold standards as expressed in the QAA Subject Benchmark Statement for Art and Design (2008). It is important to make reference to the Statement, in particular, to the threshold standard descriptors which set out what is required of a student to be eligible for an award in art and design. At the point of graduation (i.e. at Stage 3), students must demonstrate that they have the requisite skills and abilities, as outlined, in order to gain at least a pass degree. Apart from further descriptors alluding to a 'typical' pass, there are no indicators of further 'standards' relating to the classification system. It is expected that these will be articulated through the institution's / discipline's own assessment criteria.

At the end of the programme you will be able to:

A. Apply Graphic Design principles, research methodologies and analytical judgment in the generation of your research material, ideas, concepts and proposals

B. Select and utilise appropriate materials, techniques, methods and media in the realisation of your work

C. Identify and critically evaluate your practice within the contemporary cultural, historical, social and professional contexts through the use of a range of presentation skills

D. Work both independently and collaboratively (QAA, 2008).
Stage learning outcomes for Graphic Design
The following descriptors are deliberately succinct. They represent a 'nested' hierarchy of expectations of all students at each level of study so that it is taken that Stage 3 embraces Stages 1 and 2. They also represent the context in which the level standards are set. The key questions here are, 'What is it, in the broadest of terms, we want all students to be able to do at the end of the programme (Stage 3) and how do we get them there (Stages 1 and 2)?

Stage 1
Identify and utilise the skills and abilities of a graphic designer;
Articulate the focus of your work in relation to other specialisms;
Locate your work within a critical/theoretical and historical context.

Stage 2
Articulate your individual identity as a graphic designer;
Work independently and collaboratively;
Locate yourself and your work within broader issues and aspects of contemporary social practice.

Stage 3
Work effectively as a graphic designer.
Learning outcomes
The learning outcomes at module/unit level should seek to enable students to fulfill the Stage outcomes, again articulated as broadly as possible. Inevitably, unit briefs will combine creative expectations with more specific, often skill-based outcomes. The focus should be on meaningfulness rather than on measureability bearing in mind the support structures put in place to ensure that 'we haven't taken for granted what we shouldn't have taken for granted'.
Assessment criteria and grading/marking scale
The following scale is indicative and will vary depending on the nature and nuances of the discipline. Again, the categories are nested with each one incorporating the one below it. The hierarchy is based on students' abilities to integrate their thinking and progressively apply their thinking and abilities to more elaborate contexts. The scale can be applied at all three stages as the context and level of challenge is determined by the stage descriptor. Stage 3, for instance, is more challenging than Stages 1 and 2. As students progress they not only become more proficient in making and observing, say, they also get better at integrating these abilities and in more complex contexts. This is the essence of the 'spiral' curriculum.

Figure 2

The 'key words', which differentiate each level of achievement, in this instance, are based on the outcomes of an AHRB funded project (Davies and Reid, 2000 and Davies, 2006) which sought to identify discipline differences in art and design. What is important is that they are derived from the student experience of the subject. Other terms might be preferable but they should, ideally, be generated from observations of the structure of the learning outcomes of the discipline in context.

The act of assessment should be twofold. Firstly, determining whether the student has engaged with all the required learning outcomes (this is normally regarded as a pass). Then, using the assessment criteria, determining at what process level the student has been operating, given the full body of submitted work. This is not a question of adding up marks, moving from the specific to the general. It is more a question of determining an overall grade and justifying it with the available evidence against the criteria. Fine-tuning of the mark is negotiated afterwards.

As we have seen, the task of 'mapping' outcomes, curriculum and assessment can lead to a plethora of descriptors. In the search for accuracy and 'clarity', I would argue, the more we seek to map, paradoxically, the more the landscape becomes ambiguous if not for teachers then certainly for students. The use of matrices to generate assessment descriptors may appear methodical and structured but, in my experience, they produce far too many possibilities and lead to the analytical approach to assessment that Biggs, in particular, warns us off. They also attract the use of superlatives to differentiate the levels of achievement. At this point all is lost for the whole scheme becomes tautological.

Contact information
allandavies1@me.com
Biography

Allan Davies is a consultant in Higher Education Art and Design. Prior to this he was a Senior Adviser at the Higher Education Academy and before that he was Director of the Centre for Learning in Art and Design (CLTAD) at the University of the Arts. In the early days he was Head of Art and Design at what is now the University of Worcester and even further back he was programme leader for graphic information design at what was then Falmouth College of Art. During some of this he was a GLAD committee member. He is currently a QAA reviewer and was a member of the original QAA Subject Benchmark Statement for Art and Design panel.

References

Biggs, JB (2003) Teaching for Quality Learning at University, SRHE & OU Press.

Bloom, B.S. et al. (1956) Taxonomy of educational objectives: The classification of educational goals: Handbook I, Cognitive Domain, New York, David McKay.

Davies, A. and Reid, A. (2000) 'Uncovering Problematics in Design Education: Learning and the Design Entity', in Swann, C. & Young, E. (eds) Re-Inventing Design Education in the University: Proceedings of the International Conference, School of Design, Curtin University, Perth, pp.178- 184.

Davies, A (2006) '"Thinking like..." the enduring characteristics of the disciplines in art and design', in Davies, A (ed), Enhancing Curricula, 3rd CLTAD Conference Proceedings, Lisbon.

Hussey, T. and Smith, P. (2002) 'The trouble with learning outcomes', Active Learning in Higher Education, 3(3), pp. 220-233.

Price, M. O'Donovan, B. Rust, C. and Carroll, J. (2008) 'Assessment standards: a manifesto for change', Brookes eJournal of Learning and Teaching, Vol.2, No. 3.

QAA (2000) 'Art and Design Subject Overview Report 1998/2000'.

QAA (2008) 'Subject Benchmark Statement: Art and Design'.

QAA (2011) 'Outcomes from Institutional Audit (2007-09): Assessment and Feedback', 3 rd Series.

Rowntree, D. (1987), Assessing students: how shall we know them? London, Kogan Page (revised edition)

Figures 1 and 2 supplied by Allan Davies
Listing and header images: sections from photo by Paul Clark

# data[].attributes.attachment_link.title
Learning outcomes and assessment criteria in art and design. What's the recurring problem?

# data[].attributes.attachment_link.description
This paper is a critical reflection on the development of learning outcomes in art and design. It builds on a Learning and Teaching Support Network (LTSN) - funded project from 2003 that sought to provide support for colleagues seeking to make sense of the then new quality agenda.

# data[].attributes.body
https://youtu.be/-hwu3xqbMKw
Two spaces of flipped learning "group" individual. Looking at synchronous and asynchronous learning, that all students learn in different environments and levels.

# data[].attributes.headline
Grace - Notes on - How Emotion Matters in Four Key Relationships in Teaching and Learning in Higher Education

# data[].attributes.subject
Grace - Notes on - How Emotion Matters in Four Key Relationships in Teaching and Learning in Higher Education

# data[].attributes.body
" These studies reinforce that teaching is, first and foremost, a human interaction.  As humans, we are thinking and feeling creatures.  We bring ourselves, including our feelings, into the classroom.  We can express disappointment when students miss class.  We can let them see that we swoon over a particularly beautiful piece of music or get excited by the elegance of a proof.  We can show interest in students’ views. We can invite them to join us in our enthusiasm.  Part of creating the conditions for passionate thought in a classroom is allowing ourselves to share our own passion for the subject.  " P8

Does passion hold us back from teaching anything other than out lived experiences ?

# data[].attributes.body
Really interesting break down on lecturing to large and small groups and navigating the space. Along with a framework for currciculum design -
There are a number of key steps to effective course and curriculum design. One model is
as follows:
• Consider your general aims for the course/programme.
• Write specific learning outcomes (objectives): what do you want the students to learn?
• Plan the assessment framework to match your objectives.
• Plan the content, i.e. sequence of topics/readings.
• Plan the teaching/learning design – what kinds of activities will you and your
students engage in together?
• Compile a list of resources.
• Write the course outline including readings.
• Consider evaluation of the course (formative and summative) and how best
evaluation can be carried out.

# data[].attributes.body
The pedagogy of discomfort was developed in
the late 1990s by Megan Boler1
and has been
applied in various disciplines, as the following
examples show:
• to increase understanding of First
People’s (indigenous people’s) health by
non-indigenous students in Australia, in
discussing racism in theology education - see theology in education within (Brading sweetgrass post)
• in the professional development of
schoolteachers
• in ‘performing arts’ education
• in contemporary architectural education
• in teaching and understanding political
conflicts
• in immigration studies
• in decolonisation of education
• in training on gender-related violence for
youth leaders.
How can this be translated ? -
Spectating versus witnessing (how do we provide a platform for students critical voices).
Understanding and exploring anger:
here teachers are asked to unpack and
understand anger and other uncomfortable feeling surrounding the topic.
Avoiding the binary trap of innocence
and guilt: when exploring a challenging
topic such as racism, for example,
educators need to avoid the binary trap
of ‘innocent versus guilty’, as this can be
unproductive and make white students
feel blamed, lose interest or even get
defensive.
How do we overall decolonise education?

# data[].attributes.headline
Grace - Notes on "teaching creativity". (Sir Ken Robinson).

# data[].attributes.subject
Grace - Notes on "teaching creativity". (Sir Ken Robinson).

# data[].attributes.body
Argue that "value" in creativity in ideas is entirely subjective, current education still does not leave room for value to be fully subjective, it is decided by societal values. Agree that we should be teaching original thinking. It's not the requirement of creativity, but all ideas are naturally unique to the student or person. The way each brain works is entirely different, and even more so than we assume to be "normal" in neurodivergent people. The idea of creative must be expanded , as Robinson said "you can be creative mathematician, creative chef.." If creativity cannot be taught by direct instruction, how do we set up the instruction to "enable" and "inspire"?

# data[].attributes.headline
Grace - Notes from H. Fry, S. Ketteridge and S. Marshall, ed., "A Handbook for Teaching and Learning in Higher Education: Enhancing Academic Practice"- 

# data[].attributes.subject
Grace - Notes from H. Fry, S. Ketteridge and S. Marshall, ed., "A Handbook for Teaching and Learning in Higher Education: Enhancing Academic Practice"- 

# data[].attributes.body
"Students (and indeed staff) are often uncomfortable with the role of writing and theory within the subject; it is often seen as separate and unrelated." - Why? How can theory expand our teaching and learning and does using theory from other disciplines (if art is only "valid" if for the self and design only for the people, can out theories ever be interdisciplinary here?)

"Find out about any interesting institutional staff development activities for which you are eligible: attending will help you get a sense of the wider institution and meet people from across the university or college. • Ask if you can have a more experienced members" - What do you do here if your course in your institution is not valued? Do you act to show it's value?

# data[].attributes.body
This article is about inter-faculty learning project. I strongly agree that inter-faculty learning allow students to interact people from different backgrounds. Although RAVE is still not there yet, collaboration between different courses in PLP modules mean students have more chances to exchange knowledge and communicate with others from different backgrounds, solving real-world problems. In the future, workplaces are multi-faceted, and students are going to take on roles across displicine. Inter-faculty project learning is a good start to enhance the employability.

# data[].attributes.attachment_link.readable_text
The return to campus has been well documented as important in the post-disaster recovery period. And certainly, after several years of disruption to education, it has been exciting and cathartic to see university campuses full of students and the vibrancy of connections, especially when we know that young people have been prevented from partaking in various rites of passage and could be struggling to develop relationships and learn vital interpersonal skills.

Nonetheless, Hepi and the Office for Students have noted that we still have to give students a reason to attend in person and cannot pivot back to pre-pandemic learning. We must instead take a holistic approach to the student experience, use of space on campus and learning. Matt Riddle of Hepi states: "Universities need a plan that puts emphasis on the value of the learning experience. With many lecture halls standing empty, universities need to think about how they earn the commute. This means not only giving learners a sense of belonging and community but also designing learner experiences around the things that make an on-campus experience distinctive."

As a criminologist and an engineer, we use the campus and its physical spaces in very different ways. In online meetings during lockdowns, we recognised the value of, and longed for, physical place and in-person teaching. We decided to launch an innovative inter-faculty way of working to provide students with an excellent student experience. We had each wanted to work in a cross-faculty way and randomly finding one another led to this being born. After all, there is so much that different disciplines can offer each other; sometimes you just need to give it a go.

We each run a capstone module for final-year students, one being an alternative dissertation in the social sciences and the other an engineering dissertation module. As part of our new, inter-faculty way of working, both the engineering and social science students are given three-minute pitches on projects related to the development and implementation of innovative engineering products or systems. Some examples are accessible cycles, solar power refrigeration, a circular economy for campus plastic waste, or urban wind energy. Students then join teams, with the engineers learning industry-standard system development skills and the social science students there to provide consultation, scoping out of the related literature, research questions, ethical considerations and identifying concerns relating to gender, race, class, disability, age, etc.

One of the projects of particular interest to our sustainability agenda is our campus circular economy, where we recycle plastic waste into new products for sale in our university shops. Not only is this giving students a real-world problem and allowing them to devise a strategy to solve it, but students working inter-faculty will imagine completely new ways to frame the issue by utilising their different skill sets and building something new.

The modules we have designed for the project are committed to the Liverpool Curriculum Framework, spanning all of its seven strands: inclusivity; research-led learning; active learning; authentic assessments; confidence; digital fluency; and global citizenship. They also provide students with the ability to make friends and colleagues from across the university, give them a physical space to do this and encourage them to develop key personal and professional skills. The students learn through a mixture of lectures, group supervision, peer-led meetings, careers-led sessions and off-site opportunities. They also have the chance to collaborate with industry partners globally.

Over the past three years students have had limited opportunities to meet new people and develop their confidence. We wanted to provide them with incentives to develop their people skills and be exposed to new ways of thinking from peers they were otherwise unlikely to meet. We must also understand the student experience in the context of the cost-of-living crisis, with many people unable to afford socialising.

Being able to confidently communicate with those outside your field is a key employability skill, and using enquiry-based learning in this way involves developing and nurturing relationships and negotiating aspects of project work. Students are expected to manage their own projects, show leadership and creativity, influence others through collaboration and take risks. They develop resilience, independence and the ability to think outside the box. They are also asked to show what went wrong, in addition to their successes, which helps normalise failure, with all the positives that can entail.

Our modules are owned by our respective faculties with our own learning outcomes and assessments, and yet the projects collide. Students from each faculty provide consultation, support and feedback to their colleagues and peers and log this continuously to an industry standard. The project helps provide students with academic knowledge, personal development and the competencies and skills that are needed in professional life, which, in turn, can help address inequalities in graduate employment.

To make such a project happen, you will need to start with a tangible issue that needs solving; students need to see a real-world problem. Matt is chair of our Education in Sustainable Development group, and Gemma is a faculty rep for the group, thus all our options in this project centre around sustainability. Identify interested stakeholders within your department, school, faculty and institution and have conversations with your education leads. Think about practicalities such as timetabling needs and other potential obstacles to inter-faculty working. Communication is key, both among academic colleagues and students. This way of working can be challenging, and it is important that students are supported throughout by regular timetabled group supervisions.

We have had interest in this way of working from other disciplines, faculties and colleagues nationally and are happy to be contacted by any interested parties.
Gemma Ahearne is a lecturer in criminology and Matt Murphy is a senior lecturer in engineering, both at the University of Liverpool, UK. If you found this interesting and want advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the THE Campus newsletter.

# data[].attributes.attachment_link.title
Building the future: the case for inter-faculty learning

# data[].attributes.attachment_link.description
The return to campus has been well documented as important in the post-disaster recovery period. And certainly, after several years of disruption to education, it has been exciting and cathartic to see university campuses full of students and the vibrancy of connections, especially when we know that young people have been prevented from partaking in various rites of passage and could be struggling to develop relationships and learn vital interpersonal skills.

# data[].attributes.headline
Grace - Subject specific research examining how research within academia only portray certain voices in research. 

# data[].attributes.subject
Grace - Subject specific research examining how research within academia only portray certain voices in research. 

# data[].attributes.headline
QAA report From Pivot to Permanent: Examining Lessons Learned from the Shift to Online and Hybrid Teaching and Learning

# data[].attributes.subject
QAA report From Pivot to Permanent: Examining Lessons Learned from the Shift to Online and Hybrid Teaching and Learning

# data[].attributes.body
- Hybrid allows the marginalized voices of those who are not present to participate

-Hybrid can cause onsite participants to lose sight of those who are virtual. Specifically, they see each other with equal space, make eye contact while speaking to each other, but they see all of the virtual people together as “one” person on a screen

- To address the problem, we need to work on power dynamic in the following phases (1) pre-conference planning, (2) during-conference planning, (3) facilitating, and (4) sustaining community.

- Intentionally equitable hospitality implores us to extend personal invitations to marginalized persons.

- become allies who care about redressing social justice issues of others in the community

-have equitable conversations with high-profile conference participants, become heard, gain confidence, and develop reciprocal relationships with them over time

# data[].attributes.headline
Sir Ken Robinson: Can Creativity Be Taught

# data[].attributes.subject
Sir Ken Robinson: Can Creativity Be Taught

# data[].attributes.body
·      Creative arts is about process, not an event

·      About original thinking

·      Value, evaluation and judgement

Teaching - Process of instruction and mentoring process of enabling and motivating

Assessment – original idea that has value should be assessed. Therefore, we need to identify criteria for originality

# data[].attributes.headline
Learning outcomes and assessment criteria in art and design. What’s the recurring problem?

# data[].attributes.subject
Learning outcomes and assessment criteria in art and design. What’s the recurring problem?

# data[].attributes.body
- There has to be a relationship between what a student is expected to do (the intended learning outcomes) on a programme, the programme content and delivery that supports the student’s achievement and the process and judgments by which the quality of the learning is determined (assessment criteria)

- a new structure to address the sprial and holistic nature of the creative arts curriculum

- determining whether the student has engaged with all the required learning outcomes (this is normally regarded as a pass). Then, using the assessment criteria, determining at what process level the student has been operating, given the full body of submitted work

# data[].attributes.headline
Grace - Subject specific critical and ethical mapping tool for students. 

# data[].attributes.subject
Grace - Subject specific critical and ethical mapping tool for students. 

# data[].attributes.body
A template from "Laura Knight" head of social and environmental justice at UAL. A tool break down for self examination of the ethical implications of your projects. For yourself and students.

# data[].attributes.headline
Grace - Subject specific visual research 

# data[].attributes.subject
Grace - Subject specific visual research 

# data[].attributes.body
A great tool for examining the ways in which we communicate as academics.

# data[].attributes.attachment_link.title
EAP Tools for Communication booklet S Lund (1) (2)