# data[].attributes.body
social annotation tools
use more specific questions for students' informal midterm course evaluation

# data[].attributes.headline
I have used Perusall for pre-reading in graduate school at Michigan. 

# data[].attributes.subject
I have used Perusall for pre-reading in graduate school at Michigan. 

# data[].attributes.headline
My online teaching is very asynchronous, I give students at least a week to complete a set of activities that includes both required and optional tasks. I try to give students the micro (this is what you need to do this week) and also the macro (here's where we are in the flow of the course + what you are thinking ahead to.)

# data[].attributes.subject
My online teaching is very asynchronous, I give students at least a week to complete a set of activities that includes both required and optional tasks. I try to give students the micro (this is what you need to do this week) and also the macro (here's where we are in the flow of the course + what you are thinking ahead to.)

# data[].attributes.body
In the last year I've been more proactive about making the option of extension of deadlines available to students given the stresses they are combating.

# data[].attributes.headline
The concept of Specs Grading is really appealing and is something I'd like to explore further.

# data[].attributes.subject
The concept of Specs Grading is really appealing and is something I'd like to explore further.

# data[].attributes.headline
The idea of options for discussion questions is appealing in medium/large classes but feels potentially fragmenting in smaller courses? 

# data[].attributes.subject
The idea of options for discussion questions is appealing in medium/large classes but feels potentially fragmenting in smaller courses? 

# data[].attributes.headline
I have not done this with students but partial notes/outline is a strategy I've used in workshops for faculty

# data[].attributes.subject
I have not done this with students but partial notes/outline is a strategy I've used in workshops for faculty

# data[].attributes.headline
I try to do as much as I can to offer students choices around major projects and activities so that they are doing activities relevant to their personal and professional goals.

# data[].attributes.subject
I try to do as much as I can to offer students choices around major projects and activities so that they are doing activities relevant to their personal and professional goals.

# data[].attributes.headline
4 Reasons Why Every Course Should Be Designed as an Online Course

# data[].attributes.subject
4 Reasons Why Every Course Should Be Designed as an Online Course

# data[].attributes.attachment_link.readable_text
Now is the time to plan for the post-pandemic university. Too early, you say? Still in the midst of managing the low-density university amid surging COVID-19 cases?

I never said it would be easy to plan for the post-COVID university. I am saying that the time to do so is before the pandemic ends so that whenever that shift occurs, your school will be able to hit the ground running. Failure to plan now for life after the pandemic is the fastest path toward regression to pre-pandemic norms.

What should be on the list of every post-COVID-19 university?

One idea that I'd like to nominate for consideration would be a commitment to design every course as an online course. That's right, every single course. Even if there are no current plans to expand online offerings in, say, a core undergraduate residential program, my recommendation is to still design those face-to-face courses for online delivery. Each and every single one. Here's why.
No. 1: Resiliency
Obviously, a course designed for online delivery will more easily be able to pivot to remote, should we ever need to make that pivot again. We learned from COVID that smart institutions make significant investments in preparing for low-probability/high-impact events. The next black swan that causes all courses to toggle from residential to remote may not be a pandemic, but it might be a wildfire, a hurricane or an earthquake. The ability to seamlessly move between residential and online delivery modes is the best guarantee for resiliency in an uncertain world.
No. 2: Flexibility
Current plans may not envision shifting core residential programs online, but plans can change. Professors and students now have some experience with remote learning. It is less of a stretch to think about moving part or all of a degree online. We may see rapid growth in summer online offerings, complemented by online courses between the fall and spring terms. Students vote with their feet, looking for schools that offer the flexibility of both residential and online offerings. Colleges and universities can prepare for this shift by designing all courses to work both face-to-face and online.
No. 3: Quality
Every instructional designer knows that there is fundamentally no difference in the design of an online and a face-to-face course. There are not residential and online courses. Instead, there are well-designed courses and poorly designed courses. A well-designed course is built around learners and learning, rather than content. When professors work with an instructional designer, they end up with a course that is backward designed around learning objectives, not content. Good courses feature opportunities for learners to actively construct knowledge rather than passively absorb materials. Good courses utilize assessments for learning as well as evaluation. The best courses are immersive, built for faculty presence and student engagement. A course designed around the core principles of learning science and instructional design will pivot seamlessly between face-to-face and online modalities.
No. 4: Extensibility
The future of higher education belongs not to the credit hour but the competency. The fastest growth in postsecondary educational offerings is occurring not at the degree but the alternative credential. Certificates and other certifications of competency, almost always created for an online offering, are rapidly gaining acceptance by employers. This does not mean that schools should abandon their credit-bearing courses that lead to degrees. Instead, schools should offer both noncredit/nondegree online programs and credit-bearing courses and degrees. Alternative credentials are not a competitor to degrees. They are a funnel. Designing all courses for online delivery will make it easier to unpack and repack materials into alternative credentials. The name of the game is blending, remixing and reuse. Even if the original use is face-to-face, courses born online lend themselves to transformation for alternative online offerings.

# data[].attributes.attachment_link.title
4 Reasons Why Every Course Should Be Designed as an Online Course | Learning Innovation

# data[].attributes.attachment_link.description
Now is the time to plan for the post-pandemic university. Too early, you say? Still in the midst of managing the low-density university amid surging COVID-19 cases? I never said it would be easy to plan for the post-COVID university.

# data[].attributes.attachment_link.title
Teaching In Higher Ed Podcast - Teaching in Higher Ed

# data[].attributes.attachment_link.description
This is the space where we explore the art and science of being more effective at facilitating learning. We also share ways to increase our personal productivity, so we can have more peace in our...

# data[].attributes.body
Resilient Teaching Through Times of Crisis and Change MOOC

# data[].attributes.attachment_link.readable_text
Resilient teaching is the ability to facilitate learning experiences that are designed to be adaptable to fluctuating conditions and disruptions. This teaching ability can be seen as an outcome of a design approach that attends to the relationship between learning goals and activities, and the environments they are situated in. Resilient teaching approaches take into account how a dynamic learning context may require new forms of interactions between teachers, students, content, and tools. Additionally, they necessitate the capacity to rethink the design of learning experiences based on a nuanced understanding of context.

The course is designed with higher education faculty, lecturers, and graduate student instructors in mind, but may also be applicable to educators in a wide variety of instructional environments. The course is aimed at participants who may be asked to rethink how they teach in the immediate or near future due to the ever-changing circumstances of the current COVID-19 pandemic. While the creation of this course is motivated by the current crisis, we expect it will remain relevant to instructors who are faced with disruptions and change to their teaching for any number of reasons and must quickly adapt their course designs. To support learning and reflection, we are introducing a new journaling tool within the course that provides a place for participants to reflect on their own experiences. To support community growth, participants can choose to share their journal entries with the wider course community in an interactive gallery space.

# data[].attributes.attachment_link.title
Resilient Teaching Through Times of Crisis and Change | Coursera

# data[].attributes.attachment_link.description
Resilient teaching is the ability to facilitate learning experiences that are designed to be adaptable to fluctuating conditions and disruptions. This teaching ability can be seen as an outcome of a design approach that attends to the relationship between learning goals and activities, and the environments they are situated in.

# data[].attributes.headline
Universal Design for Learning in Higher Education Book

# data[].attributes.subject
Universal Design for Learning in Higher Education Book

# data[].attributes.attachment_link.title
Project MUSE - Reach Everyone, Teach Everyone

# data[].attributes.attachment_link.description
Reach Everyone, Teach Everyone is aimed at faculty members, faculty-service staff, disability support providers, student-service staff, campus leaders, and graduate students who want to strengthen the engagement, interaction, and performance of all college students. It includes resources for readers who want to become Universal Design for Learning experts and advocates: real-world case studies, active-learning techniques, UDL coaching skills, microand macro-level UDL-adoption guidance, and use-themnow resources.

# data[].attributes.headline
Another Universal Design for Learning Book

# data[].attributes.subject
Another Universal Design for Learning Book

# data[].attributes.attachment_link.description
How to Design and Deliver Online Education Using Universal Design for Learning by Katie Novak and Tom Thibodeau $29.99, paperback (ISBN: 9780989867481) $29.99, ePUB (ISBN: 9780989867498) Overview UDL in the Cloud helps educators design and deliver more accessible, engaging, and effective online learning experiences.

# data[].attributes.headline
Talk about Teaching! Leading Professional Conversations book

# data[].attributes.subject
Talk about Teaching! Leading Professional Conversations book

# data[].attributes.attachment_link.readable_text
Available Formats

ISBN: 9781483373799

Paperback

Suggested Retail Price: $35.95

Bookstore Price: $28.76

ISBN: 9781483373805

Electronic Version

Suggested Retail Price: $31.95

Bookstore Price: $25.56

Should you need additional information or have questions regarding the HEOA information provided for this title, including what is new to this edition, please email sageheoa@sagepub.com. Please include your name, contact information, and the name of the title for which you would like more information. For information on the HEOA, please go to http://ed.gov/policy/highered/leg/hea08/index.html.

1. Why Professional Conversation?

2. Power and Leadership in Schools

3. The Big Ideas that Shape Professional Conversations

4. The Topics for Conversations

6. Informal Professional Conversations

8. Conversation Activities for Implementation

Identifies topics for professional conversations that are linked to the "big ideas" or essential components of successful teaching

Links the "big ideas" to both critical research and to the author's frameworks

Identifies and describes research-based conversation skills, including setting the tone, probing, paraphrasing, and inviting thinking

Demonstrates how to create an environment that supports informal and effective professional conversations, e.g., making time for the conversations, establishing the necessary trust, reaching consensus on the essential components of successful teaching

Provides an entire chapter of very specific conversation activities or protocols organized around the "big ideas" that comprise suggested questions, materials to support the conversations, and likely outcomes of discussions

# data[].attributes.attachment_link.description
Available Formats ISBN: 9781483373799 Paperback Suggested Retail Price: $35.95 Bookstore Price: $28.76 ISBN: 9781483373805 Electronic Version Suggested Retail Price: $31.95 Bookstore Price: $25.56 Should you need additional information or have questions regarding the HEOA information provided for this title, including what is new to this edition, please email sageheoa@sagepub.com.

# data[].attributes.attachment_link.readable_text
Hypothes.is is a free social annotation tool which allows users to discuss, collaborate, organise your research, or take personal notes. Annotating can be public and open to anyone, confined to small groups or used by individuals.

What does the Hypothes.is software do?
The elevator pitch is easy.

"Visit a web page, then select some text and annotate with comments or tags. You'll see those annotations when you return to the page, and so will other Hypothesis users."

The software leverages annotation to enable sentence-level critique or note-taking on top of news, blogs, scientific articles, books, terms of service, ballot initiatives, legislation and more. It has been summarised as "a peer review layer for the entire Internet.

Annotation types

Highlights
A highlight is the digital equivalent of swiping a yellow marker over a passage of text. It anchors to its selection in the document, and quotes the selection, but there are no comments or tags. A highlight is always private ("Only me"), which means only you can see it, and only when you're logged in.

Annotations
An annotation is the digital equivalent of a marginal note. After making a selection, you use the popup New Note control to create this type of annotation. Like a highlight, it is anchored to the selected text which it also quotes. Unlike a highlight it includes extra stuff that you add: text and/or tags. The annotation can be marked so it's visible to everyone or as "Only me" to keep it private

Page Notes
A page note is like a social bookmark: it associates comments and/or tags with a document without anchoring that information to a passage within the document. A page note isn't selection-based. You create one using the New Note control on the Hypothesis sidebar's vertical toolbar. As with an annotation you add text and/or tags. Again you have the choice to make the note public or private.

Replies
Unlike a highlight, an annotation, or a page note, a reply doesn't refer to an annotated document. Instead itrefers to one of those annotation types, or to a prior reply. You use the Reply link to create a reply. Like other annotation types the reply can include text and/or tags, and must include one or the other.

10 ways to annotate with students

1. Teacher Annotations
Pre-populate a text with questions for students to reply to in annotations or notes elucidating important points as they read.

2. Annotation as Gloss
Have students look up difficult words or unknown allusions in a text and share their research as annotations.

3. Annotation as Question
Have students highlight, tag, and annotate words or passages that are confusing to them in their readings.

4. Annotation as Close Reading
Have students identify formal textual elements and broader social and historical contexts at work in specific passages.

5. Annotation as Rhetorical Analysis
Have students mark and explain the use of rhetorical strategies in online articles or essays.

6. Annotation as Opinion
Have students share their personal opinions on a controversial topic as discussed by an article.

7. Annotation as Multimedia Writing
Have students annotate with images and video or integrate images and video into other types of annotations.

8. Annotation as Independent Study
Have students explore the Internet on their own with some limited direction (find an article from a respectable source on a topic important to you personally), exercising traditional literacy skills (define difficult words, identify persuasive strategies, etc.).

9. Annotation as Annotated Bibliography
Have students research a topic or theme and tag and annotate relevant texts across the Internet.

10. Annotation as Creative Act
Have students respond creatively to their reading with their own poetry or prose or visual art as annotations.

Further detail here on the Hypothes.is blog

Other useful resources

About Sue Beckingham

A National Teaching Fellow, Educational Developer and Principal Lecturer in Computing with a research interest in the use of social media in higher education.

This entry was posted in Collaboration tools, Curation tools and tagged Hypothes.is. Bookmark the permalink.

# data[].attributes.attachment_link.title
A social and collaborative approach to annotation: @hypothes_is

# data[].attributes.attachment_link.description
Hypothes.is is a free social annotation tool which allows users to discuss, collaborate, organise your research, or take personal notes. Annotating can be public and open to anyone, confined to small groups or used by individuals. What does the Hypothes.is software do? The elevator pitch is easy.

# data[].attributes.attachment_link.readable_text
Introduction to social annotation reading tools.

How to use social annotation in an online class.

The basics

We know students may struggle to engage with assigned readings. To help remedy this, social annotation tools offer collaborative opportunities for reading, highlighting, and discussing texts online.

What is social annotation?

Social annotation (SA) is a peer-to-peer activity that allows students to collaboratively read, highlight, and discuss texts online. With advanced planning (and a little creative thinking), you can create fun and engaging SA activities that allow students to more fully engage with your classroom readings. For example, you can see an image of annotation-themed bingo activity below. In this post, we will introduce you to Perusall, a specific type of SA tool, and also offer guidelines for using SA in an online class.

What are some social annotation tools?

There are a variety of SA software tools you can use in your online classroom. One popular option is Perusall - this tool integrates with learning management systems like Canvas, and it allows instructors to upload a variety of different file types for students to collaboratively annotate and discuss. Perusall also has a suite of analytic tools that gives instructors insights into posts and student engagement.

Why should I use social annotation?

We know students may struggle to engage with assigned readings. To help remedy this, social annotation tools offer collaborative opportunities for reading, highlighting, and discussing texts online. Social annotation is a great way to get students directly talking to each other within a text. Instead of using a discussion board and quoting portions of a text, students can comment directly on the text within the context of the text. You might want to use this tool if your class requires a heavy reading load or if your students are struggling to understand key concepts in texts.

How can I use SA in my online class?

We've listed five guidelines for you to consider when integrating SA into your online class.

Guideline 1: Make sure it's the right tool for your class.

Whenever you integrate a new tool into your online class, carefully consider your students' needs and the learning objectives of the class. While social annotation offers some unique affordances for online texts - such as collaborative highlighting and discussions - it is not a solution for every problem. While Persuall is a tool you may choose to use, you will want to choose this tool with careful consideration.

Guideline 2: Make sure to provide help

Don't assume all students are comfortable using new technology like SA. When you introduce a new tool to a class, provide some guidance for how to use the basic features of the tool. A quick youtube tutorial or manual can alleviate a lot of confusion. For some initial guidance, here is a help document for students about using Perusall.

Guideline 3: Set expectations

When you introduce SA to the online classroom, it's important to set expectations for how this tool will be used. If students don't know why they are using SA and how it's benefiting their learning experience, the quality of the annotations may be underwhelming, or the tool may be underutilized. Some expectations you may want to set with SA include:

Is annotating a requirement? Or is it optional?

How often would you like students to annotate?

Are you looking for a certain number of annotations per reading?

Should annotations be a certain length?

When setting expectations, you should also consider what exemplary annotations and discussions look like. Providing examples of high-quality annotations and explaining why they are exemplary may help students in writing their own annotations.

Guideline 4: Remember to keep the dialogue going

SA tools like Perusall allow students and instructors to dialogue about the text. Pay attention to where your students are commenting, and encourage the conversation by engaging without "telling" the answer. By reading and responding to student annotations, you can build rapport with students, and you can clarify any misunderstandings that might arise.

Guideline 5: Highlight passages to scaffold learning

Perhaps there is a particular passage you want students to respond to. Or perhaps there's a passage that's difficult to understand. When you upload a new reading to Perusall, you might want to highlight certain passages for students and provide additional resources to aid their understanding. Images, videos, and discussion prompts can all be helpful ways to compliment class readings. The image below shows how one instructor exemplified a course concept using an image.

Practical tips

Make sure you are ready to support a tool like Persuall. If there is a lot of technology fatigue from students, they may not be excited to learn a new one.

Use social annotation tools to build student community. Students often generate interesting and authentic ideas in online discussions. As you are reading student discussions, consider how you might use those ideas in other parts of your class. For instance, if one student exemplifies a course concept with a personal anecdote, you might want to reiterate that anecdote for the whole class in a lecture. Incorporating student ideas into lectures and class activities will build rapport, and it will help personalize the classroom content for students.

Perusall can track student number of posts. You might want to consider the same sort of expectations for a discussion board, for example, students have to initiate 1-2 comments on a document, and comment on at least 2 other student's posts for full credit.

Resources

University of Michigan

LSA - Close reading assignments with Perusall

LSA - Collaborative writing with Perusall

Other Resources

Perusall - Learn more & Support

Ashland University - A tutorial on using Persuall as a student (it's specific to Blackboard for the first 30 seconds)

# data[].attributes.attachment_link.description
Introduction to social annotation reading tools. How to use social annotation in an online class. We know students may struggle to engage with assigned readings. To help remedy this, social annotation tools offer collaborative opportunities for reading, highlighting, and discussing texts online.

# data[].attributes.headline
Use of SA platform in a Flipped Classroom

# data[].attributes.subject
Use of SA platform in a Flipped Classroom

# data[].attributes.attachment_link.readable_text
Harvard University, Cambridge, MA, United States

Perusall LLC, Brookline, MA, United States

In this paper, we illustrate the successful implementation of pre-class reading assignments through a social learning platform that allows students to discuss the reading online with their classmates. We show how the platform can be used to understand how students are reading before class. We find that, with this platform, students spend an above average amount of time reading (compared to that reported in the literature) and that most students complete their reading assignments before class. We identify specific reading behaviors that are predictive of in-class exam performance. We also demonstrate ways that the platform promotes active reading strategies and produces high-quality learning interactions between students outside class. Finally, we compare the exam performance of two cohorts of students, where the only difference between them is the use of the platform; we show that students do significantly better on exams when using the platform.

Introduction

Getting students to read the textbook before coming to class is an important problem in higher education. This is increasingly the case as more college classes are adopting "flipped" teaching strategies. A key principle of the flipped classroom model is that students benefit from having access to the instructor (and other peers) when working on activities that, in traditional classrooms, are typically done at home (like problem sets). Moving these activities in class improves student learning as it provides them the opportunity to actively engage with the instructor and each other ( Herreid and Schiller, 2013). In a flipped class, the information transfer (traditionally accomplished by the instructor delivering a lecture during class) is moved outside the classroom to a pre-class assignment that students are expected to complete before coming to class. These pre-class assignments typically require students watch a video of a lecture online, or complete a reading. Moving the information delivery out of the classroom allows in-class time to be used for more interactive activities during which students can be actively engaged with instructors and other students.

When students are exposed to the material before class they are better able to follow material in class ( Schwartz and Bransford, 1998), they ask more meaningful questions in class ( Marcell, 2008), and they perform better on exams ( Narloch et al., 2006; Dobson, 2008; Johnson and Kiviniemi, 2009). Students report that one of the most important factors in deciding whether to participate in class is reading the textbook beforehand ( Karp and Yoels, 1976). The connection between pre-class reading and in-class participation is particularly relevant in flipped courses that rely on active in-class participation.

As pre-class reading assignments replace lectures in flipped courses and serve as the primary mechanism for information transfer, it is essential that students complete their assignments before class. Even in traditional (non-flipped) college courses, pre-class reading has been shown to be important for student learning and yet 60-80% of students do not read the textbook before coming to class ( Cummings et al., 2002; Clump et al., 2004; Podolefsky and Finkelstein, 2006; Stelzer et al., 2009). Clump et al. (2004) studied the extent to which psychology students reported reading their textbooks and found that students only read, on average, 28% of the assigned reading before class and 70% before the exam.

Other studies have looked at how much time students spend reading textbooks, and when they read. Berry et al. (2010) studied pre-class reading habits of undergraduate students enrolled in finance courses across three different universities. They found that 18% of students report not reading the textbook at all, and approximately 92% of students report spending 3 h or less per week reading. Almost half the students (43%) report reading the textbook for less than an hour a week ( Berry et al., 2010). The authors also polled the students to find out when they read. Despite instructors' recommendation to read before class, very few students actually do so: just 18% of students report that they frequently read before class; 53% report never or rarely reading the textbook before class ( Berry et al., 2010). Podolefsky and Finkelstein (2006) found that only 37% of students report regularly reading the textbook and less than 13% read before class. Instead of reading before class, students reported reading predominately in preparation for exams, to find the answer to a specific question, or to help complete homework ( Berry et al., 2010).

There are several reasons why students are not reading before class. Some studies suggest that students do not see the connection between doing well on exams and pre-class reading ( Podolefsky and Finkelstein, 2006). To strengthen the connection between pre-class reading and course grades, many instructors have implemented graded, pre-class reading quizzes ( Burchfield and Sappington, 2000; Connor-Greene, 2000; Ruscio, 2001; Sappington et al., 2002). "Just in-time teaching" (JITT) is one specific implementation of this way of handling pre-class reading ( Novak et al., 1999). With JITT, before class, students are required to answer open-ended questions about the reading online, with one question dedicated to soliciting feedback from students about what aspect of the reading they found most confusing. Instructors can use this feedback to tailor their in-class activities and instruction to the most popular areas of student confusion. Even with grade incentives, however, the rate of pre-class reading compliance is still surprisingly low. Stelzer et al. (2009) reported that even with JITT, 70% of students never or rarely read the textbook before class. Heiner et al. (2014) recorded student pre-class reading compliance in two different classes with a JITT-like implementation of short, targeted readings and associated online reading quizzes ( Heiner et al., 2014). They found that 79% of students in one class and 85% of students in the other classes reported reading the pre-class reading assignment every week (or most weeks). While these results are promising, this study (as well as the other mentioned studies on pre-class reading) relied on student-reported responses; Sappington et al. (2002) found that students' self-reported reading compliance is often distorted and invalid.

It is unclear from the literature whether there is a relationship between pre-class reading behavior and in-class exam performance. Podolefsky and Finkelstein (2006) studied the relationship between the frequency with which students report reading and course grades. They conducted this study in three different types of courses: calculus-based physics, algebra-based physics, and conceptual physics. For the calculus and algebra-based courses, they found no significant correlation between course grade and how much students reported reading. For the conceptual course, they found a moderate correlation. Smith and Jacobs (2003) looked at the correlation between time spent reading and course grade for chemistry students and also found no correlation between time spent reading (based on student self-reported data) and course grades. Heiner et al. (2014) found a statistically significant positive correlation between students' exam performance and the frequency with which students completed the online reading quiz. Because much of the research is based on students' self-reported data and because of the lack of consensus about the relationship between pre-class reading and grades, we set out to systematically study this relationship.

The research questions we address are:

(1) What are students' pre-class reading habits on a social learning platform?

(2) Which pre-class reading behaviors are predictive of student in-class exam performance?

(3) What is the efficacy of the platform in promoting student learning?

Theoretical Framework

It is generally accepted that students understand material better after discussing it with others ( Bonwell and Eison, 1991; Sorcinelli, 1991). From the social constructivism perspective, students learn through the process of sharing experiences and building knowledge and understanding through discussion ( Vygotsky, 1978). Online learning communities are virtual places that combine learning and community together ( Downes, 1999) and provide environments for learners to collaboratively build knowledge. Collaborative learning settings provide students a space to verbalize their thinking, build understanding, and solve problems together ( Webb et al., 1995; Crouch and Mazur, 2001).

Online discussion forums have been used successfully as tools to facilitate social interactions and exchanges of knowledge between learners ( Rovai, 2002; Bradshaw and Hinton, 2004; Tallent-Runnels et al., 2006). The social constructive theory of learning with technology emphasizes that successful learning requires continuous conversation between learners as well as between instructors and learners ( Brown and Campione, 1996). As a result, when designing online learning strategies, educators should create social environments with a high degree of interactivity ( Maor and Volet, 2007). The asynchronous nature of online discussion forums allows for discussion between learners and between learners and instructors at any time of day or night, and this is a major advantage over other forms of learning environments ( Nandi et al., 2009).

Beyond online discussion forums, collaborative annotation systems have recently been developed and used in education as social learning communities. Online annotation systems are computer-mediated communication tools that allow groups of people to collaboratively read and annotate material online. Many studies have shown that online annotation systems increase student learning across many different educational settings ( Quade, 1996; Cadiz et al., 2000; Nokelainen et al., 2003; Hwang and Wang, 2004; Marshall and Brush, 2004; Ahren, 2005; Gupta et al., 2008; Robert, 2009; Su et al., 2010).

Perusall: Social Learning Platform for Reading and Annotating

Perusall is an online, social learning platform designed to promote high pre-class reading compliance, engagement, and conceptual understanding. The instructor creates an online course on Perusall, adopting electronic versions of textbooks from publishers or uploading articles or documents, and then creates reading assignments. Students asynchronously annotate the assigned reading by posting (or replying to) comments or questions in a chat-like fashion.

An instructor view of the course home page is shown in Figure 1. The instructor uploads the reading material to the left-hand side of the page (under Documents) and then creates specific reading assignments from these documents which appear in the right panel.

Figure 2 shows what a student sees after opening a reading assignment and highlighting a specific passage on a page in the assignment. A conversation window opens on the right where the student can ask a question or make a comment.

Figure 2. Page of a reading assignment in Perusall. (Note: the depicted individual provided written informed consent for the publication of their identifiable image, the image of the textbook book is from OpenStax, University Physics, Volume 1. "Download for free at https://openstax.org/details/books/university-physics-volume-1," no further permission is required from the copyright holders for the reproduction of this material.)

Figure 3 shows a page that has been highlighted and annotated by students. When a student clicks on a specific highlight that highlight turns purple, and the conversation window for that highlight opens on the right.

Figure 3. Reading assignment in Perusall showing student highlights and annotations (note: the depicted individual provided written informed consent for the publication of their identifiable image, the image of the textbook book is from OpenStax, University Physics, Volume 1. "Download for free at https://openstax.org/details/books/university-physics-volume-1," no further permission is required from the copyright holders for the reproduction of this material).

When a student asks a question about a specific passage, it is automatically flagged with an orange question mark, as shown in Figure 3. Other students can respond in an asynchronous conversation.

Perusall also has an integrated assessment tool that provides both students and instructors with constant feedback on how students are engaging with the reading assignments. Finally, Perusall has a built-in tool for instructors called the Confusion Report. This report automatically summarizes the top areas of student confusion for instructors so that they can prepare class material that is targeted specifically to the content that students are struggling with the most.

Social Features

In addition to the basic highlighting and annotating functions, Perusall has a number of additional features designed to turn the online reading assignment into a social experience to encourage students to engage with the material and with fellow classmates outside of class. Several features of the software are designed to promote the social aspect of the software.

Sectioning

If the class exceeds 20 students (or another threshold set by the instructor), the software automatically partitions students in the class into groups that function like "virtual class sections." Students can only interact with and see annotations posted by others in their group (as well as any annotations posted by the instructor). This allows students to become more familiar with the other students in their group, and this familiarity helps promote more online interaction. Our prior work demonstrated that when the size of the group is too large, the overall quality of students' annotations decreases ( Miller et al., 2016), so these smaller groups prevent students from becoming overwhelmed by an excessively large number annotations and helps keep the overall quality of the interactions high.

Avatars

The avatars of other students and instructors who are viewing the same assignment at the same time appear in the top left hand corner of the screen (Figure 2). Being able to see classmates (and instructors) reading the assignment at the same time increases the social connectivity of the reading experience and encourages students to engage more with the reading (through the software).

Upvoting

Students can provide feedback on the annotations made by other students in their section by "upvoting" annotations. There are two types of upvoting in Perusall. When students would like to know the answer to a question posed by another student, they can indicate this by clicking on the orange question mark. For example, Figure 4 shows that three students clicked on the orange question mark button for that question, indicating that they too would like to know the answer. When instructors review questions in Perusall, they can pay particular attention to the questions that have been upvoted by other students.

When a student provides a particularly helpful explanation, other students can indicate this by clicking on the green checkmark. In Figure 5, five students found the explanation to the initial question to be helpful to their understanding. When students upvote other students' explanations, it helps other students find explanations that are particularly helpful to their conceptual understanding of the reading. Both of these upvoting features are designed to increase and encourage the social component of the online reading assignments and foster a sense of community within the groups.

Email Notifications

Finally, Perusall has an email notification feature that is designed to encourage the social interaction aspect of the software even when students are not logged into Perusall by letting them know when a classmate has responded to a question or comment they have made (or have clicked the question mark button for). Through the notification, Perusall encourages students to continue their conversation about the reading. Figure 6 shows an example of an email notification that a student receives when a classmate responds to his/her question. The notification encourages students to re-engage with the reading assignment by viewing the conversation and/or letting the responder know whether the response was helpful to their understanding. Students can reply to the email inside their email client, and the reply is appended directly to the conversation in Perusall, as if the student had been online.

Assessment

Perusall has an integrated assessment tool, which automatically evaluates students' participation in the reading assignment and populates an integrated gradebook (Figure 7).

The grading algorithm uses four criteria to evaluate a students' collection of annotations for any given reading assignment-timeliness, quantity, quality, and distribution-and students receive an overall score based on all four of these criteria. The grading algorithm uses machine learning to drive desirable student behavior: timely, thorough, and complete reading of the text, with annotations that demonstrate thoughtful interpretation of the subject matter. Students receive a score based on how closely their overall reading and annotating behavior matches behavior that is predictive of success in the classroom.

Instructor Tools

Besides the gradebook and individual reading assignment feedback, which provides important assessment information to both students and instructors, Perusall also assists instructors in identifying from the body of annotations the top areas of confusion so they can prepare class material that is targeted specifically at addressing these areas. To this end Perusall automatically mines questions that students are asking about a particular reading assignment and, using a topic modeling algorithm, groups questions into three to four conceptual areas of confusion. Figure 8 shows an example of a confusion report generated for a specific reading assignment. The philosophy behind the confusion report is based on Just-in-Time-Teaching ( Novak, 2011), which uses feedback from work that students do at home (like pre-class reading assignments) to inform what is done in the classroom.

Research Methods

Participants

The participants in this study were undergraduate students enrolled in an introductory physics course. We collected reading assignment data and exam data from two semesters of the course when Perusall was used [spring of 2015 (S15) and fall of 2016 (F16)]. In S15, there were 74 students and in F16 there were 79 students. Students in the S15 course were not the same students as in the F16 course. Due to the fact that the event-tracking feature of the software was not yet developed in S15, most of the analysis focuses on the F16 cohort. As a point of comparison, we also collected exam performance data during the two previous semesters of the course when a different social annotation platform was being used [spring of 2014 (S14) and fall of 2015 (F15)]. There were 72 students enrolled in the S14 course and 75 students enrolled in the F15 course. The four student populations were very similar in composition. Student populations were comprised of 48-50% premedical students and 50-52% engineering students. All four groups were 53-55% female and consisted of equal ratios of students in their sophomore, junior, and senior years of college. The four groups had a similar level of incoming physics background knowledge, as measured by the average score on the physics conceptual survey administered at the beginning of each semester [ Force Concept Inventory ( Hestenes et al., 1992) for the fall courses and the Conceptual Survey on Electricity and Magnetism ( Maloney et al., 2001) for the spring courses].

Setting

We conducted this study in the School of Engineering and Applied Sciences at Harvard University in an introductory physics course called Applied Physics 50 (AP50). AP50 is a calculus-based physics course designed for undergraduate engineering students. It is split into two courses; AP50A, a mechanics course taught in the fall, and AP50B, an electricity and magnetism course taught in the spring.

The instructor was the same for all four semesters and the same pedagogy was used each semester. AP50 met twice weekly and each class was 3-h long. In this course, all lectures were replaced by reading assignments in Perusall and class time was entirely devoted to active learning. The pedagogy was based on features from both Project-Based Learning ( Blumenfeld et al., 1991) and Team-Based Learning ( Michaelsen et al., 2002). Students worked in small groups for all aspects of the course, including assessments. There were six different types of in-class activities, each of which designed to help students master the relevant physics and get started on the projects, which were the focal point of the course.

As there were no lectures, students were expected to read the textbook on Perusall. By midnight, the night before each class, students were required to complete the pre-class reading assignment by highlighting and annotating an assigned chapter of the textbook, the content of which was the focus of the activities the following day in class. As the class met twice a week, there were typically two chapter-long reading assignments per week. Over the course of each semester there were 17 assigned chapters, with each chapter containing 34 pages on average. To receive full credit for each reading assignment, students needed to enter at least 7 timely and thoughtful annotations per chapter.

Procedure

To evaluate the efficacy of Perusall and to study how and when students were using the software, we did three different types of analyses. We first extracted, from Perusall, a number of metrics that describe student reading behavior: the amount of time students spent reading, how long before each class students logged on to Perusall, and how often they returned to the same reading assignment. We calculated student averages, per reading assignment, for each of these metrics and summarized these (in Figures 9- 13) as a way of describing how much and when, relative to class, students were reading. Second, we used the statistical software, STATA, to calculate the correlations between specific reading metrics and student exam performance so that we could determine which (if any) types of reading behaviors are predictive of exam performance. Based on these correlations, we used STATA to develop linear regression models to predict exam performance using reading behavior metrics (controlling for physics background knowledge). Third, to study the efficacy of the software in promoting student learning, we conducted a comparative study between two different types of social annotation software platforms. We compared exam performance during two semesters of AP50 when Perusall was used (S15, F16) to performance on the same exams during two other semesters (S14, F15) when a different social annotation platform was used. The exams in each of the two fall semesters (F15 and F16) were identical as were the exams in each of the two spring semesters (S14 and S15). To ensure that the students' incoming physics knowledge was the same between the two fall populations and the two spring populations, we used STATA to do a two-sample, t-test for equal means ( Snedecor and Cochran, 1989).

Figure 9. Histogram of the number of assignments students fail to complete before class, in each semester.

This study was carried out in accordance with the recommendations of the Institutional Review Board (IRB) at Harvard University, Committee on the Use of Human Subjects. The IRB classified this study as "minimal risk" and, therefore, exempt from requiring written consent from the participants.

Results

Students' Pre-Class Behavior on Perusall

Figure 9 shows the extent to which students complete pre-class reading assignments over the two semesters that Perusall was being used. In each semester, approximately 60% of students completed every one of the 17 reading assignments. Figure 9 shows that in S15, approximately 90% of students completed all but a couple of reading assignments; in F16, 95% of students completed all but a couple of reading assignments.

Perusall allows us to collect data on how much time students spend on each individual page of a reading assignment. Using these data, we can determine when a student makes it all the way through the assignment. We define a page as "read" when the time spent on that page is longer than 10 s and less than 20 min. We define a student as having completed an assignment by dividing, for each assignment, the number of pages that were read by the total number of pages in the assignment. Based on this metric, we find that 80% of students make it through at least 95% of the reading and that an additional 10% of students make it through 80% of the reading (Figure 10).

Figure 10. Histogram of the average percent of the reading assignment students complete before class.

Using the same data we find that, on average, students spend 3 h and 20 min per week reading on Perusall (Figure 11).

Figure 11. Histogram of the average number of hours students spend on the reading per week.

Figure 12 shows the average number of individual "sessions" students take to complete their reading assignments. We define a session as any cumulative pages read for longer than 10 min with at least 2 h since the previous reading session. On average, students divide their reading of each assignment in seven different sessions.

Figure 12. Histogram of the average number of reading sessions per assignment.

Figure 13 shows the average number of annotations students enter per assignment. Over the course of the semester, students wrote a total of 16,066 annotations on Perusall. On average students make 13.3 annotations per assignment-nearly twice the number that the system recommends.

Relationship between Student Reading Behavior and In-Class Performance

To study the relationship between reading behavior and in-class performance, we built a series of linear regression models predicting students' exam performance (averaged over the five exams during the semester) from the reading and annotating metrics previously discussed. These models are presented in Table 1. We control for incoming physics background by including students' pre-semester score on the Force Concept Inventory ( Hestenes et al., 1992) and the Conceptual Survey on Electricity and Magnetism ( Maloney et al., 2001). We find that students who break up their reading into more sessions do better on in-class exams than students who read in fewer sessions, even when controlling for pre-course physics knowledge and the amount of time students spend reading.

Table 1. Standardized coefficients for linear regression models predicting average exam performance using the average time students spend reading per chapter and the average number of sessions students break their reading up into as predictor variables and controlling for pre-class physics knowledge (pre-semester FCI).

Model 1 shows that we can predict 42% of the variability in students' average exam performance using only their score on the pre-semester Force Concept Inventory. If we add the average amount of time students spend reading (model 2) we can predict marginally more (43%) of the variability in exam performance although this difference is not significant. When we add to the model the average number of sessions the students use to complete the reading, we find we can predict almost 10% more of the variability in student exam performance (model 3). Increasing the number of sessions a student completes the reading in by one SD increases average student exam performance by 0.41 of a SD ( p < 0.01). None of the other reading/annotation metrics are predictive of average student exam performance.

Student In-Class Exam Performance

Finally, we compare two different cohorts of the same course and show that the cohort for which Perusall was used to deliver the pre-class reading assignments did significantly better on the same in-class exams compared to students from the previous year when Perusall was not used.

Figure 14 shows student exam performance on 10 in-class exams administered over 2 years of AP50 (five in the fall semester and five in the spring semester). While the cohort of students were different over the four semesters, Table 2 shows that the four groups of students had the same level of incoming physics knowledge at the beginning of each semester (as measured by the Force Concept Inventory and the Conceptual Survey in Electricity and Magnetism). We conducted two-sample, t-tests to confirm that the performance on these conceptual inventories was the same for the two fall groups and for the two spring groups ( p = 0.32 and p = 0.36, respectively).

Table 2. Comparison of pre-course conceptual physics survey results between the four semesters of students (S14/F15 when Perusall was not used compared to S15/F16 when Perusall was used).

The only difference in the course between the S14/F15 and S15/F16 was the use of Perusall. During the S14/F15 semesters a simpler annotation tool was used to administer the pre-class reading assignments. This annotation tool lacked many of the social and machine learning features of Perusall. Students in the S15/F16 semesters scored 5-10% better on all but two of the 10 exams compared to the students from the semesters before when Perusall was not being used ( p < 0.05). Based on a two-sample t-test, averaging over all five exams in the fall, students in the class that used Perusall scored significantly better than the class that did not use Perusall ( p < 0.05). Students in the fall class that did not use Perusall had an average exam score of 38% compared to students in the fall class that used Perusall who had an average exam score of 43% (effect size = 0.34). The same is true when we average over all five exams in the spring-students in the class that used Perusall outperformed the students from the year before ( p < 0.05). Students in the spring class that did not use Perusall had an average exam score of 41% compared to students in the spring class that used Perusall who had an average exam score of 45% (effect size = 0.31).

Discussion and Conclusion

This study explores student pre-class reading behavior on Perusall, a social learning platform that allows students to interact and discuss course material online. We find that student completion of reading assignments is substantially higher than what has been reported in the vast majority of the literature. With Perusall, 90-95% of students complete all but a few of the reading assignments before class. For comparison, most of the literature reports that 60-80% of students do not read the textbook before coming to class ( Cummings et al., 2002; Clump et al., 2004; Podolefsky and Finkelstein, 2006; Stelzer et al., 2009). One study found that, with a JITT-like implementation of pre-class reading, between 80 and 85% of students completed the reading before class ( Heiner et al., 2014), but this study was based on student-reported reading data, which has been shown to be unreliable. Using reading data from Perusall, we find that 80% of students complete 100% of the reading assignment before class. This percentage, too, is considerably higher than what is reported in the literature: Clump et al. (2004) find that students only read on average 28% of the assigned reading before class.

In addition to higher completion of pre-class reading assignment, we also find that, on Perusall, students read for longer than what is reported in the literature. Approximately 92% of students report that they spent 3 h or less per week reading the textbook. On Perusall, students spend, on average, 3 h and 20 min per week reading for this one course.

In studying the relationship between reading behavior and in-class performance, we find that the average time spent reading per chapter alone is not predictive of student exam performance. This is consistent with what has already been reported in the literature ( Smith and Jacobs, 2003; Podolefsky and Finkelstein, 2006). However, it should be noted that previous studies on the relationship between time spent reading and exam performance have all been based on student-reported data. Our study uses data obtained directly from the Perusall platform. We do find, however, that students who break the reading up into more reading sittings perform better on in-class exams than students who read in fewer sittings. This is true even when we control for the amount of time students spent reading, and consistent given the spacing effect, a well-known phenomenon in psychology: material is more effectively and easily learned when it is studied over several times spaced out over a longer time span, rather than trying to learn it in a short period of time ( Dempster and Farris, 1990).

Finally, we find that students using Perusall perform significantly better on in-class exams than students using a simple annotation tool without some of the social and machine learning features of Perusall. We recognize that this result does not indicate causality and must be interpreted carefully given the fact that other factors could be confounding the results. More research needs to be done to pinpoint exactly why students do significantly better using Perusall. Perusall has many features that the other platform did not have. For example, with Perusall assessment is built right into the platform and students get regular and timely feedback. In the other platform, assessment was provided separately by the instructors and so students received sporadic and less targeted feedback. Perusall also has many social features (sectioning, avatars, upvoting, email notifications) that are designed to improve the interactions between students. Finally, the Confusion Report makes it easier for the instructor to address main areas of student confusion in class, which both affords better targeting of in-class time to student confusion and allows students to better see the connection between pre-class reading assignments and in-class activities.

We have demonstrated the efficacy of Perusall as a social learning platform and have shown that student completion of pre-class reading assignment is substantially higher than what has been reported by other studies. In short, with Perusall we are better able to get students to complete reading assignments, and do so in a way-with spaced repetition-that leads to better outcomes. Perusall, therefore, is a useful tool for delivering content to students outside class and for building an online learning community in which students can discuss course content and develop understanding. This is particularly important in flipped and hybrid courses or any other course that relies on pre-class reading assignments.

Ethics Statement

We were covered for this research by Harvard's Committee on the Use of Human Subjects (CUHS).

Author Contributions

Several people contributed to the work described in this paper. EM conceived of the basic idea for this work. KM, BL, GK, and EM designed and carried out the study, and KM analyzed the results and wrote the first draft of the paper. All authors contributed to the development of the manuscript.

Conflict of Interest Statement

The authors developed the technology described in this article, mostly at Harvard University. Perusall.com is a commercial product based on this work. The authors are cofounders of Perusall, LLC, the company that runs perusall.com.

References

Johnson, B. C., and Kiviniemi, M. T. (2009). The effect of online chapter quizzes on exam performance in an undergraduate social psychology course. Teach. Psychol. 36, 33-37. doi:10.1080/00986280802528972

PubMed Abstract | CrossRef Full Text | Google Scholar

# data[].attributes.attachment_link.title
Use of a Social Annotation Platform for Pre-Class Reading Assignments in a Flipped Introductory Physics Class

# data[].attributes.attachment_link.description
In this paper, we illustrate the successful implementation of pre-class reading assignments through a social learning platform that allows students to discuss the reading online with their classmates. We show how the platform can be used to understand how students are reading before class.

# data[].attributes.attachment_link.description
Solutions for education Keep students engaged and connected with social annotation. Hypothesis turns any piece of digital content into a collaborative learning experience. Get Started

# data[].attributes.attachment_link.description
Motivate students with a social learning experience to increase engagement, drive collaboration, and build community through course content, including books, articles, web pages, videos, podcasts, and images.

# data[].attributes.body
My new thing this semester is putting a brief announcement in D2L at least once a week.

# data[].attributes.body
Goals and Objectives can be powerful organizational concepts for student learning - but it feels like we usually write them for accreditors first and students second.

# data[].attributes.headline
Recording of our recent "Transparent Assignments" workshop 

# data[].attributes.subject
Recording of our recent "Transparent Assignments" workshop 

# data[].attributes.body
I learned to not expect to sound like a polished broadcaster - but to try to sound natural. For me reading a script turns me into a monotone robot so I try to work off of notes instead. I don't have a magic trick beyond doing it again and again and getting more comfortable with it.

# data[].attributes.body
I've done weekly videos for a long time but this semester I tried doing video notes for feedback on student projects. It was awkward at first - I'm very used to wordsmithing student feedback which you either can't do or have to do before recording but I'm hoping students appreciate that added personal connection of hearing my voice give them feedback (I just did it so no sense of how it landed with students yet)

# data[].attributes.headline
I do a read- and think-aloud, editing activity to give students feedback on their writing...captured in Zoom and uploaded into YouTube (private channel) for sharing. I focus only on the "repeating themes" that come up in their writing so that they can focus on the 2-4 issues that impede clear and concise writing. 

# data[].attributes.subject
I do a read- and think-aloud, editing activity to give students feedback on their writing...captured in Zoom and uploaded into YouTube (private channel) for sharing. I focus only on the "repeating themes" that come up in their writing so that they can focus on the 2-4 issues that impede clear and concise writing. 

# data[].attributes.headline
Improving UDL: captioning video lectures- what is the most time efficient way?

# data[].attributes.subject
Improving UDL: captioning video lectures- what is the most time efficient way?

# data[].attributes.headline
I can see that they are forming a COI (and a community of practice) because they are taking the initiative to reach out to each other to ask questions and provide resources. They are really interested in how their colleagues are tackling problems in different contexts.

# data[].attributes.subject
I can see that they are forming a COI (and a community of practice) because they are taking the initiative to reach out to each other to ask questions and provide resources. They are really interested in how their colleagues are tackling problems in different contexts.

# data[].attributes.body
Class Discussion Ground-Rules: Cultivating a Community of Trust and Inclusion
Two primary roles of class discussions are broadening our perspectives through learning about the diverse experiences and perspectives of course members and linking personal questions, thoughts, experiences, and ideas to new knowledge learned from the readings and lectures. A primary role of the course instructor in the context of class discussions is to facilitate that linkage.
• Be willing to ‘try on’ new ideas that might be unfamiliar.
• Move up/Move back: Encourage full participation by everyone.  If you tend to speak often, move back and vice versa. Allow time and opportunity for others in the group to voice their ideas, thoughts, and opinions.
• Strive to maintain one topic at a time and avoid interrupting a person who is talking.
• Attend to and communicate about your own experiences. Use “I” statements. Use clear examples.
• Refrain from speaking for the whole group or making assumptions about the experiences of others.

# data[].attributes.headline
Weekly intro videos. Off the cuff, not scripted, not content-heavy. Just a check in and some tips for what to focus on that week.

# data[].attributes.subject
Weekly intro videos. Off the cuff, not scripted, not content-heavy. Just a check in and some tips for what to focus on that week.