# data[].attributes.body
https://cft.vanderbilt.edu/guides-sub-pages/student-evaluations/

# data[].attributes.attachment_link.readable_text
Print Version
Talking with Students about Evaluations
To motivate students to complete end-of-course evaluations and to provide useful feedback through those evaluations, the Vanderbilt Center for Teaching recommends instructors talk with their students about the importance of course evaluations and how those evaluations are used.

Designate time in class for students to complete evaluations, and let your students know why and when. (See below for more on this advice.)

T ell your students that you value their honest and constructive feedback, and that you use student feedback to make improvements to your courses. If possible, share examples of how you have changed your courses as a result of student feedback.

Let your students know that you are interested in both positive and negative feedback on the course. What aspects of the course and/or instruction helped them learn? What aspects might be changed to help future students learn more effectively?

Describe the kind of feedback you find most useful. In most cases, specific feedback with examples is more useful than general statements. See the handout "Providing Helpful Feedback to Your Instructions" from the Center for Research on Learning and Teaching at the University of Michigan for examples of specific, constructive feedback.

Remind students that evaluations are designed to be completely anonymous and that you will not be able to see any of their evaluations until after final grades have been submitted. Many students don't realize these facts.

Let students know that you are the primary audience for their feedback, but that others will potentially read their evaluations, including department and school administrators. Course evaluations play a role in personnel evaluations and in curriculum planning.

Consider including language in your syllabus that addresses student evaluations. This alerts the students to the fact that they should also pay attention to their learning experiences throughout the semester and makes them more mindful of their responses in the course evaluations. Possible examples.

Vanderbilt's Kathleen Hoover-Dempsey, Professor of Psychology, Emeritus, was interviewed in 2003 about student course evaluations. She was asked if she thought her students took course evaluations seriously. Her answer:
Yes, I do think my students take them very seriously. I think they do in part because I tell them that I take them seriously. I schedule a time when we'll do the evaluation form; I tell students in advance that I consider it to be very important, and tell them that I really want them all to be present to evaluate the course. I tell them that I read every comment and find the comments extremely useful in thinking about and improving my own teaching. When I give the evaluations forms out I repeat all of those things, and add, "You can never write too much; I value all of the feedback I get, I do read it and it is very important to me." And then I follow all of the university guidelines (like getting quickly out of the classroom after identifying who's going to collect and return them to the department office.)

So yes, I get very substantive feedback, which I really value. In many courses, perhaps especially large ones, there is likely to be at least someone who's not particularly happy with the course. Their feedback can be very, very helpful to thinking about what I might do differently in the course. I think emphasizing that we take student comments very seriously, and find them very helpful, simply increases the likelihood of getting very useful feedback from all students.

Why is it better to include time in class for student evaluations?

By setting aside 20 minutes during class for students to complete course evaluations, just like the custom when evaluations were done with pencil and paper, instructors are not only increasing the overall student response rates, but they are also increasing the likelihood that students have time to think through their responses. As a result, students will have the opportunity to produce less rushed, more thoughtful feedback, especially if this strategy is combined with the other recommended strategies below. Using class time thus may be a way for instructors to differentiate the type of serious, considered input appropriate for course evaluations from common brief and off-the-cuff input on social media, customer feedback, and other online forums. Finally, setting aside class time communicates to students the importance of evaluations in the teaching mission of the university.
Adapted from "Some Guidelines and Principles to Consider In Making Sense of Evaluation Feedback" by Kathleen Hoover-Dempsey, Professor of Psychology, Emeritus, Vanderbilt University.
It should also be noted that when setting aside time in class for students to complete course evaluations, instructors should leave the room to help ensure that students feel free to provide authentic responses.

Making Sense of Student Evaluation Feedback

Along with the fresh start of the new year, many instructors will receive an opportunity to assess their teaching skills when they receive student evaluations of their Fall courses. Making sense of student feedback can be challenging so we offer the following tips for examining evaluations.

When considering student evaluations:

Pick a good time to do so, when you will have enough time to digest at least some of the information, have privacy, and can give yourself some mental 'space' to analyze the information.

Track quantitative results. Consider how the summary rating received for each item fits with your own teaching goals and your department's expectations for teaching.

Look for patterns in students' comments-identify trends, note what you have done well and what needs improvement.

Take your experience into account. If you are new to teaching, the school, or even the course, you may still be learning about various aspects of being a professor, such as course design, teaching skills, student interaction, and departmental expectations.

Take the context and characteristics of your course into account. Research shows that student evaluations often are more positive in courses that are smaller rather than larger, and elective rather than required. Also, evaluations are usually more positive in courses in which students tend to do well.

When dealing with negative student feedback:

Know that almost all faculty members receive negative feedback at some point in their careers, including those who are senior and highly successful.

Allow yourself to acknowledge that it can feel hurtful or make you angry, but also provides a pointer toward important areas for your continued development.

When deciding how to further your development as a teacher:

Bear in mind the most frequently mentioned areas for teaching improvement in analysis of student evaluations within and across universities: 1) clearer, more specific in-class communication; and 2) clearer, more explicit organization of course content.

Consider scheduling an appointment at the Center for Teaching for a consultation to help you interpret your evaluations. Research suggests that teachers who consult with someone about their evaluations are more likely to score higher on the next set of evaluations than others who do not discuss them with anyone. To schedule a consultation on student evaluations, call the Center for Teaching at 322-7290.

When planning steps to improve the feedback you receive in evaluations, consider the following options:

Use one minute evaluations at the end of selected class sessions, asking students to note the main idea they learned that class, or two ideas about a major construct considered, or a question about content, and so forth.

Give a "midterm evaluation" of the course, using the official university form or one you have created, to check how the class is progressing while you can use the information to make changes.

Talk with the class about their interim feedback, and explicitly put into practice one of their suggestions.

Before the final course evaluation, explain to the class the importance you place on their input.

Mid-Semester Student Feedback and Other Strategies

Course evaluations can be and should be thought of as a part of a larger classroom narrative, one that focuses on improving students' learning experiences from beginning to end along two intertwined paths: student feedback and improving teaching.

Gathering Student Feedback

There are multiple opportunities to solicit student feedback throughout the semester. The feedback students provide about your teaching on their end-of-semester course evaluations is the most identifiable form of feedback and can be valuable in helping you improve and refine your teaching. Soliciting mid-semester student feedback has the additional benefit of allowing you to hear your students' concerns while there is still time in the semester to make appropriate changes.

The CFT offers a service called a Small Group Analysis, which is a method of gathering anonymous feedback from students about what is helping them learn and what is not, in a course. This completely confidential service is an excellent way to assess students' response to your teaching mid-semester. See the CFT website for more information on our SGA service.

If you're interested in gathering feedback from your students on your own, please see our " Gathering Feedback from Students" teaching guide for ideas and tools.

Lastly, for soliciting informal feedback from students on their learning throughout the semester, consider adapting some classroom assessment techniques (CATs) from our CATs teaching guide that best fit your classroom. One example of a CAT is the minute paper, during which time students take one minute to write a response to a question or statement prompt. This can be especially illuminating if the prompt is intended to collect feedback on their learning experiences in the course.

Other Mechanisms for Improving Teaching

The process of incorporating student feedback towards the improvement of your teaching can sometimes seem like a daunting process. The CFT can serve as a support system for you in this process through the following:

CFT individual consultations. We are available for consultations on any teaching questions or topics you might like to discuss.

Observations. The CFT offers classroom observations as a mechanism for instructors to get individualized feedback for a particular class.

Syllabus review. A CFT staff member can work with you to review your syllabus and consider how well your course design is accomplishing your goals.

Topic-specific consultations. CFT staff members can also consult with you around particular teaching questions, such as effective discussion approaches or assessment options.

Pedagogy-specific consultations. If you are interested in adopting particular pedagogies, such as case-based learning, service-learning, or team-based learning, the CFT can work with you to adapt that approach to your class.

The CFT hosts the Open Classroom each fall, a multi-day teaching visit event, featuring opportunities to observe and discussion teaching practices around campus.

The CFT also hosts learning communities on various teaching topics. These communities provide Vanderbilt educators opportunities to learn from and with each other as they develop their teaching skills.

Outside the CFT, peer evaluations are another way to get valuable feedback from colleagues and to potentially create a community of teachers in your department. For more information, please see our guide on the Peer Review of Teaching.

Resources on Interpreting Student Evaluations

"Student Rating Forms", a chapter from the book Tools for Teaching by Barbara Gross Davis.

Interpreting and Working with Your Course Evaluations, a resource from the Center for Teaching and Learning at Stanford University, featuring suggestions for improving one's scores on particular student evaluation questions.

Evaluating and Improving Undergraduate Teaching in Science, Technology, Engineering, and Mathematics, an on-line book published by the National Research Council (2003).]

The following articles can be found in the journal, New Directions for Teaching and Learning, Volume 2001, Issue 87, Special Issue: Techniques and Strategies for Interpreting Student Evaluations . Issue Edited by Karron G. Lewis.

Faculty Thoughts and Concerns about Student Ratings, by John C. Ory, Office of Instructional Resources at the University of Illinois at Urbana-Champaign. "Although student ratings of instruction are used to determine whether a person is teaching effectively, many people who use them are not aware of the extensive research base for them."

Encouraging Your Students to Give Feedback, by Marilla D. Svinicki, Center for Teaching Effectiveness at the University of Texas at Austin. "Giving feedback is a skill that can be learned. What are the conditions that foster that learning and the later use of that skill for feedback to instructors?"

Making Sense of Student Written Comments, by Karron G. Lewis, Center for Teaching Effectiveness at the University of Texas at Austin. "Most student evaluation instruments include a place for student comments, yet the comments are often difficult to interpret. This article illustrates these comments and uses the information for improving teaching and students' learning."

Using Midsemester Student Feedback and Responding to It, by Karron G. Lewis, Center for Teaching Effectiveness at the University of Texas at Austin. "Getting midsemester feedback from your students can help you make changes before it's too late."

Interpreting the Numbers: Using a Narrative to Help Others Read Student Evaluations of Your Teaching Accurately, by Jennifer Franklin, Center for Teaching and Learning at California State University, Dominguez Hills. "Student ratings are one of the most widely used measures in teaching today. All users should understand what the numbers mean and how they should and should not be used."

The following articles are from the former newsletter of the Center for Teaching: Teaching Forum 6:1,Fall 2003 Newsletter, "Evaluating Teaching: Student Ratings and Beyond."

Student Course Evaluations, by Anupama Balasubramanian, CFT fellow. In this article from the Center for Teaching newsletter, a Vanderbilt faculty member and teaching assistant discuss their perceptions of student course evaluations, and how to effectively use them.

From the Student's View, by Anupama Balasubramanian, CFT fellow. In this article from the Center for Teaching newsletter, eight Vanderbilt undergraduates share their experiences with student rating forms.

Interview on CFT Consultation on Student Evaluations, by Anupama Balasubramanian, CFT fellow. In this article from the Center for Teaching newsletter, former CFT Associate Director Peter Felten describes his work with student evaluation consultations.

Summaries of Research on Student Evaluations

Student Ratings of Teaching: A Summary of Research and Literature (IDEA Paper 50) by Stephen L. Benton and William E. Cashin, IDEA Center. This white paper "summarize[s] the conclusions of the major reviews of the student ratings research and literature from the 1970s to 2010. That literature is extensive and complex; a paper this brief can offer only broad, general summaries and limited citations."

Student Ratings: Myths vs Research Evidence, by Michael Theall, Center for Teaching and Learning at the University of Illinois at Springfield. Theall, a research expert in instructional design, development and evaluation, explores the myths and truths behind Student Ratings (reprinted with the permission of the Brigham Young University Faculty Center).

How To Evaluate Teaching, by Richard Felder, from Chemical Engineering Education, 38(3), 200-202 (2004). "A key to effective teaching evaluation is to collect data from multiple sources [peers, students, instructors, administrators]...making sure that all education-related activities are rated by the people best qualified to rate them."

Looking for Bias in All the Wrong Places: A Search for Truth or a Witch Hunt in Student Ratings of Instruction? by Michael Theall, Center for Teaching and Learning at the University of Illinois at Springfield, and Jennifer Franklin, Center for Teaching and Learning at California State University, Dominguez Hills. "Through a half-century of research on student ratings, the constant quest has been to prove or disprove the existence of biasing factors. What have we learned, and what has happened as a result?"

Questions Frequently Asked about Student Rating Forms: Summary of Research Findings," by Matthew Kaplan, Lisa A. Mets and Constance E. Cook, University of Michigan Center for Research on Learning and Teaching. This article answers questions such as, "What do we know about the relationship between grades and student ratings? What do student ratings tell us about teaching effectiveness?"

Flunking the Test: The Dismal Record of Student Evaluations, by Paul Trout, Montana State University. "Though most schools use them, numerical evaluations of faculty members get bad grades. They aren't accurate and they're dumbing down undergraduate education."

Student Ratings of Professors are not Gender Blind, by Susan Basow, Lafayette College. This article was originally published in the Association for Women in Mathematics Newsletter. "The ratings of male professors are unaffected by student gender, but female professors frequently receive lower ratings from their male students and higher ratings from their female students. Female professors also appear to be evaluated according to a heavier set of expectations than are male professors, and these expectations affect student ratings."

Student Ratings of Women Faculty, by Michael Theall, Center for Teaching and Learning at the University of Illinois at Springfield, and Jennifer Franklin, Center for Teaching and Learning at California State University, Dominguez Hills. This article provides research findings on interactions between instructor gender and student ratings of teaching.

Student Evaluations and Gendered Expectations: What We Can't Count Can Hurt Us, by Kelley Massoni, University of Kansas, and distributed by the Sociologists for Women in Society. "How does gender enter into students' evaluations of their teachers. Scholars who have attempted to answer this question are divided in their findings. ...This fact sheet is designed to make sense of the research on gender and teaching evaluations."

Are Student Ratings Unfair to Women? by Neal Koblitz, University of Washington, reprinted from the Association for Women in Mathematics Newsletter, Vol. 20, No. 5, September-October, 1990. "If an instructor feels compelled to put students under pressure (assigning a lot of homework, giving challenging exams), then only the most serious and mature students are at all likely to respond with high ratings at the end of the course. Most students are inclined to "punish" the instructor. There is considerable evidence that the "punishment" is more severe if the instructor is female."

Gender and Student Evaluations: An Annotated Bibliography, at the Center for Research on Learning and Teaching at the University of Michigan.

Student Evaluations: Gender Bias and Teaching Styles by Lynn H. Collings, Joan C. Chrisler, and Kathryn Quina, excerpted from Career Strategies for Women in Academe: Arming Athena (Sage Publications, 1998). "The authors discuss factors impacting student evaluations of faculty performance and steps women faculty in particular can take to ameliorate negative biases."

This teaching guide is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

# data[].attributes.attachment_link.description
Print Version Talking with Students about Evaluations Tips for Making Sense of Student Evaluation Feedback Mid-Semester Student Feedback and Other Strategies Resources on Interpreting Student Evaluations Summaries of Research on Student Evaluations Talking with Students about Evaluations To motivate students to complete end-of-course evaluations and to provide useful feedback through those evaluations, the Vanderbilt Center...

# data[].attributes.body
https://www.unco.edu/center-enhancement-teaching-learning/teaching-toolkits/course-evaluation-toolkit.aspx

# data[].attributes.attachment_link.provider_name
Center for the Enhancement of Teaching & Learning

# data[].attributes.attachment_link.description
Student learning assessment is essential to knowing if students are learning course content. Assessment data is used to track learning and make curricular changes based on students’ progress. Along with data based on course work it is important to evaluate…

# data[].attributes.body
https://www.colorado.edu/teaching-quality-framework/tools-for-teaching-evaluation#Recommendations%20on%20the%20use%20of%20SETs

# data[].attributes.attachment_link.readable_text
Many units on the CU Boulder campus have been working in partnership with the Teaching Quality Framework (TQF) Inititiative to better align their teaching evaluation practices with known scholarship on teaching evaluation by: a) examining their current teaching evaluation practices; b) identifying or creating tools that better assess teaching quality and fill gaps within their current evaluation practices in order to align the multiple measures from three key voices (peers, self, and students); and c) implementing these tools along with procedures for their use. These are designed to fulfill university requireiments - see " CU Policies Related to Measuring Teaching Effectiveness" for an annotated list of CU policies that support academic units defining their own measures of teaching quality and the use of multiple measures / multiple voices in evaluation.

The following tools are designed to align with an overarching assessment rubricthat categorizes seven dimensions of teaching, and draws from three voices (peer review, self-evaluation, and student evaluation). Working with the TQF-team, departmental teams utilizing the Departmental Action Team (DAT) model ( Corbo et al. 2016) have developed tools and processes for measuring teaching quality for the purpose of merit and / or reappointment, tenure, and promotion. Below we provide links to generic templates and department-specific examples of these materials, which currently fit into four broad categories: Peer Observation, Self-Evaluation, Student Evaluation, and Applying the Overarching Rubric to an Evaluation Form. Please note that many of the examples are still in development (notations below: "in use" = currently being used by the department in their evaluation process(es); "in review" = being reviewed and/or piloted by members in the department external to the TQF DAT team; "in development" = a near final draft is that is approaching review). As additional "in use" versions become available we will update this list.

Unstructured peer classroom observations, i.e., those that are not based on a set of core criteria, can result in inconsistency and do not always address teaching practices that are valued by a department ( AAAS 2012). For this reason, scholarly literature on teaching evaluation recommends that academic units articulate the best teaching practices for their field and define core criteria to use in the observation process ( AAAS 2012). Feedback on teaching can be more effective in promoting growth and improvement when it focuses on specific issues, contains concrete information, and is based on specific data (rather than general impressions) ( Brinko 1993). Toward these ends, departments working with the TQF have developed standardized peer observation protocols, peer evaluation plans, and associated materials. ( return to top)

These peer observation protocols are intended to provide structure and consistency to peer classroom observations. The templates and examples below draw heavily from the UTeach Observation Protocol (UTOP: https://utop.uteach.utexas.edu/) and the Oregon Teacher Observation Protocol (OTOP): Wainwright et al. 2003).

Many departments have chosen to explicitly include active learning as a category for peer observation; in departments where faculty may be less familiar with active learning, a more detailed addendum may be added to the peer observation protocol to provide additional guidance.

Peer evaluation plans should explicate departmental processes around how peer observation fits into their teaching evaluation plans, including how the peer observation protocols will be used.

In some departments, the standardized peer observation protocol is not directly used but rather is designed to provide guidance to peer reviewers around what should go into a letter. Similar to peer observation protocols, these guides should include structured, consistent categories for review.

Because peer observation materials compiled for reappointment, tenure, and promotion dossiers may differ, in some cases significantly, from what had been submitted in the past, a cover letter that can be attached to peer observation materials can provide rationale for the change.

( return to top)

Self-reflection is a key element of a complete teaching evaluation. Often the instructor's voice is lacking in the evaluation process, particularly for annual merit. Many departments are working on incorporating self-reflection into their merit processes and / or developing guidelines for writing teaching statements for reappointment, tenure, and promotion. ( return to top)

One way of incorporating the voice of the faculty member being evaluated into the evaluation process is to allow space for them to reflect on their own teaching practices in the annual merit process. Typically departments have created a list of guiding questions that align with their values and the TQF assessement rubric, from which faculty can select the most relevant to their own practices.

While a teaching statement is required for reappointment, tenure, and promotion dossiers, typically there is very little guidance on what should be included in these statements. Several departments are developing such guidance.

( return to top)

Many units on campus rely heavily on end-of-semester student evaluations of teaching (SETs, aka FCQs at CU), particularly for annual merit. While students are one of the three key voices in evaluation of teaching, over-reliance on SETs/FCQs as the primary/sole measure of student voice can be problematic (see "Role of Students in Evaluation: Student Voice" in our FAQ). The Boulder Faculty Assembly (BFA) recently recommended that FCQs (SETs) be used primarily as formative feedback rather than summative assessment, that evaluators be made aware of potential bias in FCQs, and that the omnibus questions (rate the instructor and course overall) be removed ( BFA-R-2-102918.4). In addition to rethinking how FCQs/SETs are used, many departments working with the TQF are developing / improving other forms of student voice, including classroom interviews and student letters. ( return to top)

Preliminary takeaways from ongoing pilot of new SET items at CU Boulder (fall 2020 draft)

Expanded recommendations coming soon

( return to top)

As departments explore better ways for incorporating student voice, many have begun working on guidelines for conducting classroom interviews. The Faculty Teaching Excellence Program (FTEP) at CU Boulder currently offers a Classroom Learning Interview Process (CLIP) where students are placed into small groups to discuss their responses to questions posed about their learning experiences in the course. The confidential feedback from this formative process is then shared back with the instructor.

While many departments allow faculty to submit this confidential feedback as part of their merit and/or reappointment, tenure, and promotion processes, several departments have developed modified versions that can be incorporated into their peer observation processes (see Peer Observation Protocol examples above).

( return to top)

As departments explore better ways for incorporating student voice, several have considered student letters because while many departments solicit letters from current and past students to include in a dossier for reappointment, promotion, and tenure, these solicitations often do not include guidelines to students on what to include in their letters or how to write their letters.

In collaboration with the TQF, the Mechanical Engineering Department and other departments/units have developed new solicitation letters that include specific guidelines for what students might reflect on and include in their letter. Noting that the expectation is not that students would write about every suggestion but rather pick from among the suggestions those that are most relevant to their experiences.

( return to top)

All TQF departments have worked with the TQF assessment rubric when designing their assessment materials (i.e., ensuring that the measures they develop align with the seven principles in the rubric), and some have gone on to adapt the rubic as needed to align with their discipline and use it in their evaluation processes.

While materials produced by departments are aligned with the overall rubric, departments are also developing compact representations, or summaries of the multiple data sources into a a single form that can be used as an assessment rubric. Germanic & Slavic Languages & Literatures has customized the rubric to pull evidence from self-reflection, peer observation protocols, review of syllabi/course materials, and FCQs (SETs) for annual merit evaluation of teaching. ( return to top)

( return to top)

References

American Association for the Advancement of Science. 2012. Describing and Measuring Undergraduate STEM Practices. A Report from a National Meeting on the Measurement of Undergraduate Science, Technology, Engineering, and Mathematics (STEM) Teaching. https://live-ccliconference.pantheonsite.io/wp-content/uploads/2013/11/Measuring-STEM-Teaching-Practices.pdf

Brinko, K. T. 1993. The practice of giving feedback to improve teaching: What is effective? The Journal of Higher Education 64(5): 574-593. https://www.jstor.org/stable/2959994

Corbo, J. C., Reinholz, D. L., Dancy, M. H., Deetz, S., & Finkelstein, N. 2016. Framework for transforming departmental culture to support educational innovation. Physical Review Physics Education Research, 12(1), 010113. https://doi.org/10.1103/PhysRevPhysEducRes.12.010113

Wainwright, C. L., Flick, L. B., & Morrell, P. D. 2003. Development of instruments for assessment of instructional practices in standards-based teaching. Journal of Mathematics and Science: Collaborative Explorations, 6(1), 21-46. https://doi.org/10.25891/NTKY-AX16

# data[].attributes.attachment_link.title
Tools for Teaching Evaluation: TQF Templates and Department Examples

# data[].attributes.attachment_link.description
Many units on the CU Boulder campus have been working in partnership with the Teaching Quality Framework (TQF) Inititiative to better align their teaching evaluation practices with known scholarship on teaching evaluation by: a) examining their current teaching evaluation practices; b) identifying or creating tools that better assess teaching quality and fill gaps within their current evaluation practices in order to align the multiple measures from three key voices (peers, self, and students); and c) implementing these tools along with procedures for their use.

# data[].attributes.body
https://cte.ku.edu/benchmarks-teaching-effectiveness-project

# data[].attributes.attachment_link.title
Benchmarks for Teaching Effectiveness | Center for Teaching Excellence

# data[].attributes.body
https://otl.du.edu/plan-a-course/teaching-resources/evaluating-teaching-excellence-toolkit/

# data[].attributes.attachment_link.readable_text
Faculty members at DU strive for ongoing teaching excellence and at DU we value the Teacher-Scholar model. In our efforts towards ongoing development of teaching excellence, we have provided resources and guidelines for evaluating teaching.

What is good teaching?

It is important to start with some assumptions about teaching. There is no one definition of good teaching, and good teaching comes in an array of forms. However, there is quite a bit of literature about what constitutes teaching effectiveness, in areas such as how people learn, course design, teaching methods, facilitation skills, classroom climate, educational technology, inclusive excellence, learning assessment, etc.

We know that, much like research, teaching performance is an art and skill in continual improvement and development. It is important that at DU we provide a culture where all faculty members are encouraged to continually reflect and continuously improve their teaching practice.

Evaluating Teaching
**At DU, many units are exploring more balanced approaches to evaluating teaching. Visit our Teaching Excellence Portfolio for a brief overview of some of these DU examples.**
Although there are varying opinions and research around Student Evaluations of Teaching (SETs)/Student Ratings of Instruction (SRIs), most people agree that they should not be used as the sole basis of evaluating someone's teaching performance and effort. At DU, the 2005 Teaching Task Force, the 2011 Faculty Senate Teaching Excellence Initiative, and the 2012 Renew DU Faculty Development Initiative have all recommended using multiple sources of evidence and effort when conducting annual reviews and reviews for promotion and tenure.

Evidence from Students

Although student end-of-course rating forms should not be used as the sole basis of someone's teaching performance, students can and should provide essential feedback on teaching. The resources below can help in the effective use of student rating forms. In addition, other forms of student feedback - midterm formative feedback, student letters, student learning as shown in coursework and assignments (not grades) - can provide additional evidence of teaching performance.

Evidence from Peers/Chairs

Colleagues and often provide feedback about teaching in areas such as: appropriateness of course materials and methods, and breadth and depth and relevance of content and course structure. Samples of teaching materials, syllabi, and samples of student work can be reviewed along with classroom observations.

Evidence from the Instructor

As teaching is an ongoing developmental activity, the instructor's reflection and self-assessment of teaching is valuable not only as a source of evidence, but as a tool for continuous improvement.

Additional Resources

# data[].attributes.attachment_link.description
Faculty members at DU strive for ongoing teaching excellence and at DU we value the Teacher-Scholar model. In our efforts towards ongoing development of teaching excellence, we have provided resources and guidelines for evaluating teaching. What is good teaching? It is important to start with some assumptions about teaching.

# data[].attributes.body
https://www.colorado.edu/teaching-quality-framework/sites/default/files/attached-files/cu_teaching_quality_framework_1pp_v2.06_0.pdf

# data[].attributes.headline
Include directions for how to use the site.

# data[].attributes.subject
Include directions for how to use the site.

# data[].attributes.headline
Possible infographic with hot links in a PDF that helps direct.

# data[].attributes.subject
Possible infographic with hot links in a PDF that helps direct.

# data[].attributes.body
https://frc.uccs.edu/teaching_resources/evaluation_of_teaching

# data[].attributes.attachment_link.readable_text
Evaluation of Teaching for Annual Merit Reviews and Reappointment, Promotion, and Tenure Decisions

The Faculty Representative Assembly Teaching Evaluation Task Force offers numerous recommendations for the evaluation of teaching for annual merit reviews and reappointment, promotion, and tenure. Two key recommendations are:

Regents' policy requires that each unit use at least three measures for teaching evaluations for both annual merit reviews and RPT decisions; FCQs must be used as one of those measures

Any evaluation based on the FCQ numerical ratings be used for no more than one third of the overall teaching rating of the faculty member

Read the full report.

The College of Letters, Arts & Sciences further has numerous recommendations for the evaluation of teaching in its Advisory Report.

Forms for Peer Evaluation of Teaching

Encouraging Higher Response Rates for Online FCQs

As of fall 2017 all University of Colorado Faculty Course Questionnaires will be administered online. The following resources examine the issues around response rates for online vs pencil-and-paper FCQs and offer suggestions for encouraging higher response rates.

Extra Credit Micro-Incentives and Response Rates for Online Course Evaluations: Two Quasi-Experiments; Eric D. Sundstrom, Erin E. Hardin, and Matthew J. Shaffer; Teaching of Psychology, 2016, Vol. 43(4) 276-284

The effect of incentives and other instructor-driven strategies to increase online student evaluation response rates; James Goodman, Robert Anson & Marcia Belcheir; Assessment & Evaluation in Higher Education, Volume 40, 2015 - Issue 7, p 958-970

The adequacy of response rates to online and paper surveys: what can be done?; Duncan D. Nulty; Assessment & Evaluation in Higher Education, Volume 33, 2008 - Issue 3, p 301-314

Anecdote from David Anderson, Faculty Resource Center Director:

I and some of my colleagues in the Departments of Biology and Chemistry & Biochemistry have been using micro-incentives to encourage higher response rates. In my ~150-student Organic Chemistry course, for example, I give every student in class some extra credit points based on the response rate. (We cannot determine who has or has not completed the online FCQs, so the extra credit points have to be applied to all students.) Furthermore, I prorate the extra credit points to the response rate: ≥95% response earns 5 points, ≥90% earns 4 points, and so on. Over the last two years I have averaged about a 94% response rate. Given that I have roughly 1000 points overall in my course, this extra credit amounts to less than 0.5% of the total. Very few students' letter grades are affected by this. I understand that some faculty have concerns about giving extra credit points, but they certainly help with encouraging higher FCQ response rates.

Last updated September 22, 2020
Contact Us

# data[].attributes.attachment_link.description
The Faculty Representative Assembly Teaching Evaluation Task Force offers numerous recommendations for the evaluation of teaching for annual merit reviews and reappointment, promotion, and tenure.

# data[].attributes.body
Department Teaching Survey

# data[].attributes.body
In-Person Teaching Review by Colleague

In-Person Teaching Review by Chair

# data[].attributes.body
- Lead Faculty Discussion on Focused Issue

- Summarize Pedagogy Research

- Outline Teaching Methodology

# data[].attributes.headline
Handbook of instructions for assessment options.

# data[].attributes.subject
Handbook of instructions for assessment options.

# data[].attributes.body
Attend Professional Teaching Conference

# data[].attributes.body
Publish in Teaching Journal

# data[].attributes.body
Follow-on class measurement (regression model) -- for pathway classes

# data[].attributes.body
Require TABS

# data[].attributes.headline
Handbook of instructions for assessment options.

# data[].attributes.subject
Handbook of instructions for assessment options.

# data[].attributes.body
Develop Statement of Teaching Methodology (But Probably Not -Though it came to mind)

# data[].attributes.headline
Rubric for evaluating adherence to the reflective teaching framework (college level)

# data[].attributes.subject
Rubric for evaluating adherence to the reflective teaching framework (college level)

# data[].attributes.headline
Checklist for managing their own performance.

# data[].attributes.subject
Checklist for managing their own performance.

# data[].attributes.headline
Measures of Student Learning/Knowledge - Pre-post model 

# data[].attributes.subject
Measures of Student Learning/Knowledge - Pre-post model 

# data[].attributes.body
1. Assessment Disaggregation modeles (Gamma, Gamma Gain, etc.)
2. Hake model
3. Cohen’s d (not as good but might be more familiar to some)
4. Could be class level or program level

# data[].attributes.body
Provide some of the following resources fro assessment purposes:
-​Peer Observation Protocols
-Active Learning Addendums
-Peer Evaluation Plans
-Letter Writing Guides for Peer Observation
-Peer Observation Cover Letters

# data[].attributes.headline
Measures of Student Learning/Knowledge -- Knowledge Measure (IRT 1PL model or Rasch; 2PL also possible if more data) 

# data[].attributes.subject
Measures of Student Learning/Knowledge -- Knowledge Measure (IRT 1PL model or Rasch; 2PL also possible if more data) 

# data[].attributes.body
1.   End of program assessment or end of class assessment
2. Could be used without a pretest.  Doesn’t necessarily require multiple choice

# data[].attributes.headline
Non-Survey Measure of Practices -- DART (or similar)

# data[].attributes.subject
Non-Survey Measure of Practices -- DART (or similar)

# data[].attributes.headline
Non-Survey Measure of Practices -- Manual Assessment of Inventory (observation)

# data[].attributes.subject
Non-Survey Measure of Practices -- Manual Assessment of Inventory (observation)

# data[].attributes.headline
Non-Survey Measure of Practices -- Classroom observations

# data[].attributes.subject
Non-Survey Measure of Practices -- Classroom observations

# data[].attributes.headline
Non-Survey Measure of Practices -- Canvas review

# data[].attributes.subject
Non-Survey Measure of Practices -- Canvas review

# data[].attributes.headline
Resources for Encouraging Higher Response Rates

# data[].attributes.subject
Resources for Encouraging Higher Response Rates

# data[].attributes.body
Use this Sample Syllabus Entry to explain the importance of the course evaluations

University of Colorado - Encouraging High Online Response Rates Web Site

Extra Credit Micro-Incentives and Response Rates for Online Course Evaluations: Two Quasi-Experiments; Eric D. Sundstrom, Erin E. Hardin, and Matthew J. Shaffer; Teaching of Psychology, 2016, Vol. 43(4) 276-284

The effect of incentives and other instructor-driven strategies to increase online student evaluation response rates; James Goodman, Robert Anson & Marcia Belcheir; Assessment & Evaluation in Higher Education, Volume 40, 2015 - Issue 7, p 958-970

The adequacy of response rates to online and paper surveys: what can be done?; Duncan D. Nulty; Assessment & Evaluation in Higher Education, Volume 33, 2008 - Issue 3, p 301-314

# data[].attributes.headline
Non-Survey Measures of Major Survival / Gains -- Survival model of major program path (coef of class by instructor)

# data[].attributes.subject
Non-Survey Measures of Major Survival / Gains -- Survival model of major program path (coef of class by instructor)

# data[].attributes.body
Track student success in follow-up courses/overall all courses in following semesters.

# data[].attributes.body
https://teaching.utk.edu/wp-content/uploads/sites/78/2018/10/Teaching-Evaluation-Toolbox-Redesign_AS_91418.pdf

# data[].attributes.headline
Non-Survey Measures of Major Survival / Gains -- Fitted decision tree of survival

# data[].attributes.subject
Non-Survey Measures of Major Survival / Gains -- Fitted decision tree of survival

# data[].attributes.headline
Non-Survey Measures of Major Survival / Gains -- Simple switch (in/out) of major generation by class

# data[].attributes.subject
Non-Survey Measures of Major Survival / Gains -- Simple switch (in/out) of major generation by class

# data[].attributes.headline
U Mass Program and Department-level Assessment Initiatives

# data[].attributes.subject
U Mass Program and Department-level Assessment Initiatives

# data[].attributes.body
https://www.umass.edu/oapa/program-curriculum-and-instructional-innovation

# data[].attributes.attachment_link.title
Program, Curriculum, and Instructional Innovation : Academic Planning & Assessment : UMass Amherst

# data[].attributes.attachment_link.description
The Office of Academic Planning and Assessment at the University of Massachusetts Amherst is responsible for managing the strategic planning process in academic affairs, the academic program review and approval processes, and the campus’s assessment program.

# data[].attributes.body
Have departments "cut/ignore" questions from climate survey and end of semester eval

# data[].attributes.body
https://natsci.msu.edu/faculty-staff/policies-procedures/evaluation-policy-resources/teaching-evaluation-guidelines/

# data[].attributes.attachment_link.readable_text
Preamble: The NatSci Promotion and Tenure Committee drafted guidelines in spring 2009 stipulating that the teaching skills of a candidate under consideration for promotion, tenure, or reappointment are to be evaluated using a set of instruments that include student evaluations, peer evaluations, and the candidate's teaching portfolio. In the fall of 2009, a separate ad hoc committee was charged with developing guidelines that describe how these instruments are to be implemented and interpreted, and more broadly, with recommending how the college can develop the teaching skills and instructional effectiveness of all its faculty. This document summarizes the guidelines and recommendations developed by the ad hoc committee, as revised by the NatSci RPT Committee in spring 2011.

Many faculty members in NatSci already provide high quality instruction. The guidelines and recommendations set forth here are not intended as a comprehensive critique of how teaching is currently being done. Rather, their overarching purpose is to identify teachers who are excellent, so that they may be recognized and rewarded, and to provide support and mentoring to faculty in need of improvement. More specifically, the guidelines and recommendations are intended to achieve the following:

To clarify for faculty how the quality of their teaching will be evaluated by the college and by its departments and programs.

To identify faculty in need of additional mentoring early in their MSU careers and ensure that departments provide the support those faculty need to become quality teachers.

To provide incentives that reward faculty whose teaching is consistently excellent, that encourage all faculty to be attentive to teaching quality, and that motivate expert teachers to be mentors to others.

To recognize and reward departments that foster a culture of teaching excellence.

To identify especially successful teaching practices so that they may be shared across the college.

To accomplish all of the above without placing an undue burden on faculty and staff to document teaching accomplishments or to review faculty dossiers

The responsibility for implementing teaching evaluation that accomplishes these goals rests primarily with the departments of NatSci. We recognize that these departments are a heterogeneous group, with teaching loads, course types, and student populations that differ greatly from department to department. We are therefore recommending a broad framework of guidelines that allows departments to tailor the details of implementation to their own needs and constraints

In general, we expect that departments will provide all newly hired faculty with mentoring in teaching and that they will perform peer reviews sufficient to determine whether their courses are being competently taught. Departments are also expected to provide substantive summaries of teaching accomplishments when their faculty are considered for promotion, tenure, and reappointment. The next section describes the documentation upon which departments should base their assessments of faculty teaching. The section following it describes the format of the summaries to be submitted to the college. The concluding section suggests some support programs and incentives that NatSci could provide to promote quality teaching throughout the college.

Documentation of Teaching Accomplishments: Written materials documenting a faculty member's teaching career should provide multidimensional information on teaching analogous to the documentation of research accomplishments. We recommend that the documentation in support of promotion, tenure, and reappointment decisions consist of the following items.

Student Evaluations. SIRS scores, or the equivalent, should be provided for all courses taught at MSU. Student opinion is one among many important instruments for assessing teaching performance, and uniformly low ratings can indicate a real need for improvement. However, the reasons for low scores can vary significantly and should be examined as part of the department's peer evaluation. If student evaluation scores are no longer available for some courses, a list of those courses should be provided so that the record of courses taught is complete.

A departmental evaluation of a candidate's SIRS responses (or equivalent), including an analysis of student comments, and a comparative evaluation to (the same or comparable) courses taught by others, should be included as part of the narrative response in Form D-IIIA (see below). Departments should also compile a numerical summary of SIRS scores for all courses taught since appointment (or the last RPT action), and summarize them on the NatSci RPT Numerical Student Evaluation Summaries worksheet.

Peer Evaluations. The guidelines document drafted by the NatSci Promotion and Tenure Committee stipulates that the overall evaluation of a candidate for promotion, tenure, or reappointment will be based in part on peer evaluation of a candidate's teaching performed by the home department. The college expects that peer evaluation will include both classroom visits by faculty members from the department and reviews of syllabi and assessment tools used in courses. Faculty members who perform peer evaluations should be well informed about best practices.

Peer evaluations of teaching should be annual during at least the first two years of a faculty member's service as an MSU instructor, and they should continue to be annual as long as the department has concerns about the instructor's teaching. A peer evaluation should also be performed in the year before the instructor applies for promotion. Peer evaluations of all senior faculty should be performed periodically, in order to recognize excellence and to ensure that departments remain broadly aware of their own teaching practices. The time period between peer evaluations will depend on departmental resources but we recommend that the period should not exceed five years.

The purpose of these peer evaluations is fourfold: (1) to place student evaluations in perspective, either corroborating them or providing an alternative viewpoint on the quality of teaching in the course, (2) to verify that the course goals outlined in the syllabus are clearly stated and appropriate and that the assessment tools are well aligned with those goals, (3) to review course content and exams to determine if these materials are appropriate for the course, and (4) to encourage an exchange of ideas among colleagues that enhances the quality of teaching throughout the department. Departments should develop their own processes for peer evaluations, taking advantage of best practices outlined by the National Academies (see footnote on the previous page) and remaining careful to avoid the pitfalls described therein. These processes should be well documented and well known by the faculty and the college. Departments should resist the temptation to demand that faculty adhere to a particular style of teaching and be attentive to keeping the peer evaluation process from becoming politicized.

NatSci has created peer classroom observation and teaching portfolio evaluation tools that, at the discretion of the Department, can be used to facilitate this process. These tools are attached at the end of this document.

The preparation of teaching portfolios is not intended to be burdensome and is most effectively managed by updating the portfolio each semester. Portfolios of junior faculty will naturally grow as their teaching experience accumulates. Furthermore, during the next few years, reviewers should recognize that faculty coming up for promotion, tenure, and reappointment did not have these guidelines to follow until recently and should make allowances for that fact while evaluating their teaching portfolios.

Senior faculty should not be expected to produce complete teaching portfolios within an unrealistically short time period after these guidelines are implemented. The time frame over which senior faculty should develop teaching portfolios will depend on how a department chooses to do its peer evaluations of teaching. However, the teaching portfolio should be complete before the faculty member comes up for peer evaluation.

Summarizing Teaching Accomplishments: The guidelines for promotion, tenure, and reappointment in NatSci specify that departments provide a written summary of the teaching evaluations of that faculty member (see Form D-IIIA, "Evaluation of Instruction"). That summary should devote at least one paragraph to each of the following aspects of teaching quality, based on the department's review of student evaluations, peer evaluations, and the teaching portfolio.

Clear and Appropriate Objectives. This part of the summary is based on peer evaluation of the syllabi and sample assessments provided in the teaching portfolio. Evaluators should address whether the course objectives and requirements are clearly stated in the syllabi and whether the objectives for each course are appropriate. It is also desirable for evaluators to address whether the assessments are aligned with the stated course objectives and adequately measure student learning, but this type of evaluation may be more difficult for upper division courses with specialized content.

Competence in the Classroom. This section summarizes the student evaluations and the classroom observations of the peer evaluators, who should seek to understand the reasons for either unusually high or unusually low student evaluations. Samples of written student comments can be particularly helpful in illuminating the reasons for the numerical scores. Positive student evaluations are desirable but should not weigh so heavily that fear of lower scores discourages faculty members from making changes that could improve their teaching effectiveness. If a candidate's student evaluation scores are persistently low, this part of the summary should explain the reasons for the low scores, as determined by the peer evaluators, and the steps the department and candidate have taken to improve those scores. As departments bear a significant mentoring responsibility, this portion of a candidate's summary will be one of the means used to assess a department's commitment to quality teaching.

Evidence of Excellence. This section summarizes the elements of the teaching portfolio that the candidate has provided as evidence of teaching excellence. The documentation the candidate has provided should be available to the NatSci Promotion and Tenure committee upon request. Evaluators should recognize that evidence of excellence can come in many different forms, which may include:

Evidence showing that students have made learning gains in the candidate's courses or have been effectively prepared for more advanced academic work.

Documentation of exceptional commitment to student learning.

Descriptions of innovative teaching by the candidate, which may consist of pioneering teaching methods, applications of new teaching technologies, development of new courses,or substantive revisions to existing courses.

Examples of pedagogical materials developed by the candidate, such as textbooks, websites, or lecture notes, that have been adopted in other professors' courses.

Teaching awards, as long as they come with a statement of the reason for the award.

Successful mentoring of research students.

Contributions to Teaching Culture. The last part of the summary describes how a faculty member has contributed to the broader culture of teaching, drawing on material provided in the teaching portfolio. As mentioned earlier in this document, in the section on teaching portfolios, examples of contributions may include such things as: efforts to develop graduate student teaching, mentoring a younger faculty member in teaching, working with a mentor to improve one's own teaching, collaborative teaching efforts, and participation in or direction of workshops to improve teaching skills. Evaluators are welcome to comment on how the candidate's contributions have improved the quality of teaching in their department.

Support and Incentives for Teaching Excellence: In order for these guidelines to be successful, the college must be willing to provide support to faculty members wishing to improve the quality of teaching in NatSci and incentives sufficient to motivate our busy faculty to devote some time and attention to quality teaching.

Support Programs for Faculty. Workshops on teaching and learning are offered by MSU's Academic Advancement Network, the NatSci Dean's Office, and many professional societies (e.g. American Geophysical Union, American Chemical Society, Ecological Society of America). Among the programs important to implementation of these guidelines are the following:

Training in writing an effective syllabus.

Development of a NatSci clearinghouse for sharing and disseminating successful teaching methods.

Training in evaluating student learning gains.

Training programs in peer evaluation practices.

Funding for attending teaching workshops outside of MSU

Incentives for Faculty. Evaluation of a faculty member's teaching for raises, promotion, tenure, and reappointment should be weighed by chairs and committees in direct proportion to the faculty member's assigned duties. Among the actions that the college and its departments and programs can take to motivate professors and demonstrate the college's commitment to teaching quality are the following:

Efforts to explicitly acknowledge when excellence in teaching is a factor in raising a professor's salary.

College-sponsored teaching awards for both junior and senior faculty that provide a recurring salary increase.

Rewards to departments whose summaries of teaching accomplishments demonstrate that they foster a culture of quality teaching.

# data[].attributes.attachment_link.description
Preamble: The NatSci Promotion and Tenure Committee drafted guidelines in spring 2009 stipulating that the teaching skills of a candidate under consideration for promotion, tenure, or reappointment are to be evaluated using a set of instruments that include student evaluations, peer evaluations, and the candidate's teaching portfolio.

# data[].attributes.body
https://provost.uoregon.edu/revising-uos-teaching-evaluations#m-ses

# data[].attributes.attachment_link.readable_text
The Office of the Provost and the University Senate have been working together since spring 2017 to revise University of Oregon's teaching evaluation system. Recent research indicates that student ratings may not accurately reflect teaching quality and may be inflected by bias [ 3, 4, 5, 6, 9, 10]. A research project at the University of Oregon similarly cast doubt on the reliability of numerical course evaluations [ 2].

The Association of American Universities (AAU) and other universities around the globe from University of Colorado, Boulder to University College London, England have argued that it is time for universities' ideals regarding teaching excellence to align with their policies [ 1, 7, 8]. As such, the University of Oregon has developed a holistic new teaching evaluation system that does more than simply replace problematic evaluation instruments. The new system provides the path to define, develop, evaluate, and reward teaching excellence. The goals of the new system are to ensure teaching evaluation is fair and transparent, is conducted against criteria aligned with the unit's definition of teaching excellence, and includes input from students, peers and the faculty themselves.

In August 2019 a Memorandum of Understanding (MOU) between the faculty union and the Provost communicates an update to the Collective Bargaining Agreement (CBA) and enshrines the commitment to evaluating professional, inclusive, engaged and research-led teaching practices. A new Senate committee formed in Fall 2018, the Continuous Improvement and Evaluation of Teaching (CIET) committee, oversees implementation of the April 2019 Senate legislation which includes the elements described below.

The Continuous Improvement and Evaluation of Teaching System

There are three important inputs included in the evaluation of teaching system. These include student feedback, instructor reflection, and peer review. New tools are available in each of these categories to support quality teaching and ensure that instructors receive useful and constructive feedback.

Student Feedback

Student feedback is an important part of the Continuous Improvement and Evaluation of Teaching System because students are at the center of the teaching and learning endeavor and have valuable perspectives to share. Students also have the potential to provide instructors with useful feedback to clarify which teaching practices are most beneficial to their learning experience, and what could be improved to help them learn.
Midway Student Experience Survey (M-SES)
The University Senate approved a centrally distributed Midway Student Experience Survey (M-SES) in which student feedback is only available to the instructor and is intended for mid-course adjustments or for clarifying goals and expectations with students. Campus-wide distribution began fall 2019.

Students are invited to complete the M-SES during week 4 of the term (or week 7 for the School of Law semester) between Monday at 8 a.m. and Friday at 6 p.m. In order to improve response rates and receive high-quality feedback, instructors are encouraged to provide 10-minutes of class time for students to take the survey. The results of the survey will be available to instructors the following week on Monday at noon; instructors are encouraged to debrief with their students about the feedback they received.

Instructors who have their own midway student feedback instruments and do not plan to read student feedback from the M-SES should communicate this clearly to students prior to week 4 of the term (or law semester week 7) so students know to skip that specific course when completing their M-SES surveys.
End-of-course Student Experience Survey (E-SES)
The University Senate also approved a new End-of-course Student Experience Survey(E-SES) to replace the previous student Course Evaluation. After a year of pilot testing and modifications, campus-wide deployment began fall 2019. The E-SES is focused on student learning, asks concrete questions about specific teaching practices, and inquires about student contributions to their own learning. In reports, student comments are collated by teaching practice making it easier to digest and interpret student feedback.

Analyses from E-SES pilots indicated this survey increases the number of student comments by 60% and drastically reduces the frequency of personal comments (from 21% to 1.5%). Additionally, most E-SES comments (61%), are positive, as students elaborate on teaching practices that are beneficial for their learning.

The E-SES will be available for students the last week of the term or semester prior to finals (Week 10 for the term, or Week 15 for the School of Law semester) from Monday at 8 a.m. to Friday at 6 p.m. For best response rates and high-quality feedback, instructors are encouraged to use the in-class protocol and provide 10-minutes of class time during Week 10/Week 15 for the students to complete their E-SES. Instructors can view reports the Wednesday after grades are submitted.

Implemented Fall 2019, the CIET senate committee created a process to remove discriminatory, obscene, or demeaning SES comments committee as described by the protocol. Instructors can "flag" comments for possible redaction which will be reviewed by the CIET senate committee at least three times per year between October and May, and the upcoming review date will be listed on the main Course Surveys page in DuckWeb.

Self Reflection

A crucial part of teaching evaluation is an instructor's reflection of their own teaching. The UO believes self-reflection should be a regular part of an instructor's course planning and development, and that the instructor's own voice should be available at the course level to complement student feedback from the E-SES.

To facilitate on-going self-reflection, the University Senate approved a new Instructor Reflection, which was deployed campus-wide fall 2019. The tool provides an opportunity for instructors to archive what went well and what might be improved in the future as well as how their teaching aligns with UO's definition of teaching excellence. The reflection also provides a new mechanism for the instructor's own voice to inform evaluators' interpretation of student feedback.

Peer Review

In order to improve upon peer review's efficacy as a tool for continuous teaching improvement and evaluation, the 2018-19 CAIT on Teaching Excellence and Evaluation developed a tool for units to use to reflect on their own peer review systems and practices. The tool provides questions such as "Do you ever have to rush to get a peer review done to get it into a file for midterm review, promotion, tenure etc.?" or "Are classroom observations done with a blank piece of paper and only the reviewer's personal beliefs about teaching as a guide?" It then provides concrete solutions for each concern, all of which are best practices already successfully implemented in some UO departments.

This guide for unit-level self-study is meant to support departments in ensuring their peer review practices are serving them well.

Evaluation of Teaching Criteria

Revising UO's evaluation tools is only one step of the process. In order for evaluations to matter, there needs to be a set of standards against which teaching practices can be measured. Currently, many unit policies do not define teaching quality nor do they provide basic criteria that outline the behaviors or outcomes for meeting teaching quality expectations. The August 2019 Memorandum of Understanding between the University and United Academics clarifies the minimum definition of teaching quality to be used for evaluation of teaching beginning in Fall 2020. Units are invited to modify the standards outlined in the MOU with unit-specific standards.

The MOU describes standards for professional, inclusive, engaged and research-informed teaching, which are in alignment with the UO's teaching excellence principles. The Provost's Teaching Academy has been engaging with these principles since its inception in 2016, and members have posted their professional teaching profiles describing their inclusive, engaged and research-led teaching practices. Beginning in Fall 2020, these standards and conditions apply to the formal evaluation of teaching. The Teaching Evaluation Criteria document provided by the Office of the Provost can help review committees perform these criteria-based evaluations. Units may modify the Teaching Evaluation Criteria document through the policy change guidelines outlined in the CBA. (It is available in Word for your track changes modification). Because AY20-21 is a transition year, the Office of the Provost will pay careful attention to any possibility that a substantial change in review criteria has negatively impacted a faculty member.

As units are thinking about what teaching quality means to them, we offer this document to structure departmental reflection and conversation.

Feedback

The Continuous Improvement and Evaluation of Teaching (CIET) senate committee and the Office of the Provost are interested in feedback from all stakeholders in this process, including students, faculty, staff and administrators. Use this Qualtrics form to provide your feedback on new surveys, reports or P&T related webpages and templates. You must log-in with your duck-id and password to access the Qualtrics form.
Give Feedback
Timeline

This timeline displays our progress on these initiatives in reverse chronological order.

Winter 2020:

United Academics and University sign an amendment to the August 2019 Memorandum of Understanding (MOU) providing units until Fall 2021 to make unit specific modifications to the standards described in the original MOU. Note that the standards and conditions outlined in the August 2019 MOU will still go into effect as the university's baseline definition of teaching quality in Fall 2020.
Winter Meetings with UO Faculty and Leadership

January - Department of Biology, faculty meeting

January - Department of Creative Writing, faculty meeting

January - Department of Political Science, faculty meeting

February - Department of Romance Languages, faculty meeting

February - Department of Physics, faculty meeting

February - All Unit Heads Meeting - Winter Timely Updates

February - Senior Leadership Team - Vice Presidents

March - Faculty Advisory Council

March - Lundquist College of Business - optional faculty meeting

Fall 2019

Campus-wide deployment of:

Continuous Improvement and Evaluation of Teaching (CIET) Senate Committee develops protocol for redaction of discriminatory, obscene or demeaning E-SES comments

2019/20 Teaching Excellence and Evaluation Community Accelerating the Impact of Teaching (CAIT) group formed with representation from all schools and colleges with a focus on the Teaching Evaluation Criteria document and new data dashboard.

Fall Meetings with UO Faculty and Leadership

August - College of Education, leadership meeting

August - School of Law, faculty meeting

September - Summit for Academic Leaders

September - School of Music and Dance, faculty meeting

September - School of Architecture and the Environment, faculty meeting

September - Department of International Studies, faculty meeting

September - Professional Schools Associate Deans meeting

October - Department of Women, Gender & Sexuality Studies, faculty meeting

October - School of Journalism and Communication, faculty personal committee meeting

October - Department of Special Education, faculty meeting

October - Department of Philosophy, committee meeting

October - Portland Campus, zoom meeting for faculty

November - Department of Art, faculty meeting

November - Department of East Asian Languages & Literatures, faculty meeting

November - Department of Sociology, faculty meeting

November - Department of Dance, faculty meeting

November - School of Planning, Public Policy and Management, faculty meeting

November - Division of Humanities, heads & managers meeting

November - Getting More from Peer Review workshop, for faculty and heads

December - 'First Cohort' workshop, faculty up for review under new standards Fall 2020

December - Department of History, faculty meeting

December - Department of Educational Methodology, Policy and Leadership, faculty meeting

December - History of Art and Architecture, faculty meeting

December - Department of Comparative Literature, faculty meeting

Summer 2019

United Academics and University sign a Memorandum of Understanding (MOU) enshrining professional, inclusive, engaged and research-informed teaching standards and baseline conditions for use in formal teaching evaluation.

Spring 2019

Winter 2019

March 20 - Updates on progress shared at All Unit Heads training.

February 18 - Around the O: Workplace story outlining reform efforts.

Teaching Excellence and Evaluation CAIT Pilot subgroup continues to pilot new tools:

January 30 - University Senate approved the CIET senate committee's work on a Warning and Guidance on Student Evaluations of Teaching document for faculty and personnel committees.

January 13 - Chronicle of Higher Education article highlights UO's work on course evaluation.

Teaching Excellence and Evaluation CAIT Peer Review subgroup members each selects an observation tool to experiment with for Peer Review and completes 3 mock classroom observations.

Teaching Excellence and Evaluation CAIT Pilot subgroup makes tweaks to the Midterm Student Experience Survey for week 5 and selects End of Term Student Experience Survey to Pilot in week 10. Pilot faculty will also complete Instructor Reflection.

Fall 2018

November 28 - Ginger Clark, assistant vice provost for academic and faculty affairs at the University of Southern California, hosted two discussions about USC's reform of their teaching evaluation process.

November 17 - Sierra Dawson, Austin Hocker, and Lee Rumbarger lead discussion about Teaching Evaluation reform efforts at Professional and Organization Developers Network (POD) conference in Portland, OR.

Teaching Excellence and Evaluation CAIT Pilot subgroup includes English, Honors College, Human Physiology, Lundquist College of Business, and School of Planning, Public Policy and Management. From these units:

Teaching Excellence and Evaluation CAIT provided guidance for an update on the definition of Teaching Excellence.

Teaching Excellence and Evaluation CAIT (Community Accelerating the Impact of Teaching) formed with membership from each and every school and college, as well as the three divisions of the College of Arts and Sciences.

Continuous Improvement and Evaluation of Teaching Senate Committee formed based on spring Senate Legislation.

October 3 - University Senate updated with progress report and plan for the year.

Summer 2018

Faculty teaching courses in Business, English, Public Planning, Policy and Management, Writing, Romance Languages, Education Studies, Cinemas Studies, and Human Physiology are piloting the Midterm Student Experience Survey, End-of-Term Student Experience Survey, and 10-Minute Instructor Reflection in the CollegeNET system.

Spring 2018

Pilots of updated Midterm Student Experience Survey in Qualtrics, as well as End-of-Term Student Experience Survey and 10-Minute Instructor Reflection in CollegeNET

May 1 - Town Hall for undergraduate students, 6:00-7:00 p.m., EMU Redwood Auditorium

May 2 - Town Hall for unit heads, 8:30-9:30 a.m., EMU Redwood Auditorium

May 2 - Town Hall for graduate students, 10:00-11:00 a.m., EMU Redwood Auditorium

May 3 - Town Hall for faculty, 10:00-11:00 a.m., Ford Alumni Center Ballroom

May 9 - Senate discussion of motion

May 23 -Senate motion US17/18-19 legislates the following:

The Continuous Improvement and Evaluation of Teaching senate committee will be for

Student evaluations will be anonymous

Midterm Student Experience Surveys will become available via CollegeNET

10-Minute Instructor Reflections will become available via CollegeNET

Winter 2018

Fall 2017

Meetings with stakeholders: Associate Dean's Luncheon on Teaching Evaluation, Grad Council, ASUO Exec, SWAT, Graduate Student Advisory Board, Women's Center, Dean of Student Life-Staff, Student Trustee and Senators, Women of Color Coalition, LCB Dean's Undergraduate Student Advisory Council, Dean of Students Advisory Committee, Mujeres student group, IMPACT (meetings continue through winter and spring terms)

Spring 2017

Documents

References

To learn more about the research behind our initiative to revise the UO Teaching Evaluation System, please see the following documents.

1. Aligning Practice to Policy: Changing the Culture to Recognize and Reward Teaching at Research Universities

2. Teaching, Learning and Achievement: Are Course Evaluations Valid Measure of Instructional Quality at the University of Oregon

3. Student Evaluations Can't Be Used to Assess Professors

4. Gender Bias in Teaching Evaluations

5. Teaching Evaluations and Bias

6. Is Gender Bias an Intended Features of Teaching Evaluations?

7. Undergraduate STEM Education Initiative

8. The Career Framework for University Teaching

9. Boring, A. (2017). Gender biases in student evaluations of teaching. Journal of Public Economics, 145, 27. DOI: 10.1016/j.jpubeco.2016.11.006 at https://www.sciencedirect.com/science/article/abs/pii/S0047272716301591

10. MacNell, L., Driscoll, A. & Hunt, A.N. (2015). What's in a name: Exposing gender bias in student ratings of teaching. Innovative Higher Education, 40, 291. DOI: 10.1007/s10755-014-9313-4

Updated January 2020

# data[].attributes.attachment_link.description
The Office of the Provost and the University Senate have been working together since spring 2017 to revise University of Oregon's teaching evaluation system. Recent research indicates that student ratings may not accurately reflect teaching quality and may be inflected by bias [ 3, 4, 5, 6, 9, 10].

# data[].attributes.headline
Responding to Information from Evaluations of Teaching

# data[].attributes.subject
Responding to Information from Evaluations of Teaching

# data[].attributes.body
Provide resources to help Chairs/Directors/Deans to answer the following question: How should Deans, Directors, and Heads respond when they learn about excellent teaching or unsatisfactory teaching?

https://ctlt.ubc.ca/resources/teaching/evaluation-tools/

# data[].attributes.attachment_link.authors[].name
Suggested Principles and Guidelines for the Peer Review of Teaching

# data[].attributes.attachment_link.title
Evaluation Tools | UBC Centre for Teaching, Learning and Technology

# data[].attributes.attachment_link.description
During the 2006-07 academic year, a committee co-chaired by Dr. Anna Kindler (Vice-Provost, Academic Affairs) and Dr. Joy Johnson (Nursing) revisited the ways student data are collected and used to improve teaching and learning at UBC. This committee’s objectives were to:

# data[].attributes.attachment_link.embed_code
Evaluation Tools

# data[].attributes.headline
I'd like to see us come up with a list of needs/wants for each of the groups that we need to provide resources for (i.e., non-tenured faculty, tenure-track faculty, chairs/directors, deans, institution, etc.)

# data[].attributes.subject
I'd like to see us come up with a list of needs/wants for each of the groups that we need to provide resources for (i.e., non-tenured faculty, tenure-track faculty, chairs/directors, deans, institution, etc.)

# data[].attributes.body
https://www.rit.edu/provost/sites/rit.edu.provost/files/docs/Evaluation_of_Teaching_Effectiveness.pdf

# data[].attributes.body
https://www.uwindsor.ca/ctl/sites/uwindsor.ca.ctl/files/teaching_evaluation_toolkit.pdf

# data[].attributes.attachment_link.title
Lilly Conference | Lilly Conference on College and University Teaching

# data[].attributes.attachment_link.description
The Lilly Conference Series provides opportunities for the presentation of the scholarship of teaching and learning. We invite administrators, graduate students, and faculty to join us at a Lilly event to exchange ideas and build a repertoire of skills that can be put to immediate use.