# data[].attributes.attachment_link.readable_text
Question: What activity is done by most teachers in the United States, but has almost no evidence of effectiveness in raising student test scores?

Answer: Analyzing student assessment data.

This practice arose from a simple logic: To improve student outcomes, teachers should study students' prior test performance, learn what students struggle with, and then adjust the curriculum or offer students remediation where necessary. By addressing the weaknesses revealed by the test results, overall student achievement would improve.

My own recent experiences visiting schools imply this trend continues."

Yet understanding students' weaknesses is only useful if it changes practice. And, to date, evidence suggests that it does not change practice-or student outcomes. Focusing on the problem has likely distracted us from focusing on the solution.

With the birth of large-scale state assessments and widening data availability in the 1990s, school leaders and teachers could access information on student performance that was common across schools and classrooms. Many schools also instituted standardized "interim" assessments, claiming that this periodic, low-stakes testing could help teachers identify difficult content and struggling students before the state assessment, giving both teachers and students a chance to catch up. Over time, educational testing and data companies including the Achievement Network, NWEA, and McGraw-Hill's Acuity began to sell interim assessments to schools and states, making such assessments (and their cousins, test-item banks that support formative assessment) a billion-dollar business.

Currently, a large number of teachers report they regularly get together to analyze student assessment results. In a 2016 survey by Harvard's Center for Education Policy Research, 94 percent of a nationally representative sample of middle school math teachers reported that they analyzed student performance on tests in the prior year, and 15 percent said they spent over 40 hours that year engaged in this activity. Case-study research suggests that in many Title 1 schools, this activity is a cornerstone of teachers' weekly or monthly collaborative time.

Read the full series here.

But here's the rub: Rigorous empirical research doesn't support this practice. In the past two decades, researchers have tested 10 different data-study programs in hundreds of schools for impacts on student outcomes in math, English/language arts, and sometimes science. Of 23 student outcomes examined by these studies, only three were statistically significant. Of these three, two were positive, and one negative. In the other 20 cases, analyses suggest no beneficial impacts on students. Thus, on average, the practice seems not to improve student performance.

One critical question, of course, is, why?

Observational studies suggest that teachers do, in fact, use interim assessments to pick out content that they need to return to. For instance, in a study published in 2009, Margaret E. Goertz and colleagues at the University of Pennsylvania observed teachers planning to revisit math topics using a combination of whole-group and small-group instruction.

But Goertz and colleagues also observed that rather than dig into student misunderstandings, teachers often proposed non-mathematical reasons for students' failure, then moved on. In other words, the teachers mostly didn't seem to use student test-score data to deepen their understanding of how students learn, to think about what drives student misconceptions, or to modify instructional techniques.

My own recent experiences visiting schools imply this trend continues. Field notes from teacher data-team meetings suggest a heavy focus on "watch list" students-those predicted to barely pass or to fail the annual state reading assessment. Teachers reported on each student, celebrating learning gains or giving reasons for poor performance-a bad week at home, students' failure to study, or poor test-taking skills. Occasionally, other teachers chimed in with advice about how to help a student over a reading trouble spot-for instance, helping students develop reading fluency by breaking down words or sorting words by long or short vowel sounds. But this focus on instruction proved fleeting, more about suggesting short-term tasks or activities than improving instruction as a whole.

Common goals for improving reading instruction, such as how to ask more complex questions or encourage students to use more evidence in their explanations, did not surface in these meetings. Rather, teachers focused on students' progress or lack of it. That could result in extra attention for a watch-list student, to the individual student's benefit, but it was unlikely to improve instruction or boost learning for the class as a whole.

In reviewing the research on teachers analyzing student data, I came across a small number of programs that included interim assessment as one part of a larger instructional package. While I excluded these studies from the formal review I undertook for this essay, they are notable nonetheless. One, by Janke M. Faber and colleagues in the Netherlands, focused on a program that not only contained computer-based interim assessments but also provided both instructionally focused feedback to teachers and students and personalized online student assignments. Another study, led by Jonathan A. Supovitz and colleagues at the University of Pennsylvania, examined the Ongoing Assessment Project, a program that helps teachers create assessments and examine the results, then combines this practice with professional development focused on mathematics content and student thinking about that content. Both of these studies saw positive impacts, suggesting that the analysis of data can, when combined with strong supports for improved teaching, shift student outcomes. But the small number of programs that combine the study of data with wider instructional supports limits our ability to draw real conclusions.

In total, the research in this area suggests that district and school leaders should rethink their use of state and interim assessments as the focus of teacher collaboration. Administrators may still benefit from analyzing student assessment results to know where to strengthen the curriculum or to provide teacher professional learning. But the fact remains that having teachers themselves examine test-score data has yet to be proven productive, even after many trials of such programs.

For many schools, this news is disheartening. Retooling teacher collaborative time will be a major shift-and that's assuming that schools can first identify more effective ways to help teachers improve their instruction. In our next column, we'll cover possible replacement activities.

# data[].attributes.attachment_link.title
Does Studying Student Data Really Raise Test Scores? (Opinion)

# data[].attributes.attachment_link.description
Many teachers believe that analyzing student assessment data can improve performance, but the research suggests otherwise, writes Heather C. Hill.

# data[].attributes.attachment_link.provider_name
Powerful Teaching: Unleash the Science of Learning

# data[].attributes.attachment_link.readable_text
"Powerful Teaching is a book that will benefit all teachers and, more importantly, their students."

James M. Lang / Author, Small Teaching: Everyday Lessons from the Science of Learning

More Downloads from RetrievalPractice.org

For more free downloads, including Practice Guides, Retrieval Warm Ups, Metacognition Sheets, and research reports, visit retrievalpractice.org.

# data[].attributes.attachment_link.title
Free Templates & Downloads - Powerful Teaching: Unleash the Science of Learning

# data[].attributes.attachment_link.description
Download templates and resources from the book Powerful Teaching. Take your instruction to the next level, implement evidence-based strategies, and transform learning in your classroom.

# data[].attributes.headline
Collecting Data and Designing Differentiated Learning

# data[].attributes.subject
Collecting Data and Designing Differentiated Learning

# data[].attributes.attachment_link.title
Formative Assessment: Collecting Data & Designing Differentiated Learning Experiences in Class or Online

# data[].attributes.attachment_link.description
Even though collecting informal data may be more challenging online or in a concurrent classroom composed of a group of students in class and another group online simultaneously, it's critical that teachers understand where students are and what they need to continue making progress.

# data[].attributes.headline
PK-2 = Seesaw to have a digital portfolio of student work including recordings of video/audio to see/hear their progress and share with families.

# data[].attributes.subject
PK-2 = Seesaw to have a digital portfolio of student work including recordings of video/audio to see/hear their progress and share with families.

# data[].attributes.body
Educators use PLC time to design CFAs for each math unit. These are given at the beginning of a unit so educators can design learning based on the results.

# data[].attributes.body
At the Virtual Learning Academy (LISD) we plan to utilize student surveys more for teachers to understand the adjustments that may need to be made in the courses to improve student success.

# data[].attributes.body
- Reflections
- Thoughts
- Feelings

# data[].attributes.body
This educator uses pre-assessment data to design his lessons and activities.  His learners use their own data to create their learning pathway.

# data[].attributes.headline
Involving K-2 in their data collection and goal setting

# data[].attributes.subject
Involving K-2 in their data collection and goal setting

# data[].attributes.attachment_link.title
Setting Goals with Early Primary Learners

# data[].attributes.attachment_link.readable_text
John Hattie developed a way of synthesizing various influences in different meta-analyses according to their effect size (Cohen's d). In his ground-breaking study "Visible Learning" he ranked 138 influences that are related to learning outcomes from very positive effects to very negative effects. Hattie found that the average effect size of all the interventions he studied was 0.40. Therefore he decided to judge the success of influences relative to this 'hinge point', in order to find an answer to the question "What works best in education?"
Originally, Hattie studied six areas that contribute to learning: the student, the home, the school, the curricula, the teacher, and teaching and learning approaches. (The updated list also includes the classroom.) But Hattie did not only provide a list of the relative effects of different influences on student achievement. He also tells the story underlying the data. He found that the key to making a difference was making teaching and learning visible. He further explained this story in his book " Visible learning for teachers ".

John Hattie updated his list of 138 effects to 150 effects in Visible Learning for Teachers (2011), and more recently to a list of 195 effects in The Applicability of Visible Learning to Higher Education (2015). His research is now based on nearly 1200 meta-analyses - up from the 800 when Visible Learning came out in 2009. According to Hattie the story underlying the data has hardly changed over time even though some effect sizes were updated and we have some new entries at the top, at the middle, and at the end of the list.

Below you can find an updated version of our first, second and third visualization of effect sizes related to student achievement.
Order VL for Teachers!

# data[].attributes.attachment_link.title
Hattie effect size list - 256 Influences Related To Achievement

# data[].attributes.attachment_link.description
Hattie's updated effect size list of 256 influences across all areas related to student achievement.

# data[].attributes.body
Our teachers use data to inform how they should design or adjust their courses. The adjustments and reflection do not always have to be for the students.

# data[].attributes.body
Learners can take a look at one area (tek) that they need growth and set a goal on what they want to do to show their understanding as time goes by. This is where portfolios with goal and reflection templates can be powerful.

# data[].attributes.body
Each student could create a digital portfolio that could travel with them through thier school career. This could also be used as reflective tool that could help students and teachers see thier growth.