# data[].attributes.headline
Selecting an Effective Screener - Dr. Nadine Gaab

# data[].attributes.subject
Selecting an Effective Screener - Dr. Nadine Gaab

# data[].attributes.attachment_link.title
Ask an Expert: How do we select an effective screening tool?

# data[].attributes.headline
A Brief Report of the Diagnostic Accuracy of Oral Reading Fluency and Reading Inventory Levels for Reading Failure Risk Among Second- and Third-Grade Students

# data[].attributes.subject
A Brief Report of the Diagnostic Accuracy of Oral Reading Fluency and Reading Inventory Levels for Reading Failure Risk Among Second- and Third-Grade Students

# data[].attributes.attachment_link.readable_text
The availability of psychometrically sound and usable universal screeners is a key component to successful early identification within a response-to-intervention model. The purpose of this study was to compare the diagnostic accuracy of oral reading fluency (ORF) and an informal reading inventory for identifying students considered at risk for failing a district-wide assessment. Participants included 968 students in 2nd and 3rd grade in a rural elementary school district in Minnesota. Results showed significant correlations between ORF and informal reading inventory data and between each potential screener and the criterion measure at 2nd and 3rd grades. However, ORF demonstrated higher diagnostic accuracy for correctly identifying at-risk students and resulted in 80% correct classification compared to 54% for the reading inventory data. We discuss implications for practice and directions for future research.

Figures - uploaded by Matthew K Burns

Author content

All figure content in this area was uploaded by Matthew K Burns

Content may be subject to copyright.

Discover the world's research

20+ million members

135+ million publications

700k+ research projects

Join for free

Conditions of access and use can be found at http://www.tandfonline.com/page/terms-

Downloaded by [University of Minnesota Libraries, Twin Cities] at 06:33 10 March 2015

Two better understand how teaching kindergarten students too much information contributes to off-task behavior.

View project

# data[].attributes.attachment_link.title
(PDF) A Brief Report of the Diagnostic Accuracy of Oral Reading Fluency and Reading Inventory Levels for Reading Failure Risk Among Second- and Third-Grade Students

# data[].attributes.attachment_link.description
The availability of psychometrically sound and usable universal screeners is a key component to successful early identification within a response-to-intervention model. The purpose of this study was to compare the diagnostic accuracy of oral reading fluency (ORF) and an informal reading inventory for identifying students considered at risk for failing a district-wide assessment.

# data[].attributes.attachment_link.readable_text
Susan Lambert: 0:02
If you don't know the work of Dr. Jan Hasbrouck, you are in for quite a treat. And if you do know her work, you'll already be excited to listen. Jan has been in the work of literacy for decades and still brings us relevant, timely, and helpful information. You'll enjoy hearing a bit about her Science of Reading journey. You'll learn more about what she calls a weird little assessment, and you'll be introduced to her most recent book, Student-Focused Coaching. Drop us some comments in the Science of Reading: The Community Facebook group and let us know what you think. Well, hello, Jan, thank you so much for joining us on today's episode.

Dr. Jan Hasbrouck: 0:43
Hey, Susan. I'm happy to be here. I look forward to our conversation.

Dr. Jan Hasbrouck: 1:04
<laugh> Yeah, that was a fun one to think about, because I am far along in my literacy journey at this stage of my career. I've been doing this as a professional educator for many, many decades. To think back about how I got started took me back to my own personal journey with literacy. I was one of those precocious readers. I probably had taught myself to read. But my very young mother was trying to follow all the rules. I was the eldest of three girls so I was her guinea pig, <laugh> but somehow she got the message-and we did have kindergarten back there in California, where I started school-but she was told, don't let your children, don't teach your children to read. That will happen in first grade. So don't, you're not supposed to do that. So as we know about precocious readers, those kids who basically teach themselves. She was reading to us and doing all that, not teaching us to read, but she said she was trying really hard to keep me from reading. And it was really <laugh> hard. So I don't really remember learning to read. It was easy for me and a joy throughout the process. So personally I had that experience. And right around that time, another story my mom loved to tell about me is that the first day of kindergarten, I came home that evening, and we always had dinner together, and we sat at the table and she wanted me to debrief this kindergarten experience for my sisters. And she said I announced-and I wouldn't be surprised-that I said, "I'm going to be a teacher when I grow up." <laugh> and, you know, OK! And of course, life takes turns and all kinds of things, and there were periods of time when I played around with something else. I did seriously consider possibly a medical career for a while. And because, you know, I'm a product of when I grew up-I grew up in the fifties, and for women, the choices were stay-at-home mom or you could branch out maybe and be a teacher or a nurse or librarian or something. So for medical, I thought nursing. In high school, my cousin was living with us at the time. She's my age. And we did candy-striper training together, and then went to volunteer in the high school [hospital?] with our little candy-striper uniforms and everything. And that first day we walked out and I said-

Dr. Jan Hasbrouck: 4:02
<laugh> "Not for me! I am not going into nursing!" And she just recently retired from a 40-some year career as a surgical nurse. With advanced degrees and everything. She knew that day; this is her thing. This is her passion. I knew that day <laugh> I better stick with teaching. But right around that time, also, the next probably most major thing that happened on my road to where I am today is I had really narrowed down that I was gonna be a teacher and I was excited about it. But as I was getting ready to leave high school, I had a conversation one day with a high-school English teacher that I really admired and had worked with. And so after school I went to her class and I had a life-changing conversation. Because I said to her, "Well, you know, I'm gonna be graduating soon, and I wanted to let you know that I've decided that when I go to the university next year, I'm going to study so that I can be a high-school English teacher."

Dr. Jan Hasbrouck: 5:13
And she gave me not the response I expected. She looked at me and there was a pause. And, you know, to this day, of course, I don't know if she just had a really rotten seventh-period class, or a bad headache, or whatever it was, but she looked at me and said, "Oh, Jan, no. You don't wanna be a high school English teacher. Seriously."

Dr. Jan Hasbrouck: 7:44
And I'd never seen anything like that before. And after she was done, I went right up to her and said, "What were you doing?" And she said, "It's called direct instruction. And right over there, there's some trailers and there's some folks working in there, including Zig Engelmann, and he's signing up students to learn direct instruction and you should go over and talk to him." So I did. And that was the next major step in my career. 'Cause I dropped out of the teacher-ed track and dropped into the direct-instruction track at 19 years old. A junior. And so I learned to teach using direct instruction. And what I hear now at this stage of the discussions about the Science of Reading and that wonderful [inaudible] lovely Facebook group of The Science of Reading, what I should have learned in college. There's just so many people agonizingly telling their story about being taught incorrectly, how to teach or not taught how to teach and having to discover what I should have learned in college. And I am one of a small handful of people that I learned what I should have in college. I learned how to teach from Zig Engelmann, which was not-if anybody knows anything about Zig, um, that was not always an easy process! But he was passionate and heartfelt. And he knew his stuff. And he taught us what he knew then. But you know, that was 19-many decades later. I'm still learning. The process isn't over. I have bookmarked two research studies that I hope to get to this afternoon to read. There's just new information coming about both how to teach and how to teach reading specifically and writing and the interface between reading and writing. And so I know I chose the right career. <laugh> I was right that first day in kindergarten when I came home! No doubt, no question in my mind. Because it was a good fit for me. And lucky for me, unlike some careers where my cousin, for instance-we just talked this week and she said there's no way she could have continued in the career of nursing. She just-at her age, at my age, nursing was just much too demanding, physically and emotionally. Whereas my career can allow me-I mean, I don't think I could teach a classroom of 30-some children every single day and maintain that, but I can find a way to keep involved with the profession and continue to grow myself and continue to give back to the profession. So yeah, my journey continues, but that's the trajectory.

Susan Lambert: 11:06
<laugh> I love that. And we love that you're still involved in teaching us all along the way as you're learning, too. So thank you for your commitment to this career that you chose, which is really now more than just a career. It feels a little like you're talking, this is a little passion for you, or you wouldn't be reading research articles on a Thursday afternoon. <laugh>

Dr. Jan Hasbrouck: 11:25
In between this morning, creating a webinar that I will deliver next week, and this evening, delivering a webinar to some educators-and parents apparently-in Calgary, Alberta, it's a wonderful, exciting time.

Susan Lambert: 11:44
Well, part of your professional journey includes helping us understand a lot about this-what you call the weird little assessment called ORF. <laugh> Oh, I love that you called that weird little assessment. Talk to us a little bit about-I mean, ORF has got a big place and it's often misunderstood. Can you tell us a little bit about how you've been involved in that and what you know and learned?

Dr. Jan Hasbrouck: 13:56
But when I got there, what was I supposed to do? So I started thinking that it was clear I needed help. And around that time in my mailbox, I wish I still had that piece of paper, but for some reason I got a little flyer in my mailbox that said that there was a program at our nearby university, University of Oregon, that was training in a post-graduate level course or series of courses, training teachers to work with their peer colleagues to improve educational outcomes and student outcomes and that kind of thing. And I thought, "I think maybe that's what I'm supposed to be doing! And if there's somebody over there teaching those courses...." So I called up and found out that the person teaching those courses was this brand-new professor named Jerry Tindal. And so I said, "Well, I wanna be in that program." So I went over and interviewed with Jerry, and he signed me up and the first course-that was a whole series of events that changed my life. But speaking about ORF, particularly, he had just completed his doctoral program at the University of Minnesota, where he worked with Stan Deno and Jim Ysseldyke. And Jerry's cohort of doctoral students included names that I'm sure many of your listeners would know. Other of the cohort were Lynn Fuchs, her husband, Doug Fuchs, uh, Mark Shinn, uh, Gary Germann, and on and on and on. It was this dream cohort of doctoral students. And what they were all studying together was this idea of creating very short, reliable, valid assessments that teachers could use to both quickly identify kids who might need some academic help and evaluate the effectiveness of that. So that's what Jerry had been studying as a doctoral student. One of those assessments that was created by this amazing team of people was this weird little assessment called Oral Reading Fluency. And, and I remember sitting in that class-I mean, it had just been invented; nobody heard about it; nobody was writing about it; there were some research reports, but most of them were being published just at the University of Minnesota. I mean, it was really, really early on. But Jerry was, you know, all talking about it all the time. This was the key to everything! And I was like, "I'm supposed to be learning how to go out into schools and help teachers, and all you're talking about is assessment?" It didn't quite jibe. And then when he told us about ORF: "So you have a child read aloud from an unpracticed passage. You have them read for 60 seconds. You score their errors. You calculate words correct per minute. And that's this amazing tool." And I raised my-oh, and he said, "Then you take that score, words correct per minute...what do you do with it? Well, what you do with it is you create building-level norms, and then you can find if your student is at the normative benchmark or not." And I just sat there, thinking, "First of all, 60 seconds? Are you kidding me? You know, I'm a reading specialist. I have a master's degree. I've been doing this for a long time."

Susan Lambert: 19:07
Mm-hmm.

Dr. Jan Hasbrouck: 19:08
But I did take some convincing, just to believe in the value of that ORF assessment. And so there could not be, Susan, a bigger skeptic. I know there are skeptics of ORF when I share ORF...or now I'm very rarely sharing something new with people; they've heard of it, or you know, Oral Reading Fluency, DIBELS, you know, Acadience, whatever it is they've heard about it. And they may use it. But people are as skeptical as I was about the value of it. And I get it. I sit there and listen to their skepticism and say, "Yeah, yeah! <laugh> It doesn't make any sense! But I've been using it now for...what would this be? 30-some years. I've done several research studies using it. I've used it as a practitioner. I've used it as a grandmother measuring my own grandson's progress. It works!" That team back at the University of Minnesota essentially invented the tool that we as reading professionals can use as a thermometer. It has reliability. We can trust the score. It gives us consistent scores. It has validity. It gives us good, useful information, and like a thermometer, it's super-quick.

Susan Lambert: 21:11
Yeah. Just talk a little bit about this. I know we have some more things to cover, but just this idea that oral reading fluency, I think you mentioned to me that you think the name itself sort of breeds some misconceptions.

Dr. Jan Hasbrouck: 24:11
I like that too. 'Cause I think that really is what it's measuring and why it's important. But yes, I think it's unfortunate and I spend way too much time in my webinars and trainings talking about what ORF is and how wonderful it is and how it's weird, yes, but you should trust it, but it should never have been called "fluency." Well, I wish they hadn't 'cause we have to backtrack and reexplain and all that.

Susan Lambert: 24:51
Yeah. All right. Well for those of our listeners that wanna attend a webinar with you, I'm sure you'll unpack that ORF measure at another webinar. Or we can do another recording too, if we wanna do that.

Educator 1: 25:15
Hi, I'm a teacher. And what brought me to the Science of Reading was that my students were struggling so much with my guided-reading instruction. It just did not align to what I wanted students to understand. It seemed to have been confusing them. And when I really realized by reading books and researching the Science of Reading, I understood more of why I was confusing my students and why it seemed like I was wasting their time. I was able to understand better ways to educate my students. And now I am the Teacher of the Year in my school district. So thank you to the Science of Reading.

Susan Lambert: 25:49
Well, let's segue a little bit. It was so fun to hear about you going from working with students directly to being extracted from that and doing a little bit of...I would call coaching. So that leads to your new book, co-authored book, called Student-Focused Coaching, recently published. I have it here on my desk. I actually showed it to you. I have my copy all marked up! We'll link our listeners in the show notes to the book. Tell us a little bit about the book and what the genesis for writing that was

Dr. Jan Hasbrouck: 26:21
Oh, yes. I'm happy to do that. Thank you, Susan, for letting me talk about that. That book is a culmination-I mean, I could go back really to that conversation that my boss had when he had that big idea of "somebody should be available to go out and help the teachers." He asked me to take that role. How quickly I figured out that I don't know what to do! I'm terrified, as many coaches-I now have had contact over three decades with many, many coaches, and one of the things we all share is terror of doing this job. Because no matter how skillful and experienced you are, how in the world can we have the chutzpah to walk into a colleague's classroom? Sometimes, if we're a much younger novice coach, they may have been teaching for decades longer than us. They may be teaching a content area that we've never taught before. What do we bring to the table? And what if they ask us a question we can't answer there? It's such a frightening and overwhelming role. So with that terror and being overwhelmed and frightened, I turned to Jerry Tindal as the beginning of my process to "How do I do this job?" And Jerry most importantly gave me the data message. That data has a role, not only in teaching children, but also in working with adults. So I got that message really early on. It's not all about data by any means, but coaching is enhanced by data collection too. But then I also found that there were other people studying this coaching, or there were quite a group of people back then 30 years ago, studying a process called consultation, and that research was happening primarily in special education and school psychology. So what next happened was, Jerry Tindal convinced me to drop my work as a reading coach and come work with him and work on a doctoral program. And the focus of my doctoral program was coaching or consultation. So I spent quite a bit of time reading and studying and ultimately doing some research on the coaching and consultation process. And throughout that time, I was training coaches because another thing that Jerry Tindal was quite renowned for was writing grants. So he had me writing grants and we would use those grants to fund research, but also to bring in students and pay their tuition and have them study under us. So right away in my doctoral program, I was working with people, training them to be coaches, using whatever knowledge I had...and I can tell you back in the day, I hope none of them are listening, but I was <laugh> one step ahead of them, maybe. Maybe one step ahead of them. 'Cause we were trying to figure out what this thing was. What was coaching and consultation? So whatever I had figured out, I would meet with this group-usually young women, but we had different cohorts of people over the year-to talk about what coaching is; how to do it well; what are the challenges; what are the nuances? So that was starting when my doctoral program started in 1986 and that year I was training coaches. So all of that work...and then I ended up, I wrapped up my program with Jerry at the University of Oregon and was hired at Texas A&M University. That was a big shift in every possible way. Culturally, um-

Dr. Jan Hasbrouck: 30:32
Climate change, that's right! Language change. I had to learn a whole new language in a lot of ways, but I became-I got a job as a tenure track professor in educational psychology and had a wonderful opportunity to work part-time in school psychology with Dr. Jan Hughes, who happened to have a grant to study consultation within school psychology. And the other half of that job was in special ed, and I continued to work to write grants and train students in coaching. So those two worlds-I mean, all of that worked together. One of my students in the special-ed half of that work was Carolyn Denton. And she and I wrote the first version, the original version, of the book that you now have. It was about-the publisher that we worked with wanted us to call it The Reading Coach, because at that time, most academic or instructional coaches were working in reading. And that worked very well. Carolyn's background was reading; my background was reading. So we wrote a book, actually we wrote two books, part one and part two, on The Reading Coach. And so that was used very widely during the Reading First years, a lot of Reading First mandated that schools had coaches. There were very few books for coaches. So a lot teachers, a lot of coaches used our book. But then, you know, Reading First was over, and the publisher of those two books decided to not continue to print it. So it was out of print. But I was still doing coaching training here and there. And part of my-at one time that work brought me into contact with Daryl Michel, who is just a wonderful person with a lot of interest and background himself in coaching in schools. And he had read the books that Carolyn and I wrote and was using them in his trainings. So he had a lot of experience with the model that we had created, which is called student-focused coaching. It's a particular model; it's a particular way of offering coaching. Um, so we put our heads together, Daryl and I, and said, "You know, that original book is out of print." I contacted Carolyn. And she said that she had moved on. She didn't really want to go back and do a book on coaching. So Daryl and I rewrote the book. He brought a lot of wonderful, new, fresh ideas to it, but it's also the cumulation of 30 years of work and research and practice. So that's what you, that's what you have in your hand.

Dr. Jan Hasbrouck: 35:07
OK. All right.

Susan Lambert: 35:41
Isn't there?

Dr. Jan Hasbrouck: 35:42
A lot packed in there. Yes. And it rings true to me to this day. That there is a lot of anxiety around assessment. I do think in my experience, my many decades now I've experienced, I think most people who go into teaching are... first of all, you don't go into teaching because you love assessments! "You know, that's why, that's why I wanna go into teaching!" Certainly that's not the reason. I mean, that's laughable. But also the personality type I think that generally gets attracted to teaching is people who are people-people. And they like the relationship-building and they like all of that, and data is something, you know, assessments are something they will use if they have to. But it's not the reason that they get it up and go to work every day. Having worked in school psychology and having been a researcher in a Tier 1 university educational psychology department. I know there are brilliant, wonderful people whose life is all about data and assessments and statistics and that kind of thing. And they couldn't ever imagine themselves teaching in an elementary classroom or something. So there are really different people and there are...so I think a lot of us who've become teachers have an aversion, a fear, a dislike, a distrust of assessments for a lot of reasons. Then there's the issue of-in teaching, we never have enough time. We never have enough time. Or money. Or people to do the work that we want to do. And so assessments get a little-we can feel annoyed about assessments there, too, because it's absolutely right. The most important thing we do is our teaching and we don't have enough time to do it! And then somebody usually tells us we have to take some of our time, or a lot of our time, to do assessments. And if those assessments aren't really carefully selected, and if those assessments are...if we are not well-trained in both how to do the assessment, but how to analyze the data and then what to do with that data, then it's wasted time. SoThat's what I'm seeing in school. That administrators are probably feeling pressure on themselves. They're told you need to do these assessments they don't fully understand. Nobody's taken the time to explain to them what these assessments are; how to use them; what does it mean? And even at the highest leadership level that's happening. So the culmination of that is layer upon layer upon layer of assessments. It's very rare that someone goes in and says, "OK, we're not gonna do that assessment anymore; we've got this better, more strategic, more useful assessment. We're gonna do this one instead." What's often happening, too often happening, is assessments get layered on each other. So we're doing lots of assessments. We're doing assessments that were not well-designed. I mean, going back to that weird little ORF: I didn't like it when I first heard about it, but I now love it. And I love it in part, because it is short, because it does have those psychometric properties that you have to have in order to believe and have an assessment be useful. Reliability and validity. And it is 60 seconds. I love that about it. And some of these assessments take 45 minutes, an hour and a half, days of observational assessments that some kindergarten teachers are having to do! It's just...and then at the end, what do I do with this data? So I really do understand that concern about-yeah, there is anxiety. We are probably, in most schools, I would say, doing too much testing that overlaps, repeats itself. We're using assessments that are much longer than they need to be. And our attempt to fix all that really is... Let's just, first of all, take a deep breath. Let's all have a kumbaya moment. Link arms. And agree that we don't have enough time. We don't have enough money. We don't have enough people. OK. And with that precious time and money and people, we should be doing the best instruction possible. We all agree on that, but in order to do that best assessment possible, we really do it best if we have some data. Just, just like a physician. We have these amazingly trained physicians who know all this stuff, but if you said, "Go in and take care of this patient, but you spend any time-you only have to take care of the patient, but you can't spend any time collecting data; you just have to do the care, do the surgery, give them some medicine." A physician can't do that, right? <laugh>

Dr. Jan Hasbrouck: 47:29
I know!

Susan Lambert: 47:29
She is one of my favorite people in the world!

Dr. Jan Hasbrouck: 47:31
She is amazing! And she has said she read-um, I was writing a new version, an updated version, of that book, and she expressed some interest in assessment, which not all teachers do. So I asked her to read an early draft of the book. And I said to her, "Based on your reaction, I may ask you to come on as a co-author." And she agreed to do that. So it's very much in the background, because she's full-time teaching, working as a reading coach this year, and I'm busy too. But today I was working on a webinar based on that concept. And the webinar is about educators as physicians. And it's exactly your a-ha moment: that we are responsible for the academic health and wellness, as well as the behavior and social-emotional wellness, of our children, just like a physician is responsible for the physical, and depending on their field, the psychological wellness. In order to do that well, we do need to collect data. And just like a physician, it would be malpractice if you come in with a sore throat that they say, "OK, let's do a CAT scan of your spine; let's do a biopsy of your liver; let's do-like what?! No! Let's not collect too much data! I've got a sore throat. Why don't you examine my throat and maybe take a swab? You know, so not too much data, but some data, and the right data. Use that to make a determination of what should we try. Let's try some medicine. Let's try some antibiotics. And your statement that, OK, I tried the antibiotics I've taken 'em for three weeks or whatever. I come back and say, <cough> "Still sore!" We don't say, "OK, Doctor, we take away your medical license! Clearly you have failed!" No! In medicine, the human body, getting the human body well is very complex. Getting the brain to learn to read is very complex. We try some things. Sometimes we're right. Sometimes we're a little bit, but we need to do it differently, Or more. That's what progress-monitoring data can help us do. But it is brave, 'cause it's a new question for teachers. It's it is part of the training that physicians get. They have to sit around with their colleagues and their teachers and do case studies and grand rounds. And they are grilled when they say, "OK, the patient presented with a sore throat and I did these diagnostics." And everybody says, "Why did you do those diagnostics? Why didn't you do different diagnostics?Why didn't you do this?" They're grilled upon that. And then they say, "OK." And then:"I prescribed this," and everybody jumps in: "Why didn't you prescribe this? Why didn't you prescribe-" They're, from the beginning of their work, experiencing that kind of-for good professional reasons! not to chop people down and make them feel bad, but to deepen their learning and to test their own thinking and to hear new ideas. I think that's a potential for PLCs that are happening in school.

Dr. Jan Hasbrouck: 51:04
If we brought our student data and said, "I'm doing the best I can, but this student is not making progress." Not "I'm a failure," but. "Help me, colleagues, to think about this differently. Should I have administered different assessments? Are there some interventions that I should be doing that I haven't thought of yet?" Or "I can only see this student twice a week for 20 minutes. Is there something else the group as a whole, the system as a whole, can we teach him? Can we find two more 15-minute or 20-minute slots during the week?" There's all kinds of things that we should do that mirrors-not to say that the medical world is perfect and has answered every question by any stretch of the imagination. But I think there's something about that analogy and some things about the way they've done their work that can be helpful to us.

Susan Lambert: 52:07
Yeah. That's great. Well, so we're, we're gonna look forward to another Jan book coming, sometime.

Dr. Jan Hasbrouck: 52:14
Jan and Margaret, I'm hoping!

Dr. Jan Hasbrouck: 52:23
Hmm.

Susan Lambert: 52:24
Tough one!

Dr. Jan Hasbrouck: 55:17
It's been my honor, and a great deal of fun too, <laugh> to walk back through some of the big moments of my long professional career.

Susan Lambert: 55:31
Thanks for listening, and keep your feedback coming. Want to learn more? Be sure to stay connected by subscribing to your favorite podcast app. And join our Facebook discussion group, Science of Reading: The Community. Don't forget to register for our new webinar series,The Science of Reading Is for Everyone. Over the next few months, experts including Natalie Wexler, Dr. Jan Hasbrouck, and Dr. Carolyn Strom will discuss how the Science of Reading can help all students. Educators like you will talk about their journeys with the Science of Reading and why making the change is worth it.

# data[].attributes.attachment_link.title
S5-E3: The right assessment and the right data with Dr. Jan Hasbrouck - Science of Reading: The Podcast

# data[].attributes.attachment_link.description
Today on the podcast, we're joined by literacy expert Jan Hasbrouck, Ph.D. Dr. Hasbrouck is an education consultant, author, and researcher. She opens the episode talking about her start with literacy, underscoring how she was one of the lucky one...

# data[].attributes.headline
Screening vs. Diagnostic Assessment - Dr. Stephanie Stollar

# data[].attributes.subject
Screening vs. Diagnostic Assessment - Dr. Stephanie Stollar

# data[].attributes.attachment_link.readable_text
Add to favorites
Written by Una Malcolm, M.A., OCT
Introduction: Why Assess

Assessment for learning plays a critical role in informing and driving literacy instruction. Nearly all students are capable of learning to read; while specific numbers vary, estimates indicate that approximately 95% of students are capable of learning to read (Fletcher & Vaughn, 2009). Rigorous, systematic, and explicit instruction that addresses the essential components of literacy including phonemic awareness, phonics, fluency, vocabulary and comprehension (National Reading Panel, 2000), is key to allowing all children to reach their full potential. This instruction often referred to as structured literacy, must go hand-in-hand with a comprehensive assessment system to allow educators to adjust instruction to meet the specific needs of students (Spear-Swerling, 2018). Assessment is critical for students with learning disabilities (LDs) in terms of both the prevention of reading difficulties and intervention to remediate skill gaps.

Formative Assessment: Answering Questions

Assessment for learning plays a key role in meeting the needs of all students through the creation of equitable learning environments. Assessment for learning, "is used in making decisions that affect teaching and learning in the short-term future" (Growing Success, 2010, p. 30). Multiple research reviews and meta-analyses have identified that systematic formative assessment has consistent strong, positive effects on student learning (Fuchs & Fuchs, 1986; Black & Wiliam, 2005; Hattie, 2009) across a variety of factors including student age, measurement frequency, and disability status. In essence, assessment for learning allows educators to identify where students are and inform plans to get students where they need to be.

Even within assessment for learning, multiple types of assessments exist. Growing Success (2010) distinguishes between formative and diagnostic assessments but emphasizes that "what matters is how the information is used" (p. 31). Assessment answers questions; different types of assessment tools answer different types of questions. In a Response to Intervention model, assessment purposes include universal screening, diagnostic assessments, and progress monitoring; all three of these purposes for assessment answer key questions about shaping instructional environments to meet students' needs.

1. Screening - Assessment for Learning

The development of skilled reading is a complex process that hinges on the development and the integration of cognitive and language skills. Many of these skills can be observed and measured well before children receive any formal reading instruction (Catts, 2015). With data indicating that reading trajectories are established early and are very stable (Juel, 1988), it is critical to identify reading difficulties early on in a student's academic career to inform support. Since reading remediation at an earlier age is more effective than later intervention (Lovett et al., 2017), screening plays an important role in preventing early reading difficulties from being exacerbated over time.

All students should be screened with universal screening measures three times per year (Good et al., 2011). These assessments are very brief, standardized measures that quickly and efficiently identify which children are at risk. Standardized assessments are tests that have specific procedures for administration and scoring, perhaps with scripted directions and specific scoring guidelines (Farrall, 2012). Students are assessed on several indicators of early literacy skills appropriate for their age and grade. For example, Kindergarten and grade one students may be screened on phonemic awareness, phonics, and decoding, while older students may be screened on oral reading fluency (ORF) and retell for comprehension.

Universal screening measures are widely available and very accessible. Many of the universal screeners currently available are either low-cost or free. Often there can be a fee for training or optional computer data analysis systems, but many of the testing materials are very accessible.

These assessments are technically reliable and valid, and scores are interpreted in relation to a benchmark criterion. Benchmark cutscores are not determined arbitrarily but instead are derived from research; researchers use longitudinal studies to determine what threshold a child must reach to have a strong chance of meeting future reading goals. This means a child's score allows educators to predict future reading performance with a reasonable degree of accuracy. Students who fall at or below the cutscore are at risk and will likely need additional support to allow them to meet future goals. For example, a benchmark cutscore on Acadience Reading assessments is the score needed for a child to have an 80 - 90% probability of meeting successive reading goals (Good et al., 2011). In interpreting a child's score in comparison to a benchmark score, educators can quickly and efficiently assess the level of risk of future reading achievement or difficulty before difficulties emerge or worsen.

Screening allows educators to quickly and easily catch students who are at risk. Screening does not, however, give teachers a full understanding of what the underlying cause of a problem is. Screening is akin to a blood pressure or temperature check at a doctor's office; these are simple, efficient and cost-effective ways to find problems, but abnormalities on these screeners require additional diagnostic testing to decipher what exactly is wrong.

2. Diagnostic Assessment - Assessment for Learning

Diagnostic assessments allow educators to probe deeper into the learning profiles of students in question. For many students, universal screening three times per year provides all the assessment data necessary to ensure students are making sufficient progress. For students who are at risk, though, careful analysis of reading subskills must occur to differentiate instruction and plan intervention. These assessments allow educators to ascertain which specific skills or knowledge a student has mastered, and which skills or knowledge need to be taught.

If universal screeners are similar to a blood pressure or temperature check to identify problems in a brief and cost-effective manner, diagnostic assessments are similar to more comprehensive blood tests or diagnostic imaging to dig deeper into a problem. The information from these more investigative approaches directly informs treatment, similar to how diagnostic reading assessments allow teachers to differentiate classroom instruction or plan targeted interventions.

Diagnostic assessments are longer and more in-depth than screening assessments and are typically not standardized. While screeners are quick assessments of indicators of basic skills, diagnostic assessments can be thought of as inventories of skills that directly inform instruction. Can a child segment two, three, or four phoneme words? Can a child decode words with short vowels or consonant digraphs? Diagnostic assessments allow teachers to differentiate classroom instruction and plan intervention if necessary.

Diagnostic assessments should be selected strategically. The Simple View of Reading (Gough & Tunmer, 1986) indicates that reading comprehension, the end goal of reading, is a product of two sets of subskills, word recognition and language comprehension. When selecting diagnostic assessments, teachers can probe both areas of this model to clarify a student's specific patterns of strengths and needs. When weaknesses exist in reading comprehension, is this due to difficulties with word recognition, with language comprehension, or with both?

3. Progress Monitoring - Assessment for Learning

With screening and diagnostic data in hand, teachers can differentiate core instruction and/or provide targeted intervention through intervention supports. Progress monitoring assessments allow educators to quickly measure a student's response to instruction. Instead of waiting until the next benchmark assessment several months in the future, progress monitoring allows for more rapid adjustment of instruction. While universal screening measures students' skill level with grade-level material, progress monitoring is at an individual child's instructional level to directly measure response to instruction. A grade four child reading at a grade one level would be screened at the beginning, middle, and end of the year with grade four assessments. Progress monitoring assessment, though, would be at a grade one level in order to measure the child's response to intervention.

Many universal screeners also supply progress monitoring materials. These materials are often very similar to universal screening measures in that they are brief and standardized. When selecting an assessment for progress monitoring, it is important to consider practice effects. Progress monitoring assessments typically have multiple forms so students are not tested with the same material. Progress monitoring data are usually graphed so student growth can be compared visually to an aimline, or the line between a child's current score and the benchmark goal.

Progress monitoring assessments are purposefully brief. Depending on student age and needs, this assessment may take between one and three minutes. Struggling students need to catch up quickly; Torgesen (2004) describes the "devastating downward spiral" that occurs when students do not receive timely, evidence-based intervention. Struggling readers do not have time for a "wait and see" approach. Weekly or biweekly progress monitoring allows for enough data to quickly and clearly establish patterns. Educators can quickly judge if a child's rate of progress is strong enough to meet the goal. If not, there are several options that can be considered to intensify instruction:

Increasing frequency or length of support

Decreasing group size

Making groups more homogeneous

Increasing level of explicitness

Increasing opportunities for practice

To see an example of how the Thunder Bay Catholic DSB uses tier 1, 2 and 3 reading interventions, click here to view the LD@school video The Tiered Approach

In addition to informing instruction, progress monitoring data allow for insight into students learning profiles. Research indicates that data from a child's response to intervention can give important insight into whether a child may have a learning disability. Patterns of inadequate response to evidence-based instruction, established through progress monitoring data, are key factors to consider when evaluating for a learning disability (Miciak & Fletcher, 2020; Catts & Petscher, 2021).

4. Outcome Evaluation - Assessment of Learning

Outcome assessments measure student achievement. While progress monitoring assessments show if instruction is working, outcome assessments are assessments of learning and identify if instruction worked.

These assessments are typically comprehensive assessments that measure student mastery of provincial curriculum expectations or the specific learning goals outlined. Outcome assessments often occur at the end of a period of learning, including a unit of study, a term, or a school year, for example.
Evidence-Based Assessment in the Science of Reading: Cheat sheet - Comparing and Contrasting Assessment Purposes Click here to view and download the Evidence-Based Assessment in the Science of Reading: Cheat sheet
Assessment Data and Differentiating Tier 1 Instruction

Patterns in universal screening data can support teachers in differentiating classroom instruction. Instead of using student reading levels from a running record to group students for small group instruction, universal screening data allows educators to use reading skills to more specifically build homogeneous groups of students for instruction. A grade 2 teacher, for example, could split students into several groups based on their universal screening data:

Fluent and accurate readers needing continued content and vocabulary instruction to build comprehension

Students who read accurately but need fluency instruction

Students in need of word-level decoding and phonics instruction.

Data from screeners as well as diagnostic testing for deeper probing allows for more sophisticated and specific differentiation for intervention, instruction, or enrichment since it focuses on specific skills instead of generic reading levels.

Student vs. System Analysis to Strengthen Universal Instruction

One of the strengths of universal screening is it allows educators to analyze not only individual students' growth but also the strength of classroom core instruction. If more than 20% of students are below the benchmark on universal screening, it is a clear indication that efforts to strengthen core instruction should be made. Prioritizing intervention when a large proportion of the class is at risk misses out on a valuable opportunity to strengthen the universal tier of classroom instruction. With class-wide screening data, teachers are able to better differentiate instruction and potentially implement class-wide interventions, such as Peer-Assisted Learning Strategies (McMaster & Fuchs, 2016). For example, a grade 3 teacher could implement a paired repeated reading instructional routine (Stevens et al., 2017) in response to weak fluency screening data. Instead of just problem-solving at individual student levels, teachers have the data necessary to develop action plans to strengthen core instruction for all students.

To learn more about Peer Assisted Learning, click here to access LD@school's article and video Peer-Mediated Learning Approaches.

Oral Reading Fluency and Running Records: Frequently Confused

Oral Reading Fluency assessments are commonly used in both screening and progress monitoring for developing readers. While individual assessments vary, generally Oral Reading Fluency (ORF) measures involve timed passage oral reading for one minute. Accuracy rates and words correct per minute scores are calculated and interpreted in relation to a benchmark score. ORF assessments are highly correlated with reading comprehension (Fuchs et al., 2001) and serve as a proxy for this critical outcome skill.

ORF assessments are often confused with running record assessments that are commonly used in literacy programs. While both assessments do involve the reading of connected passages, running records differ in that a text "level" is often indicated as a student's instructional or independent level based on the accuracy percentage. Fluency is typically assessed through teacher observation or rubrics. Running records do not meet desired characteristics for the selection of a screener. They do not act as brief, reliable and valid indicators of key early literacy skills, and they do not allow for interpretation of scores in relation to a benchmark criterion derived from predictive probabilities. In fact, Parker et al. (2015) found that a commonly used running record assessment had a .54 correlation with a district-provided criterion reading test. These data indicate that running records were only 54% accurate in predicting student reading performance, in comparison to the established technical validity and reliability of ORF measures.

Assessment for Learning: A Case Study
The following case study provides an example of how reading assessment can be used in the classroom to identify at-risk students, pinpoint the skills these students struggle with, and determine the appropriate intervention to get students back on track.
Maya is in grade two. As a part of the beginning-of-year screening process, her teacher administers a brief screening assessment that yields this data about Maya's reading:
Nonsense Word Fluency: Oral Reading Fluency:

Accuracy (passage reading accuracy): Below Benchmark

Words Correct (passage reading fluency): Well Below Benchmark

Maya's teacher, Ms. Kaur, recognizes that Maya is struggling and is not on track to meet future reading goals. Ms. Kaur realizes that Maya is having difficulty reading the ORF passage accurately and smoothly since she is not an accurate and fluent word reader. Based on these screening data, she decides to dig deeper with some diagnostic assessments to help her differentiate instruction. She decides to give Maya a decoding assessment to understand which specific phonics patterns she has mastered, and which patterns she needs to be taught. She learns that Maya hasn't yet mastered short vowel sounds. She also uses a phonemic awareness assessment to check her blending and segmenting, which shows skill gaps as well. Ms. Kaur realizes that Maya is having difficulty with word recognition skills. To check on her oral language skills, Ms. Kaur reads a story to Maya and asks her oral comprehension questions. Maya is easily able to answer questions about what she hears, indicating to Ms. Kaur that Maya's reading difficulties appear to be stemming from decoding and word recognition difficulties and not language comprehension difficulties.

Armed with this information, Ms. Kaur builds her small reading groups. Instead of using reading levels to group students, Ms. Kaur uses reading skills. She groups Maya with other students needing phonemic awareness and short vowel instruction.

As Ms. Kaur uses this screening and diagnostic data to plan her groups and instruction, she also continues to monitor Maya's progress using weekly Nonsense Word Fluency assessments. Since these progress monitoring assessments are one minute each, Ms. Kaur knows that this is a quick and efficient way to make sure Maya is responding to her instruction without having to wait until the mid-year screening time. If Maya is not making sufficient progress toward the benchmark score, she may need instruction to be intensified to allow her to reach the benchmark goal. Ms. Kaur graphs Maya's progress monitoring data, as well as the aimline which shows the path between Maya's current score and the benchmark. Ms. Kaur and her grade-level team can quickly and easily judge if Maya is progressing at a fast enough rate to close the gap.

About the author:

Una Malcolm is a doctoral student in Reading Science and also has a Master's degree in Child Study and Education. She is a member of the Ontario College of Teachers. Una has trained extensively in several evidence-based teaching methods, such as Direct Instruction, Phono-Graphix and Lindamood-Bell. She is an Acadience Reading K-6 (formerly DIBELS Next) Mentor.

References

Black, P. & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7 - 74. doi: 10.1080/0969595980050102

Catts, H.W., Nielsen, D. C., Bridges, M. S., Liu, Y. S., & Bontempo, D. E. (2015). Early identification of reading disabilities within an RTI framework. Journal of Learning Disabilities, 48(3), 281-297.

Catts, H.W., Petscher, Y. (2021). A cumulative risk and resilience model of dyslexia. Journal of Learning Disabilities, doi.org/10.1177/00222194211037062

Eunice Kennedy Shriver National Institute of Child Health and Human Development, NIH, DHHS. (2000). Report of the National Reading Panel: Teaching Children to Read: Reports of the Subgroups. U.S. Government Printing Office.

Farrall, M. L. (2012). Reading assessment: Linking language, literacy, and cognition. John Wiley & Sons, Inc.. https://doi.org/10.1002/9781118092668

Fuchs, L.S. & Fuchs, D. (1986). Effects of systematic formative evaluation: A meta-analysis. Exceptional Children, 53(3), 199 - 208.

Fuchs, L., Fuchs, D., Hosp, M. & Jenkins, J. (2001). Oral reading fluency as an indicator of reading competence. Scientific Studies of Reading, 5(3), 239 - 256.

Fletcher, J.M. & Vaughn, S. (2009). Response to intervention: Preventing and remediating academic difficulties. Child Development Perspectives, 3(1), 30 - 37. doi: 10.1111/j.1750-8606.2008.00072.x

Good, R.H., Kaminski, R.A., Cummings, K.D., Dufour-Martel, C., Petersen, K., Powell-Smith, K.A., Stollar, S. & Wallin, J. (2011). Acadience Reading K-6 Assessment Manual. Acadience Learning Inc.

Gough, P.B. & Tunmer, W.E. (1986). Decoding, reading, and reading disability. Remedial and Special Education, 7(1), 6 - 10. https://doi.org/10.1177/074193258600700104

Hattie, J. (2009). Visible learning. Routledge.

Juel, C. (1988). Learning to read and write: A longitudinal study of 54 children from first through fourth grades. Journal of Educational Psychology, 80(4), 437 - 447. https://doi.org/10.1037/0022-0663.80.4.437

Lovett, M., Frijters, J.C., Wolf, M., Steinback, K.A., Sevcik, R.A., Morris, R.D. (2017). Early intervention for children at risk for reading disabilties: The impact of grade at intervention and individual differences on intervention outcomes. Journal of Educational Psychology, 109(7), 889 - 914. https://doi.org/10.1037/edu0000181

McMaster, K. L., & Fuchs, D. (2016). Classwide intervention using peer-assisted learning strategies. In S. R. Jimerson, M. K. Burns, & A. M. VanDerHeyden (Eds.), Handbook of response to intervention: The science and practice of multi-tiered systems of support (pp. 253-268). Springer. https://doi.org/10.1007/978-1-4899-7568-3_15

Miciak, J. & Fletcher, J.M. (2020). The critical role of instructional response for identifying dyslexia and other learning disabilities. Journal of Learning Disabilities, 53(5), 343 - 353. https://doi.org/10.1177/0022219420906801

Ontario Ministry of Education. ​​(2010). Growing Success: Assessment, evaluation and reporting in Ontario's schools: Covering grades 1 to 12. Ministry of Education.

Parker, D.C., Zaslofsky, A.F., Burns, M.K., Kanive, R., Hodgson, J., Scholin, S.E. & Klingbeil, D.A. (2015) A brief report of the diagnostic accuracy of oral reading fluency and reading inventory levels for reading failure risk among second and third grade students. Reading & Writing Quarterly, 31(1), 56 - 67. doi: 10.1080/10573569.2013.857970

Spear-Swerling, L. (2018). Structured literacy and typical literacy practices: Understanding differences to create instructional opportunities. Teaching Exceptional Children, 51(3), 201-211.

Stevens, E. A., Walker, M. A., & Vaughn, S. (2017). The effects of reading fluency interventions on the reading fluency and reading comprehension performance of elementary students with Learning Disabilities: A synthesis of the research from 2001 to 2014. Journal of Learning Disabilities, 50(5), 576-590. doi: 10.1177/0022219416638028

Torgesen, J.K. (2004). Avoiding the devastating downward spiral: The evidence that early intervention prevents reading failure. American Educator, 28, 6-19.

# data[].attributes.attachment_link.title
Evidence-Based Assessment in the Science of Reading

# data[].attributes.attachment_link.description
Written by Una Malcolm, M.A., OCT Introduction: Why Assess Assessment for learning plays a critical role in informing and driving literacy instruction. Nearly all students are capable of learning to read; while specific numbers vary, estimates indicate that approximately 95% of students are capable of learning to read (Fletcher & Vaughn, 2009).

# data[].attributes.headline
Dr. Don Compton - Screening All Students

# data[].attributes.subject
Dr. Don Compton - Screening All Students

# data[].attributes.attachment_link.readable_text
All Resources | Expert Interview

Sign in to share, email, rate, comment or favorite.

In this interview, Dr. Donald Compton explains the purpose of universal screening, describes how it fits into a multi-tiered intervention system, and discusses using screening to identify students at risk for reading problems and place them into appropriate interventions.

# data[].attributes.attachment_link.description
All Resources | Expert Interview Sign in to share, email, rate, comment or favorite. In this interview, Dr. Donald Compton explains the purpose of universal screening, describes how it fits into a multi-tiered intervention system, and discusses using screening to identify students at risk for reading problems and place them into appropriate interventions.

# data[].attributes.attachment_link.readable_text
As a research lab, we do not provide clinical care, such as screening and diagnosis of dyslexia or other language-based learning disabilities. We acknowledge the importance of screening and early intervention and have provided helpful screening tools and advice.

If you are interested in presentations related to reading development, dyslexia, or screening for early literacy milestones, please visit our Events page.

# data[].attributes.attachment_link.title
Screening for Reading Impairments - Gaab Lab

# data[].attributes.attachment_link.description
As a research lab, we do not provide clinical care, such as screening and diagnosis of dyslexia or other language-based learning disabilities. We acknowledge the importance of screening and early intervention and have provided helpful screening tools and advice. If you are interested in presentations related to reading development, dyslexia, or screening for early literacy milestones, please visit our Events page.

# data[].attributes.attachment_link.description
This white paper summarizes changes to DIBELS measures and provides evidence to support the use of early screening tools such as DIBELS to screen for dyslexia and help educators intervene before significant reading difficulties materialize, thus reducing the number of students characterized with learning disabilities.

# data[].attributes.attachment_link.readable_text
We're excited to announce the launch of the Acadience Help Center!

Find the answers to your questions about Acadience assessments, data services, and more with our new searchable Help Center.

# data[].attributes.attachment_link.title
Unlocking Potential. Changing Lives. | Acadience Learning

# data[].attributes.attachment_link.description
Acadience Learning exists for a single reason: to improve outcomes. We help realize the potential of every student, every teacher, and every school.

# data[].attributes.attachment_link.readable_text
The authors of IDAPEL® at Acadience Learning are pleased to announce the continued research release of IDAPEL for research partners for the 2022-2023 school year. IDAPEL is a series of French-language formative measures designed to assess the early literacy skills of students learning to read in French in grades K through 5. IDAPEL assesses the fundamental early reading skills of phonemic awareness, alphabetic principle, accuracy and fluency with connected text, and text comprehension.

As diagnostic tools, the IDAPEL measures are sensitive to student growth over time, and facilitate the early identification of students who may be at risk of insufficient reading gain. The measures are time efficient and easy to use, and invaluable for progress monitoring student skill, and for evaluating student response to intervention.

The measures are currently being used with French-language native speakers and students enrolled in French-language immersion programs. Preliminary benchmark goals have been established for native French-speaking students learning to read in French as their native language. We continue to analyze data collected with French immersion students for the purpose of establishing benchmark goals for this population of students.

IDAPEL is available to participating research release partners. The IDAPEL assessments materials are available to participating sites for free download through the Acadience Learning website. Participating sites are responsible for printing copies of the testing materials as needed.

# data[].attributes.attachment_link.title
IDAPEL Research Release | Acadience Learning

# data[].attributes.attachment_link.description
The authors of IDAPEL® at Acadience Learning are pleased to announce the continued research release of IDAPEL for research partners for the 2022-2023 school year. IDAPEL is a series of French-language formative measures designed to assess the early literacy skills of students learning to read in French in grades K through 5.

# data[].attributes.attachment_link.readable_text
This manual includes information about DIBELS 8, how DIBELS 8 differs from previous editions of DIBELS, how to administer and score DIBELS 8 subtests, and how to use DIBELS 8 data to inform instructional decision-making.

# data[].attributes.attachment_link.description
This manual includes information about DIBELS 8, how DIBELS 8 differs from previous editions of DIBELS, how to administer and score DIBELS 8 subtests, and how to use DIBELS 8 data to inform instructional decision-making.

# data[].attributes.attachment_link.title
aimswebPlus & aimswebPlus Canadian Edition

# data[].attributes.attachment_link.description
aimswebPlus screens and monitors the reading and math skills of PreK–12 students. With its robust set of standards-aligned measures, aimswebPlus is proven to uncover learning gaps quickly, identify at-risk students, and assess individual and classroom growth. aimswebPlus also…

# data[].attributes.attachment_link.readable_text
Start measuring progress now. No strings attached. No credit card required.
Get Started Now
Teacher Deluxe

Start using the full version now. Only $49.99 per year! Restricted to 200 students / one teacher per building. (If you exceed these numbers, check out the District version to better meet your needs!)

Subscribe Now

# data[].attributes.attachment_link.description
Start measuring progress now. No strings attached. No credit card required. Get Started Now Start using the full version now. Only $49.99 per year! Restricted to 200 students / one teacher per building. (If you exceed these numbers, check out the District version to better meet your needs!) Subscribe Now

# data[].attributes.attachment_link.description
FastBridge's valid and reliable assessments help educators identify students' academic and social-emotional behavior (SEB) needs faster, align the right interventions at the right time, and measure whether interventions are helping students catch up-all in one platform and in up to half the test time.

# data[].attributes.headline
Accuracy of student performance while reading leveled books rated at their instructional level by a reading inventory

# data[].attributes.subject
Accuracy of student performance while reading leveled books rated at their instructional level by a reading inventory

# data[].attributes.attachment_link.description
www.researchgate.net needs to review the security of your connection before proceeding.

# data[].attributes.headline
Fountas and Pinnell Benchmark Assessment System: Doesn't Look Right, Sound Right, or Make Sense

# data[].attributes.subject
Fountas and Pinnell Benchmark Assessment System: Doesn't Look Right, Sound Right, or Make Sense

# data[].attributes.attachment_link.title
Fountas and Pinnell Benchmark Assessment System: Doesn't Look Right, Sound Right, or Make Sense | Reading Rockets

# data[].attributes.attachment_link.description
My district, like so many others, uses Fountas and Pinnell's Benchmark Assessment System (BAS) as the primary measure of student reading progress. But if you've had doubts about the assessment and the ways BAS data is often used, you're on to something...

# data[].attributes.attachment_link.title
Becoming a Literacy Leader Workshop Series

# data[].attributes.headline
Jamey Peavler - Maximizing the Benefits of Small Group Instruction

# data[].attributes.subject
Jamey Peavler - Maximizing the Benefits of Small Group Instruction

# data[].attributes.headline
Dr. Timothy Shanahan - Do Learning Centers and Seatwork Improve Reading Achievement?

# data[].attributes.subject
Dr. Timothy Shanahan - Do Learning Centers and Seatwork Improve Reading Achievement?

# data[].attributes.attachment_link.readable_text
I am an elementary literacy coach. A trend I am seeing in our K-2 classrooms are center activities not aligned to measurable outcomes. My question is, in a room of 24 first graders, when the teacher is pulling a small group to deliver targeted instruction, what does research say is best for what the other students to be doing? I'm struggling to find a model that we can confidently start driving towards. I am often asked about what the "other students" should be doing while teachers meet with small groups. I refer to What Works Clearinghouse studies to see gains for different programs and approaches. I can't find anything that quantifies students' reading improvement for independent reading vs. a computer program. For example, teachers will tell me they "put" students on a computer program daily for 20 minutes while they work with a small group because the teacher likes the program and thinks it is effective. Some of these programs do show decent gains on the studies on What Works Clearinghouse, yet I'm not a fan of computer-assisted reading programs. I struggle to respond to that because I can't find research support that refutes it over gains from independent reading. Can you help me to explain to teachers the difference or benefit to having students read independently over being put on a computer program if there is one?
Shanahan responds:

Wow! Thanks a lot, ladies.

You've used up my weekend in researching an answer to these questions.

No, I don't mean I've been reading the research on seatwork (okay, I did a bit of that).

But there are an amazing number of websites that promise they can teach you how to say, "I don't know" and still sound intelligent.

Unfortunately, they counsel that the dummy who doesn't know should offer to research the question.

My problem is that I've researched it and still don't know the answer. (Do I still sound intelligent?)

There isn't much research on this topic.

But go to Amazon, type in "learning centers," and an plethora of books pops up. They are all great-apparently; their blurbs describe them as "effective" and say they "really work." (What they work at is not clear from the blurbs.)

There are some studies on seatwork, particularly in mathematics. This neglect is surprising given the vast amounts of classroom time devoted to seatwork. One study found that kids spend as much as 70% of their instructional time on their own, which varied by subject area (Fisher, et al., 1978); and more recent studies have produced similar results (NICHD Early Child Care Research Network, 2005; Vaughn, et al., 2002).

The studies that do exist tend to evaluate the effectiveness of seatwork, rather than pointing out successful seatwork routines that increase learning.

For example, back in the 1980s Paul Sindelar and his colleagues tested different amounts of direct teacher guidance and seatwork activities. They devoted 25%, 50%, and 75% of the time to teacher-led instruction and found that the more time with the teacher the better the kids did at learning math. And, the opposite-the more seatwork, the less learning.

Is it a good idea to have kids spending so much time working away from the teacher?

According to the studies, no.

Studies report that kids are less likely to be engaged in learning when working on their own (Cohen, 1994; Cowen, 2016; Gump, 1967; Kouno, 1970); and amount of seatwork has been found to be negatively related to learning (Seifert & Beck, 1984).

One study even reported that the best readers did reasonably well with seatwork and other independent activities, but lower readers learned substantially less from such activities (Connor, Morrison, & Petrella, 2004). They needed the teacher time.

The remaining research on seatwork focuses less on what kinds of assignments, centers, or programs lead to the most learning, and more to how teachers might manage seatwork better so that the kids won't be as disruptive. In other words, their goal is identifying which routines keep kids busiest, not which ones teach best.

What these studies are all saying is that seatwork should be thought of not as a productive part of the school day, but as a necessary evil.

When it comes to academics, kids simply don't learn that much on their own-except perhaps for the ones who are really good learners anyway-in which case, seatwork could be characterized as an effective way to make sure the lower kids don't catch up!

The second questioner above hoped that I could point out research showing that independent reading was better than computer work. I know of no studies that make that comparison, but generally independent reading tends to have a pretty small impact on reading achievement. Seatwork studies suggest that the more learning comes when kids are interacting with others; not working by themselves-even when that self-time might be reading on their own.

It is possible that there are benefits to working with digital programs, since they provide some interaction (and some do have supporting research). However, even the best of those programs tend to require lots of teacher involvement if they are to make a learning difference-though that isn't always how they are used in classrooms.

I do like the idea of having kids read during their downtime, but I would link this closely with their reading instruction. Thus, I might have the kids reading something silently prior to coming to group, so that they are ready to discuss; or I might have kids follow up their reading with me, by practicing fluency with a partner (which can help if the texts are hard enough and the kids stay on task). And, either of these activities could involve a social studies or science book, not just the "reading books," which might take advantage of some of the slack time in those classes.

Research is very supportive of cooperative learning groups, so you might have some luck combining that with project learning activities, but cooperative learning requires some real knowledge. You'd have to study up on that if you wanted to really be successful.

Writing is another activity that can fill such space. Perhaps doing a writing lesson prior to meeting with reading groups, and then having kids working on their compositions while you work with groups would be useful (again, as long as they stay on task).

Overall, my message would be to minimize this kind of independent work, seatwork, center time, or on-one's-own computer assignments. It's better for the kids to work with the teacher and to work with each other.

Brush up your skills in working with larger groups including how to maximize student-to-student and student-teacher interactions and choral or every-student-responds responses.

Seatwork is like homework; it is best focused on applying what students already have learned.

When you do meet with small groups, try staggering the schedules-talking with one while the other reads, then moving to the other group to interact with them, while the first group is annotating. Or, if you meet with groups in a particular order, always give yourself 5-10 minutes between groups to interact with the kids who have been doing seatwork on their own.

I'd love to tell you that there are terrific research-supported workbooks, seatwork activities, computer programs, stations, or learning centers. There just aren't. Given that practitioners need to be experimental, trying out various routines to evaluate their impact on learning and behavior.

Those books I noted earlier might be a good place to start. But wherever the ideas come from for your seatwork routines, you should be skeptical and data-oriented in evaluating their effectiveness and usefulness, and willing to try lots of alternatives.

Or, I could just say, "That's a really interesting question."

# data[].attributes.attachment_link.title
Learning Centers and Seatwork and Reading Achievement | Shanahan on Literacy

# data[].attributes.attachment_link.description
Teachers are often told to schedule small group instructional time. The idea is that this will force differentiation and higher reading achievement. In reality, it increases repetition and entails the use of lots of approaches not found to help reading achievement.

# data[].attributes.headline
Geva - Psychological Assessment of Culturally and Linguistically Diverse Children and Adolescents

# data[].attributes.subject
Geva - Psychological Assessment of Culturally and Linguistically Diverse Children and Adolescents

# data[].attributes.attachment_link.authors[].name
Esther Geva PhD (Author), Judith Wiener PhD (Author)

# data[].attributes.attachment_link.title
Psychological Assessment of Culturally and Linguistically Diverse Children and Adolescents: A Practitioner’s Guide

# data[].attributes.attachment_link.description
Up to 90% off Textbooks at Amazon Canada. Plus, free two-day shipping for six months when you sign up for Amazon Prime for Students.

# data[].attributes.headline
Farrall - Reading Assessment: Linking Language, Literacy and Cognition

# data[].attributes.subject
Farrall - Reading Assessment: Linking Language, Literacy and Cognition

# data[].attributes.attachment_link.title
Reading Assessment: Linking Language, Literacy, and Cognition

# data[].attributes.attachment_link.description
Reading Assessment: Linking Language, Literacy, and Cognition : Farrall, Melissa Lee: Amazon.ca: Books

# data[].attributes.attachment_link.title
Part 2: Becoming a Literacy Leader Workshop Series

# data[].attributes.attachment_link.title
Do We Stop Making Sense When We Assess With Nonsense Words?

# data[].attributes.attachment_link.description
Assessing with nonsense words is powerful, yet controversial. This blog responds to frequently asked questions about the Acadience Reading Nonsense Word Fluency assessment.

# data[].attributes.attachment_link.readable_text
Oscar Wilde, in The Importance of Being Earnest, wrote "The truth is rarely pure and never simple" (Wilde, 1898/1990, p. 10).

We can say the same thing about science. To arrive at absolute truth in science is rare because scientific knowledge constantly evolves, but we do know there are certain essential findings that constitute sufficient causal evidence to guide us to make decisions about health, the environment-and also about reading instruction.

The Importance of Evaluating Research

In reading instruction, educators want to know what practices they can trust that will improve reading outcomes. This means we want confirmed research that makes causal connections clear between "treatment practices" and the outcomes.

However, to understand statements of causal relationships for effective practices in reading research, we need to be certain of the methods used. Dr. Keith Stanovich explained that "the experimental method provides the critical support for making causal claims" (Stanovich, 2003).

So, what do we look for in evaluating research? The Science of Reading Defining Guide, recently released by the Reading League (2022), provides a detailed chart explaining each of the three important components one should look for in examining research studies:

1. A study design that is experimental or quasi-experimental. Experimental designs include random assignment to treatment and control groups; quasi-experimental designs do not use random assignment but participants are usually compared to groups with similar characteristics.

2. A detailed description of the methodology, the population, and any assessment instruments used. Detail is necessary for replication and generalizability because one study alone is not sufficient to have confidence in the findings.

3. Acceptance in a peer-reviewed journal. This last component is the quality check done by multiple, independent scientists well versed in the nature of the content of the study.

For a more complete description of these three components, download the Reading League's Science of Reading: Defining Guide.

As reading educators, we often hear about inherently appealing ideas that sound promising but then turn out not to be supported by evidence. Examples include suggestive articles indicating the use of specialized fonts for children with dyslexia or special, colored lenses to improve reading. Despite numerous studies of their efficacy, neither of these approaches has been supported experimentally. Close reading, while emphasized in the Common Core State Standards, also has not been proven in empirical, peer-reviewed studies.

Many of us also remember the ubiquitous influence of learning styles that took hold in education despite the lack of any evidence to its effectiveness, and even after studies showed matching instruction to students' learning styles did not improve reading outcomes. Just because a practice sounds good does not mean it is effective.

We have spent our careers dedicated to guiding teachers' professional learning about reading- Linda as co-founder and past president of CORE and Holly as the founder of the University of Florida Literacy Institute. As a result, we have closely followed advances in reading science, and we've done our best to separate the effective from the simply appealing.

What Does the Research Say?

This brings us to the central question of this blog: Is there research that confirms the efficacy of pseudowords, which some refer to as nonwords, when used during instruction, not just for assessment?

The idea seems to make sense because if children can decode pseudowords, we can make the conclusion that they have developed grapheme-phoneme correspondences. Or can we?

Often pseudowords (or nonwords) and nonsense words are used interchangeably even though they are not quite the same. We define these terms as follows:
P seudoword (or nonword): a string of letters that mimics a real word both orthographically and phonologically but is not a real, meaningful word that exists in the language (e.g. gom)

Nonsense word: orthographically and phonologically random and does not match known orthographic patterns (e.g. qif)
On January 27, 2022, during a webinar, "The Science of Reading and English Language Learners," with colleagues Drs. Elsa Cárdenas-Hagan and Claude Goldenberg, Linda asserted with a high degree of certainty that while the use of pseudowords in assessment is well documented, using them in instruction is not.

Then Linda saw Holly's comments on the utility of teaching with pseudowords on the " Science of Reading-What I Should Have Learned in College" Facebook group. She cited the findings from a dissertation study (Cardenas, 2009), in which phonics instruction that included the use of pseudowords turned out to be more effective than instruction with only real words.

Having read that dissertation, Linda wondered about the nature of the assessment used. If children were tested using only pseudowords in the measure then it would be a close measure of what was taught rather than a general measure of phonics skills.

One of Holly's own doctoral students (Madsen, 2014) had conducted a similar study, building on Cardenas's findings. Her study did not find significant differences between instruction with and without pseudowords, but the kids in both intervention groups did better than the controls.

Madsen drew on a theory called recombinative generalization. This is a linguistics concept, which suggests that learning is supported when language units are combined in novel arrangements. The idea is that using pseudowords gives students additional opportunities to practice with new grapheme-phoneme correspondences arranged in novel ways, so a student who has learned that m represents /m/, i represents /ĭ/, and p represents /p/, would benefit from the practice afforded by decoding the pseudowords "mip" and "pim."

Another argument in favor of using pseudowords is that it can serve as practice for reading multisyllabic words, since most syllables are essentially pseudowords (e.g., tas, tic).

Dr. Hoover

Reaching Out to the Research Community

After seeing Holly's post, Linda proceeded to reach out to reliable and well-known experts in the science of reading research community to get a clearer picture of the use of pseudowords instructionally.

Dr. Goldenberg

Not surprisingly, most responded by saying they knew of no confirming research to justify the use of pseudowords for instruction.

Dr. Wesley Hoover explained that a nonword (some use pseudoword and nonword interchangeably) is "a property of whether it is known by the person encountering it and not an exclusive property of the letter sequence encountered" (personal correspondence January 28, 2022). What this means is that when children are decoding words that are not in their vocabularies during phonics instruction, the word is a nonword to them.

Dr. Cárdenas-Hagan

Dr. Hoover went on further to explain that "in learning to read, set for variability is important." Set for variability is the mental flexibility that allows a reader to discern the mismatch between a decoded word and its actual pronunciation. This is because a word may have multiple pronunciations, but having the word in a child's vocabulary will facilitate correct pronunciation, which can lead to the ability to make other grapheme-phoneme correspondences and generalize to other words. Pseudowords, according to Dr. Hoover, do not necessarily afford this opportunity. Furthermore, Dr. Hoover indicated "in using real words teachers have the opportunity to advance competencies in both word recognition and language comprehension, the two critical components of reading comprehension" (personal correspondence January 31, 2022).

Dr. Claude Goldenberg, a prominent researcher on English language learners, pointed out that just because it may not be wrong to teach with pseudowords, it doesn't mean it is helpful and may even be counterproductive in particular for English learners.

Dr. Ehri

Dr. Elsa Cárdenas-Hagan, another prominent researcher focusing on English learners, is also concerned about using such words when teaching English learners who need to grow their English vocabularies.

Having heard all these counter arguments, Linda reached out to Dr. Linnea C. Ehri, because if anyone knows decoding instruction, it is Dr. Ehri.
Although our training method worked with a number of kindergarteners, it may not be the best way of designing instruction. Rehearsing nonsense syllables over and over is boring and tedious, especially for young children. Our method was selected mainly for experimental purposes. More interesting ways of teaching deciphering skill need to be identified and evaluated for their effectiveness. (p. 12)

Dr. Odegard

Sure enough, she responded with two research studies. The most recent study with Brazilian students focused on decoding grapheme-phoneme subunits as well as two-letter CV pattern words of low frequency and missing diacritics (Sargiani, Ehri & Maluf, 2021). In addition, in an earlier study, Dr. Ehri used pseudowords in instructing young children and found positive effects (Ehri & Wilce, 1987).

Dr. Ehri generously sent Linda a set of real words and pseudowords by way of example when training children to practice reading units composed of CVCs: "bat, bet, bit, bot, but, tab, teb, tib, tob, tub." Meanings were not included in this type of set; instead, the purpose focused on processing grapheme-phonemes "to map spellings to pronunciations" (Personal correspondence January 31, 2022).

Dr. Stollar

However, Dr. Ehri also stated that when the central purpose is to connect meanings to spellings and pronunciation, it is not necessarily appropriate to include pseudowords. Furthermore, in the earlier 1987 study, Ehri and Wilce stated the following:

Dr. Tim Odegard at Middle Tennessee State University's Tennessee Center for the Study and Treatment of Dyslexia also concurred with Dr. Ehri but with a caveat. Dr. Odegard explained that the need for pseudowords is "born out of the necessity for readers who need a lot of repetitions to consolidate learning." However, he went on to say with regard to English learners, "not all ELL students will require as much practice as others. The rate of learning and the need for considerable practice to consolidate learning is an individual difference" (personal correspondence January 31, 2022). At his center, the goal is to have children learn to read with real words.

From a practical consideration, mixing real words with pseudowords could provide extra practice with grapheme-phoneme decoding while children are progressing to the point when many more words become available for decoding practice. Dr. Odegard indicated that too often children do not get sufficient practice decoding so adding pseudowords could augment the real words. Thus, from a design standpoint, there is logic to this approach. However, when children are reading words in sentences and in decodable or connected texts, which should be a regular part of decoding instruction, using real words is preferred. Furthermore, more time reading texts and more interleaving (mixed practice) as children learn more grapheme-phoneme correspondences may resolve the sufficiency of practice concern.

Dr. Stephanie Stollar recently published a blog post about the use of pseudowords in assessment. In it, she cautioned against using them in instruction. She said, "The instant word recognition required for reading comprehension depends on the synthesis of graphemes, phonemes, and word meanings. Instruction that uses nonwords works against this goal."

Dr. Stollar goes on to suggest that assessing English learners with pseudowords is appropriate because "for students to become accurate and fluent readers of English, an underlying awareness of the phonics structure of English words is necessary in order to apply word attack skills when encountering unfamiliar words in text."

Further Research Needed

It would be worthwhile to see a study examining instruction using pseudowords compared to instruction using real words, both with and without increased connected text reading. Some curricula, such as SIPPS and Being a Reader, to name two, provide a large amount of practice with words, sentences, and encoding and reading connected texts, but also provide work with syllable parts (see the SIPPS Challenge Level Scope and Sequence) that are themselves pseudowords (e.g., mem or ber from remember).

Until further peer-reviewed research is conducted, we may not have a complete answer to the appropriate use of pseudowords in instruction. But Dr. Ehri's cautionary conclusion-that it depends on purpose-makes the most sense.

Finally, one consideration is the tyranny of time. Schools have limited time with children and what time does exist for literacy should include both developing automaticity with word recognition and the development of strong language comprehension, including vocabulary knowledge, oral language and background knowledge.

If we want children to improve both word recognition and language comprehension, we should maximize the instructional time that produces both outcomes. This alone is an argument to teach decoding as much as possible with real words, assess using pseudowords, and ensure application to texts to acquire the meanings of the words. In this way, children will bond grapheme-phonemes correspondences to spellings and to meanings for later automatic retrieval.

Unlike the prior examples about specialized fonts, colored lenses, and learning styles, which have been disproven by reliable research studies, the use of pseudowords in teaching reading will benefit from further research. Since this idea has taken hold, especially in some circles for children with dyslexia, more complete peer-reviewed studies could either advance or dispel the use of this practice except in certain situations.

Reflecting on the Importance of Being Earnest about the Science of Reading

This exchange about pseudowords reminds us to be cautious in asserting something with certainty before we have exhausted as much research as possible. To address this topic carefully, we followed the guidance provided in the Science of Reading Defining Guide by the Reading League (2022):

1. Disagree respectfully

2. Identify best practices from multiple studies

3. Dig deeper and seek clarification

4. Have courage to reconsider

Holly's post started Linda down this path of exploration and once she read Dr. Ehri's studies, the use of pseudowords, with cautions, can make sense. It's important to take into account what Dr. Hoover and Dr. Odegard said, in particular because we have the urgent job of helping children become both automatic with word recognition and build their language comprehension as well.

We are also mindful of what Dr. Cardenas-Hagan and Dr. Goldenberg have said, and the importance for English learners to have enough English words in their oral vocabularies to be able to bond the spellings to the pronunciations and to meanings.

Finally, we are incredibly grateful to Dr. Ehri who responded to multiple emailed questions in order to provide clarity and nuance to this discussion.

We are all still on a journey, and to paraphrase Algernon in The Importance of Being Earnest when he explained further why the truth is "rarely pure and never simple": "Modern life would be very tedious if it were either, and modern [science] a complete impossibility."

***
To learn more about this Collaborative Classroom blog series, Structured Literacy: Unpacking Nine Key Topics for Transforming Reading Instruction and Outcomes for Readers, read the introduction.
For more information about Dr. Wesley Hoover, see research on the Simple View of Reading and Hoover WA, Tunmer WE. The Simple View of Reading: Three Assessments of Its Adequacy. Remedial and Special Education. 2018;39(5):304-312. doi: 10.1177/0741932518773154

The actual quote by Algernon is "The truth is rarely pure and never simple. Modern life would be very tedious if it were either, and modern literature a complete impossibility!" It can be found in Act 1, page 10.

***

Footnotes

References

Cardenas, Jessica M. (2009). Phonics instruction using pseudowords for success in phonetic decoding. [Doctoral Dissertation, Florida International University]. FIU Electronic Theses and Dissertations. Retrieved from https://digitalcommons.fiu.edu/etd/139.

Ehri, L. C., & Wilce, L. S. (1987). Cipher versus cue reading: An experiment in decoding acquisition. Journal of Educational Psychology, 79(1), 3-13. Retrieved from https://doi.org/10.1037/0022-0663.79.1.3

Madsen, K. M. (2014). The use of pseudowords for decoding instruction. [Doctoral dissertation, University of Florida]. UF Digital Collections. Retrieved from https://ufdcimages.uflib.ufl.edu/UF/E0/04/59/36/00001/MADSEN_K.pdf.

Sargiani, R. de A., Ehri, L. C., & Maluf, M. R. (2021). Teaching beginners to decode consonant-vowel syllables using grapheme-phoneme subunits facilitates reading and spelling as compared with teaching whole‐syllable decoding. Reading Research Quarterly, 57(2), 629-648. Retrieved from doi:10.1002/rrq.432.

Stanovich, P. J. & Stanovich, K. E. (2003). Using research and reason in education: How teachers can use scientifically based research to make curricular & instructional decisions. National Institute of Child Health and Human Development; Department of Education; and Department of Health and Human Services. Retrieved from https://files.eric.ed.gov/fulltext/ED482973.pdf.

Stollar, S. (2022, Mar 23). Do we stop making sense when we assess with nonsense words? Retrieved from https://www.readingscienceacademy.com/blog/.

The Reading League. (2022). Science of reading: defining guide. Retrieved from https://www.thereadingleague.org/what-is-the-science-of-reading/.

Wilde, O. (1990). The Importance of Being Earnest. Dover Publications. (Original work published 1898.)

# data[].attributes.attachment_link.title
The Use of Pseudowords: The Importance of Being Earnest about the Science of Reading - Center for the Collaborative Classroom

# data[].attributes.attachment_link.description
Is there research that confirms the efficacy of pseudowords when used during reading instruction, not just for assessment? We explore this question in this blog post-the sixth in our series Structured Literacy: Unpacking Nine Key Topics for Transforming Reading Instruction and Outcomes for Readers.

# data[].attributes.headline
ORF: Essential for Reading Comprehension - Dr. Jan Hasbrouck

# data[].attributes.subject
ORF: Essential for Reading Comprehension - Dr. Jan Hasbrouck

# data[].attributes.attachment_link.title
Jan Hasbrouck - Reading Fluency: Essential for Reading Comprehension

# data[].attributes.headline
Grade 4 Literacy Block (with links to instructional resources)

# data[].attributes.subject
Grade 4 Literacy Block (with links to instructional resources)