# data[].attributes.headline
PBIS Tiered Fidelity Inventory (TFI) Manual 

# data[].attributes.subject
PBIS Tiered Fidelity Inventory (TFI) Manual 

# data[].attributes.headline
Sign In Sheet for: PBIS- Get to Know the New PBIS Surveys

# data[].attributes.subject
Sign In Sheet for: PBIS- Get to Know the New PBIS Surveys

# data[].attributes.attachment_link.description
Google Forms - create and analyze surveys, for free.

# data[].attributes.attachment_caption
Using Data for Decision Making 10-25-23 PD

# data[].attributes.attachment_link.description
Find how-to videos, walk-throughs, quick tips, and recorded webinars related to the applications and specific surveys you use.

# data[].attributes.attachment_link.readable_text
Tiered Fidelity Inventory (TFI)

In this video we guide you through having a team score the TFI as well as give you tips and advice on what to have in place and how to solve tricky situations.

# data[].attributes.attachment_link.description
Video: In this video we guide you through having a team score the TFI as well as give you tips and advice on what to have in place and how to solve tricky situations.

# data[].attributes.attachment_link.readable_text
Over the last three months, Teach by Design has focused on implementation. We talked about the seven core elements of successful implementation. Then, we talked about how to measure your implementation - both how closely you did what you said you'd do and whether you saw positive outcomes because of it. We highlighted two surveys: the Tiered Fidelity Inventory (TFI) and the School Climate Surveys for students, family, and staff. Well, if you take a survey, you're going to end up with data. If you know anything about us at all, once you end up with data, we're going to tell you to make some decisions. Which brings us to this very article. How do you use data not just to make decisions, but also to make effective solutions?

‍
Ironically, I set out to highlight research data demonstrating the way decisions are more effective when people use data to inform them. The research I found included next to no percentages, statistics, or standard deviations. I landed on real life, human statements from focus groups and interviews conducted in nine high schools across the United States.1 Researchers identified these schools by their commitment to continuous improvement and their user of data in their decision-making processes. Researchers expected to hear all the ways teachers depend on data as they make decisions that influence the way they teach in their classrooms. What they found was: teachers are more likely to rely on their intuition than solely information downloaded from a database.

‍
Over the course of 186 interviews about their school's practices, 98 interviews about their school's culture, and 19 focus group discussions, researchers found 40% of teachers used systematic data to make their decisions. That seems pretty good. The sentiment changes a little when you find out just as many teachers only used their intuition or gut feelings when they made decisions about their classrooms. You can almost feel how a researcher - someone dedicated to collecting and analyzing data - might react when they hear a teacher describe their decision-making process this way:
"I think that once you've been doing a job for a couple of years, I think all teachers have instincts and you can kind of like feel when things go right or wrong. Now I know that that's something that's pretty vague, but really I think good teachers have that. Almost like an intuition...‍

When I think about an administrator's reaction to student behavior, everyday decisions about reading groups, or small choices about where to put someone on a seating chart, the critical side of my brain hopes there are data involved at some point. On the other hand, using data as the only source of information has its own set of problems. Data are flat. They give you facts about something, but no soul. They give you a snapshot of a moment, but no story. Data come to life when you know the context they came from and the stories that created those numbers. That's why 15% of the people these researchers interviewed - the 15% who used a combination of data and intuition - might be onto something. In his TED Talk, Sebastian Wernicke tells a perfect example of how this exact idea led to the development of two very different TV shows.

A Tale of Two Shows

Roy Price is a senior executive with Amazon Studios. He's responsible for picking the TV shows Amazon will produce. Specifically, he wants to pick shows that generate between a 9 and 10 rating on IMDB's scale. These are the kinds of shows where once the season ends you find yourself Googling when the next season premieres. These are shows like The Wire, Game of Thrones, and Breaking Bad. In one effort to create the next binge-worthy series, Roy and the good folks at Amazon released eight different pilot episodes to their audiences for free. Roy watched millions of people as they watched each episode and he analyzed all their data. When the results came in, the data told him that Amazon should create a sitcom about four Republican US Senators. They created that show. Do you remember it? Me neither.
Over at Netflix, Ted Sarandos has a similar mission to create the next series you can't stop watching. His approach is a little different. He takes the mountain of data already available to Netflix - ratings, viewing histories, what people say they like and don't like - and he analyzes all of it. Armed with that information, as well as his experience working in this medium and his instincts about good content, Netflix decided to produce a drama series about one US Senator.

Yeah. THAT drama series about one US Senator.
House of Cards was one of the very first binge-worthy shows. It received critical acclaim, 33 Emmy nominations, eight Golden Globe nominations, and an 8.8 on that coveted IMDB scale.

How can two people, both good at their jobs, both using data to inform their decisions, end up with wildly different outcomes? I'll let Sebastian Wernicke tell you more about it.

By looking at data, we find details about something we'd otherwise miss if we relied solely on our own experiences. We break down a problem to its smallest parts to know every in and out of the situation. That's just one step of several in a decision-making process. The second step Sebastian talks about in his TED Talk is the part where we take all these pieces and we put them back together into a solution. Data do a great job of breaking a problem apart, and a lousy job of telling us what to do next. Our brains are good at that. It's our experience, the intangible ways we relate to the data, that lead us to solutions better suited to our specific contexts.
What does this mean for you? Right, we were talking about surveys.
At the end of the school year, your team has a mountain of data at its disposal and a responsibility to create a plan to improve next year. You could take the pieces needing improvement and just slide them over to your action plan verbatim. Or, you could use those pieces, alongside your own experiences, to inform the solutions you come up with as a team. I sat down with Bert Eliason, Celeste Rossetto-Dickey, and Danielle Triplett to find out how they guide teams through the action planning process. Here are a few of our best tips to help you find that balance between using data and using your gut.

Create an Action Planning Agenda

First things first: Every meeting needs an agenda. If someone schedules a meeting and doesn't have a set of things to talk about, run...run far, far away. An agenda lets everyone know what's included in the discussion as well as what isn't. Give people the chance to prepare for the discussion by collecting their thoughts ahead of time and bringing their questions ready to ask. Your action planning agenda might look different, but here are some topics you should be sure to consider:

1. Review the big picture including national, statewide, or district-level data.

2. Revisit your team's annual action plan

3. Review the end-of-year data for your school

4. Discussion time!

5. Select and prioritize items for next year's action plan

6. Assign items to team members and set goals for when they will be completed

7. Plan how to share decisions with stakeholders after the meeting

Data do a great job of breaking a problem apart, and a lousy job of telling us what to do next...It's our experience, the intangible ways we relate to the data, that leads us to solutions better suited to our specific contexts.

Data Points and Gut Checks

The example agenda above relies on data to inform decisions. Wherever there are data, there are gut reactions to it. Here are the data points we suggest reviewing and examples of how to incorporate your experiences into your discussion time.

The Big Picture Review

At the beginning of your meeting, start by looking at the bigger picture of what's happening nationally, statewide, and in your own district. Your school isn't the only one implementing PBIS; check out how other schools' rate their own fidelity.

The Data Points

To know what's happening in the big picture, there are a few places you can look. Nationally, schools taking the TFI score around 74% on Tier I implementation, 69% on Tier II, and 62% on Tier III. If you're curious about how that looks across each TFI subscale, check out the full evaluation brief here.
Every year, PBISApps publishes national ODR averages and every SWIS school has the option to include that average as a line on their Average Referrals per Day per Month report. Check out more information about the 2017-18 national averages here.
Your school's district coordinator knows how implementation is going locally and the state coordinator can send you information about PBIS implementation at the state level.

Gut Check

When it comes to looking at how other people are doing, our natural instinct is to compare ourselves to them. Once you know how other schools in your district, state, or the country score on the TFI, the immediate next question is: What's our TFI score? Before you answer that question, take a minute to reflect on the national, state, or district data you just saw. Some questions to ask are:

What would it feel like to be in a school like that?

What resources do schools have to assist with implementation?

Does the average seem high or low? Why?

What have I heard from other schools in our district or state? Does their experience relate or contrast to the data I see?

What changes happened at the district or state level affecting overall implementation?

What was my experience with the district coordinator like this year?

The Annual Action Plan Review

Your team started out the year with a plan. Your plan consisted of goals you set and some number of things you were going to do to accomplish them. You've looked at this plan nearly every team meeting and it's time to revisit that plan, one more time.

Data Point

The only real data you need here is the original action plan. If you never wrote a formal plan, somewhere you wrote down your goals for the year. Pull them out. Read them.

Gut Check

Ever walk into the kitchen and forget why you went in there in the first place? I'm asking you to remember what things were like seven months ago. I know. It's a big ask. Think back to those first few days of schools, fresh off summer break...

How did I feel to be at school?

What were my hopes?

What was the first team meeting like?

How did those feelings, hopes, and meetings change over the course of the year?

Did we commit to too much or could we have done more?

End-of-Year Survey Data Summary

This summary is critical. Knowing how you end the year informs every goal you have moving into next year. From this data, you'll decide how much improvement you want to see, and where you'll focus your effort. Take the time to summarize the data ahead of your meeting. Here's how.

Data Point

PBIS teams should expect to have two types of survey data to review: fidelity and outcome data. The TFI is your fidelity data, the School Climate Surveys are your outcome data. Someone from the team should take the time ahead of your meeting to review these data and bring a summary to your action planning meeting. Conducting an initial review ahead of time makes the meeting so much more productive. Decide who will summarize the information and give them the time to do it.

‍
If the data summary task falls on you and you don't know where to start, here's what you do.

1. Start with the survey's Total Score Report. Compare your school's average to the national, state, or district average.

2. Move to the Subscale Report. Find which subscales scored best and which ones scored lowest.

3. Finally, look at the Items Report. For the subscales with the best scores, write down the top three items you think your team should commit to sustaining next year. For the subscales with the lowest scores, write down the three items you'll want to be sure the team discusses further on how to improve them next year.

Gut Check

It's time to move beyond the scores and start asking yourself how you relate to the data you see. The data describe a school you just spent the last nine months working in. Now is your moment to tell your stories, describe your experiences, and highlight your highs and your lows.

What was my contribution to the way implementation worked this year?

How did my classroom feel?

Is there a subscale or an item that stands out to me as something I relate to?

What connections can I make between the datasets?

Was there an event, or something significant that happened during the year I think shifted our implementation for the better or worse?

What did I hear from my grade level team?

How did the playground feel? How would I describe the hallways or the cafeteria?

Now What?

It's time to make some decisions. Ideally, the discussions you have in your meeting bring you to themes. Within that theme, select and prioritize the items for your action plan. Don't forget to include items around your team's logistics.

Is anyone leaving the team next year?

Is there a change in roles on the team?

Will the current meeting schedule (day, time, location) work next year?

Are there concerns related to administrator support or turnover to be addressed?

Once you have the items selected, decide on your next steps. The action planning meeting is a great time to reflect and talk about what worked and what didn't work so well. Don't get so lost in the discussion that you forget about the true meeting purpose: Come up with a plan - complete with goals and assignments - to share with your staff and with families.
When you do it well, your action planning meeting should incorporate both the data you collected throughout the year as well as your lived experiences as teachers and PBIS team members in your school. An expert perspective is incredibly valuable to making effective decisions. You are the experts in your building. Connect the data points to everything you know about your students, your resources, your building, and your implementation. Create the solutions that work for your context, not just the ones spit out of a database.

‍
Special thanks to the following people for their contributions and their time given to this article:

Celeste Rossetto Dickey, M.Ed.

Celeste has worked in education since 1979. She spent seven years as an elementary and middle school teacher and another thirteen years as an elementary school counselor. She's worn many hats in the PBIS world: PBIS Coordinator, PBIS Trainer, MTSS Coordinator. She currently works half time with the PBIS Applications Training Team. Celeste is passionate about assisting all students to be successful using the PBIS/MTSS framework in schools.

Danielle Triplett, M.Ed

Danielle started her career in education as a middle school language arts teacher. With her insight into the important role systems play within schools and classrooms, she became a PBIS coordinator in the Gresham-Barlow school district. Today, Danielle is a research assistant at Educational and Community Supports working on an instructional alternative to exclusionary discipline.

Bert Eliason, Ed.D.

Bert came to PBISApps after spending 18 years as a teacher and 15 years as a middle school principal. As an administrator, he was actively involved in implementing PBIS. He currently works as a research associate on the PBISApps Training Team. He often writes PBIS implementation, the use of PBISApps, and racial disproportionality in school discipline. He is a Northwest PBIS Network board member and a strong believer in proficiency-based education and equitable outcomes for students and staff.

1. Ingram, D., Louis, K. and Schroeder, R. (2004). Accountability Policies and Teacher Decision Making: Barriers to the Use of Data to Improve Practice. Teachers College Record, 106(6), pp.1258-1287.

# data[].attributes.attachment_link.title
PBISApps | Teach By Design-The Best Decisions Take Guts and Data

# data[].attributes.attachment_link.description
Using data to make decisions is important. What role does your intuition play in the process?

# data[].attributes.attachment_link.readable_text
Get the Most Out of Every Meeting with TIPS

We all make decisions every day. Ideally, the decisions we make are informed by facts. Using data to make decisions is critical as we get to the root of a problem; it's also a central reason why we developed SWIS in the first place. From the first person trained to the last, we expect SWIS facilitators to guide teams to use referral data to make better decisions. We assumed it would be an easy fit into a team's regular meeting agenda. Well, you know what they say about assumptions...

Did team members really know how to incorporate data in their decisions? What actually happens during a typical team meeting?
"We didn't want to come across as pretentious; it's just a meeting. I thought: 'The federal government wants to pay us to come up with a model for adults to make decisions?' It seemed so simple. What we found was: The adults working together didn't have clear, predictable routines for organizing their data for decision making. They had the data, but they didn't know what to do with it."

Researchers from the University of Oregon and the University of North Carolina at Charlotte looked closely at how team-based problem solving plays out in schools to see if they could come up with a clear model everyone could follow for improving the effectiveness of their meetings. That model became known as the Team Initiated Problem Solving (TIPS) model. TIPS co-author Anne Todd, described the beginning of the study this way:

So, what is TIPS? Here's how it works:

​TIPS is a problem-solving model established within a standard set of meeting foundations. It's a series of steps anyone can use to move from identifying a problem to implementing a solution. Schools involved across the eight years of research studies received coaching along the way to be sure they implemented the model the way it was meant to be implemented.

​Did TIPS make a difference? In short, yes.

On average, teams involved in the first four years of the study improved their decision making by 86%. They defined their problems with greater precision, used data throughout more of their process, and created more complete action plans than they did prior to using the model.[1]​

The funding for TIPS research ended its eighth and final year in 2016. The co-authors are wrapping up their findings on whether the model had an impact on student outcomes (spoiler alert: it did) and have a manuscript exploring those findings currently under review. Until we can read all about it, we sat down with Anne to find out what teams should be doing right now to get more out of their team meetings. Here are her Top 5 TIPS Tips for Teams:

1. Give the meeting a purpose.

Have you ever gone to a meeting and spent the first few minutes trying to figure out who's running it and what you're going to discuss? Teams using TIPS establish meeting foundations as their very first step. As Anne says, "The meeting foundations are really critical. They get agreement among team members about why they meet, when they meet, what decisions they'll make, and how all of their decisions are documented." Meetings should have a regular agenda format and every member should know their role. Ask your teams how they take minutes and where those minutes get saved so everyone can access the information later. Laying this groundwork will save your teams time and headache down the road.

2. Name and Train a Data Analyst

TIPS researchers quickly learned the team meeting typically was the first time anyone in the group had seen the data they were about to discuss. Team members would work together to analyze the information collaboratively and come up with ideas for defining the problem to solve. While that process feels inclusive, it also takes time. Enter the data analyst. This is the team member responsible for reviewing data before the meeting, looking for potential problems to be discussed, getting the issues on the agenda, and bringing the data to the meeting so everyone can follow along. The data analyst keeps the meeting focused on generating solutions rather than admiring the problem.

3. Administrators Get to Wear Their Administrator Hat

With an administrator on your team, you have access to the person who makes decisions about budget, schedule, and personnel. So why give that person any other role in a meeting? Many of the teams Anne met assumed the administrator would also facilitate the meeting. "When we told administrators they weren't going to be the meeting facilitator, we watched their shoulders relax. We gave them permission to be leaders and to make decisions rather than run a meeting." However you select your meeting facilitator, let the administrator stay off the nomination list.

4. Backups, Backups, Backups

Just when you have the roles and process down, someone on your team inevitably leaves. Turnover in schools is real and getting a new person oriented to the process takes time. Do your teams have a system in place to keep the momentum going while bringing a new team member up to speed? Anne says, the key is to give every role a backup. Make sure there is at least one other person on the team who knows what another team member does.

What is the problem behavior?

Where is the problem happening?

When is the behavior most likely to occur?

Who is engaged in the behavior?

Why do students likely engage in the behavior?

5. Get Precise

During your SWIS Facilitator Training, you likely learned a key component of TIPS: defining the problem with precision. A problem is never fully defined until you can describe not just the what of the issue, but also the where, when, who, and why of it. Teams use Core Reports to identify the school's current reality and to ask: Is there a problem? If you can't find a problem, that's not a bad thing; celebrate successes where you find them. If there is a red flag, it's time to get precise. Take full advantage of the SWIS Drill Down Tool to look at the possible problem in context.

If you or the teams your support are the kind of people who like a visual for working through this kind of process, our PBISApps Training Team has prepared a SWIS Drill Down Tool Worksheet. Teams start with one red flag from the SWIS Core or Additional Reports and use filters in the Drill Down Tool to identify the context for the problem they're working to define.

Get More Information

While the TIPS project is coming to an end, you have access to TIPS materials, right now! Head over to pbis.org. Take a look in the Data-based Decision Making section under the Topics menu. Toward the middle of the page you'll find a section called Team-based Decision Process and a giant TIPS graphic. Check out the descriptions, resources, and training videos! Here you'll find video overviews, worksheets, training slides, meeting videos and more. Check it out!

About Anne W. Todd​

Anne W. Todd has had a full teaching and research career focusing on systems, data and practices at all three tiers of support. A University of Oregon scholar since 1984, her passion is to move research to best practices that are feasi​ble, practical, and efficient. Anne teaches with excitement as she works with schools, districts and departments of education to adopt TIPS as a problem-solving and decision-making model for both social and academic student data.

‍

​1. Todd, A. W., Horner, R. H., Newton, J. S., Algozzine, R. F., Algozzine, K. M., & Frank, J. L. (2011). Effects of team-initiated problem solving on decision making by Schoolwide behavior support teams. Journal of Applied School Psychology, 27(1), 42-59. doi:10.1080/15377903.2011.540510

# data[].attributes.attachment_link.title
PBISApps | Teach By Design-Top 5 TIPS Tips for Teams

# data[].attributes.attachment_link.description
​Team Initiated Problem Solving (TIPS) co-author Anne Todd gives 5 things you can do right now to get more out of your team meetings.​

# data[].attributes.attachment_link.title
Center on PBIS | Resource: TIPS Meeting Minutes Template

# data[].attributes.attachment_link.description
Word Doc: Meeting form to be used by school teams using the Team Initiated Problem Solving (TIPS) method. Useful for keeping the conversation focused to problem solving.

# data[].attributes.headline
5 Questions Every Team Should Ask About Schoolwide Behavior 

# data[].attributes.subject
5 Questions Every Team Should Ask About Schoolwide Behavior 

# data[].attributes.attachment_link.title
PBISApps | Teach By Design—5 Questions Every Team Should Ask About School-wide Behavior

# data[].attributes.attachment_link.description
Data is a four letter word, but it doesn’t have to be one that causes your team anxiety. Embrace a data routine and start looking forward to using your data to improve decision making and outcomes for students!

# data[].attributes.attachment_link.readable_text
Tiered Fidelity Inventory (TFI)

Helps teams identify (a) which items will be the focus of implementation efforts for the coming month, and (b) what the specific action(s) will be, who will lead in completing the action, and a date by which the action is expected to be completed.

# data[].attributes.attachment_link.description
Form: Helps teams identify (a) which items will be the focus of implementation efforts for the coming month, and (b) what the specific action(s) will be, who will lead in completing the action, and a date by which the action is expected to be completed.

# data[].attributes.headline
The Second Year Itch: Why Does Implementation Fail?

# data[].attributes.subject
The Second Year Itch: Why Does Implementation Fail?

# data[].attributes.attachment_link.readable_text
​​​​​​Once upon a time, there was an elementary school called Lincoln. It was one of four elementary schools in its district. One August morning, Lincoln staff met the district office to be trained on a new initiative ready to make their school a better place. They spent the whole year implementing new strategies, pouring over new materials, attending conference sessions, reviewing data, making team-based decisions, and creating action plans. Everything was going so well. Then, in the summer, just before the next school year, Lincoln found out its administrator transferred to another school and their dedicated counselor would be shared with among the other three elementary schools. That year, the team stopped meeting as frequently. The new administrator suggested modifications to the way Lincoln improved their practices. By the third year, the new initiative everyone was excited about became the old initiative they used to do.

With so much research focused on how to help schools sustain implementation, a group of researchers wanted to know what they could learn from schools who simply stopped implementing. What were their experiences? Could those experiences shed some light on the ways in which others implement new strategies district-wide? In short, the answer is YES.

Th​​e Study

5,331 schools, 1420 districts represented across 37 states[1]

Researchers took a closer look at how these schools implemented School-Wide Positive Behavioral Interventions and Supports (SWPBIS) from their first through fifth years. The large number of schools included in the study means schools had varying degrees of support from state and local administration, a wide range of resources allocated, and all demographics represented. After analyzing this giant pile of data, it turns out every school fit neatly into one of four groups of implementers:

Sustainers who maintained their high degree of implementation across all five years

Slow Starters who struggled to implement during the first three years, but slowly rose to meet Sustainer levels by years four and five.

Late Abandoners who implemented pretty well in the first three years and then dropped out in years four and five.

Rapid Abandoners who had a successful first year and then, surprisingly, dropped out the second year never to start back up again.

​If you think about schools you work with who have implemented a new initiative or new practices, they likely fit into one of these categories.

What Ca​n You Do?

This study gives us a start to figuring out how to setup schools to be Sustainers and what to do if you think they may be dabbling in the Abandoner groups.

Create a Community for Your Sch​​​o​ols

For a lot of us, in anything we do-finding a new gym, trying a new restaurant, hiring a mechanic-we want to be able to connect with people who have already done it. We want to ask them: What is your experience? How did it go for you? It can be scary to be the first to do something; most people want to be part of a community. Creating the right community for the schools in your district takes strategy. Find something schools have in common:

Then, group those schools together as a cohort. That group of schools might feel bigger than you expected: According to the study, on average, sustaining schools were part of a cohort the size of 41% of their district (McIntosh et al., 2016)! But think about it: As a community, schools will go through training together, share resources with each other, and reach out for advice from one another. Just knowing they're not alone in this, they might be more likely to weather the tough times and keep going.

What might a community look like for your district?

Get Your Ducks in ​​a Row

With a larger group of schools going through their first year of implementation together, you will need get your resources all lined up, ready to go. Make sure you have enough coaches to work with the number of schools you have. Training events should be offered throughout the year so everyone is able to attend. Print enough materials to get you through. Deliberately schedule team check-ins rather than waiting for teams to reach out. When schools not only feel supported, but actually are supported, they are successful.

How would you support 41% of the schools in your district all implementing in their first year together? ‍

Schools get excited about implementing that first year. There's a lot of energy that goes into its success. If teams find out at the end of that year, after all that effort, ​they didn't meet their goals, it can be a real blow. Remind your schools that it's normal to take three to five years to fully implement a new initiative.

Pay Attention to​ the Second Year

Schools give it their best effort the first year they start a new strategy. Resources are available. Coaches work closely with teams. There might even be enthusiasm from staff to get started. Schools implementing SWPBIS take surveys as a status check to find out how well they're implementing these strategies. Schools meeting or exceeding criterion on those surveys were doing really well. Most schools in the study implemented at or above criterion in the first year. The second year stands out as a turning point - for Slow Starters and Rapid Abandoners in particular.

​Slow Starters saw their scores go down further from their first year. Rapid Abandoners no longer met criterion as they had in the previous year and actually saw their scores drop significantly. While both groups struggled with implementation in the second year, one pushed through and found success; the other stopped and never came back.

While the research is still new, and there is more to explore, we asked Dr. Rhonda Nese what schools can do to count themselves among the sustaining group rather than the 58% of schools who abandoned the practice.

"Schools get excited about implementing that first year. There's a lot of energy that goes into its success. If teams find out at the end of that year, after all that effort, ​they didn't meet their goals, it can be a real blow. Remind your schools that it's normal to take three to five years to fully implement a new initiative."

​Action Planning - If after the second year, scores still aren't where you expected them to be, focus on finding the items you consistently scored a zero. Pick a few and work on those. These victories create momentum.

Share your data with staff - Research tells us sharing data with staff is a key contributor to schools sustaining practices​[2]. To improve scores, talk to the people in your building. Show them what you've been up to and how you've seen it make a difference in your building. Ask them for their perspective on how you can improve.

Reach out to the cohort - Now is the time to take full advantage of the community you created. Talk to other schools about your struggle. You're likely to find out they've been through something similar, too. Steal their strategies and see if they work for your school.

Which schools do you support right now who are in the middle of a tough second year?

Create a large community of first-year implementers. Give schools a support system to work through the harder times together.

Supporting a large community isn't an easy task. Make sure you have the resources you need to provide the training, materials, and on-going coaching to every school in the group before you get started.

Pay attention to implementation in the second year. Look for the schools who are struggling to see the results they expected after their first year. Don't let them give in to that second-year itch. Remind them of the process and coach them through what they can do to weather the storm.

Dr. Nese has three ideas for creating Sustainers out of a year-two slump.

If there is anything this new research can teach us it's this: There is no standard experience for implementing new strategies. Sometimes everything clicks. Sometimes it feels like two steps forward and one step back. Whatever a school's experience, another school has been there, too.

1. McIntosh, K., Mercer, S., Nese, R. and Ghemraoui, A. (2016). Identifying and predicting distinct patterns of implementation in a school-wide behavior support framework. Prevention Science, 17(8), pp. 992-1001. doi: 10.1007/s11121-016-0700-1.​

2. McIntosh, K., Kim, J., Mercer, S. H., Strickland-Cohen, M. K., & Horner, R. H. (2015). Variables associated with enhanced sustainability of school-wide positive behavioral interventions and supports. Assessment for Effective Intervention​, 40(3), 184-191. doi: 10.1177/1534508414556503.

# data[].attributes.attachment_link.title
PBISApps | Teach By Design-The Second Year Itch: Why Does Implementation Fail?

# data[].attributes.attachment_link.description
Research tells us how to set up schools to sustain PBIS implementation...and what to do if you think they may be dabbling in abandoning everything.

# data[].attributes.headline
Local PBIS Coordinator Information Form 

# data[].attributes.subject
Local PBIS Coordinator Information Form 

# data[].attributes.attachment_link.title
PBISApps | Local PBIS Coordinator Information Form

# data[].attributes.attachment_link.description
Form: Form to be filled out to become a Local PBIS Coordinator.

# data[].attributes.attachment_link.title
PBISApps | Teach By Design-Teach by Design's 2023 Year in Review

# data[].attributes.attachment_link.description
Here are some of our reflections on a year's worth of research findings, practical strategies, and a whole lot of GIFs.

# data[].attributes.headline
Let's Talk About Behavior Trends: What the Data Say and How to Address Your Classroom Realities 

# data[].attributes.subject
Let's Talk About Behavior Trends: What the Data Say and How to Address Your Classroom Realities 

# data[].attributes.attachment_link.readable_text
Do you watch ? I cannot get enough of this show. Abbott Elementary is a TV show shot in a mockumentary-style similar to The Office. It's about a group of teachers at a public elementary school called Abbott Elementary and how they overcome the same problems facing educators today. In this week's episode, Janine, a second grade teacher at Abbott played by the incomparable Quinta Brunson, meets with a student's parent to discuss how the student's small disruptions have escalated throughout the year and how she could use some help figuring out how to make the rest of the year a little better. The meeting doesn't go as planned and instead of identifying solutions, the meeting ends when the student's mother calls her a bad teacher and leaves. The moment leaves Janine shaken and questioning her capability as a teacher.

The thing about sitcoms is they tend to present and solve a problem in the span of 30 minutes. Real life doesn't quite work that way. However, the problem Janine faced in this episode and the emotions it elicited sure do reflect current realities in classrooms across the country.

Recently, the Institute of Education Science (IES) released a new set of data from its School Pulse Panel, "a study collecting information on the impact of the COVID-19 pandemic from a national sample of elementary, middle, high, and combined-grade public schools." Of the 846 schools responding to the most recent survey, 84% agreed or strongly agreed that student's behavioral development was negatively impacted by the pandemic. Respondents said incidents of classroom disruptions, disrespect, and technology violations are all up since the beginning of the pandemic. ¹

Teachers are telling us they are overwhelmed by behavior. It's time for us to help them address it.

Well, we talk about behavior support all the time over here. We also happen to have large datasets that help us make sense of what's going on. Every year, we release a summary of the previous year's discipline data trends.

I wondered what last year's data could tell us about where to focus our solutions.

What's Happening Nationally?

Nationally, the trends in overall discipline data continue to stay relatively stable. ²

Discipline Data Trends

Reader, I know you see that 2019-20 school year and the way it spikes up. I see it, too. The 2019-20 school year was a partial year; school happened like usual until March 2020 and then data collection stopped. So, while we include it in the graph because there are enough data to put it there, it is also unlike the other years in the report. Moving on.
The picture didn't quite line up for me with the experiences from teachers across the country. I expected to see a big increase this year. While I see the slightest trend up, these data are mostly stable since 2016-17. So, what's going on? If there's one thing we know about data, it's that one picture can't tell us everything.

Let's talk about what isn't included in this graph. First, these are trends for "major" referrals only - referrals for behaviors handled by an administrator, outside the classroom. What about the behaviors teachers handle in their classrooms without sending students to the office? The survey data from NCES told us it's those classroom-managed behaviors that have increased over time and those behaviors aren't included in this picture.

Second, this is a national dataset. We're looking at averages based on thousands of schools. Those averages give us a good idea of what's happening overall, but they don't tell us what's happening exactly where you are. The good news is you have access to your own referral data to tell you more about what's happening in your building.

What Do Your Data Tell You?

Open SWIS and check out your Year-end Report. There you'll find a year-by-year comparison of the number of referrals per 100 students, for administrator-managed behaviors as well as classroom-managed behaviors, and all referrals combined! As you look at the trends:

Do the averages trend up? Down? Remain relatively stable?

What do you notice?

Do your data confirm the overall feeling you have in your school?

What surprises you?

In that same report, you'll find graphs showing what percentage of your total referrals were written for each behavior. Write down the top five behaviors from the report. For example, in 2021-22, the top 5 behaviors were:

Inappropriate Language (24% of the total referrals)

Defiance (22%)

Physical Aggression (12%)

Harassment (12%)

Tardy (8%)

% of Total Referrals by Behavior

Do the same behaviors take up the same percentage of your referrals?

Which behaviors are happening more frequently now? Less?

How does the difference you notice match up with the way your school feels now compared to then?

Head back to the top of the page and select the 2018-19 school year (or other full year of data) from the School Year drop down menu. Checkout that same behavior report and list out the top five again.

We know data help us identify precisely what's happening. We also know data fall flat until we can give them life through our lived experiences. The stories from your classrooms frame the context where these behaviors happen, so keep telling them. Eventually, conversations will turn to, "Now what are we going to do about it?"

Before you start immediately dosing up the behavior support you provide, you need to confirm you have the basic foundation nailed. Here are six questions to help you verify the core components of schoolwide support are in place.

Who's on Our Leadership Team?

Creating a positive school climate

Using data to drive decisions, evaluate implementation, and share out how everything is going

Establishing how to select, train, and coach new staff on how you support behavior in your school

So much about how your school supports student behavior depends on the people leading the effort, specifically your schoolwide leadership team. This is the team of people from your school who work together and commit to:

For your leadership team to be in place, it needs to have a coordinator, an administrator, a family member, and staff representation across grade levels. At the high school level, maybe you include a student on your team, too. Double-check you have all members present on the team and everyone has the time available to attend most meetings.

Did We Teach Expectations?

You probably have a set of schoolwide expectations-the social, emotional, and behavioral skills you hope to see from everyone. Maybe they even spell out an easy-to-remember acronym like SOAR or PRIDE. Perfect! Did you teach students what those expectations mean and when was the last time you reminded them? Sometimes it seems obvious what Be Safe means on the playground, but until you explicitly teach students what it means and what it looks like, it's open to their interpretation. Let them know what you're looking for and how specific behaviors will make the space feel more welcoming. Ask students to join in the conversation by sharing about the expectations they experience outside of school, at home, or in their community.

Teaching expectations isn't a one-and-done practice. It's something you need to do throughout the year. Reminders about what each expectation means and what it looks like when students are with you makes it more likely they'll know exactly what to say when someone asks them, "What does PRIDE mean here in our school?"

Do All Staff Know How to Respond to Behavior?

When student behavior aligns with your schoolwide and classroom expectations, it's important to let them know. The continuum of strategies you use for acknowledging those moments should be accessible and to all staff. Acknowledgements can be tokens or even simple statements like, "Hey, I see you looking up with your pencil in hand. Thanks for paying such close attention."

Along with a continuum of strategies to acknowledge student behavior, you have a continuum of consequences for when behaviors don't align with expectations. That continuum includes your school's referral process. Make sure everyone in your building knows when to manage behaviors in class and when to send students out of class for more support.

Do Students Know Our School and Classroom Routines?

Every school a set of procedures, schedules, and routines that create consistency and make the day-to-day functions more predictable for everyone. Within those practices are the procedures, schedules, and routines happening in every classroom in your school. The more predictable you can make your routines, the more likely it is for behaviors to align with expectations. How closely do your classroom routines and expectations match up with your schoolwide systems and practices? Make sure all students know how to turn in assignments, how those assignments can be made up, what happens when they walk in at the beginning of class, what needs to happen at the end of class, and every procedure in between.

Do Our Students and Their Families Have a Say?

Numerous studies confirm that actively engaged families play a major role in improving academic and behavior outcomes for their students. ² When students are engaged in their education, they are more hopeful about their future, more empowered, more connected to their school, and they begin to think more critically about the systems around them. ⁴ Including student and family perspectives in the support you provide is essential to your PBIS implementation. There are lots of ways to leverage your existing practices to get these perspectives shared and included in your conversations.

Are We Using Data?

We started this article by sharing data, so it only seems right to end it by asking you how you use data in your school. If behavior is a concern in your building, track it. Make sure all staff know how they can use referral data as a way to communicate the daily experiences they have in their classrooms. If you aren't tracking classroom-managed behaviors, why not start now?

Speaking of starting...

We get it. The last three years disrupted so many of our regular routines, including assessing implementation fidelity. If it's been awhile since you took the Tiered Fidelity Inventory (TFI) or the School Climate Survey (SCS), get them back on your calendar. These assessments coupled with referral data give your team the information it needs to address the problems you face more precisely.
A cliff hanger? Yes, but part two is coming next week. In our April podcast episode learn how to dose up the support you provide to all students by taking advantage of some specific high-leverage practices.
These six questions help you answer the larger questions we ask all the time in PBIS implementation: Are we doing what we said we would do? And whenever we ask that question, the odds are high there is a fidelity assessment you can take to help you answer it. Guess what? In this case, the TFI is exactly the assessment you need. The first 15 questions on the survey assess how many of the core features of schoolwide support you have in place at your school: leadership teams, expectations, acknowledgements, consequences, routines, family partnerships, and of course, data-driven decisions. The TFI will help you find out which ones are solid and which ones need some attention. When you've firmed up your schoolwide foundation, you're ready to start layering on additional support.

1. Institute of Education Sciences, (2022) School Pulse Panel May Survey [Data set]. National Center for Education Statistics. https://ies.ed.gov/schoolsurvey/spp/.

2. PBISApps. (2022). SWIS Data Summary 2021-22 School Year. University of Oregon. https://www.pbisapps.org/resource/swis-data-summary-2021-22-school-year.

3. Jeynes, W. (2005). A Meta-Analysis of the Relation of Parental Involvement to Urban Elementary School Student Academic Achievement. Urban Education, 40(3), pp.237-269.

4. Lac, V., & Cumings Mansfield, K. (2018). What Do Students Have to do with Educational Leadership? Making a Case for Centering Student Voice. Journal Of Research On Leadership Education, 13 (1), 38-58. doi:10.1177/1942775117743748

# data[].attributes.attachment_link.title
PBISApps | Teach By Design-Let's Talk About Behavior Trends: What the Data Say and How to Address Your Classroom Realities

# data[].attributes.attachment_link.description
Supporting student behavior continues to be a top concern in schools across the country. Before layering on additional support, we have to confirm the basics are firmly in place.

# data[].attributes.headline
5 Questions Every Team Should Ask About Racial Disproportionality 

# data[].attributes.subject
5 Questions Every Team Should Ask About Racial Disproportionality 

# data[].attributes.attachment_link.readable_text
Editors Note: This is the second article in our four-part guest author series focused on using questions to guide challenging team conversations. The first article exploring school-wide decisions is available . This month's guest author is Dr. Ruthie Payno-Simmons, from Michigan's Integrated Behavior and Learning Supports Initiative (MiBLSi).
I have always engaged in equity work as an educator. I recall my early teaching days. I participated in ongoing action research around what it meant to teach for social justice. I used the research findings to provide learning spaces for students in ways that centered their voices, lived experiences, learning interests, and identities.[1] Everyone has a story about what led them to go into education. These are three of my experiences that influenced my approach and focus as an equity-oriented educator.

Why don't prominent historical figures look like me?

Did only a few people who look like me make important contributions to society?

What is wrong with Black people?

What is wrong with me?

I want to become a doctor. Will I be able to go to college?

Will I graduate from high school?

Will I be a successful adult?

As an African American student, most of my teachers did not reflect my racial identity. My textbooks focused on the contributions of the dominant culture. History books portrayed people representative of my race in ways that reinforced negative or deficit stereotypes. During Black History month, we always saw images of and learned about the same two or three individuals: Martin Luther King, Rosa Parks, and Frederick Douglass. While these three incredible people contributed to making our world more just, I spent most of my K-12 years learning about a history that did not reflect my culture or contributions from people who looked like me. I experienced the pressure to conform to dominant cultural expectations, what scholars call assimilation that occurs in schools.[2] These experiences led to negative racial and gender identities, and the following set of questions that I wrestled with during my K-12 experience:

Then, in high school, something extraordinary happened. I met Mrs. Cunningham, an African American female who became my 9th grade English teacher. I intentionally took drama for four years because she captured my imagination about what a professional woman of color could be. She made me feel better about being a brown girl. She gave me hope that I could be successful. She engaged in my life actively and showed genuine care and concern.

My second influential experience occurred in 11th grade. My American History teacher Mr. Lewis, an African American male, consistently taught about the contributions of Black people. I began to hear a very different history than the one from my K-10 experience. Every day, I couldn't wait to get to his class. I sat on the edge of my seat in this American history class, waiting to learn something new about people who looked like me.

As a teacher, administrator, and parent, I observed adults fearing five and six-year-old African American male students. They were negatively labeled, frequently sent to the hallway or office, viewed as guilty, and rarely proven innocent. Additionally, they experienced more suspensions than any other ethnic or racial group of students.

How can I ensure that students of all backgrounds see historical figures that look like them?

How can I infuse multicultural literature across content areas?

How can I help people interrogate their conceptions of Black people?

What's wrong with people and their thinking? How did they get there?

Because of Mr. Lewis' class, I majored in history as an undergraduate student. I wanted to learn more about history as it occurred and from multiple perspectives. I also wanted to learn about the accounts of others around the world and how we were all connected. Majoring in history was my third influential experience; it opened my eyes to truths I did not know. It was liberating. I was empowered as I learned more about systems of power and oppression. Majoring in history played an important role in increasing my positive racial and gender identities. I learned that Black free people were living in North America before the Mayflower. I learned that women wrote the Declaration of Sentiments and collectively fought for women's rights. I learned about the brutality of slavery and the intentional efforts put forth to maintain the status quo of systemic oppression following the abolishment of slavery. Majoring in history had an enormous impact on my teaching. During teacher preparation, my questions shifted to:

As a teacher, administrator, and parent, I observed adults fearing five and six-year-old African American male students. They were negatively labeled, frequently sent to the hallway or office, viewed as guilty, and rarely proven innocent. Additionally, they experienced more suspensions than any other ethnic or racial group of students. As a parent of three Black males, I experienced low expectations, gatekeeping, and micro racial aggressions, such as being surprised by their intelligence and "good behavior." During parent-teacher conferences, we often heard comments such as "he is a good boy" and "he is so articulate." On my critical reflection journey, I became more aware of how power and privilege work to maintain harmful practices. As a school administrator, I participated in meetings where adults from dominant culture made decisions about school systems to support students they did not understand culturally. Collectively, these experiences fuel my passion and desire to assist educators in centering equity in education and school discipline systems. Educational equity occurs in systems where "educational policies, practices, interactions, and resources are representative of, constructed by, and responsive to all people so that each individual has access to, meaningfully participates in, and has positive outcomes from high-quality learning experiences, regardless of individual characteristics and group memberships."[3]

How do I continue to interrogate my own biases and assumptions?

How can I help teachers and administrators understand issues of bias and systems of oppression?

How can I ensure that all students have access to equitable educational opportunities?

How can I help educators and school communities interrogate their conceptions of Black people and others from non-dominant cultures?

In my current role as an equity implementation specialist and consultant, my questions evolved to:

With the documented success of Positive Behavior Intervention Supports (PBIS), significant discipline disproportionality of African American students continues to exist across our nation. Scholars have linked these longstanding disparities to systemic racial bias.[4] I have worked with many caring and dedicated adults in PBIS schools who, when faced with the realization of such educational inequities, responded by saying, "I care about all of my students. How do I begin addressing issues of racial discipline disproportionality?"

One way to start is with a problem-solving process.

This four-step process is similar to the Team Initiated Problem Solving model.[5] The questions associated with each step are influenced by the work happening at the Great Lakes Equity Assistance Center. Using this process, teams ask these five critical questions to understand why disproportionate discipline occurs for certain groups of students in their systems.

Question 1: Is there an inequity problem in our school?

This question aims at surfacing those who don't benefit from the way things are. It allows teams to view discipline disparities as a systemic inequity rather than a deficit characteristic of a group of students based on identity markers. Teams can look at Referrals by Ethnicity Reports and Risk Ratios in SWIS as a starting point to identify if there is a discipline disparity. For example:

An elementary school leadership team looks at their Risk Ratios and discovers African American students are more than four times more likely to receive a referral than students from all other groups.

Next, the team reviews Referrals by Ethnicity and sees that African American students make up only 15% of student enrollment, but received 60% of total referrals. This indicates an overrepresentation in discipline. White students make up 64% of the students enrolled and only 21% of the office referrals. This suggests underrepresentation in discipline.

The team views the higher risk ratio and total referrals of African American students compared to lower risk and outcomes of all other groups as a red flag.

1. Where does the inequity occur?

2. When does it happen?

3. Why does the inequity happen?

The next step is to follow up and dig deeper to understand more about what in their systems affects this group the most. The team narrows the problem using the SWIS drill-down tool to understand more specifically the inequity related to African American students:
As of October 1, first and third grade African American students (boys and girls) are referred most often from the classroom for disruption. The motivation appears to be for task avoidance and peer attention. These referrals occur more frequently on Tuesdays and Wednesdays.

Answering these questions yields the following precise inequity statement:

Question 2: What is it about our people, policies, and practices that contribute to this inequity?

As school leadership teams develop goals and objectives, these critical questions focus on systems, not on fixing students or teachers. Remember, the disparity is linked to systemic inequity and not the fault of students or their perceived circumstances. During this part of the process, teams should use multiple data sources, such as the Tiered Fidelity Inventory, fidelity use of asset-based instructional practices, and stakeholder survey data. The data should assess systems, practices, and outcomes.

Discipline policies

Feedback and acknowledgement

Student/family/community involvement.

Remember our leadership team from the first question? Now, the team seeks to understand what it is about their people, policies, and practices that contribute to the inequity. The school's TFI scale and subscale scores are above the 70% threshold; the lowest score was their implementation score.

They drill down into their TFI score and notice three items' scores related to implementation are not sufficiently in place:

Question 3: Which equitable practices can address this discipline inequity?

When teams develop actions plans to address racial disproportionality, they should select activities and tasks that directly support implementing equity strategies. Likewise, strategies selected should align with the team's goal. School leadership teams should leverage the PBIS Cultural Responsiveness Field Guide: Resources for Trainers and Coaches to consider strategies aligning with their goal of providing cultural elaboration to PBIS systems.

Engage in professional learning that focuses on inclusive practices for student and family engagement

Create an equitable way of soliciting parents, students, and community members to participate in meetings where decisions are made about behavior and other systems

Professional learning for the newly expanded leadership team

Schedule future meetings at times optimal for the group to increase inclusivity and participation

In the case of the example above, the three items not fully implemented are great opportunities to include multiple and diverse perspectives in understanding more about the students' experiences as well as understanding more about ways in which people, practices and policies in the system are harmful or serving as barriers. One strategy aimed at addressing the team's goal would be to "expand the leadership team and the problem-solving continuous improvement process to include parents, students, and community members representative of the student population."

Possible activities to support the strategy are:

Question 4: Did the equity strategies decrease our discipline disparity?

Schools should use multiple data sources to monitor and check to see if the plan is working. Various data sources can include TFI data, asset-based instructional practices data (i.e., universal design and culturally responsive practices), stakeholder survey data, climate data, etc. The data should assess systems and methods in seeking to understand student outcomes rather than only looking at outcome data as a measure of determining equitable outcomes.

Question 5: Who is at the table when decisions are made about our school systems?

Schools should intentionally include diverse stakeholders and multiple perspectives at all levels of PBIS implementation and continuous improvement processes. This question surfaces who is present and included and who is not as systems are created and maintained. It also reveals whose experiences and identities are privileged and whose are not.

Addressing disproportionality is a complex process, and we are all still learning how to do this work in meaningful and practical ways. However, the complexity of this work should not deter us from having courageous conversations about our systems. Nor should it stop us from changing our systems to benefit all students, especially those who have been historically marginalized. These questions serve as a starting point that can support equity efforts within our PBIS systems. So, lean into the problem-solving process, knowing that by asking questions such as the five highlighted above and applying strategies to mitigate disproportionality that you are tackling systemic inequities head-on.

Resources from the Field

Schools often implement PBIS from a highly technical approach. While implementing PBIS from a technical perspective is an essential part of the process, contextual and critical considerations are equally important when seeking to address disproportionate outcomes. the National PBIS center provides tools of technical and adaptive approaches.

This multi-component approach supports a focus on implementing a preventative, multi-tiered, and culturally responsive behavior framework, engaging academic instruction, equity-oriented policies, the use of neutralizing implicit bias in discipline decisions, and systematic problem solving using disaggregated data.

This guide provides a process and rationale for engaging in the data review process for examining disaggregated data, action planning, implementation evaluation, recommendations for addressing disproportionality, and cultural elaboration within the big ideas of PBIS.

This field guide outlines an integrated framework to embed equity efforts into school-wide positive behavioral interventions and supports (SWPBIS). The goal of using this guide is to make school systems more responsive to the cultures and communities that they serve.

The Great Lakes Equity Assistance Center provides a wealth of accessible literature and tools for practitioners supportive of centering equity in educational policies and practices as well as cultivating critical reflection and learning around issues of power, privilege, and implicit bias. These resources can support incorporating a crucial component missing from the problem-solving process. This critical component involves having multiple and diverse perspectives involved in the problem-solving process. It also involves a shift from focusing on fixing students and teachers to asking, "whose interests are being served well by our systems, and whose are not?" It also involves interrogating systems by asking, "what is it about our organizational system, people, policies, and practices that result in inequities?"

1. Damico, J., & Riddle, R. (2004). From Answers to Questions: A Beginning Teacher Learns to Teach for Social Justice. Language Arts, 82(1), 36-46.

2. Paris, D., & Alim, H. S. (2017). Culturally sustaining pedagogies: Teaching and learning for justice in a changing world language and literacy series (pp. 1 online resource).

3. Kyser, T. S., & Skelton, S. M. (2019). The fundamentals of educational equity. Equity Digest, 2(3), 4.

4. Carter, P. L., Skiba, R., Arredondo, M. I., & Pollock, M. (2017). You can't fix what you don't look at. Urban Education, 52(2), 207-235. doi:10.1177/0042085916660350.

5. Newton, J. S., Horner, R. H., Algozzine, B., Todd, A. W., & Al- gozzine, K. (2012). A randomized wait-list controlled analysis of the implementation integrity of team-initiated problem-solving processes. Journal of School Psychology, 50, 421-441.

# data[].attributes.attachment_link.title
PBISApps | Teach By Design-5 Questions Every Team Should Ask About Racial Disproportionality

# data[].attributes.attachment_link.description
Addressing disproportionality is a complex process...however, the complexity of this work should not deter us from having courageous conversations about our systems.

# data[].attributes.attachment_link.title
PBISApps | Teach By Design—The October Catch

# data[].attributes.attachment_link.description
This 15-hour in-person and online hybrid training prepares and certifies participants to provide services and supports as a SWIS Facilitator to local schools interested in implementation of the School-Wide Information System (SWIS).

# data[].attributes.headline
The October Catch: Changing ODR Trajectories 

# data[].attributes.subject
The October Catch: Changing ODR Trajectories 

# data[].attributes.attachment_link.readable_text
As part of using discipline data in decision making, Office Discipline Referral (ODR) data can be used as a screener to identify students in your building who could benefit from targeted behavioral supports in addition to what is offered at the universal level.[1],[2] Using ODR data as a screening measure in October is a technique we call The October Catch.

Consider your data from last year: If you knew by the end of October you could have identified half of the students who ended the year with 6 or more referrals, wouldn't you? The number of ODRs students receive by the end of October is a highly accurate predictor of their cumulative ODR trajectory through the remainder of the year. Research shows 50% of students who received 6 or more total ODRs during the course of the year already received 2 or more ODRs by the end of October; 79% had 2 or more ODRs by the end of December.[3] See Figure 1.

You may already have processes in place to refer students for additional supports: teacher referral, even ODR counts. This research suggests you have the ability, right now, to identify and refer some students well before they have the time to accumulate​ 4, 5, 6 or more referrals. As you generate the SWIS Referrals by Student report this October, pay close attention to the students in your building with 2 or more referrals. Consider what you know about each student. Look at additional screening data available to you. Talk about what targeted supports you have available in your building and which students would benefit from enrollment.

Again in November and December, generate the Referrals by Student report. This time, check to see if there are new students with more than 2 referrals. Remember, 79% of students who ended the year with 6 or more referrals had at least 2 referrals by the end of December. Be sure you also check in on the students you referred for targeted supports; how has their referral trajectory changed or not? If your school has a Check-In Check-Out program, were any students enrolled in October and have data in your CICO-SWIS account? If not already, consider tracking these students in CICO-SWIS to better use their point data in decision making.

As a team, you have multiple resources informing your decision-making process. Use the ODR data you've collected as a piece of puzzle to predict earlier which of your students would benefit from additional supports.

1. Tobin, T. J., Sugai, G., & Colvin, G. (1996). Patterns in middle school discipline records. Journal of Emotional and Behavioral Disorders, 4, 82-94
2. Tobin, T. J., & Sugai, G. M. (1999) Using sixth-grade school records to predict school violence, chronic discipline problems, and high school outcomes. Journal of Emotional and Behavioral Disorders, 7, 40-53
3. McIntosh, K., Frank, J. L., & Spaulding, S. A. (2010). Establishing research-based trajectories of office discipline referrals for individual student. School Psychology Review, 39, 380-394.​​​​​​​​​​​​​

# data[].attributes.attachment_link.title
PBISApps | News-The October Catch: Changing ODR Trajectories

# data[].attributes.attachment_link.description
Waiting until students receive 6 or more ODRs to identify them for additional supports may mean you're waiting too long. Learn how to use SWIS to interrupt this trajectory in the fall.

# data[].attributes.headline
Consistent Process + Reliable Data = Better Results 

# data[].attributes.subject
Consistent Process + Reliable Data = Better Results 

# data[].attributes.attachment_link.readable_text
I volunteered at last year's environmental studies commencement ceremony here at the University of Oregon. One of the department's traditions is for each student to say what they plan to do after graduation. I listened as one by one, graduates rattled off job offers and newly acquired research funding and positions with environmental conservancies. When I graduated, I had a degree in journalism and an open room at my parent's house. I moved home after graduation and spent the summer with my family, which included going to my brother's soccer tournaments with his club team.

There was a kid named Cody on that team. We sat with his parents during most games. One day, as I talked about my lack of a job and my ever decreasing bank account, Cody's mom mentioned they needed a temp in her department at the University of Oregon. I sent my resume to the business manager the next day and they hired me over the phone.

It's been 14 years, and I'm still here...thanks to my brother playing on the same soccer team as Anne Todd's son.

Do you ever think about the random string of events that led you to where you are today? Like how you juuuuuust missed that 3-car fender bender because you couldn't find your keys so you left the house five minutes late. The way a small choice ripples to a larger impact on our lives is known as the butterfly effect. Science will tell you it's all part of chaos theory and using the theory to create order to our lives is a misuse. However, if outcomes are just a series of choices strung together, we should take a closer look at the choices we make to see if we can't create the outcomes we want.

Origin of a Referral

We're big fans of the office discipline referral (ODR) around here. An ODR, in our eyes, isn't a punishment. (It turns out, it's really hard to punish someone with a piece of paper.) An ODR tells a small story about a student's behavior that someone saw and how that person reacted to it. It's a piece of data. According to research, it's a good measure for predicting teacher perceptions, student perceptions, chronic behavior issues, and academic outcomes.[1] In nearly every study where ODRs are included as a data source, you'll find this sentence...or something close to it:
"...ensure that all school's existing referral systems (are) sufficient to support meaningful and usable referral data..."
Your various systems work together to make sure you're collecting the right information in the right way at the right time. When it comes to improving student outcomes, that butterfly effect starts with your discipline systems. Here's how...and we'll throw in a strategy for intervening earlier with students needing additional support, just for good measure.

Using Consistent Procedures...

Every school has a way of handling student problem behavior. That way boils down to four parts working together to create a strong discipline procedure:

1. A consistent referral process

2. Definitions for problem behaviors

3. A referral form to document the event

4. Proactive strategies for responding to problem behaviors

When every referring adult agrees to these four components, you can be sure your data will be consistent. Our example starts with the procedures schools use with the School-Wide Information System (SWIS). All schools using SWIS agree to use a common referral form with defined categories and labels. Each school has a procedure for when to handle behaviors in the classroom and when to send a student to the office. The referral process also defines what happens next. If nothing else, SWIS schools have a documented, consistent way to manage student behavior. Each referral can be compared reliably to every other referral collected at the school; and if the sample is large enough, we can compare each referral to all other referrals collected nationally in the SWIS database.

Leads to Reliable Data...

In the same schools use their SWIS data to look for patterns, researchers also wanted to see what trends they could find in the national database.[2] As you might expect, there are common themes all over the place. The first that stands out is the behaviors most likely to get an elementary or a middle school student referred - fighting and defiance, respectively. Taken at face value, it's not so interesting, but keep going with me and you'll see how it fits.

In addition to research questions, PBISApps also comes out with national trend data based on the referrals entered in SWIS during the previous school year. For example, in 2017-18, the median average number of referrals written per day in elementary schools was about 1 referral for 500 students. During that same school year, about 8% of elementary schoolers, 13% of middle schoolers, and 13% of high schoolers received 2 or more referrals.
Still with me?
Students with more than two referrals typically need additional support for their behavior; these students often don't appear on anyone's radar until late in the year after they've accumulated their third, fourth, or fifth referral. If schools could intervene earlier, they'd have the ability to change the trajectory of a whole group of students' entire year for the better.

So Teams Can Anticipate Problems...

Enter two studies - one in 2014, one in 2010 - aimed at determining how early we can predict which students will be referred to the office more than twice during the year[3],[4]. Turns out, we can predict that, and we can do it as early as October. In fact, 50% of elementary students and 71% of middle school students who ended the year with six or more referrals had at least two referrals by the end of October. (Whew, that was a mouthful.) Also, if students had at least one referral by October for one of two behaviors, their odds of accumulating more than six referrals by the end of the year went up. Can you guess which two behaviors they were?

Fighting and defiance: The most referred behaviors in elementary and middle school.

This means, you have an opportunity to break the cycle and help students build a new path.

And Improve Student Outcomes

Start by finding out which students have one or more warning flags by the end of October:

Two or more referrals

At least one referral for fighting

At least one referral for defiance

Save these names and check in on them again at the end of November. If any student continued to rack up referrals, rally your Tier 2 and Tier 3 teams around them and see what support they might need.

Do you see what we just did? All of these trends and patterns and predictions are all possible because of where we started: consistent procedures. The agreements you make about your discipline procedures directly affect your ability to support students to achieve better outcomes. With all our talk about using data to make decisions, we should spend a little more time talking about the systems surrounding those data - and that's just what we'll do over the next three months. Join us for our upcoming series as we explore creating efficient referral processes, defining subjective behaviors, and ensuring equitable responses to problem behaviors.

1. Irvin, L., Tobin, T., Sprague, J., Sugai, G., & Vincent, C. (2004). Validity of Office Discipline Referral Measures as Indices of School-Wide Behavioral Status and Effects of School-Wide Behavioral Interventions. Journal Of Positive Behavior Interventions, 6(3), 131-147. doi: 10.1177/10983007040060030201
2. Spaulding, S., Irvin, L., Horner, R., May, S., Emeldi, M., Tobin, T., & Sugai, G. (2008). Schoolwide Social-Behavioral Climate, Student Problem Behavior, and Related Administrative Decisions. Journal Of Positive Behavior Interventions, 12(2), 69-85. doi: 10.1177/1098300708329011

3. Predy, L., McIntosh, K., & Frank, J. (2014). Utility of Number and Type of Office Discipline Referrals in Predicting Chronic Problem Behavior in Middle Schools. School Psychology Review, 43(4), 472-489.

4. McIntosh, K., Frank, J., & Spaulding, S. A. (2010). Establishing Research-Based Trajectories of Office Discipline Referrals for Individual Students. School Psychology Review, 39, 380-394.

# data[].attributes.attachment_link.title
PBISApps | Teach By Design-Consistent Process + Reliable Data = Better Results

# data[].attributes.attachment_link.description
Your school's systems work together to make sure you're collecting the right information in the right way at the right time. When it comes to improving student outcomes, everything starts with your discipline systems.

# data[].attributes.headline
School Climate Survey Administration Part 1: Preparation 

# data[].attributes.subject
School Climate Survey Administration Part 1: Preparation 

# data[].attributes.attachment_link.title
PBISApps | School Climate Survey Administration Part 1: Preparation

# data[].attributes.attachment_link.description
Video: ​In this video Celeste Rossetto-Dickey demonstrates how to administer the School Climate Survey with tips on who should take which survey.

# data[].attributes.attachment_link.title
PBISApps | Equity Report: Drill Down Tool

# data[].attributes.attachment_link.description
Video: In this video Bert Eliason explains how to get the data to answer some questions by digging deeper related to​ equity using the Drill Down tool in SWIS.

# data[].attributes.headline
Proven Ways to Make School-wide Feedback Work 

# data[].attributes.subject
Proven Ways to Make School-wide Feedback Work 

# data[].attributes.attachment_link.readable_text
Have you ever filled out one of those customer satisfaction surveys circled at the bottom of your receipt? The other day, a Sephora employee offered me some free samples if I could take five minutes to fill out a survey about my experience that day. I'm a sucker for a sample and I didn't have anywhere I had to be, so I said "Sure!" I scored our experience on various scales of 0 to 10 and told them how Amy helped us find the perfect curl cream for our wavy-haired kiddo at a discounted price.

...and Reader, the samples were worth the time. That leave-in conditioner smelled divine!

I'm not the first person to fill out a customer satisfaction survey at Sephora. Every year that company collects more than 800,00 surveys to follow-up on in-store customer experiences and to identify where employees might need more training. By integrating customer feedback in their decisions, Sephora's customer loyalty ratings are up 6%, which is among the highest in the industry. Allegra Stanley, Sephora's Vice President of Customer Loyalty, says they've been so successful because they remain committed to delivering what customers tell them they want. "It's not about what their loyalty demonstrates to us, but what we can deliver to our clients that creates the most meaningful and connected experience with our brands."[1]

In the same way Sephora uses customer feedback to improve their practices, it makes sense that your team would ask for your school-wide community's perspectives to better inform your decisions.

Researchers tell us it's actually risky not to ask.

According to those researchers, "when individuals cannot contribute their perspectives to the pursuit of a collective effort, their silence takes an emotional toll. If this silence occurs because the system does not allow people to speak up, people believe themselves undervalued and disrespected."[2] On the other hand, inviting students to join in your evaluation process does more than provide them with a platform to share their experience. Researchers found it can be a transformational experience - one that encourages deeper reflection and fosters a greater sense of connection to the program itself.[3]

An easy place to start soliciting feedback from your larger school community is through the School Climate Survey (SCS) Suite. Each survey provides a member of your school community with a space to share their perspective on how it feels to show up at school every day. There's a survey for elementary students, two surveys for middle and high school students, a survey for families, and a survey for school personnel.

Each climate survey is available for folks to take online in PBIS Assessment. As soon as someone submits their feedback, it's added to your totals so you can see the data in real time. There are a variety of reports available including:

Total Score Report

Subscale Report

Items Report

Disaggregated reports based on demographic data like race/ethnicity, gender, sexual orientation, and grade

Using an app makes summarizing all this feedback easier and PBIS Assessment is free... even the training!

The feedback you get from the SCS offers insights you may have never considered before. For example, in a sample of 721 elementary, middle, and high schools, the climate survey item students scored the lowest was: There is an adult at school who will help me if I need it.[4] That's a tough thing for a student to open up and say face-to-face to an adult. It's a lot safer to share in an anonymous survey.

Getting students, families, and school personnel to take the SCS is the first step. What do you do with all those data once they come in? Well, you go through it all one step at a time.

Maybe we could start our action plan by coming up with how to create a safe space where we invite students to talk openly about their experiences. What would that space look like? What would it feel like? Who would we invite?

Step 1. Close the Loop

When you've taken a satisfaction survey, did anyone ever tell you what they planned to do with your feedback? It's rare. When the climate survey windows close, let your schoolwide community know what you plan to do next. Are you reviewing the scores at your next team meeting? Let them know. Are you planning to share some the survey reports with everyone? Tell them when. Will you add the information to your regular communications with students, families, and personnel? Get the deadline on someone's calendar. Close that loop and let your community know what you're going to do next.

Step 2. Paint a Big Picture

Each of the climate surveys comes with a total score report, but if you ask me, the real gems are in the subscale reports. These reports break the surveys up into sections so you can see which systems require more attention. Here's an example. As a high school team, you sit down to review the results of the extended version of the climate survey your students took, and you see this. What are your takeaways?

The first thing I notice is there are some positive notes to celebrate; I also see there's quite a bit of room to improve. Specifically, our students are telling us they don't feel like they fit in at our school and they don't feel as supported by the adults in our building as we'd like them to be. Those are hard topics to talk about face-to-face. Maybe we could start our action plan by coming up with how to create a safe space where we invite students to talk openly about their experiences. What would that space look like? What would it feel like? Who would we invite? The list of students we ask to participate might depend on what our data continue to tell us.

Step 3. Break Up the Data

Checking on how your SCS data break out demographically gives you another lens into your students' experiences. It could be, on average, all students in your school share a similar perspective on climate. It could also be that a specific group of students has a very different experience in your school than everyone else.

Every climate survey comes with a way to break the data out demographically. So, do that! In the example above, when we look at the data across grade levels, we notice that 8th graders scored our school climate the lowest out of everyone. As we put our focus groups together, maybe we could create one focus group specifically for 8th graders to share with us (and each other) what we could do to improve.

Step 4. Check Other Sources

The SCS information is one piece of the puzzle. There are so many data sources you have at your fingertips. How does this feedback compare with your discipline and behavior data from the SWIS Suite? How does it match with the Tiered Fidelity Inventory (TFI) scores you submitted as a team? Check in with other teams in your building. What are the stories their data tell? Don't stop with one dataset; check on other sources.

To understand how your systems and practices affect the people in your building, you need to ask. Soliciting feedback from your schoolwide community is the one way to ensure your implementation meets their needs and generates the outcomes you expect. How do you wade through all that data? Start by reviewing the big picture. What do the subscales tell you about where you can focus your attention? Dive deeper into the dataset and check on individual groups. Does everyone experience your school the same way or is there one group experiencing things differently? Finally, what does this dataset make you curious about in other data sources? Connect the dots between the feedback you have in front of you and the other datasets available in your school. The amount of information you collect could be overwhelming, or you could just take it one step at a time.

[1] Danziger, P. (2020). How To Make A Great Loyalty Program Even Better? Sephora Has The Answer. Forbes. Retrieved 3 March 2022, from https://www.forbes.com/sites/pamdanziger/2020/01/23/how-to-make-a-great-retail-loyalty-program-even-better-sephora-has-the-answer/?sh=31917035287a

[2] Mason,S. (2022). Disempowerment evaluation and the risks of avoiding stakeholder feedback. Counseling Outcome Research and Evaluation, 1-10.

[3] Odera, E. L. (2021). Capturing the Added Value of Participatory Evaluation. American Journal of Evaluation, 42(2), 201-220.

[4] La Salle, T. P. (May, 2020). How are Schools Using the School Climate Surveys? University of Oregon. www.pbis.org

# data[].attributes.attachment_link.title
PBISApps | Teach By Design-Proven Ways to Make School-wide Feedback Work

# data[].attributes.attachment_link.description
So, you asked your students, families, and staff to take a survey about your school climate. What do you do with all the data you collected? You go through it all one step at a time.

# data[].attributes.attachment_link.description
Manual: This manual helps you navigate and use the PBIS Assessment application.

# data[].attributes.attachment_link.description
Get to know your PBIS implementation with our free, online application for entering surveys like the TFI and the School Climate Survey.

# data[].attributes.headline
Overview & Logic of District-wide Implementation 

# data[].attributes.subject
Overview & Logic of District-wide Implementation 

# data[].attributes.attachment_link.readable_text
Initiating, expanding, and sustaining PBIS at the school level requires systemic support from the district. This session will provide an understanding of how districts can build internal capacity to implement PBIS district-wide. District leaders will share their experiences with developing district-wide systems supporting implementation, developing their workforce, and striving for sustainability.

# data[].attributes.attachment_link.title
Center on PBIS | Video: [Session I1] PBIS Forum 2021: Overview & Logic of District-wide Implementation

# data[].attributes.attachment_link.description
Initiating, expanding, and sustaining PBIS at the school level requires systemic support from the district. This session will provide an understanding of how districts can build internal capacity to implement PBIS district-wide. District leaders will share their experiences with developing district-wide systems supporting implementation, developing their workforce, and striving for sustainability.

# data[].attributes.attachment_link.readable_text
When you talk about your PBIS implementation, there are two questions to regularly ask:

Are we doing what we said we would do?

How does our implementation affect the people in our school?

Taking the Tiered Fidelity Inventory (TFI) informs the first question; taking the School Climate Surveys (SCS) informs the second.

What Does the SCS Do?

The SCS is a suite of five surveys to measure student, staff, and family perceptions of school climate. Each survey is a brief, valid, and reliable way to ask stakeholders how your school feels to them.

The SCS is also an anonymous, multiple-response survey. That means, once you open the survey windows, you'll get a series of links to send around to the affiliated groups. It should only take them around 10-20 minutes to complete. As responses come in, PBIS Assessment summarizes them into reports addressing each group, providing you with an overall picture of your implementation's impact on the people learning, teaching, and participating in your school every day.

Which Surveys Should I Use?

There are five surveys available to take within the SCS suite. Open a survey window in PBIS Assessment for as many of them as are appropriate for your school:

Elementary: For students between grades 3-5. (If your school includes grade 6, have them complete the elementary survey, too.)

Secondary (brief) and Secondary (extended): For students between grades 6-12.

School Personnel: For teachers, administrators, and anyone else working in your school.

Family: For parents, caregivers, and guardians of students enrolled at your school.

When Should I Take the SCS?

Schools take the SCS once or twice during the school year. If you take it once, you can easily watch how perceptions change from year-to-year. Taking the SCS twice a year helps you notice the major and minor changes in perceptions from the beginning of the year to the end.

What Happens After I Submit the SCS?

PBIS Assessment takes every climate survey submitted within a given window and summarizes them into the reports your team needs to improve school climate in your building. The reports available for the School Climate Survey vary depending on the survey you take.

The Total Score Report gives you an average score your participants gave as they took the survey. The closer a score is to five, the more positive participants feel about your school's climate.

A Subscale Report is available for the Family and School Personnel surveys. Subscales relate to specific areas of your school's climate that participants feel are working well or could be improved.

Drill down further into the data by reviewing the Items Report and identify some of the individual elements to add to your action plan.

Take another look at the data based on the demographics of the people who took the survey. See how your school feels to students, families, and school personnel depending on their race/ethnicity, gender, grade, or (for middle/high schools only) sexual orientation. To make sure responses remain anonymous, if a demographic subgroup has fewer than five respondents associated with it, the results won't display. However, their responses are always included in the Total Score, Subscale, and Items reports.

The School Climate Survey helps teams gain insight into how school personnel, students, and their families feel when they walk into your building. Use their responses as a starting point to create the kind of place where everyone wants to be.

For more details on taking the SCS, check out these resources.

# data[].attributes.attachment_link.description
The SCS is a suite of four surveys to measure student, staff, and family perceptions of school climate. Each survey is a brief, valid, and reliable way to ask stakeholders how your school feels to them.

# data[].attributes.attachment_link.description
Video: In this video Megan Cave shows assessment coordinators how to add users to organizations in PBIS Assessment.​​​​

# data[].attributes.headline
Building Momentum for PBIS Implementation in High Needs Districts 

# data[].attributes.subject
Building Momentum for PBIS Implementation in High Needs Districts 

# data[].attributes.attachment_link.title
Center on PBIS | Resource: Building Momentum for PBIS Implementation in High Need Districts

# data[].attributes.attachment_link.description
PDF: Building district capacity for implementing PBIS in high need school districts is challenging. This practice brief summarizes the lessons learned one year into an intensive 5-year partnership with Southbridge Public Schools, a district identified as high-need.

# data[].attributes.attachment_link.description
PBISApps empower educators to make data-driven decisions in real time for the best student outcomes. The SWIS Suite: Powerhouse reports for schools. PBIS Assessment: Measure effectiveness and school climate for free. PBIS Evaluation: Reports for districts, regions & states.

# data[].attributes.attachment_link.readable_text
Dr. Kent McIntosh shares four tips to sustain PBIS, gleaned from a 5 year study involving data from 860 schools. Also included is four checks to see if your school is on the right track.

Classroom PBIS

School-Wide

Data-based Decision Making

# data[].attributes.attachment_link.title
Center on PBIS | Video: Four Tips for Sustaining PBIS

# data[].attributes.attachment_link.description
Dr. Kent McIntosh shares four tips to sustain PBIS, gleaned from a 5 year study involving data from 860 schools. Also included is four checks to see if your school is on the right track.

# data[].attributes.headline
Drilling Down District Data: Analyzing Reach and Fidelity of PBIS Implementation 

# data[].attributes.subject
Drilling Down District Data: Analyzing Reach and Fidelity of PBIS Implementation 

# data[].attributes.attachment_link.title
Drilling Down District Data- Analyzing Reach and Fidelity of PBIS Implementation

# data[].attributes.headline
PBIS District Systems Fidelity Inventory (DSFI)

# data[].attributes.subject
PBIS District Systems Fidelity Inventory (DSFI)

# data[].attributes.headline
PBIS Cultural Responsiveness Field Guide V2

# data[].attributes.subject
PBIS Cultural Responsiveness Field Guide V2

# data[].attributes.headline
PBIS Apps Local Coordinator Information Form 

# data[].attributes.subject
PBIS Apps Local Coordinator Information Form 

# data[].attributes.body
https://ohioleadership.org/news-and-events/showcase?mc_cid=f64216e268&mc_eid=6505a9471f