When Analyzing Student Outcomes, Don’t Lose the Forest for the Trees

A large part of our work is spent around small conference tables with various configurations of teachers, instructional coaches, school administrators, and district leaders, engaged in what we call “instructional problem solving.” These are structured conversations designed to use the best evidence available to identify strengths, understand what isn’t working, and make distinct commitments for improvement. We include the modifier instructional because understanding what happens in the classroom between teachers, students, and the content, is critical to improving academic outcomes. There are other types of problem solving that matter, but this where we have placed our bets.

In the course of hundreds of these sessions, we have become acutely aware of a common habit that routinely impedes instructional problem solving—fixating on individual student outcomes without absorbing the big picture the data are revealing. We have learned this is not an intentional behavior but often an unconscious defense mechanism for facing hard truths. The good news is being aware of this tendency goes a long way toward reducing its power.

One of the core visuals we provide to teachers in their midyear portfolio is designed to illustrate the overall effectiveness of their Tier 1 instruction (i.e., the materials, tasks and interactions that all students encounter routinely, which can be small or whole group, and is typically differentiated to some degree based on formative assessment). In the MTSS framework, about 80% of students should respond favorably to core instruction, meaning they are learning quickly enough to stay on pace with their peers. When the ratio falls below 80%, the framework requires that we ask what can be strengthened in the core before assigning interventions.

When we begin working with a school, the reality is we find very few classrooms where 80% of students are on pace to make a year’s growth. It is more typical to see around 50%. However, if we open a conversation by asking the teacher, “What do you see in your student outcomes?” the response, more often than not, sounds something like this: “Well, Johnny, at the top of the list here, hasn’t been coming to class. Rosa completely rushed her assessment. Damien, bless his heart, can’t sit still long enough to read directions. I can’t get Belinda to take her head off the desk. Cindy reads so well but gets test anxiety. She probably needs medication, poor thing.” And so on. They will tick through the sad story of every student on the page if we let them. It is a rare occasion when a teacher responds by saying, “What I’m currently doing is not serving the needs of most.”

While it is important to understand the challenges in each student’s life, there is a danger in starting the problem solving at a granular view. If we don’t first zoom out and look for patterns, not only might we place the burden of change on the individual student, we will certainly miss powerful opportunities to uncover issues with core instruction that are within the control of the teacher. Left unchecked, we end up with overwhelmed ESE specialists, bloated intervention budgets, and plenty of blame to go around when students never get “back on track.”

The primary purpose of an MTSS framework is to use data, not to determine which interventions are needed, but to determine how we can prevent the need for intervention in the first place through solid core instruction.

Here is one thing that every teacher can do to move in this direction. When you get your next round of progress monitoring data: Cover up your student names (literally) for the first pass. Big picture, what percentage of students are on pace to make a year’s growth? Next, ask which subgroups (e.g., race, gender, disability, prior year achievement level) are growing the most and least quickly? What is different about how you routinely interact with the students in these groups? What is different about the content they are exposed to or the tasks they are asked to engage in?

Make a hypothesis about something you can change in the classroom that is likely to benefit the most students. Only then, uncover the names and use your situational awareness of the students who are growing least quickly to determine who is likely to need intervention in addition to stronger Tier 1 instruction.

Want to learn how K12 Lift can help your teachers make meaning of their student growth data? Click here to start a conversation.


Read Next: Confronting Difficult Data: Grieve, Reflect, Act