Statistics Without Context

This article is part of the “Teaching Deep” series (checkout #TeachingDeep on Twitter). The purpose of these posts, as outlined in the introduction, is to perform a sort of deep cleanse of my teacher soul, to reflect on the practice and the art of teaching. Each post will be grounded in an examination of something that I’m doing (or have recently done) in my classroom as part of my teaching practice. Typically, it will be something that I’m not super-thrilled about, but not always. Perhaps you’ll identify. Hopefully, you won’t!

Recently, I was in a conversation with my neighbor. He  works in the warehouse for a chain of grocery stores across Texas. He said to me, “Last week we moved 400,000 units.”

I lifted my brow, nodded my head, and sipped my coffee. “Wow,” I said, “400,000, huh?”

I didn't really know what to say. I’m a teacher and a writer. That number means nothing to me. (I don’t even know if I’ve reported it accurately here, to be honest.) It’s a statistic without context.

As a teacher of writing and research, I caution my students about statistics. I tell them that you’ve got to know as much about a statistic as you possibly can. You've got to investigate the context.

• Who is reporting it?
• How was it obtained?
• How does it compare to other measures?
• What is it really measuring?
• Etc.

Keep asking questions about those statistics until you’re sure that you know everything about what they could possibly mean. Do not simply assume that you know what it means.

In short: BEWARE STATISTICS.

Thinking about my neighbor’s statistic, I had no idea how big a “unit” was and how this compared to what the warehouse normally does. An answer to one of these questions would be helpful. Fortunately, he did provide me with more context, but more on that later.

Right now, I want to talk about teaching.

As teachers and parents, we deal with context-free statistics all the time: Grade Reports. Grades, usually, are statistics reported without much context. I open my son’s grade report and I see a series of letters and numbers, but I have very little idea what they actually mean. It might say, for example, that he earned a “92” in “Math.” Naturally, my head starts jumping to the questions that I would tell my student-writers to ask:

• What does the "92" mean?
• What do we mean by "Math"?
• What are they doing in "Math"?
• How was the number recorded?
• How does the number compare to other measures?
• Etc.

As parents, we often take a look at the grade report and interpret it based on a series of assumptions that we’ve carried over from our own days as students:

• A 92? That must mean that he got 92% of math questions correct. Not bad!
• A 92? That’s an A-! Pretty good, especially considering his parentage…
• A 92? Hm. What’s going on with the other 8 points? Why is my kid deficient? Shouldn’t he know all of the math now that he’s in third grade?

We tend to bring our own baggage, good or bad, to our children’s grade reports, and we use that baggage as the foundation for our interpretation of a statistic without context.

Older students do the same thing.

Each year, I have a conversation with students where I ask them what a grade means. I might say, “What’s an ‘A’? What does that mean?” The first response is always: “It means that you worked hard.”

“Really?” I say. “Have you ever earned an A without working very hard? Or maybe you know someone who earns A’s without putting in a whole lot of effort?”

The students nod their heads. They all know who that kid is who doesn't seem to do much, yet she always seems to make the grade.

“Well, that’s a counterexample,” I say. “If you know someone who has earned an A without working hard, then an A can’t necessarily mean that you worked hard. It might mean that, but it might not.”

We continue this conversation, unpacking their assumptions, frustrating them with counterexamples and more questions about the context.

STUDENTS
Perhaps the A means that you know a lot of the material that was covered.

ME
But sometimes we get A’s when we guess.

STUDENTS
Maybe it just means that you answered a certain percentage of the questions correctly on tests.

ME
Okay, but have you ever earned a grade for something that wasn’t a test, like following directions or even just being present? Or maybe part of your grade is about whether or not you handed in an assignment on time?

Eventually, the students give up and we all pretty much agree on this basic fact:

The grade means whatever the teacher says it means.

You’ll get no argument from me there.

I will write about this topic much more in the future, thinking about the ways that I’ve both succeeded and failed in creating meaningful grading practices. (As part of this #TeachingDeep series, an examination of my grading practices may not be very kind to me. We’ll see…😬)

But, right now, I’d rather look at a positive change that is happening today. My son’s school, in an effort to think meaningfully about learning during the COVID-19 pandemic, has decided to eschew traditional grade reports. Instead, the school has scheduled parent-teacher conferences so that we can have a conversation about our child’s learning.

On their face, and from a teacher's perspective, these conversations may look cumbersome—15 or 20 minutes with each parent—but the advantages are obvious. Consider, for example, that ’92’ in ‘Math.’ In this conversation, I will be able to ask questions about Gus’s performance in Math. I will be able to learn more about the context so that I can find what those grade report statistics really mean.

As I ask these questions, I might find that that number is incredibly meaningful. Gus is doing a good job, but we need to think more about his approach to number theory: he needs a more open and conceptual mindset rather than a fixed mindset that looks for recipes.

Or maybe we’ll find that the 92 reflects things that have little to do with Math. Maybe, for example, his math skills are great, but he doesn’t follow directions well.

Or maybe we’ll find that that number values something that we don’t find particularly valuable. It could, for example, value speed in arithmetic which many studies have debunked as a good way to learn math.* Fine. I’m not going to put too much stock in a "92" in "Math" if that’s what those 8 points value.

Because this conversation is baked into the system, I don’t have to feel like a nagging or intrusive parent as I try to figure this stuff out. The conversation is welcome. It's the modus operandi; it's, to steal a new cliché, the new normal.

If all goes well, then we’ll have an open and honest conversation about our child’s performance against a set of standards or values that the school (and/or his teacher) have deemed essential. We will be able to think about those standards and values, reinforce the ones that need reinforcing, and de-stress the ones that are less important to his development.

A conversation allows for the give-and-take that provides context and nuance around a grade report. It allows the learner (or, in this case, the learner’s parents) to think about what the report really means, what it really values.

Our conference with our son’s teacher is this afternoon. I’m looking forward to it!

As for those 400,000 units that my neighbor’s warehouse was moving, that was 25% more than their busiest period last year: the weeks leading up to Thanksgiving. How’s that for some context?

A FOOTNOTE!

* Check out youcubed for some thoughts about speed and depth. You might also read Jo Boaler's Limitless Mind (especially, "Chapter 5: Why Speed Is Out and Flexibility Is In!") for a thorough—but readable—treatment of this topic.

Questions/Comments/Ideas?

Did you identify with something above? Did it strike a nerve? Do you have something to say? Get in touch on Twitter and please make sure to use the #TeachingDeep hashtag!

Also, please consider subscribing to my email list to get more posts about teaching, writing, and parenting in your inbox!