YouCubed is Sloppy About Research

I.

There’s a lot of sloppy talk of science that gets tossed around education. Every teacher knows this — or least, I hope they do — and I try not to get too worked up about it. I also try not to get worked up by people who wear backpacks on crowded subways. Not that it’s OK, but lots of people do it (the backpack thing) and picking any one person to bark at hardly seems like it would help.

The other thing is that people can be really passionate about sloppy science. This happens all the time, it’s nothing special about education. But passion makes it hard to talk critically about the research without it seeming like you’re attacking everything else that the person is passionate about.

I think a lot of the time it’s because we assume that the attack on the research isn’t really about the research, it’s about everything else it supports. It’s like, Why are you bothering to poke holes in [research that supports X]? You’d only do that if you were really against [X]. 

Which leads to an interesting question. Is it possible, at all, to avoid this trap? Is it possible to critique sloppy use of research without being heard as if you’re trashing a person, their organization, and everything they stand for?

II.

What if you ask a lot of rhetorical questions — does that help?

Anyway, let’s talk about YouCubed. I think there’s something that — if we slow down, turn off passion, turn on curiosity — we can agree is a serious mistake. Here’s a popular quote from their popular page, Mistakes Grow Your Brain:

When I have told teachers that mistakes cause your brain to spark and grow, they have said, “Surely this only happens if students correct their mistake and go on to solve the problem correctly.” But this is not the case. In fact, Moser’s study shows us that we don’t even have to be aware we have made a mistake for brain sparks to occur.

When teachers ask me how this can be possible, I tell them that the best thinking we have on this now is that the brain sparks and grows when we make a mistake, even if we are not aware of it, because it is a time of struggle; the brain is challenged and the challenge results in growth.

Indeed — this does sound really, really surprising! So, applied to math, it sounds like if you solve an equation incorrectly you learn from that experience (brain sparks?) in a way that you wouldn’t if you had solved it correctly. The mistake you made causes struggle — even if it’s not a struggle that you’re aware of. You might not feel as if you’re struggling, but your brain is on account of the mistake.

The citation of Moser is very clear, so, ok, let’s go upstream and check out Moser. Though a lot of research is behind paywalls, a lot of it isn’t and a quick Google Scholar search gives us a copy of the paper, here.

The study was an fMRI (EEG, see below. -MP) study. Here’s my understanding of the paper. The researchers wanted to know, on a neurological level, what’s different about how people with a growth mindset or a fixed mindset react to mistakes. So they gave everybody a test, to figure out if they had a fixed or a growth mindset. Then they hooked subjects up to fMRI EEG machines. While in the machines, participants worked on a task that yields a lot of mistakes. Researchers recorded their neural activity and then analyzed it, to see if there was a meaningful difference between the fixed and growth mindset groups, after making errors.

Spoilers: they found a difference.

Also: the whole paper is premised on people being aware of the errors that they made. This is not a subtle point, buried in analysis — the paper mentions this like nine times, including towards the end where they write, “one reason why a growth mind-set leads to an increased likelihood of learning from mistakes is enhanced on-line error awareness.”

This is so clearly different than what the YouCubed site says that I’m starting to doubt myself. The paper seems to be entirely about what happens when you realize you’ve made a mistake. Yet it’s cited as supporting the notion that you learn (“brain grows”) from mistakes, even if you aren’t aware of them.

Is there something that I’m missing here?

III. 

This doesn’t seem to me like an isolated issue with YouCubed.

A while ago Yana Weinstein and I made a document together trying to collect errors in YouCubed materials, explain why, and suggest improvements. I don’t have much else to say about that, except that if you’re interested you might check it out here.

IV.

What strikes me about YouCubed is that the errors just seem so unnecessary. The message is a familiar one, and I’m OK with a lot of it: don’t obsess over speed, think about mindset, don’t be afraid of mistakes. But there’s this sloppy science that gets duct taped on to the message. What purpose does that serve?

There’s also the question of why so few people in the math education community talk about this. I mean, it’s not like we lack the critical capacity. Every so often I see people in math education whip out their skeptical tools to tear apart a piece of research. Why not with YouCubed?

I don’t want to be cynical, but I want to be truthful. The first reason, I think, is because the message of YouCubed (besides the science) is widely admired. A lot of teachers love it, and nobody wants to ruin a fun time.

But I don’t think we have to worry about that. We can talk about the science of YouCubed in a way that doesn’t entangle the rest of the YouCubed message.

The second reason is, I think, that YouCubed and Jo Boaler’s popularity makes it difficult for the most visible people in math education to seem critical — no one wants to turn on one of their own. Especially since Boaler has often been subject to unreasonable attacks in the past — nobody wants to be unfair, cruel or sexist to her.

I’m actually very sympathetic to that. But this is also why it’s important for people who aren’t part of the research or PD world to have platforms to discuss ideas. We don’t have the reputation or the connections to lose, and so we can take a closer look and ask, wait, does that really make sense?

Corrections:

Actual neuroscientist Daniel Ansari points out that I don’t know the difference between an EEG and an fMRI, which is true.

Excellent post. One thing: Moser et al. used EEG not fMRI

— Daniel Ansari (@NumCog) October 17, 2017

 

Advertisement