Why Mythbusting Fails: A Guide to Influencing Education With Science

By Michael Pershan and Benjamin Riley. We wrote it together, and Ben posted it over at the Deans for Impact blog. I think there are a lot more educators than scientists who read my blog, but pretty much everything but the details apply to an educator interested in helping scientists understand classroom teaching better. Or your daily life, political persuasion, arguments with a friend, etc. – MP

The persistence of neuromyths

“If it disagrees with experiment, it’s wrong,” physicist Richard Feynman said. “In that simple statement is the key to science.”

By this measure, the learning-styles hypothesis has failed too many times to count. Experiment after experiment has shown that matching the form of instruction to a student’s preferred “style” of learning – such as auditory or visual – does not improve a student’s understanding. As a result, the vast majority of cognitive scientists are certain that learning styles have been debunked.

And yet many educators still believe learning styles are important. What gives? Why don’t they trust the science?

One possibility is that educators are simply unaware of the research undermining learning styles’ usefulness. If this is the problem, then there is a simple solution: just spread the word!

To that end, recently a group of 30 learning scientists – including Steven Pinker, Hal Pashler and others – published a letter to inform teachers that learning styles is a “neuromyth” that “create[s] a false impression of individuals’ abilities, leading to expectations and excuses that are detrimental to learning in general, which is a cost in the long term.”

But what if educators are presented with this information…and they still don’t change their minds?

Take Terry J., a lifelong public educator from Canada. Terry read the scientists’ letter, but it failed to convince her to change her practice. “I know that I am a visual learner,” she said. “There is research that supports the idea that learning styles matter, and there is research that says learning styles are bunk.”

Perhaps Terry just “hates scientists.” But this plainly isn’t true – Terry is very interested in science and research that can inform education practice. In fact, she helps design professional development for her province, and she and her team use educational research to plan their workshops. She even consults with a research foundation located just down the hall from where she works.

Terry’s seen the reports, but – in a striking bit of symmetry – she believes that when it comes to learning styles, it’s the research community that is uninformed. The perspective of the letter-writing neuroscientists, she said, is just “based on different assumptions and interpretations.”

The scientists who signed this letter would of course find much to critique in Terry’s views. But if their letter was intended to persuade anyone, surely Terry sits squarely within the target market. She is a public servant, protected from profit motives, with a longstanding interest in applying educational research in practice.

If these scientists can’t persuade Terry, who can they?

The challenge of cultural cognition

Terry’s resistance to scientific authority might seem bizarre – certainly it will cause some learning scientists to shudder. Yet this sort of resistance is not unique to education. In fact, it’s easy to find examples of resistance to scientific insights even in fields that are thought to be highly scientific, such as medicine. And, even more paradoxically, this resistance to scientific evidence can coexist with a strong trust in science and scientists.

How can this be? The answer may lie with what some researchers describe as “cultural cognition.”

Cultural cognition describes how we interpret certain facts and evidence through the lens of our existing values. Usually, we accept scientific claims as true because, overall, most of us trust science and scientists. But – in rare but notable cases – our stance on a scientific matter comes to take on a larger, much more personal meaning. Beliefs about science can become entangled with our self-identities, even if they didn’t start out that way.

Take climate change. Despite mounting evidence and a clear scientific consensus on the relationship between human activities and the rise of global temperatures, beliefs about the cause of global warming are growing more polarized in the U.S., rather than less. From the cultural-cognition perspective, this is largely because beliefs about global warming have become statements about who we are – namely, whether we self-identify as environment-protecting liberals or industry-defending conservatives. This means that increasingly Americans not only disagree on the level of risk posed by global warming, but whether there is even a scientific consensus.

This outcome was not inevitable. But many climate-science advocates emphasize the (perceived) ignorance or anti-science attitudes of those who don’t understand global warming or its causes. In doing so, these advocates effectively insult precisely the people they wished to persuade. The tragic result? The normal bipartisan trust in science has become “polluted” on climate change, in part because of communication strategies that have antagonized existing values, and activated cognitive defenses.

The perils of threatening teacher autonomy

So now let’s return to education and learning styles. What we want to suggest – tentatively and with caveats – is that we run the risk of polluting the environment on learning sciences in the same way people have polluted the climate-change communication environment.

In particular, we worry that some researchers do not fully appreciate the importance that educators place in their own autonomy. Teachers are the ultimate deciders of what takes place in their classrooms, an autonomy that provides them with a major source of professional satisfaction. Teachers may not receive high wages or status, but they do receive tremendous psychic rewards when students appear to learn as a result of their decisions. And educators possess a great deal of (reasonable) sensitivity in protecting this autonomy, as there is a long history of “outsiders” seeking to tinker with what happens within districts, schools and classrooms.

So the scientific consensus that learning styles do not exist will become irrelevant if educators come to see their beliefs about learning styles as critical to their professional autonomy. And one way to heighten the risk of that happening is through talk of what science demands teachers do or believe.

Now, the caveats. We don’t have any direct evidence that teachers currently believe in learning styles because they see it as necessary to establish their professional identities. Nor are we familiar with any evidence tracking changes in educator beliefs about learning styles over time. For all we know, educators who read the recent letter from 30 scientists denouncing learning styles as a “neuromyth” are busy reshaping their beliefs and changing their practice.

But suppose that our analysis is correct. This poses a very delicate dilemma for advocates of learning science (ourselves included). After all, scientific evidence should have weight, and if educator autonomy extends to believing in myths, well, that’s undeserved autonomy.

Is there no hope for change?

Teaching learning science to educators

Let’s return to Terry J. She read the letter by leading scientists denying the existence of learning styles, but nevertheless continued to believe in the debunked hypothesis. Attempts to persuade Terry to abandon this neuromyth may backfire if they emphasize her obligation to accept the burden of scientific evidence, especially if Terry sees this as a threat to her professional autonomy. This is a vexing challenge.

There is a way forward. We need more teaching – and less preaching – to influence the beliefs of educators such as Terry. To achieve this, advocates of learning science should borrow from the playbooks of good science teachers. These teachers do not prioritize getting students to reject their existing beliefs, but instead seek to foster new scientific knowledge in their students. They replace scientific misconceptions, rather than debunk them.

But how should learning science be taught to teachers? We urge learning-science advocates to ask three questions before attempting to influence teachers.

1. What do educators already believe about how learning takes place, and why?

A bedrock principle of cognitive science is that we learn new ideas by reference to what we already know. Effective teachers are eager to understand their students’ existing beliefs so that they (the teachers) can use prior beliefs and understanding to develop new knowledge.

We think advocates of learning science should be more curious about why teachers believe what they believe, including learning styles. Math teacher Dylan Kane provides a great example of this curiosity in action. He recently conducted a short, non-scientific poll of his followers on Twitter — many of them educators – to learn more about where enthusiasm for learning styles might stem from.

The contrast suggests that while learning styles are popular, what’s really popular is instruction involving multiple modalities. Perhaps some teachers who express a belief in learning styles “really mean that they try to use a variety of representations and activities in class,” as Kane wrote in a subsequent blog post. This is something to encourage in education; good teachers know the value of teaching their students in more than one way.

Of course, Kane’s poll was not scientific, and we don’t know how many of the respondents are practicing teachers. But we suspect most advocates of learning science advance their arguments against learning styles with even less data regarding the existing beliefs of their intended audience (educators). If so, their attempts to build new knowledge in educators may be premised on a misunderstanding of what educators believe, or why. That’s a recipe for an unproductive dialogue.

2. What scientific insights about learning are important for educators to understand?

Another principle from cognitive science is that our decisions are guided by mental models and representations. The most common forms of science communication focus on making evidence-based information available, and assume this information will be incorporated into the recipient’s mental model as a matter of course. Usually, it isn’t, and recipients simply retain their existing ways of seeing the world.

Instead of simply sharing evidence or information, we suggest advocates of learning science spend more time helping teachers understand models of learning they can employ in the classroom. Happily, many scientific principles are useful for teaching, but here we consider just one: dual coding.

Dual-coding theory states that the mind processes words and pictures along different pathways. Researchers have found that instructors can present more information to students by distributing it across both words and pictures. Instruction that does this in artful and in complementary fashion often will reach more students than instruction that does not.

As some learning scientists have aptly observed, dual coding covers similar territory as learning styles. For that reason, advocates of learning science would be wise to introduce dual coding as an alternative to learning styles. We doubt teachers will immediately reject learning styles – students rarely discard their initial (mis)conceptions right away – but over time, if dual coding proves effective in the classroom, teachers may find that learning styles has lost its appeal. Once again: it is better to replace ideas than to debunk them.

3. How might we create opportunities for teachers to practice their understanding of learning science?

Practice is essential for learning, but not all practice is equally effective. A great science teacher (or teacher of any subject, for that matter) provides students with many opportunities to practice their new understanding in structured ways.

This presents a real challenge for advocates of learning science. Blog posts, op-eds, and social media all have a role to play in raising scientific awareness, but we know that real learning requires more. How can we foster opportunities for teachers to try new science-informed practices – and receive useful feedback as they do?

There are no easy answers to this question, but our hunch is that at a minimum it will require learning scientists to approach educators with more humility. Instead of attacking myths, scientists need to approach educators as professional colleagues. As colleagues, teachers and scientists have much to learn from and teach each other about what works in a classroom.

Learning scientists can help practicing teachers improve by inviting them to attend learning-science conferences, collaborating on rapid-cycle research projects, and by providing direct professional development in the local schools where teachers teach. And whenever these interactions take place, we hope learning scientists will listen to teachers, and learn from their experiences in the classroom to inform future research.

This sort of approach might reach Terry J. in a way that info-spreading never could. Terry loves science, research and education. It’s hard to imagine her saying “no” to an opportunity to collaborate with scientific researchers in a collegial way. Will anyone offer her the opportunity?

Building bridges between science and teaching

We believe we are at a crucial moment in the relationship between learning science and education. More than ever, there are organizations and individuals seeking to share the fruits of scientific research with educators. We suspect there have never been as many books about learning science in the hands of teachers as there are today. We should celebrate this development.

At the same time, we worry about the danger of backlash. “Teachers must reject the learning styles ‘neuromyth’” is a provocative headline. It’s also polarizing. The more teachers are told what they must do or believe, the greater the risk that they will become antagonistic toward learning science, or even research generally. We should endeavor to prevent that from happening.

Scientists know a lot about the cognitive processes that can lead to learning – or not. Educators know a lot about the instructional processes that can lead to learning – or not. By building on their respective knowledge bases, and treating each other with mutual respect, we can foster the scientific profession of teaching.

Reading Research: The Case of Mrs. Oublier

I.

A Revolution in One Classroom: The Case of Mrs. Oublier (link) is an oft-cited piece of education research by David K. Cohen. It’s a case study of just a single teacher (Mrs. O) and her math teaching, at a time (the ’80s) when California lawmakers sought to radically transform math teaching in the state.

Mrs. Oublier is a pseudonym, oublier meaning “forgotten” in French. She’s earned this pseudonym for thinking her teaching had undergone a revolution, though in the eyes of Cohen she hardly changed any of the important stuff. I guess the point is that she oublier-ed to make these changes? Or that reformers didn’t help her make them?

Anyway, a lot of the fun of the piece is seeing the funhouse-mirror ways in which Mrs. O interprets those cutting-edge ideas about manipulatives, small group work, and estimation. And Cohen has serious things to say about why policy-makers never quite reached Mrs. O in the way they intended to, though I might question some of his conclusions.

Another thing that’s interesting about this piece is what it’s not: a representative sample from the teaching population. It’s the story of one teacher. Cohen tells us that Mrs. O’s story matters, but why should we believe him?

There’s no denying that Cohen tells a good story. But isn’t research supposed to be more than a good story?

II. 

Mrs. O has been teaching second grade math for four years. The kids like her; colleagues like her; administrators think she’s doing a great job.

As a student, Mrs. O hadn’t liked math much, and she didn’t do too well in school. When she got to college, though, she started doing better. What changed? “I found that if I just didn’t ask so many why’s about things that it all started fitting into place,” she tells Cohen. So, that’s not a great start.

And yet, Mrs. O tells Cohen that she’s interested in helping her students really understand math. She also tells him that she’s experienced a real revolution in her teaching, a departure from the traditional, worksheet+drill methods she used when she began. On the basis of his observations, Cohen is strongly inclined to agree with her on this.

In the centerpiece episode, Cohen catches Oublier in the midst of a fairly ridiculous lesson. Oublier wants to teach her students about place value (so far so good). To do this, she wants to introduce another base system (debatable, but not necessarily a disaster). So Oublier gives each kid a cup of beans and a half-white/half-blue board.

Mrs. O had “place value boards” given to each student. She held her board up [eight by eleven, roughly, one half blue and the other white], and said: “We call this a place value board. What do you notice about it?”

Cathy Jones, who turned out to be a steady infielder on Mrs. O’s team, said: “There’s a smiling face at the top.”

On a personal note, I have been teaching 3rd and 4th Graders for four years and the idea of giving kids those little cups of beans gives me minor terrors. What if the cups spills? How early do you have to get to school to set up the beans? What if a kid eats a bean?

Anyway, after Mrs. O has ensured that all the kids noticed that their boards are half-white and half-blue, she starts the game. The game is supposed to be about grouping and regrouping in place value systems, but it’s really entirely about beans. She calls out a command, and the kids add a bean. At no time does she connect the beans to numbers.

According to Cohen, this was no accident, as Mrs. O wasn’t really a fan of making numbers explicit in her activities:

This was a crucial point in the lesson. The class was moving from what might be regarded as a concrete representation of addition with regrouping, to a similar representation of subtraction with regrouping. Yet she did not comment on or explain this reversal of direction. It would have been an obvious moment for some such comment or discussion, at least if one saw the articulation of ideas as part of understanding mathematics. Mrs. O did not teach as though she took that view. Hers seemed to be an activity-based approach: It was as though she thought that all the important ideas were implicit, and better that way.

Oublier is a huge believer in manipulatives — in fact, the transition from worksheets to manipulatives seems to be a big part of what her “revolution” entailed. For Mrs. O, kids learn through the physical manipulation of the objects. As in, learning is the direct result of touching beans:

Why did Mrs. O teach in this fashion? In an interview following the lesson I asked her what she thought the children learned from the exercise. She said that it helped them to understand what goes on in addition and subtraction with regrouping. Manipulating the materials really helps kids to understand math, she said. Mrs. O seemed quite convinced that these physical experiences caused learning, that mathematical knowledge arose from the activities.

Oublier tells Cohen that she relies heavily on a textbook, Mathematics Their Way, and that this text was the major source of some of her new ideas about physical activities and teaching math. From poking around, it looks like the whole text has been posted online, including the lesson that Mrs. O was caught teaching. Here’s what the bean-counting activity looks like in the text:

Screenshot 2017-06-11 at 8.55.38 PM.png

OK, now the next page of that activity:

Screenshot 2017-06-11 at 9.11.52 PM.png

But you won’t believe what’s on the page after that:

Screenshot 2017-06-11 at 9.12.43 PM.png

This is sort of getting repetitive so I’ll just skip ahead five pages:

Screenshot 2017-06-11 at 9.13.37 PM.png

Cohen comes down pretty hard on this curriculum, and on Mrs. O for using it:

Math Their Way fairly oozes the belief that physical representations are much more real than symbols. This fascinating idea is a recent mathematical mutation of the belief, at least as old as Rousseau, Pestalozzi, and James Fenimore Cooper, that experience is a better teacher than mere books. For experience is vivid, vital, and immediate, whereas books are all abstract ideas and dead formulations.

I’ve focused on the manipulative episode, but that’s just part of her teaching that’s detailed in the piece. According to Cohen, Oublier generally seems to adopt the exterior of cutting-edge math teaching while sort of missing their points. She asks kids to estimate, but doesn’t give them chances to think or share ideas. She uses manipulatives, but doesn’t really ask kids to think much with them. She puts kids into small groups, but basically uses this as a classroom management structure. She avoids numbers and abstraction wherever possible.

This was certainly not what California’s math reformers had in mind.

III. 

The point, for Cohen, is that California’s math reformers let Mrs. O down. But how, exactly?

I found myself needing more context for the California reforms than Cohen provides. Fortunately, the journal issue in which Mrs. O originally appeared was entirely dedicated to the California math reforms. (In fact, every piece in that issue was a different in-depth case study like Mrs. O.)

Cohen actually leads off the issue with a helpful summary of the aims and methods of the 1985 math reforms (link). At their center was a document, the California Math Framework. The Framework called for a transformation of math teaching away from rote memorization and drill, and towards a focus on conceptual understanding, teaching kids to communicate about math, problem solve, work in groups, make sense of math, etc.

So far, nothing new. Reform groups like NCTM have been pumping out these documents for a century.

What was new was the muscle California chose to employ. The state education office said that they would only reimburse districts for textbooks that met the standards of the Framework. And then they actually followed through by rejecting all the texts that publishers initially submitted. Eventually, the state got what they wanted and created an approved list of textbooks for districts to choose from.

(As Alan Schoenfeld notes in his Math Wars piece, California — along with Texas and New York — determine what gets published nationally because of the size of their markets. The publishers basically design their books for the big states, and the rest of the country gets dragged along. So California’s reform muscle had national implications.)

This was half the plan. The other half was to change the state tests for kids so that they also reflected the vision of the Framework. The idea was that if textbooks and tests were in place, teachers would come around all on their own.

I missed this the first few times, but this is why Cohen dwells so much on Oublier’s textbook choice. Oublier’s favored Math Their Way text was not an accepted California text, and Oublier’s district had adopted something else. Oublier likes Math Their Way, though, so she just uses that in her classroom instead. None of her superiors seems to mind either.

In other words, that entire “change teaching by making a list of textbooks” plan was sort of stupid. It failed to account for the ability of teachers to get other textbooks if they wanted to.

The fundamental assumption of the policy seemed to be that teachers need permission, or perhaps incentives, to teach in new ways. As Cohen points out — over and over — this is not the case. Teaching in fundamentally different ways implies believing that you should teach differently as well as knowing how to do so.

It’s pretty simple, actually: if you want to change teaching, you can’t ignore the teachers.

IV. 

Even as Cohen critiques the California reforms, he still seemed to me pretty cheery about the potential for policy to impact reform.

First, he really does seem to give a lot of agency to math textbooks. He keeps on talking about the influence of the Math Their Way book on Mrs. O. On the one hand, the book’s influence on her comes at the expense of the Framework’s reach. At the same time, if a textbook can really have such a strong impact on a teacher, then the premise of the California reforms has been upheld. If you’re a reformer reading Cohen, I imagine that your mind starts wandering: imagine what would’ve happened if we could’ve gotten the right book in her hands!

Beyond Cohen’s implicit optimism about textbook reform, he also wonders aloud about the possibility that a bit of incentive-engineering could have steered someone like Mrs. O towards better teaching:

“The only apparent rewards were those that she might create for herself, or that her students might offer. Nor could I detect any penalties for non-improvement, offered either by the state or her school district.”

These two sources of optimism, when put in context, seemed a bit dated to me. Cohen published this article in 1990, just after NCTM published its Principles and Standards for School Mathematics in 1989. This was, in many ways, a higher-profile go at California’s Framework, and (surprisingly to all involved) it took off, becoming a blockbuster for NCTM.

In the 90s, NSF would fund the development of new math texts that were aligned with the NCTM standards. My sense is that they didn’t live up to the expectations of the textbook-optimists. The texts were just texts, tools that teachers could use well or poorly depending on their understanding of math and of teaching.

It turns out: textbooks can’t transform teachers.

(Textbooks, it also turns out, can become highly visible targets of controversy, and nearly all use of the reform textbooks became contentious in the 90s. So that seems like it needs to be part of the textbook-reform calculus.)

Cohen seems to think that Math Their Way transformed Mrs. O, but he also thinks that she didn’t really revolutionize her teaching. The changes were cosmetic. And there’s a huge difficulty determining how the text impacted because of the plain fact that she chose this curriculum. Presumably, she chose it because she was disposed to. It fit with her understanding of math and of teaching. It didn’t fundamentally challenge her, and I see no reason to think that a text has any such power of a teacher, even when imposed.

Cohen’s other musing — about incentives — has echoes in No Child Left Behind and performance pay reforms. These reforms have also failed to live up to the dreams of the reformers, as all reforms do, and teaching chugs along, mostly as it has.

At times, it seemed to me that Cohen believes that the fundamental problem, for Mrs. O, is that her views on the nature of math remain unchanged:

…however much mathematics she knew, Mrs. O knew it as a fixed body of truths, rather than as a particular way of framing and solving problems. Questioning, arguing, and explaining seemed quite foreign to her knowledge of this subject. Her assignment, she seemed to think, was to somehow make the fixed truths accessible to her students.

I’m not particularly sympathetic to this critique. Math, among other things, is a fixed body of truths (theorems, facts, relationships) that we ought to help students know.

But forget that for a moment. Cohen sometimes seems to think that this isn’t just a problem for Mrs. O, but the root problem. If we could just help Oublier see that math isn’t quite as she thinks it is — that it’s dynamic, a source of puzzles, it’s about thinking and not just about knowing — then her teaching really would undergo a real revolution.

This seems to be where we are, right now, in math education reform. We’re not trying to save the world with NSF-funded textbooks, and we’re not hoping to incentivise great teaching. We believe, like Cohen, that the fundamental problem is one of learning, and that the fundamental problem is a fundamental problem, some ambitiously big thing that, if we can help teachers attain, the rest of their teaching will fall into place.

Right now, one version of the “fundamental problem” is productive struggle. NCTM has included this in their latest set of reform standards, the Principles to Actions standards. And if you’re in Baltimore this July, you can attend a three-day summer institute focused on productive struggle. The workshop promises to show how productive struggle is tied to every dimension of effective math instruction, from planning to feedback to wider advocacy.

I don’t think I believe in this sort of reform either. Cohen keeps drawing comparisons in this piece between teacher and student learning — both are challenging, he says, both take time. And that’s true. But imagine if we treated students like teachers. In other words, imagine if instead of teaching math to kids we had a workshop a few times a year where we tried to fundamentally alter their conceptions of math, and then sort of hoped that the rest of their math learning would just fall into place.

I know the comparison isn’t exactly direct, or fair, but I don’t believe that any knowledge can be altered by changing one fundamental element. Knowledge isn’t really structured that way, it seems to me. It’s not built on a foundation. To alter teaching you’d have to alter it broadly, not centrally. And broad change just can’t happen in a three-day workshop.

The final source of optimism that Cohen raises is that maybe Mrs. O represents progress for math reform. Though she hasn’t seemed to internalize the message of the reform, this sort of messy progress is what progress actually looks like.

I have no way of knowing if that’s true, but it certainly strikes me as possible. I haven’t read more recent work of Cohen’s. I wonder if, looking back on the last 30 years of reform, he’s still as optimistic.

V.

Hey, wait a second! This is just a single case study. We were swept along in this gripping tale (aptly summarized) and assumed she represented some larger trend, but that’s just the illusion of focus. Cohen’s fooled us, then, hasn’t he? Maybe Mrs. O means nothing at all. (Or, at least, nothing beyond her own case.)

There are two things that temper this sort of skepticism. First, the journal that published Mrs. O also published four other case studies in the same issue (open version). So in addition to the case of Mrs. O, you also get the case of Carol Turner, Cathy Swift, Joe Scott, and Mark Black.

(Unclear if the other pseudonyms are also supposed to be deeply meaningful. Mark Black, because policymakers treat him like a black box. Cathy Swift, because the reforms were too fast! The other two stump me. Maybe they’re anagrams? Joe Scott = COOT JEST.)

Five case studies are only a bit better than one, but these other four cases present a lot of the same mixed-success-at-best themes as Mrs. O’s case. That helps.

The other thing that tempers skepticism about Mrs. O’s relevance is that Cohen actually also identified the “forgotten teacher” problem in a very different piece of research.

That other piece is called Instructional Policy and Classroom Performance: The Mathematics Reform in CaliforniaThis time around, Cohen and his team do pretty much the opposite of “sit in the back of a classroom and watch.” They survey 1,000 California elementary teachers. They ask teachers to rate how frequently they employ various instructional activities in class. Hey, they ask, wouldn’t it be nice if all these teacher responses really pointed to two types of teachers? We could call them “traditional” and “reform-friendly”…

Err, did I say “traditional”? I meant “conventional”:

Screenshot 2017-06-12 at 10.51.01 PM.png

 

Anyway, Cohen’s group also asked teachers what professional learning opportunities they had, in relation to the math reforms. (I love that ‘Marilyn Burns’ is an option.)

Screenshot 2017-06-12 at 10.52.30 PM.png

 

What they find basically supports Cohen’s take in his Mrs. O piece — reform is possible, but only when it focuses on professional development that targets teacher learning:

Our results suggest that one may expect such links when teachers’ opportunities to learn are:

  • grounded in the curriculum that students study;
  • connected to several elements of instruction (for example, not only
    curriculum but also assessment);
  • and extended in time.

Such opportunities are quite unusual in American education, for professional
development rarely has been grounded either in the academic content of schooling or in knowledge of students’ performance. That is probably why so few studies of
professional development report connections with teachers’ practice, and why so many studies of instructional policy report weak implementation: teachers’ work as learners was not tied to the academic content of their work with students.

Some people love the Mrs. O piece, but hated the sort of study that we previously read here, the one about teacher-centered instruction for first graders. First, because they rely on teacher responses to survey questions, and how much can you really learn from that? Second, because the statistical work can hide researcher assumptions that then become tricky to dig out. Third, because with scale comes quality control issues. You really no longer know what you’re dealing with.

To which, we might ask, why did Cohen produce exactly this kind of study when it came to evaluate the success of California’s reforms?

I talk to just as many people, though, who hold the complete opposite view. To them, something like the Mrs. O study is useless, as it doesn’t help us identify the causal forces at work. Maybe the reform failed Mrs. O, but compared to what? There are no controls, and without some sort of random assignment to a treatment can we really be sure that a focus on teacher-learning would make the difference Cohen said it would?

Is it too soft of me to say that both critiques are right?

It’s not my job to study teaching, but it sure seems hard. Every research approach has trade-offs. The way I see things, it’s best to use multiple, incompatible approaches to study the same things in teaching from wildly different perspectives. Why? Because of how it’s possible to take wildly different incompatible perspectives on teaching.

At one point, Cohen points out that Mrs. Oublier seemed comfortable living in contradiction:

Elements in her teaching that seemed contradictory to an observer therefore
seemed entirely consistent to her, and could be handled with little trouble.

But there really isn’t anything strange here at all. Everyone is willing to live with some contradictions in their lives. Contradictions can be unlivable, but they can also be productive — in teaching, in life, but also in research. Intellectually incompatible perspectives can be desirable.

Anyway, enough about all this. What should we read next?