Study an example, see the world

I’ve been a math teacher in New York City since 2010, a few months after I graduated from college. It’s the only job I’ve ever had, besides for little things over the summer when I was a teen. (In order: babysitter, camp counselor, Pepsi vendor at Wrigley Field, tutor. All kind of relevant to teaching, come to think of it.)

Though I teach math, math didn’t feel easy for me as a student. It was never where I shined. An exception was geometry, with its heavy emphasis on proof. Proof felt natural for me in a way that algebra didn’t.

When I began teaching, I realized that for many students the situation is reversed — it’s proof that feels unnatural and cumbersome. Writing a proof involves combining statements in ways that seemed to mystify many students. This was especially true early in my career.

After a few years of hitting my head against the wall, I started to understand what made this such a difficult skill to teach. Proof is the closest that mathematics comes to writing, and writing itself is impossible without reading. How can a student who has never read an essay possibly write one? I concluded that my students needed to read more proofs.

It took me a few more years to understand how exactly to pull this off in class. My big frustration was that my students wouldn’t devote enough attention to the proof examples I shared. I would distribute a completed proof and ask the class to read it with care. Very often, it seemed that they missed the whole point of the proof. They couldn’t read it carefully yet — they didn’t know how.

Now, things go better when I share proofs in class. One big difference is I have a much better understanding of all the subtle conceptual understandings that go into a proof, many which were invisible to me at first. (In teaching, it can be trouble when a topic comes naturally to you.) There are many aspects of a proof that I need to help them uncover.

Besides for a better understanding of the subtleties of proof, I’ve learned to structure my activities in sturdier ways. I’ve learned to design these activities so that they have three parts:

  • The proof example
  • Comprehension questions about the example
  • Proof-writing practice, with the example as a model

I didn’t come to this structure on my own, by the way. I came to it through reading about Cognitive Load Theory (where these are sometimes called “example-problem pairs”) and especially from seeing it in some especially well-designed curricular materials:

Screenshot 2017-11-30 at 6.49.21 PM.png

Also:

Screenshot 2017-11-30 at 6.50.45 PM.png

 

(In fact, I didn’t really understand how to make my own example activities until I saw many models in these curricular materials. I needed examples, myself.)

So, for instance, I created this proof example for my students this year:

DM8ANAWWsAACjSC.jpgLooking back, the example isn’t perfect. It ended up being a bit visually crowded, and it might have been better to eliminate some of the letter-abbreviations. In class, I actually covered up each stage of the proof to focus their attention on each part.

In any event, this activity shows a lot of what I’ve learned about teaching proof. I knew I wanted to make explicit the complicated two-stage structure of some congruence arguments, so I worked hard to create a pretty clear example for my students. I then called on students to answer a trio of analysis questions about the proof — there’s a lot to notice, and students don’t yet know how to notice the underlying structure of this kind of proof all on their own. Finally, I ask students to use what they’ve noticed on a related pair of problems, so that students see that there’s something here that’s generalizable to many different kinds of diagrams.

Even when my proof activities aren’t structured so rigidly, I try to include variety and a chance to practice. Here is a simpler activity, but I still call for students to do a bit of proof-completion in the second prompt:

DM8ANArXkAItunW.jpgSometimes when I talk to other teachers about examples, they tell me they’re worried that kids will just try to unthinkingly copy the model. I do know what they mean, but it’s not what I see with my kids. I think that part of the reason is that I reserve example-analysis for when I worry that the math is going to be difficult, even overwhelming for many students. There is certainly a way to misuse these activities, and perhaps if I used these sorts of tasks on less complex material I would see unthinking imitation.

One of my jobs is to help students see things that they can’t yet see — things like the logical structure of a good mathematical argument, or the way just a tiny bit of information about a shape can guarantee a whole lot more. When things don’t come naturally to my students, what I’m learning to do is to design an activity that opens up a little window into the mathematics so they can learn to see new things.

Addendum (1/6/18): I just came across this lovely line from Paul Halmos:

A good stack of examples, as large as possible, is indispensable for a thorough understanding of any concept, and when I want to learn something new, I make it my first job to build one.

Advertisements

Teaching, in General

I. 

If you give a quiz covering lots of different topics, you’re going to get a lot of different mistakes. Which leaves you with a dilemma: how do you address those mistakes?

Yesterday’s quiz in geometry was a review quiz, so the topics were from all over the place:

  • angles in isosceles triangles
  • inscribed angles in a circle
  • area of triangles, parallelograms and trapezoids
  • congruence proofs

As expected, kids distributed their not-quite-there work fairly evenly across these topics. (OK so that’s not true, there were a lot of issues with the congruence proofs. There always are and always will be. Sigh.)

Here were two bad options for returning the quiz:

  • Try to address all the issues with individual comments. First, it’s a game of whack-a-mole that is guaranteed to drive me insane. Second, what should I do? Try to leave perfect hints? Say nothing, and let kids figure out on their own what they did wrong? Show them the correct way to answer the question, and thereby eliminate anything for the kids to actually think about when I return the quizzes?
  • Pick just one thing to focus on. Reteach that one thing in a careful way, then return the quizzes and ask kids to revise.

The second of the two options is great when there the mistakes are in the same galaxy. (I wrote about this in a post, Feedbackless Feedback.) But, I’m realizing now, this isn’t a terrific move when the mistakes are distributed across many topics. Because on what basis should I pick something to focus on reteaching? Any choice would be equally bad.

II. 

While reviewing the class’ quizzes, I found myself falling into written comments, at least until I figured out what else to do with the quizzes.

I used to write long, wordy comments that were essentially hints on the margins of the page. (“Great start! Have you tried multiplying both sides of the equation by 3?”) I came to dislike those sort of comments, as they just focus focus focus attention all on THIS problem. But I don’t particularly care about whether a students gets this problem correct; I care about the generalization.

What I’ve fallen into is, whenever possible, writing a quick example that’s related (but not identical) to the trouble-problem (the problem-problem) on the page. I do this below on the second question:

image-3.jpg

Then, I ask kids to revise the original on the basis of the example (or anything else they realized).

After writing a few of these example-comments, I realized I was taking a lot of time doing this, and repeating myself somewhat. I also realized that I don’t know if I could repeat this on every page for the congruence proofs, as the problem itself was reasonably complex:

image-2.jpg
Hard to read. I use the highlighter to flag errors, but this student highlighted the triangle himself.

I wasn’t sure what to do. Then, I remembered something I had read from Dylan Wiliam — I think it’s in Embedded Formative Assessment. His idea there was that you can give all the class’ comments to everyone, and then kids have to decide which comments apply to them.

I thought, OK, I can work with this. So I quickly (quickly!) made a page of examples, one for every mistake I saw on the quiz:

ums-copier@saintannsny.org_20171115_095927 (1)-page-001.jpg

My routine in class went like this:

  • Hand out the examples for revision.
  • Hand back the quizzes with comments.
  • Search for an example that’s relevant to your mistake.
  • Call for revision on the basis of the examples. Work with friends, neighbors. Of course, I’m available to help.
  • Then, try the extension task.

This was my first time trying this, but I thought it went well. Solid engagement, really good questions, no unproductively stuck students.

When you do something good in teaching, you never really know if it’ll work again, but I’ve got a good feeling about this one. It feels like a lot of what has already worked for me, but in a better order.

III.

Harry Fletcher-Wood is very nice and has a lot of interesting thoughts about feedback. As such, Harry and I very nicely disagree about a pretty interesting question about feedback: how can you teach people how to give better feeedback?

The usual caveats apply: I am not a teacher teacher, but Harry is involved in teacher education, and I have no idea if I’m right on this.

In any event, Harry recently published a really cool post where he tried to synthesize a lot of the research on feedback into a decision tree:

Feedback-decision-tree-1.6

Now, this is awesome as a synthesis. But just because something is a good description of feedback doesn’t mean that it’s useful prescriptive advice. My favorite example of this comes from Pólya’s strategies for mathematical problem solving. Alan Schoenfeld has a nice way of putting it in Learning to Think Mathematically — the strategies have descriptive, but not prescriptive validity:

In short, the critique of the strategies listed in How to Solve It and its successors is that the characterizations of them were descriptive rather than prescriptive. That is, the characterizations allowed one to recognize the strategies when they were being used. However, Pólya’s characterizations did not provide the amount of detail that would enable people who were not already familiar with the strategies to be able to implement them.

In other words, just because a heuristic is a good description of practice doesn’t mean that it is an effective pedagogical tool. And that’s precisely my concern with Harry’s decision tree.

Feedback is a high-level concept that describes a TON of what happens in teaching. And any guidelines for how to give feedback effectively are also going to be high-level in a way that reminds me of Pólya’s moves like “find a simpler problem” or “draw a picture.”

And just as Pólya’s moves struggle because they aim to guide problem solving in geometry, algebra, topology, etc., all areas of math, Harry’s decision tree seems to me an attempt to guide feedback in all areas of teaching — math, history, medical school, etc.

Of course, Harry doesn’t intend for this to be the only thing guiding students, but neither did Pólya. My question is whether these generalizations themselves are helpful, beyond whatever ways that teacher educators can make them concrete and specific for teachers.

But what’s the alternative?

I don’t know yet. I can say a few things now that I couldn’t a few years ago:

  • I think domain-specific — math-specific, history-specific — generalizations will be more useful than domain-general ones.
  • I think that the generalizations can productively come in the form of instructional routines.

And, with this post and the other one, I now have two generalizations I can make about giving feedback in math class.

First: if there’s a problem that a lot of students have trouble with, consider a reteaching/revising cycle like the one in this image:

Screenshot 2017-11-15 at 1.24.03 PM

Second: if mistakes are sprinkled across too many topics, consider something like the revision routine I described in this post.

IV.

My bet is that a lot of knowledge about teaching looks like this. It’s not that there isn’t knowledge about teaching that accrues, but that we look for ways to scale things out of their contexts. Then we call those things myths and talk about how we have to kill ’em.

In general, generalizations about teaching are hard to come by. But nobody teaches in general. All teaching is intensely particular. These kids. These schools. This idea.

Some people are skeptical of the possibility of making generalizations about teaching, and the vast majority of people are cheery about making sky-high generalizations that cross every context. There’s a middle position that I want to find. There’s a sweet spot for knowledge about teaching, though I don’t know if we’ve all found it yet.

A Worked Example Workaround?

When you decide to explain some math to a kid, how should you explain it? Step-by-step, or all at once?

There’s an issue with step-by-step explanations: kids have to remember what you’ve already said in order to understand what comes next. This means that there’s often a lot to hold in their head!

There’s an issue with fully worked-out examples: by not developing math slowly, in full view of the student, you make it seem as if the solution was dropped out of the sky. It can present a false picture of math: as constituted entirely of an encyclopedia of procedures that mathematicians memorize, look-up, and employ on canned problems.

I was thinking, today, about whether there is a way to get the best of both worlds. My mind wandered towards my 8th Grade class. We’re studying slope, so I launched class by putting two right triangles under the document camera. Which is steeper?

Students debated, thought some more, offered good and better approaches. I kept a record of their ideas, which I scanned after class:

pic1

I’m wondering, what if I started tomorrow’s class by projecting this image back on the board? I’d say, Here’s what we figured out yesterday. I’d like to give you two new triangles to look at today. I’m going to keep yesterday’s work up, in case it’s helpful.

Would that give a distorted view of mathematics and its development? Would that give students the benefits of worked-out examples?

Too Much of A Good Thing

Greg Ashman:

If more guidance makes minimally guided approaches more effective then why not use a fully guided approach? Won’t that be still more effective? It is an argument that plays out again in the book and one that offers little comfort to proponents of open-ended problem solving in high school maths classes.

But, Jordan Ellenberg:

The difference between the two pictures is the difference between linearity and nonlinearity, one of the central distinctions in mathematics…Mitchell’s reasoning is an example of false linearity—he’s assuming, without coming right out and saying so, that the course of prosperity is described by the line segment in the first picture, in which case Sweden stripping down its social infrastructure means we should do the same.

But as long as you believe there’s such a thing as too much welfare state and such a thing as too little, you know the linear picture is wrong. Some principle more complicated than “More government bad, less government good” is in effect. The generals who consulted Abraham Wald faced the same kind of situation: too little armor meant planes got shot down, too much meant the planes couldn’t fly. It’s not a question of whether adding more armor is good or bad; it could be either, depending on how heavily armored the planes are to start with. If there’s an optimal answer, it’s somewhere in the middle, and deviating from it in either direction is bad news.

Also, John Sweller:

That is not to say that there are no disadvantages to the use of worked examples. A lack of training with genuine problem-solving tasks may have negative effects on learners’ motivation. A heavy use of worked examples can provide learners with stereotyped solution patterns that may inhibit the generation of new, creative solutions to problems.

Greg’s argument is, “If a bit is good, isn’t a lot better?” But this sort of falsely linear thinking isn’t compelling, no matter what you think about direct instruction.

 

How did 69 turn into 29?

Last year, while reading and writing about cognitive load theory, I came across something weird that I couldn’t explain. A paragraph from Greg Ashman’s latest reminds me of this puzzle. It’s really small and inconsequential, but it’s been bugging me. Maybe you can figure it out.

He writes:

One of my PhD supervisors did an experiment in the 1980s. Undergraduates were given as series of problems. Each problem involved a starting number and a goal number. The participants had to get from the first number to the second using only two moves which they could repeat: multiply by three or subtract 29. The problems were designed so that each one was solved by alternating the steps. Although the students could generally solve the problems, very few ever worked out the rule.

Great. Multiply by three, or subtract 29.

Except you go back to that paper, and it’s actually subtract 69.

Screenshot 2016-09-11 at 6.31.37 PM.png

Where did Greg get the “subtract 29” from? I don’t know, but it could be from this piece by Sweller in 2016.

Screenshot 2016-09-11 at 6.33.29 PM.png

Anyway, totally unimportant. Completely uninteresting. But. Did he forget? Was it a typo? Did he decide — as so many before — that 69 is a funny number to talk about in classes?

If you see me and I’m looking pensive, this is probably what I’m thinking about.

Cognitive Load Theory’s Changing Take on Motivation

In 2012, John Sweller (of Cognitive Load Theory fame) sat for an interview about his work. The conversation turned to motivation, and Sweller made it very clear that motivation was beyond the scope of CLT.

“One of the issues I faced with Cognitive Load Theory is that there are at least some people out there who would like to make Cognitive Load Theory a theory of everything. It isn’t. […] It has nothing to say about important motivational factors…It’s not part of CLT.”

Later in the interview he expands on this point.

“Cognitive Load Theory works on the assumption that the students are fully engaged, fully motivated, that their attention is being directed. Cognitive Load Theory has nothing to say about a student who is staring out the window and not listening.”

When I started researching Sweller’s work, I was fascinated by these later interviews, because I saw them as conflicting with his earlier publications. I thought this represented an important shift in his thinking, one that connects to his dismissal of “germane load” from his theory.

That’s what I thought when I wrote the essay. But does the claim hold up?

The first time Sweller writes about motivation is in Sweller & Cooper, 1985.

pic1

I was talking to Greg Ashman about this passage, and Greg made a great point. He argued that this early passage is not necessarily in conflict with Sweller’s later interviews. Why not? CLT may consider motivational factors, but it’s not what CLT is about. After all, they didn’t even measure motivation as part of this experiment. True, you need to motivate students to participate in the study, but that’s hardly the same thing as studying motivation!

In Sweller, van Merrienboer and Paas 1998, motivation comes up again (as it non infrequently does in van Merrienboer’s work).

pic1

Now, again, it’s true that this mentions motivation. And, at first, I thought that this conflicted Sweller’s later take. Sweller says in that interview that CLT assumes that students are fully motivated. If students are already fully motivated, then why talk about possible negative effects of motivation?

But this still might not conflict with Sweller’s later statements. After all, this is merely speculating on a possible way worked examples might impact motivation negatively. This does not mean that CLT is about motivation or that its study is part of CLT work.

The best support for the story I told in the essay, I think, comes from van Merrienboer & Sweller 2004 . Motivation makes it into the abstract:

pic1.png

“Complex learning is a lengthy process requiring learners’ motivational states and levels of expertise development into account.” Doesn’t that mean that we’re no longer just assuming high levels of motivation in CLT research? And this attention to motivation is called “a recent development in CLT.” So, surely, motivation is part of CLT’s research. No?

I think the clearest statement of motivation’s place in CLT comes in the “discussion” section of this piece:

pic1

“Four major developments in current CLT research were discussed…research to take learners’ motivation and their development of expertise during length courses or training programs into account.

This is the evidence I was confronting, and I’m sure there is more than one way to read it. My read, however, is that this is a claim that CLT research included motivational factors, and that this conflicts with Sweller’s later statements. After all, would Sweller say in 2012 that learner’s changing expertise isn’t a part of CLT research? Certainly, he wouldn’t, as the expertise-reversal effect is still an important part of CLT’s work. Motivation might have continued to be part of CLT, but Sweller changed his mind. That’s my read.

My claim was never that motivation was a core concern of CLT. But I do think that Sweller’s thinking about motivation and CLT shifted in a way that illuminates his development. It’s a shift that I think tells us something about how a major task of scientists of learning is to manage complexity, to decide what to study and what to ignore. (And how it is, to an extent, a choice.) And I do think that Sweller’s thinking about motivation helps illuminate the much more significant change in his thinking about germane load.

As always, I might have gotten this wrong. But this is why I think that there’s something interesting about motivation in CLT.

Cognitive Load Theory and Why Students Are Answer-Obsessed

It’s true: math education doesn’t give a ton of attention to Sweller and cognitive load theory. Math education researchers who are aware of Sweller are most familiar with his attack on problem-based, experiential, discovery and constructivist learning (“An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential and Inquiry-Based Learning“). As Raymond mentioned on twitter, those within math education who are likely to recognize Sweller are equally likely to dismiss him and his work.

Part of this, I think, has to do with focusing on the wrong aspects of Sweller’s work. Ask 100 people what the key idea of Sweller’s work is, and I bet 99 would say: it’s easy to overload the working memory of students. For learning, it’s important not to. So, don’t. An important but limited insight. (We’re trying not to overload anyone!)

The last 1 person out of the 100 is me. As far as math education is concerned, I think the key idea of Sweller’s work is about problem solving, not cognitive load. Here is that key idea: problem solving often forces a person into answer-getting mode, and answer-getting mode is incompatible with learning something new.

(“Answer-getting” mode also has to do with expectations that students have about math class and the sorts of activities they think are valued in mathematics. Sweller shows it has a cognitive element too.)

Sweller’s early work was with number puzzles. Participants in his studies solved the puzzle successfully, but never came to notice a fairly simple pattern which was sort of the “key” to finding any solution. Why? There were two reasons:

  1. When you’re looking for the solution to a problem, your attention is massively restricted to those things that are directly relevant to finding the solution. Lots of important details of the scenario or environment get ignored.
  2. Attention is a zero-sum game. There’s only so much that a person can notice. A person focused on finding the solution is unable to focus on much else.

(For more, read this part of my essay.)

I have found this to be absolutely true and deeply insightful. The first time the idea really hit me was during Christopher Danielson’s talk, titled “What’s the Difference Between Solving A Problem and Learning Mathematics?” There is a difference. Sweller helps us get specific about some of the reasons why.

These limitations of problem solving guide my daily classroom work. My 8th Graders are wrapping up their study of linear functions and moving on to exponential functions. Yesterday, I found myself wanting my students to start thinking about the differences between linear and exponential graphs and patterns. I took this image from David Wees’ project and displayed it on the board:

Screenshot 2016-04-13 at 6.05.12 AM

In the past, my first instinct would have been to pose the problem as quickly as possible. “What are the coordinates of point B? of point A?” I would then give my students time to think, and I would have expected some learning to have occurred.

Now I know that this could be a particularly bad way to ask my students to begin their work. They probably wouldn’t notice what I want them to notice. Instead, they’d probably go into that answer-getting mode that focuses all their resources in an unproductive way:

Screenshot 2016-04-13 at 6.08.40 AM.png

Another key insight of Sweller has to do with how to avoid ensnaring students in this unproductive struggle. One suggestion of Sweller’s is to ask less-specific questions. These nonspecific questions don’t funnel attention in the way specific questions do, and they therefore don’t overload students in quite the same way.

Sweller first described the power of nonspecific questions with regards to angle problems. Rather than asking students to find a particular angle, he asked “Calculate the value of as many variables as you can.”

Screenshot 2016-04-13 at 6.15.32 AM
Sweller, Mawar & Ward, 1983. 

 

With my 8th Graders, yesterday I began class with two nonspecific questions. I asked these questions so that they’d notice as much about the diagram as possible and start putting together some of the pieces about exponential relationships.

My first question: “What do you notice?” I waited for lots of hands to go up, and then I quickly called on three students. (I find it’s important to move quickly here — not so interesting to rattle through everyone’s noticings.)

My second question: “Study the diagram and find something to figure out.” I asked students to do this in their heads, alone. Then, “Talk to your partner — come up with at least two different things to figure out, then as many as you can.” (What counts as something “figured out”? We’ve done this routine many times, so my students know from experience.)

Here is an incomplete list of what my students calculated/figured out from the exponential graph:

  1. The y-coordinates are doubling
  2. The y-axis is going up by 4
  3. The slopes are changing between each pair of points
  4. The graph is non-proportional
  5. The next coordinate would be (6, 64)

If my students had mentioned, at this phase, that the coordinates of B were (2,4) we would have moved on. Since they hadn’t, and since they were saying so many smart things, I decided that this would be a great time to ask a third question:

“What are the coordinates of point B? point A?”

My students were able to answer these specific questions, but that’s hardly the point. Sweller’s research suggests that you can’t use problem-solving success as a gauge of whether kids have learned something or not.

I do think, though, that the reasons my students gave for their correct answers are revealing. Some students, in justifying their answers, mentioned that you could be sure that point A was at (0,1) because the y-coordinate seems to be 1/4 of the way up to 4. Other students then pointed out that (0, 1) fits the general pattern. What’s interesting is that this first observation — the position of point A up the axis — never came up in the first two questions I asked. That makes sense, because that way of looking at the position of point A has nothing to do with the exponential pattern.  In fact, it’s the sort of hyper-focused response that you’d only expect to hear when a very specific goal has been set by the teacher — find the coordinates of point A. Otherwise, that’s not the thing that’s worth noticing here (probably). It misses the forest for the trees in the way people do when they are focused on achieving a narrow goal.

The second response, though, showed that some of my students had started making good connections. They justified the coordinates of points A and B based on the general pattern.

All this suggests to me that while some of my students are ready for working on specific problems, many of them aren’t yet there.

Asking more nonspecific problems isn’t the only recommendation that Sweller makes, of course. He’s better known for recommending the heavy use of worked-out examples and explanations in class. We do those too, though probably not as often as Sweller would like. Still, there’s more to Sweller’s theory than worked examples.

The key idea here is that specific questions cause students to chase specific goals. Chasing a goal isn’t always helpful for learning. On the one hand, I think this makes the case for developing a specific question more slowly, asking students to notice before posing a problem. On the other, this calls for us to be more cautious and deliberate about how we use problems in our teaching, especially in the early stages of teaching a new idea.

 

Cognitive Load Theory is More Than Worked Examples

For the last few months, I’ve been working hard on an essay about John Sweller’s cognitive load theory. This is, by no means, a comprehensive essay about CLT. I wanted to tell a very specific story in the piece — about how Sweller came to invent his theory, how he changed it so that it could better embrace greater complexity in classroom learning, and how he ultimately restricted the boundaries of his theory to avoid this complexity.

Something that I don’t talk much about in the piece are the implications of CLT for teachers of math. CLT is highly active in arguments about how best to teach math, and many who identify as “traditionalists” cite CLT to support their views. This, in turn, leads those who identify as “progressives” to seek to discredit CLT. I have no desire to negotiate this terrain.

Discussion of CLT, I find, often focuses on one specific teaching recommendation: worked examples. See, for example, Deans for Impact’s The Science of Learning report:

Screenshot 2016-04-10 at 12.17.58 PM.png

A closer look at the work of CLT, I think, complicates the focus on worked examples in several ways.

First, there are other ways that Sweller and CLT identifies for reducing cognitive load. In particular, Sweller has found that problems with non-specific goals (i.e. more open questions) are helpful for reducing cognitive load. You don’t often hear this aspect of Sweller’s work come up in debates, but I think that’s a shame, because I think both progressives and traditionalists could support the use of these sorts of questions.

Second, there was a period of Sweller’s career when he trained his eye on learning more complex skills in classroom environments. Though he eventually moved away from this work, during this time he noted that there can be issues with worked examples, when put into practice. For example, in 1998 he wrote (with his co-authors) that “A lack of training with genuine problem-solving tasks may have negative effects on learners’ motivation.”

“A heavy use of worked examples can provide learners with stereotyped solution patterns that may inhibit the generation of new, creative solutions to problems…For this reason, goal-free problems and completion problems…may offer a good alternative to an excessive use of worked examples.”

Further, work by researchers had found that worked examples can be bested by “completion” problems, where there is thinking left for students in the task. This is the work of van Merrienboer, which I also write about in the essay. Here’s a quote about worked examples from his research:

“…students will often skip over the examples, not study them at all, or only start searching for examples that fit in with their solution when they experience serious difficulties in solving a programming problem. … [In completion problems] students are required to study the examples carefully because there is a direct, natural bond between examples and practice.”

So CLT research has at least two alternatives to worked examples for novice learning: open questions and completion tasks. And research within CLT has identified motivational or practical issues with excessive use of worked examples — these are from papers that Sweller himself wrote.

(The truth is that, depending on how complex the skill we’re trying to teach is, van Merrienboer’s line of thinking opens up a great deal of possibilities beyond worked examples. While he’s opposed to throwing novices into the deep end, well, everyone should be opposed to that. Instead, he wants to find authentic, motivating tasks that are manageable for novices. For more, see his “Ten Steps to Complex Learning.”)

I don’t think it’s surprising that “worked examples” have earned outsized attention by educators. This is the same thing that happens when educators embrace research, in general. A few years ago I read Jack Schneider’s From the Ivory Tower to the SchoolhouseThe book is about why some research catches on with teachers, while most does not. He identifies four key characteristics of research that makes the jump to practitioners:

  1. Perceived Significance: It needs to be perceived as coming from reliable, important names. (e.g. “a bunch of Harvard researchers just found that…”)
  2. Philosophical Compatibility: The research needs to be in sync with the beliefs of the educators who embrace and share it.
  3. Occupational realism: It needs to be easy to put in immediate use.
  4. Transportability: It needs to be easy to share — tweetable, even.

While Sweller doesn’t have a name-brand research pedigree that is recognizable to us in the US, worked examples otherwise fits this framework perfectly. It’s a practice that is very realistic (most teachers are already using lots of worked examples and explanations), it’s very easy to share the idea, and for those who traditionalists who have embraced it it is very much ideologically safe.

That’s not a criticism of traditionalists who embrace worked examples — it’s just a point about how research gets shared in education. “Worked examples,” like “growth mindset” or “project-based learning,” fit Schneider’s framework quite well.

What this means, though, is that you have to listen carefully to hear about anything beyond worked examples when people talk about CLT. But this emphasis on worked examples does not fairly represent Sweller or CLT. There are a host of additional ideas and techniques that his and others’ CLT research has found: open questions, completion tasks, and motivational and practical issues with worked examples in practice.

You can’t really hope to change the way people talk about anything in education, let alone research. You can hope to dig a bit deeper and find a bit of understanding beyond the noise, though. That’s what this project has been about, for me. I’m excited to share it, and I’ll continue to add some thoughts about CLT over the next few weeks.

Dissent of the Day

I said [Three Act problems] are most valuable to me before learning skills, or rather as the motivation for learning skills. I don’t expect that students will just figure everything out on their own, though. Act one helps generate the need for the tools I can offer them here in act two.

-“Teaching With Three Act Tasks: Act Two,” Dan Meyer

I’ve been thinking about it, and I think I disagree with Dan’s take here. I think there are important differences between providing instruction during, before or after a tough mathematical experience, and that instruction during a problem is often bound to be lost in the flood of ideas that a mind is awash in.

Here’s where I’m coming from. Over the past few class periods, my 4th Graders have been working on a lovely little activity. We watched a short video showing Andrew papering his cabinet with sticky notes. How many sticky notes would it take to cover the entire thing?

pic1

I showed this video, and was disappointed by the tepid response from my students. Then I asked my students to estimate the number of stickies it would take to cover the cabinet. More blahs. And then I clarified that we’re trying to figure out how many stickies would cover the entire cabinet, and my kids exploded with ideas and excitement: “Wait, can you give us time to figure this out?”

Really, really great stuff.

While walking around, I noticed some kids getting lost in their calculations. Lots of great ideas, but constantly losing the thread.

IMG_4026 Other kids, though, used diagrams to preserve their line of thought. These kids, even if they were less computationally sophisticated than other students in class, were finding relatively more success in the problem.

IMG_4028

When I noticed this, I realized that this sort of diagramming was an important mathematical idea that I should make explicit to everyone. When pairs called me over to help them make sense of their confusing calculations, I made the suggestion: here’s a diagram, here’s how you can use it, this could help with where you’re stuck.

No dice, so I decided to pause class and say it to everyone: hey all! I noticed that the tricky thing isn’t just the calculations, but trying to keep track of what you’ve figured out and what you still need to work on. Diagrams can help, here’s a diagram, here’s how you can use it, you might try this.

As I walked around some more, I poked around to see if pairs had adopted my suggestion. No dice, still.

Bell rings, kids hand in their work, that’s that for the day.

The next day, I start class by saying, “I noticed a lot of us got stuck on the problem yesterday. We’re going to keep on working today, but here’s something that might help: here’s a diagram, etc.”

Dice!

What happened? Hard to know, of course, but here’s what I’m thinking: the first time around, my kids had a million mental distractions. Some were wondering if their calculations were right. Others were just trying to get a grip on a plan of attack for the problem. Others were trying to remember where on their page they had written their current tally of the stickies on the front and back.

In other words, these kids had a lot to think about during this problem, and they weren’t really able to dedicate the brain space needed to understand a new and unfamiliar strategy.

This is also how I make sense of something I’ve noticed in my Algebra 1 class. I haven’t yet given these kids activities that explicitly address the “cover-up” method for solving equations, but I keep trying to bring it up when kids ask me for help with equations in class. The thing is, it never seems to stick.

It seems to me that if we think “just-in-time” instruction works particularly well, my kids should be able to hold onto this method a bit better than they currently do. After all, they have a clearly felt need for some new bit of math (they called me over, right?) and they are getting the instruction during their felt moment of need. Super-duper effective setting for instruction, right?

But then it doesn’t stick. And I think it’s for the same reason that my 4th Graders didn’t take up the “draw a picture” suggestion: they’re too mentally distracted to really focus on the new idea and properly learn it. After all, learning a new idea in all its proper generality can be a pretty heady bit of work. When my kids call me over for help with their equations, they’re potentially thinking about many other mathematical things — where am I in the problem? did I make a mistake by subtracting? what’s 4 divided by 6? — and often can’t focus on the strategy itself.

This, then, is a sort of dissent against the Three Act model of instruction. New mathematical ideas are not best introduced in the middle of a problem if they’re going to get the mental real estate they deserve. Students are often productively distracted by a difficult problem, and unable to focus on the strategy or tool at hand.

The thing that works better, in my experience, is following up a tough experience with a new idea or tool. This seems to me closer to ideal. The students get to spend of time struggling with a tough problem, which I think is valuable all on its own; they thoroughly understand the problem context, since they spent careful time on it; when I introduce a new idea after this experience, they are in a strong position to focus on this tricky new idea itself rather than the million other things it takes to comprehend this new tool.

As Dan Schwartz writes:

This report is based upon work supported by the National Science Foundation under REC Grant 0196238.

OK fine, but he also writes:

Instruction that allows students to generate imperfect solutions can be effective for future learning.

But instruction that comes in the heat of the moment is not looking towards the future — it’s coming during the chaotic present, a time when the student’s mind is being bombarded with many tricky ideas that are specific to a particular problem context. I don’t think that’s a great time to introduce a new idea, but tomorrow might be.