Excerpts from my outbox: Where are all the good readings about feedback?

Question: Do you have any recommendations for good (i.e. accessible, useful, reasonable) readings on feedback?

Answer: First I’ll get the self-recommendations out of the way.

I did a blog series about feedback a few years ago. It’s definitely idiosyncratic, but it might be more readable than some of the academic stuff that I’m about to recommend. Also, there are some pretty detailed case studies as “chapters” in this series, so that might be helpful:

Series On Feedback and Revision

I really do have a lot of writing scattered around about feedback, and I can’t help but recommend more of my own stuff.

• “Better-luck-next-time” feedback is bad: https://blog.heinemann.com/mpershan-feedbackupdate-5-12
• On feedback research, and why it tends to be useless: http://www.learningscientists.org/blog/2016/3/29-1
• Same: https://problemproblems.wordpress.com/2016/03/29/we-still-dont-know-what-feedback-works-bonus-track/
• On thinking of feedback as more than just comments/conversations: https://problemproblems.wordpress.com/category/feedback/
• In particular: https://problemproblems.wordpress.com/2017/05/26/feedbackless-feedback/
• A specific routine I use for whole-class feedback: https://problemproblems.wordpress.com/2018/11/28/theres-nothing-passive-about-worked-example-activities/
• A mistake plus discussion about activities that would address the mistake, which is how I like thinking of feedback: http://mathmistakes.org/how-many-more-minutes/
• More posts along those lines: http://mathmistakes.org/finding-the-area-of-a-rectangle/and http://mathmistakes.org/5-out-of-6-books/

In terms of readings…well, I think honestly most stuff is not very good. If you end up reading some of my writing on this you’ll find out why I think that is. In short, I just don’t believe in feedback as a teaching concept. Conceptually it’s far too broad a category. Feedback can only (I think) be successfully studied in lab conditions that artificially restrict interactions between students and learning materials — so we end up hearing that feedback is very important, but really there’s very little useful to say about how or why. In teaching, feedback is constant but not always productive for teaching.

Dylan Wiliam is the standard recommendation, though I feel his notion of “formative assessment” isn’t much better than “feedback” in terms of covering a huge amount of teaching territory. (It does emphasize that student work should be used to impact teaching, but remains an incredibly broad category.) That said, I was definitely influenced by his piece in the NCTM research handbook, which he reworked later into a series of books — this is his Embedded Formative Assessment. I think the most useful things in this book are some of the ideas of specific formative assessment techniques. It’s still pretty jargony. I don’t love it, but it’s probably the best thing out there.

Valerie Shute wrote a long literature review. That’s on the researchy side of things. Again, I think it suffers from all the challenges of trying to organize a vast and unwieldy teaching concept, but I do find it useful as evidence that the research literature is a mess.

Both Wiliam and Shute cite Kluger & DeNisi’s meta-analysis and reviews of the feedback literature, because they did a great job underscoring how much of a mess the literature was/is. They’re the ones that found that roughly 1/3 of studies found that feedback had a negative impact on learning.

Another canonical citation is Ruth Butler’s experiment with gradeless comments — though honestly in this day and age we should be wary of psych studies with eye-popping results that haven’t been carefully replicated, and I don’t know if this has been. That’s just my editorial. Raymond wrote up the paper in a very nice way.

If I’m thin on readings that I haven’t written, that’s largely because I grew very unhappy with the dominant ways of talking about feedback. Once you take the lesson from Wiliam that kids don’t learn from feedback — they learn from thinking about feedback, or using feedback to do something else — then there’s no reason to think that grades or comments are any more “feedback” than your next classroom activity.

The thing that does matter is (a) figuring out what kids know and (b) crafting activities that are responsive to what kids know. If feedback is good for anything, it would have to be good for that. And that really has nothing to do with grades or comments on pages.

But in some ways, it’s hard to put the genie back in the bottle. People already think of feedback as comments/grades. So I don’t really know what to do with most of the existing writing about feedback, because I don’t know if I really want to talk about feedback as it’s usually defined at all.

 

Some new thoughts on hints

 

A couple years ago I gave a talk (above, and here) about hints.

(What’s the deal with cardigans? Seems like they were totally in for a few years and now (for a fashion-forward fellow) they are practically unwearable. Maybe they were never in, I don’t know.)

Thanks to a conversation with Dave today, I started thinking again about hints, and I think I have something to add about hint-giving.

So, picture yourself in a classroom, a kid waves a hand and gets your attention. ‘Can I have some help? I’m totally stuck.’

You walk over: tell me your symptoms? when did it start? and then?

(My point being, you assess the situation…)

OK but once you have a sense of what’s going on, I think there are roughly three possibilities I experience and three corresponding suggestions I can make for how to react.

Possibility One: I need to teach the student something, so I sketch a quick example.

When an individual student needs to learn something new in the middle of the task, it’s never ideal.

I used to handle these moments by trying to nudge the kid along with questions about the task at hand. I’ve come to think that this is a mistake, and now I try to avoid it. Instead, I try to quickly write a related problem with a relevant, worked out solution.

Here’s a snapshot of what I mean. This student called me over because she was totally stuck on solving -1.7x = 4.3x + 3.6, and after some gentle questioning I saw what she meant. Knowing a bit about what this kid was already comfortable with and where she could go next, I quickly wrote a totally different problem [-2x = 5x + 7] and wrote a solution exemplifying what she might do for the problem at hand. (She still made mistakes, which I highlighted. Sorry for the overstuffed picture.)

I do this for two reasons.

The first is that, when one aims to nudge kids along at the task at hand, one runs the risk of ending up in a Polya-esque recursive suckhole of questions, digging the student deeper and deeper into their own brain, until the math is buried beneath piles of questions.

I call this ‘Polya-esque’ because when I think about the sort of interactions I try to avoid one-on-one, I often think of this passage from How to Solve It:

If the teacher, having watched sharply, cannot detect any sign of such intiative he has to resume carefully his dialogue with the students. He must be prepared to repeat with some modification the questions which the students do not answer. He must be prepared to meet often with the disconcerting silence of the students…

“Do you know a related problem?”

…..

“Look at the unknown! Do you know a problem having the same unknown?”

…..

“Well, what is the unknown?”

“The diagonal of a parallelpiped.”

“Do you know any problem with the same unknown?”

“No. We have not had any problem yet about the diagonal of a parallelpiped.”

“Do you know any problem with a similar unknown?”

…..

That’s issue #1 with focusing on the task

The second is that I worry that it’s very hard for a person to learn something from thinking about just one problem. I want to leave students with a chance to think about a whole problem, not just the little scraps that I didn’t solve for them via nudges. (These conversations just are “better-luck-next-time” interactions.)

Ideally, a kid has more than just one example and one practice problem to learn something new. Then again, ideally a kid isn’t missing crucial knowledge that’s keeping them from doing math in class. We’re well past ‘ideally.’

So, the first situation is that a kid really needs to learn something new. In the heat of the moment, I don’t try to feed them each step or to weave a series of vague questions into a meaningful discovery. Instead, when a kid is stuck because they need to learn something, I try to teach them that thing.

Possibility Two: I need to help the student make a connection, so I remind them of a similar problem.

This is the territory that is closest to what I was describing in my talk. If I’ve done a nice job with my teaching, the kids have some memorable examples, ideas, problems or techniques to refer back to when trying something on their own. That way, when a kid tells me that they’re stuck but I don’t think they’re missing something crucial, I can lead with…

• Remember the diagram we were studying at the start of class? This problem is actually really similar to that one.
• So this is a complex area problem, and there are always basically two options: add some lines to cut the shape up, or use negative space. Which do you want to try here?
• I see you solved this equation by adding two to both sides. Why not do something similar here?

Of course, this only works if the students have some prior knowledge. I often lead with this and see if I get a catch. If I don’t, maybe I’ll start thinking about Possibility One.

Possibility Three: I need to help a student realize that they can handle this on my own, so I redirect the student back to the problem.

Sometimes the only thing a student doesn’t know about some math is that they know it. (Which is something that they need to know.) In situations like these, my job is to either reassure the student that they’ve started down a good path, that they aren’t breaking math, or to deflect the question in a way that puts the work back on the kid.

I feel as if there isn’t much more to say about this possibility — it’s the one that math educators generally love to talk about, because it’s the most fun. And, come on, it is fun. How cool is it that the following interaction actually works, ever?

Student: Hey I’m totally stuck.

Teacher: OK what if you weren’t stuck?

This absolutely works, but only sometimes. To get roughly precise, it only works (roughly) a third of the time, because there are two other possibilities.

Other moves that are fun when we’re just trying to redirect attention back to the problem:

• What have you tried so far?
• What haven’t you tried?
• Why did you write this?
• Talk for a bit about this.

There’s not much magic here. When a student is in a situation like this it’s often just about getting them back into the problem. They probably got nervous about something and stopped early. I do that all the time when I’m doing math, it’s totally normal. Lending confidence is one of the many little things that a teacher can do.

***

Cardigans or not, I do need more sweaters, though at this point I could easily just wait until the fall.

Hints are a nice slice of teaching. Not too much, not too little. And it’s this interaction pattern that I have so many times a day. Question, response, question, response. It’s so easy to experiment and try out a slightly different pattern of response. If it works, I’ll likely try it again, and again, and then one year passes and then it’s another, and slowly a new pattern of interaction has replaced the old.

Some people that I talk to don’t like the connotations surrounding the word itself, ‘hint.’ I get that, and to that I offer three replacements that correspond to three possibilities. When a kid is stuck, in most cases I respond with either encouragement, a reminder, or a new example. If that’s simpler to talk about, then let’s stick to that.

Teaching, in General

I. 

If you give a quiz covering lots of different topics, you’re going to get a lot of different mistakes. Which leaves you with a dilemma: how do you address those mistakes?

Yesterday’s quiz in geometry was a review quiz, so the topics were from all over the place:

• angles in isosceles triangles
• inscribed angles in a circle
• area of triangles, parallelograms and trapezoids
• congruence proofs

As expected, kids distributed their not-quite-there work fairly evenly across these topics. (OK so that’s not true, there were a lot of issues with the congruence proofs. There always are and always will be. Sigh.)

Here were two bad options for returning the quiz:

• Try to address all the issues with individual comments. First, it’s a game of whack-a-mole that is guaranteed to drive me insane. Second, what should I do? Try to leave perfect hints? Say nothing, and let kids figure out on their own what they did wrong? Show them the correct way to answer the question, and thereby eliminate anything for the kids to actually think about when I return the quizzes?
• Pick just one thing to focus on. Reteach that one thing in a careful way, then return the quizzes and ask kids to revise.

The second of the two options is great when there the mistakes are in the same galaxy. (I wrote about this in a post, Feedbackless Feedback.) But, I’m realizing now, this isn’t a terrific move when the mistakes are distributed across many topics. Because on what basis should I pick something to focus on reteaching? Any choice would be equally bad.

II. 

While reviewing the class’ quizzes, I found myself falling into written comments, at least until I figured out what else to do with the quizzes.

I used to write long, wordy comments that were essentially hints on the margins of the page. (“Great start! Have you tried multiplying both sides of the equation by 3?”) I came to dislike those sort of comments, as they just focus focus focus attention all on THIS problem. But I don’t particularly care about whether a students gets this problem correct; I care about the generalization.

What I’ve fallen into is, whenever possible, writing a quick example that’s related (but not identical) to the trouble-problem (the problem-problem) on the page. I do this below on the second question:

Then, I ask kids to revise the original on the basis of the example (or anything else they realized).

After writing a few of these example-comments, I realized I was taking a lot of time doing this, and repeating myself somewhat. I also realized that I don’t know if I could repeat this on every page for the congruence proofs, as the problem itself was reasonably complex:

Hard to read. I use the highlighter to flag errors, but this student highlighted the triangle himself.

I wasn’t sure what to do. Then, I remembered something I had read from Dylan Wiliam — I think it’s in Embedded Formative Assessment. His idea there was that you can give all the class’ comments to everyone, and then kids have to decide which comments apply to them.

I thought, OK, I can work with this. So I quickly (quickly!) made a page of examples, one for every mistake I saw on the quiz:

My routine in class went like this:

• Hand out the examples for revision.
• Hand back the quizzes with comments.
• Search for an example that’s relevant to your mistake.
• Call for revision on the basis of the examples. Work with friends, neighbors. Of course, I’m available to help.
• Then, try the extension task.

This was my first time trying this, but I thought it went well. Solid engagement, really good questions, no unproductively stuck students.

When you do something good in teaching, you never really know if it’ll work again, but I’ve got a good feeling about this one. It feels like a lot of what has already worked for me, but in a better order.

III.

Harry Fletcher-Wood is very nice and has a lot of interesting thoughts about feedback. As such, Harry and I very nicely disagree about a pretty interesting question about feedback: how can you teach people how to give better feeedback?

The usual caveats apply: I am not a teacher teacher, but Harry is involved in teacher education, and I have no idea if I’m right on this.

In any event, Harry recently published a really cool post where he tried to synthesize a lot of the research on feedback into a decision tree:

Now, this is awesome as a synthesis. But just because something is a good description of feedback doesn’t mean that it’s useful prescriptive advice. My favorite example of this comes from Pólya’s strategies for mathematical problem solving. Alan Schoenfeld has a nice way of putting it in Learning to Think Mathematically — the strategies have descriptive, but not prescriptive validity:

In short, the critique of the strategies listed in How to Solve It and its successors is that the characterizations of them were descriptive rather than prescriptive. That is, the characterizations allowed one to recognize the strategies when they were being used. However, Pólya’s characterizations did not provide the amount of detail that would enable people who were not already familiar with the strategies to be able to implement them.

In other words, just because a heuristic is a good description of practice doesn’t mean that it is an effective pedagogical tool. And that’s precisely my concern with Harry’s decision tree.

Feedback is a high-level concept that describes a TON of what happens in teaching. And any guidelines for how to give feedback effectively are also going to be high-level in a way that reminds me of Pólya’s moves like “find a simpler problem” or “draw a picture.”

And just as Pólya’s moves struggle because they aim to guide problem solving in geometry, algebra, topology, etc., all areas of math, Harry’s decision tree seems to me an attempt to guide feedback in all areas of teaching — math, history, medical school, etc.

Of course, Harry doesn’t intend for this to be the only thing guiding students, but neither did Pólya. My question is whether these generalizations themselves are helpful, beyond whatever ways that teacher educators can make them concrete and specific for teachers.

But what’s the alternative?

I don’t know yet. I can say a few things now that I couldn’t a few years ago:

• I think domain-specific — math-specific, history-specific — generalizations will be more useful than domain-general ones.
• I think that the generalizations can productively come in the form of instructional routines.

And, with this post and the other one, I now have two generalizations I can make about giving feedback in math class.

First: if there’s a problem that a lot of students have trouble with, consider a reteaching/revising cycle like the one in this image:

Second: if mistakes are sprinkled across too many topics, consider something like the revision routine I described in this post.

IV.

My bet is that a lot of knowledge about teaching looks like this. It’s not that there isn’t knowledge about teaching that accrues, but that we look for ways to scale things out of their contexts. Then we call those things myths and talk about how we have to kill ’em.

In general, generalizations about teaching are hard to come by. But nobody teaches in general. All teaching is intensely particular. These kids. These schools. This idea.

Some people are skeptical of the possibility of making generalizations about teaching, and the vast majority of people are cheery about making sky-high generalizations that cross every context. There’s a middle position that I want to find. There’s a sweet spot for knowledge about teaching, though I don’t know if we’ve all found it yet.

Feedbackless Feedback

I.

Not all my geometry students bombed the trig quiz. Some students knew exactly what they were doing:

A lot of my students, however, multiplied the tangent ratio by the height of their triangle:

In essence, it’s a corresponding parts mistake — the ’20’ corresponds to the ‘0.574’. The situation calls for division.

Half my class made this mistake on the quiz. What to do?

II.

Pretty much everyone agrees that feedback is important for learning, but pretty much nobody is sure what effective feedback looks like. Sure, you can find articles that promise 5 Research-Based Tips for great feedback, but there’s less there than meets the eye. You get guidelines like ‘be as specific as possible,’ which is the sort of goldilocks non-advice that education seems deeply committed to providing. Other advice is too vague to serve as anything but a gentle reminder of what we already know: ‘present feedback carefully,’ etc. You’ve heard this from me before.

As far as I can tell, this vagueness and confusion accurately reflects the state of research on feedback. The best, most current review of  feedback research (Valerie Schute’s) begins by observing that psychologists have been studying this stuff for over 50 years. And yet: “Within this large body of feedback research, there are many conflicting findings and no consistent pattern of results.”

Should feedback be immediate or delayed? Should you give lots of info, or not very much at all? Written or oral? Hints or explanations? If you’re hoping for guidance, you won’t find it here. (And let’s not forget that the vast majority of this research takes place in environments that are quite different from where we teach.)

Here’s how bad things are: Dylan Wilam, the guy who wrote the book on formative assessment, has suggested that the entire concept of feedback might be unhelpful in education.

It’s not looking like I’m going to get any clarity from research on what to do with this trig quiz.

III.

I’m usually the guy in the room who says that reductionist models are bad. I like messy models of reality. I get annoyed by overly-simplistic ideas about what science is or does. I don’t like simple models of teaching — it’s all about discovery — because I rarely find that things are simple. Messy, messy, (Messi!), messy.

Here’s the deal, though: a reductionist model of learning has been really clarifying for me.

The most helpful things I’ve read about feedback have been coldly reductive. Feedback doesn’t cause learning . Paying attention, thinking about new things — that leads to learning. Feedback either gets someone to think about something valuable, or it does nothing at all. (Meaning: it’s affecting either motivation or attention.)

Dylan Wiliam was helpful for me here too. He writes,

“If I had to reduce all of the research on feedback into one simple overarching idea, at least for academic subjects in school, it would be this: feedback should cause thinking.”

When is a reductive theory helpful, and when is it bad to reduce complexity? I wonder if reductive theories are maybe especially useful in teaching because the work has so much surface-level stuff to keep track of: the planning, the meetings, all those names. It’s hard to hold on to any sort of guideline during the flurry of a teaching day. Simple, powerful guidelines (heuristics?) might be especially useful to us.

Maybe, if the research on feedback was less of a random assortment of inconsistent results it would be possible to scrap together a non-reductive theory of it.

Anyway this is getting pretty far afield. What happened to those trig students?

IV. 

I’m a believer that the easiest way to understand why something is wrong is usually to understand why something else is right. (It’s another of the little overly-reductive theories I use in my teaching.)

The natural thing to do, I felt, would be to mark my students’ papers and offer some sort of explanation — written, verbal, whatever — about why what they did was incorrect, why they should have done 20/tan(30) rather than 20*tan(30). This seems to me the most feedbacky feedback possible.

But would that help kids learn how to accurately solve this problem? And would it get them to think about the difference between cases that call for each of these oh-so-similar calculations? I didn’t think it would.

So I didn’t bother marking their quizzes, at least right away. Instead I made a little example-based activity. I assigned the activity to my students in class the next day.

I’m not saying ‘here’s this great resource that you can use.’ This is an incredibly sloppy version of what I’m trying to describe — count the typos, if you can. And the explanation in my example is kind of…mushy. Could’ve been better.

What excites me is that this activity is replacing what was for me a far worse activity. Handing back these quizzes focuses their attention completely on what they did and what they could done to get the question right. There’s a time for that too, but this wasn’t a time for tinkering, it was a time for thinking about an important distinction between two different problem types. This activity focused attention (more or less) where it belonged.

So I think, for now, this is what feedback comes down to. Trying to figure out, as specifically as possible, what kids could learn, and then trying to figure out how to help them learn it.

It can be a whole-class activity; it can be an explanation; it can be practice; it can be an example; it can be a new lesson. It doesn’t need to be a comment. It doesn’t need to be personalized for every student. It just needs to do that one thing, the only thing feedback ever can do, which is help kids think about something.

The term ‘feedback’ comes with some unhelpful associations — comments, personalization, a conversation. It’s best, I think, to ignore these associations. Sometimes, it’s helpful to ignore complexity.

Transformation Feedback that Worked, I Think

I felt good about how I handled feedback today in class. The class has been studying transformations in 8th Grade, and I wanted kids to practice visualizing different transformations. The task was to match shapes with transformations.

The above work is something I saw a lot. My kids had a pretty good sense of the different motions that translations, rotations and reflections were describing, and overall were pretty accurate.

Rotations around centers besides (0,0), though? Kids didn’t seem to know how to handle that. Some treated a rotation around (2,0) as just another rotation around the origin. Others seemed to get the quadrant of the image correct, but the coordinates weren’t right.

I decided that this would be what I’d focus the start of my next lesson on. I’d start with a pre-feedback activity, give some quick comments on the work, and then ask kids to revise their work.

This was the diagram I started class with:

I was trying to figure out what prompt to start with. I thought about just asking kids, “Where would the image be?” but this idea had problems. First, I wanted to know a bit more about how they thought because I wasn’t sure exactly where the problem was with their thinking about this. Second, I knew that kids would have trouble with this based on their previous work.

Instead, I posted the image and asked kids to figure out as much as they could about the diagram. I figured that this vaguer prompt would be more revealing while also giving me a chance to respond to ideas and help kids learn how to think about this sort of rotation.

(I also thought about just telling kids, here’s how you do this. I think that would have been plenty effective in this case, but I also like giving kids chances to have interesting mathematical ideas, and this seemed like a chance for that.)

Here was the annotated picture from under the document camera.

The three ideas on the top were from kids. After hearing them I pointed out that some of these were correct, assuming that the transformation was around the origin.

Then I said, this is why I brought this image into class, because I noticed from your classwork that rotations not around the origin are hard. I have something that I think will help, I call it drawing a tether. And then I connected points A and B. Does this help you figure anything else out?

Very quickly a kid said, yeah, now I’ve got this and there was a lot of chatter about where the other points would go. I annotated those, and summarized that adding the tether would help. I also said that you might figure out where the other points are relative to B, and this led to more mathematical chatter and more asserting of where the points are. (Is that the sort of evidence I can use to know if a class is working out? Mathematical chatter?)

Then I said, I want you to try out these ideas while revising your classwork. I highlighted ones that could be revised, and I’d like you to draw a revision on the page. From past experience, I wanted to keep the timing of this quick, so I said you’ve got 5-10 minutes, and if you don’t start quickly then you probably won’t get a chance to learn much now.

Here was some of the revised work.

Not everyone got feedback about rotation around (2,0). These kids get feedback about reflections around y = x and y = -x (which is good, because we need to know those for the next activity we’re working on).

The questions that I’m left with are about how sure I can be that this lesson went as well as I felt it could be. What’s my evidence? Is it the way the kids were acting? Was it their engagement? Or their resulting work? Should I have done a quick exit assessment — would that have supported my claim that this worked?

And does “this lesson worked” mean that it just worked in the classroom? Or can it mean that the planning worked, that I made moves on purpose and with confidence?

How do you really know if a lesson worked?

Is Feedback A Chore?

But Wiliam offers two pieces of guidance that I think can create an effective, practical framework for using comments-only feedback.

1. Students should do something with the feedback — and if it’s important to us, we should prioritize instructional time for them to do so.
2. Opportunities for feedback should be structured such that the feedback is transferable beyond the task itself.

This framework suggests that much of the feedback I give is ineffective. If I don’t prioritize instructional time for students to respond to it, students who most need that feedback are unlikely to make effective use of it. And many tasks that I give feedback on are unlikely to lead to transferable learning, instead focusing student attention on concrete features of the task that will not support their learning in the future.

This is from Dylan Kane, who is one of the best classroom bloggers out there.

I’ve been grappling with the same issues Dylan brings up. I’ve recently written three pieces that try to get at my current approach. They also are my attempt to grapple with the limitations of research on feedback.

Feedback – We Still Don’t Know What Works

We Still Don’t Know What Works: Bonus Track

Beyond “Better-Luck-Next-Time” Feedback

Dylan’s post captures the idea that giving feedback is a chore, a regiment like dieting that we can discipline ourselves to keep. This attitude makes a lot of sense if feedback is an instructional “add-on,” something that goes over and above teaching. It’s extra, unnecessary, but (somehow) crucial.

My view is that it’s far more helpful to think about teaching routines that more naturally feature feedback, but are something more than “giving feedback.” This year I’ve nailed down one such routine in which written comments are just one helpful component: I give feedback to the class via an activity, and then use comments to connect that general, transferable lesson to my kids’ specific work.

It’s also my view that current research on feedback treats it like an “add-on.” I might be wrong, but I think that this is partly due to the lineage of “feedback.” It entered our lexicon through research on learning that did not grapple with classroom contexts. We teachers need to get better at expressing a view of the work that is truer to the work. When feedback is just slapped on to our teaching, it feels like a chore because it is a chore, because it’s sole purpose is to justify our judgement.

Is feedback good for learning? Are pencils good for learning? Feedback is the wrong thing to focus on. The right thing to focus on are the patterns of our teaching that we keep coming back to. Some of these involve written comments, others don’t.

Ultimately, we might need to stop thinking in terms of feedback. Meanwhile, we should look for routines that don’t make writing comments feel like a burden.

We Still Don’t Know What Feedback Works: Bonus Track

I wrote a piece about research on feedback — how it’s helped me, how it hasn’t — for the Learning Scientists blog. I worked hard on the piece, you should go read it. I also had a great time working with Megan and Yana — both learning science researchers — who run the Learning Scientists blog, and you should check out their work too.

Over at their blog, I make an argument that research on feedback has not, so far, been able to make real recommendations for the classroom. This is inherent in the way the work has been done — mostly in lab settings. Laboratory work usually simulates classroom environments where feedback is occasional, spotty, and easy to identify. Most k-12 classrooms — even ones with mediocre teaching — are knotty webs of interaction. Our classrooms are rich with feedback.

Our question isn’t whether or how to give feedback. After all, we’re going to give lots of verbal feedback, and every teacher ends up giving written feedback too. Sometimes we give immediate feedback; other times its delayed. We respond to so many ideas and actions that, in teaching, we end up giving a little bit of everything, when it comes to feedback. The question is: how do we structure all this feedback so that it actually helps learning? What form should it take, and what routines can help us give feedback that advances learning?

Go read the piece! It’s not long.

There was something else I had to say about this, but it didn’t belong in the piece. It’s about whether “feedback” is the right thing for us all to be talking about in the first place.

After all, there was a time before people talked about giving or receiving feedback. It’s a relatively recent development, actually. Check out the term’s frequency in the Google database:

The term originated in engineering contexts, and was only brought into education (and wider usage) later. Here’s Dylan Wiliam on the early history of the term:

Jumping ahead a bit, Wiliam argues that the move from engineering to education was not an entirely smooth one:

It’s been a while since I revisited this, there was a time when I was able to convince myself (mostly through Google Book searches, admittedly) that it was Skinner and the Behaviorists (solid band name) that helped shepherd the term into wider usage. Skinner’s version of feedback functioned a great deal as feedback does in engineering.  Feedback, for Skinner involved a bit of pain, a bit of pleasure, slowly conditioning a rat or person towards some greater truth. In these sorts of settings, yes, it makes some sense to talk about feedback all on its own. Learning has been immensely simplified into a positive or a negative association, so talk of feedback is clear and distinct. There is not major ambiguity as to what we are talking about.

In a classroom, though? If a teacher tells me that he “gave students some feedback” that could mean they did any of the following things: graded and returned their work; had a conversation one-on-one about an assignment; yelled at a kid for crossing a line; explained in a whole-class setting why a certain common answer was wrong; wrote a comment on a paper, without a grade; praised an answer; praised a person; sent a report card. And then we ask, “What’s the most effective way to give feedback?”

This is insane, and ultimately untenable. We can’t talk about how to give effective feedback for the same reason we can’t talk about how to effectively build a table.

What I’m putting my hopes into — and I said this at the end of my Learning Scientists piece — is in expanding the lens through which we look at feedback. If we are interested in creating opportunities for rich interactions between teacher and student that help learning, we need to describe whole routines of instruction that create these moments. The moment of feedback is a part of these routines, but the only way to make sense of them is to consider them as bits in entire movements of teaching. (Feedback is just an aria.)

Am I the only person to point this out? Hardly. I’ve borrowed this rant from others. Check out Kluger and Denisi, Valerie Shute, or Dylan Wiliam. They all call for expanding the scope of what we study — formative assessment, or feedback for learning, or formative feedback or whatever, but we have to study something more substantial than just feedback.

This is all pretty theoretical (see why it got cut?) but practically, here’s what it means: if someone tells you how to give effective feedback, do not believe them. Instead, try to find the larger routine where that effective feedback might thrive.

Essay: On Visual Patterns and Feedback

can you find a pattern in every direction?

Last summer I wrote an essay about how feedback and the math that visual pattern problems can help students learn.

Looking back, I don’t think this essay ever worked entirely, as a piece of writing.As my initial excitement about the piece soured, I never got around to giving it the big edit that it needed. Still, there are some good ideas in there that it helped me to figure out.

Here’s the essay: On Visual Patterns and Feedback

Here’s an excerpt:

I knew what I wanted to help Toni see. She was looking for a pattern in the growth, but she was having trouble getting specific about it. I wanted to ask a question that would draw Toni’s attention to helpful features of the pattern’s growth and help her get specific about precisely how this shape is changing.

This would involve a bit of guessing on my part, though, since I didn’t really know what question would work!

My first question was a promising dud: “Can you see the previous step in the following step?”

To which Toni responded, “no.”

I tried again, this time directing her attention more directly: “Do you see the second picture in the third? Imagine that you were building the third picture from the second. Where would you put the extra bricks?”

Bingo. She grabbed her pencil and started sketching.

Why did that question work? I think it’s because it encouraged Toni to see the static picture on the page as a changing thing. Toni had lots of experience playing with blocks and adding on parts to existing doodles. By asking her to think of one picture in the next, I helped direct her thinking to this analogy, and she was able to see the pattern’s growth in a useful way that related to things she had lots of experience with.

Like I said, an interesting failure. Enjoy! Let me know if you find parts of this useful.