Feedbackless Feedback

I.

Not all my geometry students bombed the trig quiz. Some students knew exactly what they were doing:

Screenshot 2017-05-26 at 3.12.57 PM

A lot of my students, however, multiplied the tangent ratio by the height of their triangle:

Screenshot 2017-05-26 at 3.19.05 PM.png

In essence, it’s a corresponding parts mistake — the ’20’ corresponds to the ‘0.574’. The situation calls for division.

Half my class made this mistake on the quiz. What to do?

II.

Pretty much everyone agrees that feedback is important for learning, but pretty much nobody is sure what effective feedback looks like. Sure, you can find articles that promise 5 Research-Based Tips for great feedback, but there’s less there than meets the eye. You get guidelines like ‘be as specific as possible,’ which is the sort of goldilocks non-advice that education seems deeply committed to providing. Other advice is too vague to serve as anything but a gentle reminder of what we already know: ‘present feedback carefully,’ etc. You’ve heard this from me before.

As far as I can tell, this vagueness and confusion accurately reflects the state of research on feedback. The best, most current review of  feedback research (Valerie Schute’s) begins by observing that psychologists have been studying this stuff for over 50 years. And yet: “Within this large body of feedback research, there are many conflicting findings and no consistent pattern of results.”

Should feedback be immediate or delayed? Should you give lots of info, or not very much at all? Written or oral? Hints or explanations? If you’re hoping for guidance, you won’t find it here. (And let’s not forget that the vast majority of this research takes place in environments that are quite different from where we teach.)

Here’s how bad things are: Dylan Wilam, the guy who wrote the book on formative assessment, has suggested that the entire concept of feedback might be unhelpful in education.

It’s not looking like I’m going to get any clarity from research on what to do with this trig quiz.

III.

I’m usually the guy in the room who says that reductionist models are bad. I like messy models of reality. I get annoyed by overly-simplistic ideas about what science is or does. I don’t like simple models of teaching — it’s all about discovery — because I rarely find that things are simple. Messy, messy, (Messi!), messy.

Here’s the deal, though: a reductionist model of learning has been really clarifying for me.

The most helpful things I’ve read about feedback have been coldly reductive. Feedback doesn’t cause learning . Paying attention, thinking about new things — that leads to learning. Feedback either gets someone to think about something valuable, or it does nothing at all. (Meaning: it’s effecting either motivation or attention.)

Dylan Wiliam was helpful for me here too. He writes,

“If I had to reduce all of the research on feedback into one simple overarching idea, at least for academic subjects in school, it would be this: feedback should cause thinking.”

When is a reductive theory helpful, and when is it bad to reduce complexity? I wonder if reductive theories are maybe especially useful in teaching because the work has so much surface-level stuff to keep track of: the planning, the meetings, all those names. It’s hard to hold on to any sort of guideline during the flurry of a teaching day. Simple, powerful guidelines (heuristics?) might be especially useful to us.

Maybe, if the research on feedback was less of a random assortment of inconsistent results it would be possible to scrap together a non-reductive theory of it.

Anyway this is getting pretty far afield. What happened to those trig students?

IV. 

I’m a believer that the easiest way to understand why something is wrong is usually to understand why something else is right. (It’s another of the little overly-reductive theories I use in my teaching.)

The natural thing to do, I felt, would be to mark my students’ papers and offer some sort of explanation — written, verbal, whatever — about why what they did was incorrect, why they should have done 20/tan(30) rather than 20*tan(30). This seems to me the most feedbacky feedback possible.

But would that help kids learn how to accurately solve this problem? And would it get them to think about the difference between cases that call for each of these oh-so-similar calculations? I didn’t think it would.

So I didn’t bother marking their quizzes, at least right away. Instead I made a little example-based activity. I assigned the activity to my students in class the next day.

ums-copier@saintannsny.org_20170515_143541_001.jpg

I’m not saying ‘here’s this great resource that you can use.’ This is an incredibly sloppy version of what I’m trying to describe — count the typos, if you can. And the explanation in my example is kind of…mushy. Could’ve been better.

What excites me is that this activity is replacing what was for me a far worse activity. Handing back these quizzes focuses their attention completely on what they did and what they could done to get the question right. There’s a time for that too, but this wasn’t a time for tinkering, it was a time for thinking about an important distinction between two different problem types. This activity focused attention (more or less) where it belonged.

So I think, for now, this is what feedback comes down to. Trying to figure out, as specifically as possible, what kids could learn, and then trying to figure out how to help them learn it.

It can be a whole-class activity; it can be an explanation; it can be practice; it can be an example; it can be a new lesson. It doesn’t need to be a comment. It doesn’t need to be personalized for every student. It just needs to do that one thing, the only thing feedback ever can do, which is help kids think about something.

The term ‘feedback’ comes with some unhelpful associations — comments, personalization, a conversation. It’s best, I think, to ignore these associations. Sometimes, it’s helpful to ignore complexity.

Advertisements

Transformation Feedback that Worked, I Think

I felt good about how I handled feedback today in class. The class has been studying transformations in 8th Grade, and I wanted kids to practice visualizing different transformations. The task was to match shapes with transformations.

pic1.png

The above work is something I saw a lot. My kids had a pretty good sense of the different motions that translations, rotations and reflections were describing, and overall were pretty accurate.

Rotations around centers besides (0,0), though? Kids didn’t seem to know how to handle that. Some treated a rotation around (2,0) as just another rotation around the origin. Others seemed to get the quadrant of the image correct, but the coordinates weren’t right.

I decided that this would be what I’d focus the start of my next lesson on. I’d start with a pre-feedback activity, give some quick comments on the work, and then ask kids to revise their work.

This was the diagram I started class with:

pic1.png

I was trying to figure out what prompt to start with. I thought about just asking kids, “Where would the image be?” but this idea had problems. First, I wanted to know a bit more about how they thought because I wasn’t sure exactly where the problem was with their thinking about this. Second, I knew that kids would have trouble with this based on their previous work.

Instead, I posted the image and asked kids to figure out as much as they could about the diagram. I figured that this vaguer prompt would be more revealing while also giving me a chance to respond to ideas and help kids learn how to think about this sort of rotation.

(I also thought about just telling kids, here’s how you do this. I think that would have been plenty effective in this case, but I also like giving kids chances to have interesting mathematical ideas, and this seemed like a chance for that.)

Here was the annotated picture from under the document camera.

img_4278

The three ideas on the top were from kids. After hearing them I pointed out that some of these were correct, assuming that the transformation was around the origin.

Then I said, this is why I brought this image into class, because I noticed from your classwork that rotations not around the origin are hard. I have something that I think will help, I call it drawing a tether. And then I connected points A and B. Does this help you figure anything else out?

Very quickly a kid said, yeah, now I’ve got this and there was a lot of chatter about where the other points would go. I annotated those, and summarized that adding the tether would help. I also said that you might figure out where the other points are relative to B, and this led to more mathematical chatter and more asserting of where the points are. (Is that the sort of evidence I can use to know if a class is working out? Mathematical chatter?)

Then I said, I want you to try out these ideas while revising your classwork. I highlighted ones that could be revised, and I’d like you to draw a revision on the page. From past experience, I wanted to keep the timing of this quick, so I said you’ve got 5-10 minutes, and if you don’t start quickly then you probably won’t get a chance to learn much now.

Here was some of the revised work.

img_4276

Not everyone got feedback about rotation around (2,0). These kids get feedback about reflections around y = x and y = -x (which is good, because we need to know those for the next activity we’re working on).

img_4274

The questions that I’m left with are about how sure I can be that this lesson went as well as I felt it could be. What’s my evidence? Is it the way the kids were acting? Was it their engagement? Or their resulting work? Should I have done a quick exit assessment — would that have supported my claim that this worked?

And does “this lesson worked” mean that it just worked in the classroom? Or can it mean that the planning worked, that I made moves on purpose and with confidence?

How do you really know if a lesson worked?

Is Feedback A Chore?

But Wiliam offers two pieces of guidance that I think can create an effective, practical framework for using comments-only feedback.

  1. Students should do something with the feedback — and if it’s important to us, we should prioritize instructional time for them to do so.
  2. Opportunities for feedback should be structured such that the feedback is transferable beyond the task itself.

This framework suggests that much of the feedback I give is ineffective. If I don’t prioritize instructional time for students to respond to it, students who most need that feedback are unlikely to make effective use of it. And many tasks that I give feedback on are unlikely to lead to transferable learning, instead focusing student attention on concrete features of the task that will not support their learning in the future.

This is from Dylan Kane, who is one of the best classroom bloggers out there.

I’ve been grappling with the same issues Dylan brings up. I’ve recently written three pieces that try to get at my current approach. They also are my attempt to grapple with the limitations of research on feedback.

Feedback – We Still Don’t Know What Works

We Still Don’t Know What Works: Bonus Track

Beyond “Better-Luck-Next-Time” Feedback

Dylan’s post captures the idea that giving feedback is a chore, a regiment like dieting that we can discipline ourselves to keep. This attitude makes a lot of sense if feedback is an instructional “add-on,” something that goes over and above teaching. It’s extra, unnecessary, but (somehow) crucial.

My view is that it’s far more helpful to think about teaching routines that more naturally feature feedback, but are something more than “giving feedback.” This year I’ve nailed down one such routine in which written comments are just one helpful component: I give feedback to the class via an activity, and then use comments to connect that general, transferable lesson to my kids’ specific work.

It’s also my view that current research on feedback treats it like an “add-on.” I might be wrong, but I think that this is partly due to the lineage of “feedback.” It entered our lexicon through research on learning that did not grapple with classroom contexts. We teachers need to get better at expressing a view of the work that is truer to the work. When feedback is just slapped on to our teaching, it feels like a chore because it is a chore, because it’s sole purpose is to justify our judgement.

Is feedback good for learning? Are pencils good for learning? Feedback is the wrong thing to focus on. The right thing to focus on are the patterns of our teaching that we keep coming back to. Some of these involve written comments, others don’t.

Ultimately, we might need to stop thinking in terms of feedback. Meanwhile, we should look for routines that don’t make writing comments feel like a burden.

We Still Don’t Know What Feedback Works: Bonus Track

I wrote a piece about research on feedback — how it’s helped me, how it hasn’t — for the Learning Scientists blog. I worked hard on the piece, you should go read it. I also had a great time working with Megan and Yana — both learning science researchers — who run the Learning Scientists blog, and you should check out their work too.

Over at their blog, I make an argument that research on feedback has not, so far, been able to make real recommendations for the classroom. This is inherent in the way the work has been done — mostly in lab settings. Laboratory work usually simulates classroom environments where feedback is occasional, spotty, and easy to identify. Most k-12 classrooms — even ones with mediocre teaching — are knotty webs of interaction. Our classrooms are rich with feedback.

Our question isn’t whether or how to give feedback. After all, we’re going to give lots of verbal feedback, and every teacher ends up giving written feedback too. Sometimes we give immediate feedback; other times its delayed. We respond to so many ideas and actions that, in teaching, we end up giving a little bit of everything, when it comes to feedback. The question is: how do we structure all this feedback so that it actually helps learning? What form should it take, and what routines can help us give feedback that advances learning?

Go read the piece! It’s not long.

There was something else I had to say about this, but it didn’t belong in the piece. It’s about whether “feedback” is the right thing for us all to be talking about in the first place.

After all, there was a time before people talked about giving or receiving feedback. It’s a relatively recent development, actually. Check out the term’s frequency in the Google database:

Screenshot 2016-03-29 at 3.12.11 PM

The term originated in engineering contexts, and was only brought into education (and wider usage) later. Here’s Dylan Wiliam on the early history of the term:

Screenshot 2016-03-29 at 2.53.01 PM.png

Jumping ahead a bit, Wiliam argues that the move from engineering to education was not an entirely smooth one:

Screenshot 2016-03-29 at 2.55.26 PM.png

It’s been a while since I revisited this, there was a time when I was able to convince myself (mostly through Google Book searches, admittedly) that it was Skinner and the Behaviorists (solid band name) that helped shepherd the term into wider usage. Skinner’s version of feedback functioned a great deal as feedback does in engineering.  Feedback, for Skinner involved a bit of pain, a bit of pleasure, slowly conditioning a rat or person towards some greater truth. In these sorts of settings, yes, it makes some sense to talk about feedback all on its own. Learning has been immensely simplified into a positive or a negative association, so talk of feedback is clear and distinct. There is not major ambiguity as to what we are talking about.

In a classroom, though? If a teacher tells me that he “gave students some feedback” that could mean they did any of the following things: graded and returned their work; had a conversation one-on-one about an assignment; yelled at a kid for crossing a line; explained in a whole-class setting why a certain common answer was wrong; wrote a comment on a paper, without a grade; praised an answer; praised a person; sent a report card. And then we ask, “What’s the most effective way to give feedback?”

This is insane, and ultimately untenable. We can’t talk about how to give effective feedback for the same reason we can’t talk about how to effectively build a table.

What I’m putting my hopes into — and I said this at the end of my Learning Scientists piece — is in expanding the lens through which we look at feedback. If we are interested in creating opportunities for rich interactions between teacher and student that help learning, we need to describe whole routines of instruction that create these moments. The moment of feedback is a part of these routines, but the only way to make sense of them is to consider them as bits in entire movements of teaching. (Feedback is just an aria.)

Am I the only person to point this out? Hardly. I’ve borrowed this rant from others. Check out Kluger and DenisiValerie Shute, or Dylan Wiliam. They all call for expanding the scope of what we study — formative assessment, or feedback for learning, or formative feedback or whatever, but we have to study something more substantial than just feedback.

This is all pretty theoretical (see why it got cut?) but practically, here’s what it means: if someone tells you how to give effective feedback, do not believe them. Instead, try to find the larger routine where that effective feedback might thrive.

Essay: On Visual Patterns and Feedback

pic1.png
can you find a pattern in every direction?

Last summer I wrote an essay about how feedback and the math that visual pattern problems can help students learn.

Looking back, I don’t think this essay ever worked entirely, as a piece of writing.As my initial excitement about the piece soured, I never got around to giving it the big edit that it needed. Still, there are some good ideas in there that it helped me to figure out.

Here’s the essay: On Visual Patterns and Feedback

Here’s an excerpt:

I knew what I wanted to help Toni see. She was looking for a pattern in the growth, but she was having trouble getting specific about it. I wanted to ask a question that would draw Toni’s attention to helpful features of the pattern’s growth and help her get specific about precisely how this shape is changing.

This would involve a bit of guessing on my part, though, since I didn’t really know what question would work!

My first question was a promising dud: “Can you see the previous step in the following step?”

To which Toni responded, “no.”

I tried again, this time directing her attention more directly: “Do you see the second picture in the third? Imagine that you were building the third picture from the second. Where would you put the extra bricks?”

Bingo. She grabbed her pencil and started sketching.

Why did that question work? I think it’s because it encouraged Toni to see the static picture on the page as a changing thing. Toni had lots of experience playing with blocks and adding on parts to existing doodles. By asking her to think of one picture in the next, I helped direct her thinking to this analogy, and she was able to see the pattern’s growth in a useful way that related to things she had lots of experience with.

Like I said, an interesting failure. Enjoy! Let me know if you find parts of this useful.