Another Thought About Assessment

One of the ideas that has been pinging around my brain recently is that the order of questions on a quiz or a test has a pretty large, but unmeasurable, effect on student performance. Not performance of the whole group, mind you. I am thinking on the granular, individual level. I am now reading¬†Thinking Fast and Slow by Daniel Kahneman (it’s pretty fantastic so far) and he just described an experiment that got me thinking (quickly!) and I put the book down to bang out this quick post.

He describes a survey done of some students where one group of students saw these two questions in order.

  • How happy are you these days?
  • How many dates did you have last month?

When presented in this order, researchers saw essentially zero correlation between the answers to the questions. Kahneman concludes that dating is not the measurement by which students assess their own happiness. However, when the questions were reversed there was a remarkable correlation. He concludes that the first question gets students focused on that particular aspect of their lives and colors how they view the question about their own happiness.

Does something similar happen to many of our students? If an early question or two seems comfortable and familiar, does this build a sense of confidence and help lead to better results? I often find myself moving at least one of the questions that I anticipate to be a challenge toward the front of the test. My reasoning is that I want them to engage these richer questions while they still have more energy and more time to wrestle with the ideas built-in. After reading this brief description by Kahneman I am now doubting myself. I wonder if I would see better performance from some of my students if I intentionally arrange the test so as to build confidence and, perhaps, give some more built-in clues for later on in the assessment. The real problem with this type of thinking, and the type of thinking I have been more traditionally doing, is that my sense of which problems might pose a challenge do not always correlate to my students’ point of view. I think I want to have a conversation about this with my classes. My three subjects this year have such different sets of students that I suspect I will get pretty different feedback on this issue. That might serve me, and my students, well.

Any thoughts? Drop me a line here or over on twitter where I am @mrdardy

 

Seeking Better Feedback From Students

At this school and my last one there is a formal process for students to voice their opinions/concerns/suggestions through a course evaluation form. At both schools I have noticed that not enough students take this process as seriously as we want them to. The forms are well-intentioned and detailed in their questions, but they are all Likert-scale questions (with some blank space for expanding answers) and they are hand written forms. I suspect that many students are self-conscious about their handwriting being identified. I think that one of the results of this is that the only handwritten extensions we tend to see are from students who are happy and want to share nice words in depth (we love it when we see these!) or when they are especially unhappy and want to share unkind words in depth (happily, these are much less common.) However, I always walk away feeling that these could be better and more useful. I would love to have a process where these are done online anonymously as I suspect that we will get more willingness to expand on answers. I also want questions that are aimed a little more directly at the concerns of our math classrooms rather than well meaning general feedback forms.

I am convinced that the process is a meaningful one. Letting our students know that their voices and opinions matter is a powerful thing. Letting them know that the adults who have been evaluating them need to hear back on how we are doing levels the playing field (at least a little bit) and this tells our students that they can be part of an honest feedback loop where we can grow based on their experiences/opinions/successes/failures. But I also know that the process can be better than the one we have.

I know I have read a few posts along these lines and I am off to the MTBoS search engine this morning to see what I can find. I would love to hear from anyone reading this about their successes, and failures, in elucidating meaningful feedback from their students.

As always, feel free to drop a comment here or pick up the conversation over on twitter where I remain @mrdardy

Thanks in advance for any wisdom.