One of the ideas that has been pinging around my brain recently is that the order of questions on a quiz or a test has a pretty large, but unmeasurable, effect on student performance. Not performance of the whole group, mind you. I am thinking on the granular, individual level. I am now reading Thinking Fast and Slow by Daniel Kahneman (it’s pretty fantastic so far) and he just described an experiment that got me thinking (quickly!) and I put the book down to bang out this quick post.
He describes a survey done of some students where one group of students saw these two questions in order.
- How happy are you these days?
- How many dates did you have last month?
When presented in this order, researchers saw essentially zero correlation between the answers to the questions. Kahneman concludes that dating is not the measurement by which students assess their own happiness. However, when the questions were reversed there was a remarkable correlation. He concludes that the first question gets students focused on that particular aspect of their lives and colors how they view the question about their own happiness.
Does something similar happen to many of our students? If an early question or two seems comfortable and familiar, does this build a sense of confidence and help lead to better results? I often find myself moving at least one of the questions that I anticipate to be a challenge toward the front of the test. My reasoning is that I want them to engage these richer questions while they still have more energy and more time to wrestle with the ideas built-in. After reading this brief description by Kahneman I am now doubting myself. I wonder if I would see better performance from some of my students if I intentionally arrange the test so as to build confidence and, perhaps, give some more built-in clues for later on in the assessment. The real problem with this type of thinking, and the type of thinking I have been more traditionally doing, is that my sense of which problems might pose a challenge do not always correlate to my students’ point of view. I think I want to have a conversation about this with my classes. My three subjects this year have such different sets of students that I suspect I will get pretty different feedback on this issue. That might serve me, and my students, well.
Any thoughts? Drop me a line here or over on twitter where I am @mrdardy