Rock, Paper, … Chi-Square!

In my last post I talked about how my students are benefiting from my pals in the MTBoS. Well, here I am to testify again. We are just about to wrap up our study of the Chi-Square distribution in our AP Statistics class. I used to start this unit with a little document I created based on an article in Malcolm Gladwell’s Outliers. In his book he posits the idea that there is a disparity in birth date distribution for players on a junior national hockey team. In the document I linked to I put the roster information into an EXCEL spreadsheet and I just displayed the data and asked my students to notice things. I felt like I needed to stop using this because in each of the past two years I had a number of students who read the book in an English elective and they gave away the surprise before enough conversation happened. So, I put out a twitter call for help and Bob Lochel (@bobloch) chimed in and directed me to a super helpful post over on his blog. I had seen the cool applet for playing Rock, Paper, Scissors over at the New York Times. So, I borrowed heavily from Bob (and made sure to credit him during our class discussions) and off we went to the computer lab. I prepared a handout to help organize my students and I set them loose. I asked (as you can see on the handout doc) my students to play 24 time in four different contexts. Play with random moves generated by a random integer generator or play with your gut instincts and try each against the two modes of the machine on the Times’ website. The NYT claims that the ‘robot’ plays either as a novice with no pre-programmed knowledge of how the game is played or as an expert with data gathered from other players. The novice learns your patterns as it plays you while the expert calls on a large data set of how people behave. 24 repetitions is probably not enough for the novice computer but I had some time constraints that I was trying to work around. After both of my classes played, I created a document with the data on all of the results. The next day I displayed the data and we had a pretty great conversation about the results. An important note – some of my AP Stats kiddos cannot count because the data did not come in in multiples of 24. Sigh

So I tried to start the conversation with  a simple question – Should you do better when you think about the game or should you do better by random number generation? This lead to a quick decision that the expected value of a random number generator would be an equal distribution of 8 wins, 8 ties, and 8 losses for each set of 24. Now the table is set for the important principles of the Chi-Square test. Let’s talk about the difference between observed results and expected results. We also had a great conversation about how it appeared that the random number generator actually outperformed many people – especially in the expert mode. We talked about the fact that the expert mode was trying to predict behavior and how the randomness involved here might actually play in our favor.

In the week plus since this experiment I have been able to refer back a number of times to this experiment and it feels like my students have a pretty good handle on their task here. We have our unit test tomorrow so I hope that my optimism will be supported by some data.

In addition to thanking Bob Lochel I also want to thank a new twitter pal, Jennifer Micahelis (@MichaelisMath) who engaged me in a conversation about this experience and prompted me to gather my thoughts and write about it. I definitely will revisit this experiment the next time I teach this unit.

24 thoughts on “Rock, Paper, … Chi-Square!”

  1. Very interesting post. I like how having something like this activity at the beginning gives you something that you can continue to draw back on and talk about as you continue through the unit right up until the test.

    1. Thanks for dropping by! I was able to refer back a few times, but mostly I felt that the benefit was opening the discussion of expected results versus observed results.

  2. I would like to know the results of the test scores for your students. Does to paper, rock, scissor game truly give the students a richer understanding of the Chi-Square test? What question did the students have in regards to the results that they saw?

    1. The test results this year were solid, but unspectacular. It is, of course, awfully hard to tell sometimes what works well in building some lasting connections. They stumbled a bit on technical stuff in the multiple-choice portion of their test but they did a really nice job on a challenging free response question this year.

  3. I really liked how your students were engaged in this activity. It sounds like the students really understood the concepts as you were able to refer back to it. We also did this activity in class. Similar to your class, a group discussion was conducted as many questions were asked and thought about. I really like this activity and would use it in my future classroom. Thanks!

    1. Thanks for dropping by! I am glad to hear that it went well with your class, I will definitely call on this activity in the future when I teach statistics. It’s a keeper.

  4. I like the thought about how the random number generator could work to our advantage. We know that the computer picks up on our tendencies, but since the random number generator doesn’t have tendencies, it could allow us to beat the computer. At the same time the random number generator could still create our “tendency” and cause us to lose even more.

    1. What was fun in class was to watch the students wrestle with the idea that somehow randomness, blind luck, would somehow outperform our instincts.

  5. Hey Jim! I just got done doing the same activity as your students with the chi-squares. I was wondering if you had covered any definitions before doing this activity in class? Definitions such as the p-value, df, or what the chi-square number meant. To me, it was confusing having to figure those out on my own and I was just curious as to how you approached this lesson.

    1. We left the conversations about the chi-square statistic and its p-values until later. They were already familiar with the idea of degrees of freedom and expected values from our study of the t-distribution and our study of probability. I wanted this to be a way into those conversations about chi-square and a way to mathematize the idea of distance from expected values.

  6. I just finished doing this same activity and I am wondering how well the students actually did on the exam and if they were able to fully understand the material within the lesson or if they were unable to see the connections from the rock paper scissors game to the actual statistics. Interested to see how the data from the exam worked out.

    1. Thanks for dropping by! I commented above on the unit test performance this year. I saw some deep understandings developing but some small technical issues plaguing some of the students. I have some faith that ‘getting their hands dirty’ this way, playing around with seeing the difference between expected outcomes and actual data will have some lasting value.

  7. “So I tried to start the conversation with a simple question – Should you do better when you think about the game or should you do better by random number generation?”

    Should is an interesting word choice here. Its strange to me that when it comes to the discussion of games like this the lack of reference to the idea of skill. Rock/Paper/Scissors is a skill game that consists mostly of mind games; so when you talk about doing better (more wins+less losses) what you really are asking is if people playing are good (skilled) enough to be better than RNG or bad enough that they would be better off rolling a die.

    1. I was hoping to bait them into a discussion about the supposed benefits of our experience with this game. The mind game thing still came into play when we discussed this together. many students reported a strategy of switching when they lost to the play that the computer had just made. So, the instinct to psyche out the opponent exists even when the opponent is a machine.

  8. I really liked the activity overall. It was interesting to see the results of the random inputs compared to what a random data set is supposed to look like. I think you made an excellent point about random entries potentially being an advantage because they go against what the computer is predicting. What do you think was the best part of the lesson for your students? Also, if you had more time to commit to this lesson, what more would you do with your students?

    1. Thanks for dropping by – you raise some interesting questions. I think that if I had a longer time with my students, like a lab period type of setup, I would have gathered and compared data while we were still together. I would have had them throw up some dot plots or bar graphs to display their results. This could have generated some great conversation right away about the differences between expected and observed counts. I think that with any activity like this, the best part is that the students are not listening to some outside expert right away. They have the opportunity to think about these ideas and bounce questions off of each other first. Then, hopefully, they will be interested in listening to me as we unlock some of the technical aspects of the statistical testing process together.

  9. We just replicated your experiment in our Math 229 at GVSU. We had the peculiar result that, with 24 random attempts generated by 10 students, the students won about 27.5% of the time, and the computer won about 35.5% of the time. I think we need a larger sample size!

    Thanks for sharing this interesting activity!

  10. My college teacher-prep class just modeled this activity. It was quite fun and interactive! As I compared our results to your classes’ results, I find it interesting that your students were much more successful against the veteran computer with a random strategy (our data revealed 65 wins to 84 loses). I suppose I’m glad that your students’ data supported the idea that a random strategy should hypothetically perform better against the veteran computer.

    1. Thanks for dropping by and thanks for joining my list of blog followers! It’s hard to be glad about the notion of random decision making outperforming humans, isn’t it? It’s nice when a logical argument seems true, though.

  11. I really liked this activity! It helped me better understand and think about statistics in a way I haven’t before, since I haven’t ever taken it yet. So, since a sample size of 24 didn’t seem big enough, how big do you think it should be? My class did this activity and we had similar but still different results – our class lost more games as a whole, but we did have a smaller class so we had less data to work with.

    1. I am kind of torn about this since our class periods are not very long. I did not want it to become boring so I don’t want to have TOO many repetitions, but I also do not want to be too subject to the whims of small sample size fluctuations. I kind of chose 24 since it is a multiple of 3 and seemed kind of ‘big enough’
      Perhaps I could ask them to play more games but limit the number of strategies. If I assign just two or three strategies to each student and share the wealth I could ask for more trials without worrying about time constraints.

  12. Having gone through this activity just now in a teacher prep course, I was surprised by the similarity in some of the conclusions as well as a few different conclusions. We had not considered the fact that randomness might be an advantage against the veteran computer. I wonder if there would be a statistical significance between a player’s gut results and random results if the player used quite a few runs. I definitely enjoyed this activity and can see myself using it if/when I teach (AP) Statistics!

    1. Thanks for dropping by! I can tell you that the conversation about randomness being a superior strategy did not occur to me until I started looking at the data the kiddos generated. As I mentioned above in a comment, this activity is definitely a keeper for our AP Stats class.

Leave a Reply

Your email address will not be published. Required fields are marked *