Testing and the Wisdom of Crowds
The other day, I gave my Advanced Technical Writing students a quiz. One question: “Name three things you should do before starting any research project.”
The answers weren’t in their text. I had not told them what these things should be in prior classes. In fact, the question had not come up–which is one reason I asked it.
What I was doing was something of an experiment on the wisdom of crowds and an attempt to make a point about authority and the weakness of the multiple-choice test when it is the sole means of evaluation.
Instead of grading the quizzes, I created a chart of their answers, grouping them into eight headings (any one of which could legitimately be on any list of top three tasks). The next day, I reported back to the class, listing to top three (and giving all of the students credit for the quiz). The wisdom of the group of 20 students was that one should:
- Consider the question;
- Consider what tools are available for researching the question; and
- Create a plan for developing an answer to the question.
Simple? Yes. An appropriate list? Yes.
But would this be the list that I, as teacher, would have given?
My list would have included something about considering audience–but it would not necessarily have been a better list. In fact, the more I think on it, the list the students came up with is better than mine would have been, for consideration of audience could fall within consideration of the question.
Which is the point.
The students, working independently and without “authority,” came up with a list as good as (better than, actually) any I could have created, even if different. Their group wisdom proved at least as great as (if not greater than) my individual wisdom, for all my experience and preparation.
For the wisdom of crowds to work, there can’t be a leader whose views influence the individuals. In this case, I had given no hint of what I might believe, only that I wanted three elements to their answers. When people defer to one person’s authority, they do not act as individuals, therefore their aggregate answers aren’t really an aggregate at all, but simply the reflection of the one. In such cases, there is no wisdom of the crowd. Of course, there are certainly cases where one does not want to rely on the wisdom of crowds, for the crowd may itself be limited, but it is certainly one way of producing knowledge and can work well as one aspect of education.
When we judge education simply by seeing if students can parrot back answers that we have already decided are correct, we stick ourselves in a rut, believing in the one true answer and ignoring the wide range of possibility that is out world. A multiple-choice test can be useful, yes, but it puts forward a view of knowledge that is extremely limiting and confining to students attempting to really learn. A multiple-choice test is the opposite of the quiz I gave, for the students are asked to choose between things presented rather than presenting things they have chosen.
Both student-generated knowledge and teacher-presented knowledge are needed for education to be complete. But we have moved towards “testing” as the single means of evaluating education and knowledge. However, as my students showed on that quiz (and in many other ways), real learning goes way beyond what can be encapsulated on a test.
My students learned a great deal through that quiz… through the discussion that ensued when I gave them back a graph and not individual test papers. They now know a little better that there are different kinds of authority, and generally a number of possible answers to even a simple question. They are learning to set up their own research projects and even ‘process documents’ in ways that reflect the possibility of multiple routes to any one solution… or solutions.
Learning is something that students do, not something that teachers give them. Evaluation, then, needs to take into account actual student thought and action, not simply their ability to memorize what teachers have spoon-fed them.
Until we reclaim that, no number of attempts to reform our schools will succeed.