According to a recent French-American study, a very compelling reason exists for ditching evaluations: they discriminate against women. Photo: Stocksy
Should universities ask students to rate their professors? Given that students hand over bucket-loads of money for their degrees, surely they have a right to complain if the product is faulty. And shouldn't teachers be open to criticism? After all, the rest of society seems to have accepted that the nuances of human relationships can be reduced to the crushing banality of a thumbs-up or thumbs-down. Why quarantine the university from the tyranny of the amateur reviewer?
According to a recent French-American study, a very compelling reason exists for ditching evaluations: they discriminate against women. In the words of economist Anne Boring, the lead researcher on the paper, they "do not measure teaching effectiveness", are strongly "biased by gender" and "disadvantage female instructors". Even worse, these evaluations are then used by universities when determining who to hire, promote or retain, which means that the sexism of students is partly determining female academics' career advancement.
The researchers conducted statistical tests on two different groups of French and U.S. university students. In the French case students were assigned both male and female instructors for a course. At the end of the course they were made to sit a final exam, regardless of which instructor they had. As the purpose of student evaluations is to measure teaching effectiveness, you could assume that the students' final grade would reflect on the quality of the teaching. The results? The students of male instructors on average did worse on the final exam, BUT male students rated male instructors more highly across the board.
In the American test, students took an online course with either a male or female instructor, but in half the cases the male instructors used women's names and vice-versa. It was a kind of virtual drag. The instructors gave similar feedback on assignments and returned students' work at exactly the same time. Yet when it came to assessing their instructors, women gave instructors they believed to be male higher evaluations than instructors they thought to be female. Yup, even for something objective like returning work on time, women instructors received scores that were 16% lower.
As someone who has taught both history and law at universities for over ten years, this is all depressingly familiar. In my experience, gender bias is also apparent in how students are likely to describe you. Disaffected students are more likely to call female instructors 'combative' while our male colleagues get called 'challenging'; we get 'bossy' while they get 'authoritative'. A British study also found that female faculty are more likely to be characterised as 'approachable, helpful or nice' while male faculty are hailed as 'funny, brilliant, genius or arrogant.'
Women are also crucified if they fail to meet the gendered expectation that they be supportive or caring. Intellectual labour must always be accompanied by emotional labour. Oh, and attractiveness. In America instructors get ranked in one online survey for 'hotness'.
Of course, anything that is bad for all women is worse for women of colour. American studies have consistently found that Hispanic and African-American instructors receive lower course ratings than white instructors. In one survey African-American instructors were found to rate higher than white instructors in categories relevant to teaching quality like 'enthusiasm for' and 'knowledge of' the subject, and 'care about students'. Yet these same instructors received lower scores for the more important items like 'overall teaching ability' and 'overall value of the course'. The authors argue it's an outcome that can only make sense if you take into account the racial bias of students.
So if we accept, as studies suggest, that student evaluations are just vast hateful grids of racism, sexism and other intergalactic evils, why do we still have them? Can we measure teaching quality while excluding students' prejudices?
Anne Boring says no. It is simply not possible to adjust for the bias in student evaluations because the evaluations themselves also depend upon so many factors (like class size, level, format, physical characteristics of the classroom). Professor Philip Stark also notes there's a sampling bias, as only very happy or very unhappy students are likely to fill them in. On top of that, students are much more likely to rate a lecturer well if they give high marks, which in turn leads to grade inflation on the part of professors - and sloppy teaching.
Frankly, I would love to see student evaluations abolished. This is partly because students are biased, but it's also because students are learners; they are not customers and nor are they reviewers assessing the university and its staff like a restaurant on Urban Spoon.
Student evaluations are a product of the transfer of market logic to pedagogy, whereby education becomes a product (a degree) rather than a process of learning to think critically.
Like any commodity, it is the student-consumer who judges the quality of the product, who passively tastes the fruits of the academic's labour. If it pleases them, then they'll buy it and the product is judged to have quality. If the course is too conceptually challenging or the teacher doesn't meet their expectations of gender or race, then they discard it and the teacher is considered to be faulty. As a result, the delights of learning how to question and be seriously intellectually challenged are sacrificed to superficial pleasures. Teachers behave like coddling parents, and universities look more and more like crèche (check out Sydney University Library's new sleeping pod, for instance. It can be found right near the blinking idiocy of the television on level 4.)
Abolishing student evaluations would not mean abandoning teacher-assessment. As Philip Stark has argued, it would be very easy to replace the forms with peer evaluation or a faculty review of the materials that instructors use in their classes, like syllabi, handouts and samples of students' work.
The current research is just another addition to a voluminous literature highlighting the damage done to universities through student evaluations. In the interests of equality as much as pedagogy, no teacher should ever have to open those dreaded manilla envelopes at the end of semester again.
Alecia Simmonds is a researcher in law at UTS and a history lecturer at New York University-Sydney. She is the author of Wild Man (Affirm Press).