Email
When Education Guardian published its university league tables last week, one of the key indicators used was data from the National Student Survey. This proved controversial. "It's an optional survey, so many universities who don't shove it in their students' faces are penalised, and so only the complainers bother to fill it in," read one comment on EducationGuardian.co.uk. Another accused the tables of being, as a result of using such indicators, "a hotchpotch of subjectivity".
It is not the first time the NSS has come in for criticism. Last year, Lee Harvey quit as the Higher Education Academy's (HEA)director of research and evaluation after a letter by him was published in the Times Higher Education magazine criticising the survey as "bland" and "methodologically worthless". There have also been claims that some institutions have pressurised students into giving positive answers. One lecturer at Kingston University was recorded telling students to up their scores because "if Kingston comes bottom ... no one is going to want to employ you because they'll think your degree is shit".
Union backing
But even those universities that have not scored well in the four years that the survey has been published continue to urge their students to complete it. "We do take the results of the NSS very seriously and note the responses in each section," says Janet Hartley, pro-director for teaching and learning at the London School of Economics, which came fourth from bottom in terms of teaching satisfaction last year. And while in the early years of the survey a couple of student unions chose to boycott it on the grounds that it was an arbitrary measure, this year it has the backing of every union.
"We maintain that the data are still not very useful on a national basis," says Ant Bagshaw, education officer at Cambridge University student union (Cusu), which held out the longest against the survey. "But for internal use, some of them are quite interesting."
The NSS was commissioned in 2005 by the Higher Education Funding Council for England on behalf of all the UK's funding bodies as part of a revised quality assurance framework. The aim was to gather feedback on course quality to make institutions more accountable for the experience they were delivering to students. It was also designed to help inform the choices of future applicants to higher education.
Carried out by the polling organisation Ipsos Mori, it asks final-year undergraduates to rate 22 elements of their higher education experience, relating to the quality of teaching, assessment, academic support, organisation, resources and personal development, as well as to their overall satisfaction. These responses are made available to prospective students on the website Unistats.com. Survey respondents can also add comments, which are forwarded to participating universities, colleges and student unions. These are said by institutions to offer useful qualitative information, and also a chance for students to air praise or gripes. One such comment on last year's survey read simply: "9am lectures!"
Nearly 220,000 students completed the survey last year, including, for the first time, more than 6,000 studying HE courses at further education colleges. The size of the sample and quality of the data produced was what finally convinced Cusu to back it. One question asks how far students feel assessment criteria have been made clear to them in advance. "If you break that down by subject, you can see which subjects are really not explaining what is expected of students in an exam in the way that they should," says Bagshaw.
Janet Beer, vice-chancellor of Oxford Brookes University and chair of the NSS steering group, argues that the survey allows national comparisons that are useful both for prospective students and for identifying common areas of weakness.
"You find art and design students uniformly grumpy and philosophy students tend to be quite cheerful," she says. She suggests that this could be to do with class sizes in the two disciplines - philosophy classes tend to be small, and art and design large - but the upshot has been that the HEA's art and design subject centre is now looking closely at how to improve feedback.
Lowest marks
In fact, the quality of feedback is one issue that the NSS has raised across all disciplines. It is an area that has consistently scored the lowest marks in the survey, prompting a number of institutions to review their assessment and feedback methods.
The fact that so many institutions are now reviewing the way they do things as a result of the survey has finally proved its credentials. It is also why the National Union of Students backs it so enthusiastically, especially as it has often led to students being asked to help find solutions to the issues raised.
There are still complaints that it is difficult to devise a survey that is appropriate for every institution. Some in collegiate universities, where teaching is divided between the college and university, argue that it can give only a partial picture, while the LSE argues that, by covering only final-year undergraduates, it doesn't take into account the experiences of postgraduates.
Even universities that score well argue that it is only part of a package of ways of measuring students' experiences. Janice Kay, deputy vice-chancellor at the University of Exeter, which was in the top three last year for both teaching quality and satisfaction, says: "You can see the usefulness of it, as long as one listens to what students are saying and then does something about it."
For Beer, the fact that so many institutions seem to be doing just that has proved many doubters wrong. She says: "I'm absolutely convinced it's a force for good."
• See our rankings of universities at Education.Guardian.co.uk/university guide
;