The Professor Will See You Now: MPs' surveys
4 min read
In an occasional series, Professor Philip Cowley offers a political science lesson for The House’s readers. This week: the 10 per cent problem
Who was the most successful prime minister?
Asked to grade the postwar prime ministers on a range from 0 (“highly unsuccessful”) to 10 (“highly successful”), one recent study of MPs’ views – published shortly before the election – saw Margaret Thatcher come top, with an average response of 7.8. Not far behind came Clement Attlee and Tony Blair (both 7.4).
These rankings can wax and wane as reputations change. Compare the latest scores to a similar survey by the same project in 2013. Ten years ago, Gordon Brown was bottom out of 12. He’s the biggest climber in the last decade, now ninth out of 16. Yet the top three have remained identical and most scores have not changed very much.
The responses are unsurprisingly often driven by partisan bias (but then who among us isn’t?). Labour MPs grade Labour PMs more positively; the same, mutatis mutandis, for the Conservatives. Given the radically different partisan mix in the House, if the survey were redone today, it would see Thatcher do less well, Attlee and Blair better. That said, there was cross-party consensus about the bottom place: Liz Truss scored 0.7. The next lowest was Eden (3.2).
There is one other aspect of the research that was interesting, if depressing for those of us on this side of the parliament-academic divide. The latest paper is based on responses from 65 MPs. That is, 10 per cent of the House. When the researchers did a similar study 10 years before, they had a response rate of around 25 per cent.
MPs can be sampled – we don’t need everybody to have replied – but there can come a point at which a sample becomes too small
Just like a survey of the public, MPs can be sampled – we don’t need everybody to have replied – but there can come a point at which a sample becomes too small. This paper made efforts to check that their respondents are broadly representative (which they are), and the results are in line with other surveys, but it speaks to a wider issue: it is getting harder and harder to survey parliamentarians.
I understand why. Politicians are busy people with an almost infinite set of demands on their time. While some people might be flattered to be asked to take part in an academic study, politicians generally are not. They get bombarded with requests from school kids, university students, lecturers, pressure groups, and various randoms. If they said yes to everyone, they’d spend their entire day doing nothing else but filling in surveys or giving interviews. Plus, there’s always a risk that it’s not a genuine project with anonymity guaranteed (as this one was), but an attempt to stitch them up. Why risk it?
A similar problem applies to those who want to interview MPs. I owe my career to the kindness of hundreds of MPs who had given up their time to talk to me. But colleagues tell me this, too, is getting harder. It’s all a long way from the 1970s when one famous study managed to interview 521 MPs (an 83 per cent response rate) with face-to-face interviews ranging from 30 minutes to an astonishing five hours.
But this should worry MPs too. They are becoming harder to reach by academics at exactly the time that quantitative data about their behaviour is becoming increasingly easily available. With relatively little effort, I can now track how an MP is voting, or what questions they are asking, or how often they are turning up to a select committee.
But this runs the risk of research being based purely on observable behaviour, lacking the rounded understanding of the institution that you can only get from asking people questions, whether it’s face-to-face or via questionnaires. Anyone who knows anything about parliament knows that the observable data only gets you so far.
For your sake and mine, please don’t put the survey straight in the bin.
Further reading: Royal Holloway Group PR3710, The good, the not so good, and Liz Truss: MPs’ evaluations of postwar prime ministers, Political Quarterly (2024)
PoliticsHome Newsletters
Get the inside track on what MPs and Peers are talking about. Sign up to The House's morning email for the latest insight and reaction from Parliamentarians, policy-makers and organisations.
