The Professor Will See You Now: by-elections
Illustration by Tracy Worrall
4 min read
In an occasional series, Professor Philip Cowley offers a political science lesson for The House’s readers. Here: it’s not the by-election winner but the trend that counts
Cowley’s Law of By-Election Analysis is that too much attention is paid to who wins. Winning and losing is obviously not a trivial part of the electoral process: it matters to the candidates; it matters to (some) constituents; in a small number of cases it may make a substantive difference to the government’s majority.
But these are not the reasons by-elections attract the attention they do. The main reason we are interested in by-elections is because we see them as a measure – albeit an imperfect measure – of a party’s standing in the country. And if a party wins a constituency by 5, 50 or 500 votes, the political lesson and significance should essentially be the same as if they lose it by 5, 50 or 500 – yet it never is.
Prior to Batley and Spen, for example, there were claims that the Starmer project was finished if Labour lost. Labour won by 323 votes. The idea that a leadership would be doomed if a party lost a seat by 300 votes, but all is well if it wins by 300ish is – by any objective standard – insane. The same applies to the fallout from Uxbridge, held by the Conservatives by just 495 votes. The lessons we learned from that by-election should not have radically altered had 250 people voted differently. Had the Conservatives managed narrowly to hold Mid Bedfordshire or Tamworth, the coverage would have been vastly different, but the real lessons would have been the same as when Labour won narrowly.
I could point out that the late great David Butler made a similar argument in print as long ago as 1949. Focus (with appropriate caution) on the turnover in votes, not the outcome. No one seems to have listened to him then, so they probably won’t listen to me now. But by-elections are a noisy enough signal at the best of times. There is no need to make them even less useful by focussing on the wrong thing.
Knowing that seats were more likely to change hands, party managers became less willing to allow 'unforced' by-elections
Butler was writing at a time when by-elections were far more frequent than they are today. The post-war peak hit 61 between 1959 and 1964. Fewer than half of the 61 were caused by the death of the sitting Member. The majority were appointments to various external bodies or to the Lords, along with a handful of resignations. Only six saw a change in party control – although one of those was Orpington, an early sign of a more volatile electorate.
Once by-elections became riskier, their numbers dipped. Knowing that seats were more likely to change hands, party managers became less willing to allow “unforced” by-elections. While every four- or five-year parliament between 1945 and 1979 saw at least 30, that has been true of only one parliament since 1979 – and that blip was caused by the mass resignation of Unionist MPs in 1985 as a protest against the Anglo-Irish Agreement.
But no government – however good its party management – can stop its MPs dying; the Grim Reaper became the most common cause of by-elections. As a very interesting research paper just published in the Journal of Elections, Public Opinion and Parties notes, more than half of the by-elections since 1979 have been caused by the death of Members.
Seen over the longer term, 19 by-elections so far this parliament is on the low side. But this is a figure already higher than in all but one of the parliaments since 1992 – and the figure may yet rise higher.
Perhaps less obvious is what is causing this. Death has not been the main cause of by-elections since 2010. Of those so far this parliament, just four have been the result of a Member’s death. The last 11 by-elections have all been triggered by resignations or the result of a recall petition – or in some cases resignation in the face of an incoming recall petition.
Your further reading for this week: A Middleton, Turnout, government performance and localism in contemporary by-elections, Journal of Elections, Public Opinion and Parties (2023); D Butler, Trends in British By-Elections, The Journal of Politics (1949)
PoliticsHome Newsletters
Get the inside track on what MPs and Peers are talking about. Sign up to The House's morning email for the latest insight and reaction from Parliamentarians, policy-makers and organisations.
