Decoding a voter's poor handwriting is subjective – let's enlist AI to help with the Bradfield recount

The Guardian

21-07-2025

Liberal candidate Gisele Kapterian has appealed her narrow loss to Nicolette Boele in Bradfield to the court of disputed returns. According to Professor Anne Twomey, no questions of law are raised in Kapterian's challenge. Rather, the court is being asked to determine more mundane questions. Is that 1 actually a 7; is that 6 an 8? and so on.
Cases like this present an almost absurd disjuncture between the banality of the evidentiary questions – is a 1 a 7? – and the weight of the consequences: who shall represent over 100,000 citizens in parliament?
Australian elections are global exemplars of fairness. All adult citizens are required to be enrolled and to turn out to vote. Ballots can't be hacked in Australia: they are tangible things, physical pieces of paper, with handwritten markings. Scrutineers from both sides carefully observe the count. The entire electoral process is administered by a politically neutral agency, the Australian Electoral Commission.
The prospect of an Australian election being determined by competing subjective assessments about poor handwriting is jarring, at odds with the rigorous procedural fairness so valued and so manifest in Australian electoral law and practice.
AEC officials no doubt do their best, and must make thousands of these judgments in the course of closely scrutinised counts like the one in Bradfield. Indeed over the course of a career, many AEC officials must become some of the most experienced handwritten-digit distinguishers in the world. Nonetheless, challenges to their initial judgments can be made under section 281 of the Commonwealth Electoral Act, reserving disputed ballots for determination by a more senior AEC official. Further challenges about the formality of these ballots can form the basis of a petition to the court of disputed returns.
The last time the courts considered questions about ballot formality was in 2007 from the seat of McEwen. The resulting federal court case produced one of the more unusual judgments one will find in Australian law reports. Mitchell v Bailey (No 2) contains a lengthy tabular schedule, listing the disposition of 643 reserved ballots and – in 153 instances – reasons for Justice Richard Tracey's assessments about ballot formality differing from those of the AEC. Examples include comments such as 'Notations reasonably resemble numbers. In particular, three of them can be recognised as figures 7, 6, 5.' Why? How? Presumably, they just did to Tracey, just as they did not to AEC officials. No criticism of the late justice is intended; the point is to highlight just how subjective and hence seemingly unfair these assessments – and election outcomes – can appear.
Tracey's observations and principles were adopted in their ballot formality guidelines. Given the rather exhaustive guidance provided by Mitchell v Bailey, what's left to dispute? Highly subjective judgment calls about handwriting is all that is left to fight over, something akin to Australia's version of the United States's unedifying hanging chad episode in the 2000 presidential election.
Can we do better? Could an election really be decided this way? That one person's 7 is another person's 1?
Here's a modest proposal. For decades we've been training computers to recognise handwritten digits, principally for making mail processing and delivery more efficient. Massive datasets of real, handwritten digits have become one of the touchstones of machine learning, test beds for refining algorithms and global competition among researchers. The best algorithms have 99.82% accuracy in recognising digits. And the AEC itself uses digital scanning to process Senate ballot papers.
The outputs of digit recognition algorithms are probabilities, summing to 1 over the 10 possible digits, collapsing to a probability of almost exactly 1.0 on one candidate digit in most cases, but more spread out when processing poor, ambiguous handwriting.
Algorithmically derived, rigorously validated on massive datasets spanning hundreds of thousands of handwritten digits, these probabilities could be a useful alternative – or at least a guide – in helping a judge determine whether a 1 is a 7, whether a 2 is an 8, and so on, and ultimately as to whether a ballot is formal or not. One simple decision rule might be to classify an ambiguous mark as that digit recognised with probability above 0.5, consistent with courts' reliance on the 'balance of probabilities' as a decision rule in many legal settings.
Humans remain very much 'in the loop' in this process. The algorithms only assist in the hard cases: the ballots subject to dispute. Further, while digit recognition algorithms process single digits, ballot formality turns on whether a unique, valid enumeration of preferences or sequences of digits can be discerned, an assessment that considers not just individuals marks but the ballot as a whole.
Surely this guidance could not only help the court in a practical sense (parsing hundreds of reserved ballots) but offer reassurance to the parties – and the voters of Bradfield – that the election is being finally decided rationally and with a degree of objectivity, consistent with the fairness integral to Australian elections.
Simon Jackman is an honorary professor at the University of Sydney, specialising in elections, public opinion and data science