No matter who you voted for in the Australian federal election, chances are at some point on Saturday evening as the results started to come through you asked, what good are polls if they don’t produce reliable data?
It was certainly a question for a number of commentators during the live broadcast on the ABC, and it landed solidly in the conversation around my living room and on social media across the country.
When politics are shaped by continuous polling; when political leaders and policies change in response to preferences expressed in quick polls, the quality of polling data really matters.
For the two years leading into the election, Labor was ahead in every single Newspoll. And even though the old adage says the only poll that matters is the one on Election Day—even exit polls were way off.
Nine has referred to the results of their own Nine YourGov Galaxy Exit Poll as “disastrously wrong” as it predicted a “decisive ALP victory”.
Beyond getting quite the shock (good or bad, depending on your perspective) on election night, the impact of all this inaccurate polling on the outcome of the election itself is not yet clear.
Was the ALP vote down because some voters assumed the outcome was a sure thing, so they could use their ballot to indicate other priorities?
Were polls just wrong because voters’ intentions differ from their behaviour on the big day? Or because polling techniques are outdated and the rise of mobile phones is disrupting access to representative samples, which traditionally rely on landline phone lists?
Strategists and pundits are out in force doing this post-game analysis but one thing is for certain—when we don’t have good data for decision making, the outcome of those decisions can sometimes come as a shock.
The gap in gender data can be just as misleading
The confusion triggered by incomplete data goes well beyond the outcome of this election. In her new book, Invisible Women, Caroline Criado Perez has written extensively about the data gap that exists in relation to men and women. We know much more about men’s life experiences than we do women’s, and this is known as the gender data gap.
Perez’s book outlines case after case, industry by industry, where data used for decision making does not reflect the experience, physiology or circumstances of women, with sometimes disastrous, and usually shocking, results. From the design of car airbags and seatbelts to shelf height and office temperatures, Perez outlines how data used to inform critical aspects of everyday life, typically fail to account for 51% of the population: women.
It should come as little surprise, then, that the same inaccuracies of data that exist in political polls and car design also exist in another area of global significance: poverty data. Right now, global measures of poverty and inequality are simply not accurate enough. In poverty measurement, the household has traditionally been considered the appropriate unit of measurement, with assumptions made that household resources are evenly shared among household members.
Anyone who lives in a household with others knows this isn’t always true. Whether it is who gets the last piece of food when it is scarce or who is responsible for the cooking and cleaning, the dynamics inside the household mean individual circumstances are not all the same.
Using information from one person in the household as a proxy measure for the situation of all people inside that household results in deeply inadequate data
These different experiences in the home shape access and opportunities, including participation in a myriad of activities outside the house—from school to religious celebrations to paid work.
Using information from one person in the household as a proxy measure for the situation of all people inside that household results in deeply inadequate data that doesn’t account for the specific and unique experiences of men and women.
The world built according to the average male measurements can be deadly for women
A more accurate way to measure poverty
And so, while election pollsters are thinking about how to overcome limitations to the current approaches to polling, partners at the International Women’s Development Agency and the Australian National University, with strategic funding support from the Australian Government, are working to deliver a new measure of poverty, ready for global use by 2020: the Individual Deprivation Measure (IDM).
The IDM overcomes limitations in current measures of poverty by surveying all adult members of a household to gather individual and gender sensitive data
The IDM overcomes limitations in current measures of poverty by surveying all adult members of a household to gather individual and gender sensitive data. The IDM gathers data on 15 dimensions of life, including some particularly relevant to women like family planning, and the ability to share their voice in the community.
I have no doubt that the 2019 federal election will generate lessons across the political spectrum. While I cannot predict yet what those will be I sincerely hope that a stand-out lesson for people from all political persuasions will be the recognition that data matters and when decisions are taken on the basis of limited data, surprising outcomes occur.
As we focus our attention on understanding what happened with current polling methods, we should take the opportunity to also acknowledge limitations in other measures relied on in policy making, like poverty measurement.
While the two may seem unrelated, the lessons in both point to a fundamental truth: our inability to accurately capture people’s experiences and beliefs, can alter how we view and respond to the world.
To make sure we understand constituencies and beyond, and take the right actions to deliver on our intentions, we need accurate data. Without it, we are heading for a shock.
