Census delays and missing data – Expert Reaction

One in seven New Zealanders did not fully complete the 2018 Census, according to information from Statistics New Zealand.

Statistics New Zealand chief executive Liz MacPherson provided information to Government yesterday indicating 700,000 Kiwis either didn’t partake in the survey at all, or did not fully complete the survey.

The release of the 2018 data has already been delayed several times.

The SMC asked experts to comment on their use of census data and the impact of these issues. Please feel free to use these comments in your reporting.

John McDermott, Executive Director, Motu Economic & Public Policy Research, comments:

“Of the 15 research staff at Motu, six will use Census or Census-related data nearly every day. The Census is an extremely valuable resource for research because it collects information on small populations, on small areas and from the same people across multiple domains. People in Aotearoa are very mobile and the frequency of Census collection, therefore, needs to be higher than in more stable countries so that we know what is changing and where.

“This research feeds into policy decisions that make a difference to the future wellbeing of New Zealanders. Without official Census data, anecdote and intuition would reign without evidence to support or debunk partisan vagaries. Just in the last year, our researchers have used Census data to examine income inequality, migrants’ settlement experience, the gender pay gap, the housing market, the cost of poor [quality] housing, household greenhouse gas emissions, and firm productivity.

“Nobody can deny that the 2018 Census was problematic. Secure data labs around Aotearoa have echoed with the sound of raging introverts stressing over how to achieve their research targets with the delay and less relevant data that won’t be taken as seriously, because in 2019 what politician cares what was happening six years ago?

“Even if StatsNZ makes sensible technical adjustments to turn the available 2018 Census data into representative national statistics, the microdata (which is the main form of the Census data that we use) will be of lower quality, making the census a less valuable research resource. However, nothing can fill the gap that would be left if Census data were not collected.

“After the 2011 Christchurch Earthquake, the Census was deferred by two years. What’s done is done but the best way forward to mitigate problems that have been generated is to return the Census to the old timeline and run it in 2021 (though this would require a law change). That’s enough time for the lessons from 2018 to be taken on board, StatsNZ to be resourced to ensure a successful census distribution and collection, and for researchers to gain a handle on the data before it all changes again.”

No conflict of interest.

Peter Crampton, Professor of Public Health, Kōhatu – Centre for Hauora Māori, University of Otago, comments:

“The Census is an essential tool for good policymaking that has a focus on equity, and for monitoring social and economic outcomes.

“Government, policymakers, iwi, community group and interested citizens are dependent upon high quality, authoritative data on which to base their analyses, plans, proposals and funding formulas. For example, in the health sector, the major funding formula for all DHBs is entirely dependent upon accurate population counts and other demographic data. Administrative data, the Integrated Data Infrastructure (IDI), and other data sources can often provide high-quality data that are useful for many purposes, but they do not provide systematic and complete demographic and household information for the total population, which must come from the Census.

“It is possible to predict which population groups are most likely to be undercounted in the Census. Typically these groups include a preponderance of marginalised, disenfranchised, high-needs people, whānau and communities — those for whom good social policy is of the highest priority. If these groups become invisible in the Census then policymaking becomes doubly difficult and some of the least advantaged communities are at risk of becoming further marginalised.

“My experience of Statistics New Zealand over a number of censuses is that the staff who lead the census process are expert, dedicated and completely committed to achieving a successful outcome.”

Conflict of interest statement: I have used census data for research purposes for about thirty years. There was no external funding source for preparing these comments. The views, opinions, findings and conclusions or recommendations expressed in these comments are strictly those of the author. They do not necessarily reflect the views of the institution where the author currently works. The comments are presented not as policy, but with a view to inform and stimulate wider debate. Peter Crampton is a member of the Government’s Health and Disability System Review panel; the views expressed here are his own and not those of the panel.

A joint statement from National Institute of Demographic and Economic Analysis, University of Waikato:

“The delayed release of the 2018 New Zealand Census of Populations and Dwellings, and the ongoing questions about the coverage and quality of data are significant concerns. Censuses form a critical component in processes of planning for, and responding to, issues that are fundamental to political participation and social and economic wellbeing: electoral boundaries, the planning of health, education and other social services, and the work of researchers seeking to address social, economic and environmental challenges.

“To be effective, censuses must be universal, comprehensive, simultaneous and regular in frequency. In New Zealand, there is already a problem with frequency because the Christchurch earthquakes delayed the 2011 census to 2013, sacrificing frequency to maintain the other three principles. The 2018 Census has not achieved universality, comprehensiveness and simultaneity: the participation rate is less than 90% (compared to 92.9% response rate in 2013); some 5% were partial responses (compared to 2% in 2013); and these gaps are being filled with data from other sources not captured simultaneously nor designed for census use.

“There are significant equity implications from Census 2018. We know from New Zealand and international experience that response rates vary amongst population groups and those most likely to be undercounted are those experiencing significant inequities. The government has made a priority commitment to achieve equity and address wellbeing but the sectors who have the greatest responsibilities to achieve equity—health, education, social support—will not have the information they need to assess and respond to current and future needs. Judging from past experience, the impacts of a significant undercount that feeds into government activity will be felt most sharply by Māori and Pacific peoples.

“StatsNZ are confident that they will be able to produce high-quality census data through the extensive use of government administrative data. However, such data is collected for different purposes, under different circumstances with different categories and substantial inaccuracies. StatsNZ will also be using individuals’ information from their 2013 Census forms. It is unclear whether New Zealanders are comfortable with, or even aware of, how StatsNZ are using their personal data to address missing data in Census 2018.

“Accurate, up to date and inclusive data is fundamental to our work as social scientists. At the National Institute of Demographic and Economic Analysis, our research includes topics such as child health equity and child wellbeing; housing and urban regeneration; iwi demography and governance; and diversity, migration and demographic change (including changing household and family structure). These projects, which are publicly funded and have specific outcomes for government and community activity, rely on the information provided by the Census and is impacted by the shortcomings that are currently apparent. The delays, coverage and quality issues in the 2018 Census will make it much more difficult to deliver on our commitments, limits our ability to identify recent population patterns and changes or evaluate underlying causes of societal challenges, and reduces the impact that our research can have on public policy and community activity.”