AI company CEOs have called for international cooperation to slow AI development, set safety standards, and manage risks.
This comes in the wake of recent AI hacking incidents, and alarm raised by a resigning employee of AI company Anthropic.
The SMC asked experts to comment.
Dr Andrew Lensen, Senior Lecturer in Artificial Intelligence, School of Engineering and Computer Science, Victoria University of Wellington, comments:
Is a global agreement to slow AI development achievable and how would this work? What kinds of precedents can we draw from?
“I would love to see a global agreement to slow AI development happen, but it’s really hard to see in the current geopolitical environment. Climate change is an existential risk, but the Paris Agreement’s aim to not see temperatures rise beyond 1.5c wasn’t met. We have seen success on issues like the ozone layer (the Montreal Protocol), but that was where there was a clear alternative and the economic cost of taking action was much less significant.
“The closest analogue we have in AI is probably the Bletchley Declaration and the 2024 Seoul and 2025 Paris AI Summits, but, a bit like the COP climate change conferences, they don’t seem to have given the systemic change that we deserve.
“I’m not advocating giving up, of course — the best way for New Zealand to try and push for change would be to re-establish our moral and ethical leadership in this space, and band together with other middle powers, as suggested by Canadian PM Mark Carney in Davos this year.”
How does NZ compare to the rest of the world in terms of AI adoption and concerns around harm?
“We are a real mixed bag! Some numbers coming out of AI providers like Anthropic suggest that we are among the highest-uses of AI like Claude in the world, but survey after survey has shown that we are also the most distrusting and worried about AI globally.
“I see our distrust/worry as a good thing, not a bad one; there are many good uses of AI technology, but if used naively (AI for AI’s sake), there is a real risk of unintended consequences and unnecessary costs and resource usages. We can’t compete to build these huge models, so positioning ourselves as critical and responsible adopters would be a much better plan.”
Amidst all of the noise, what should we be focusing on?
“We see two polarised views: either AI is going to make us extinct, or it’s all hype and we should just ignore the fear-mongering. It’s important to be a bit sceptical that the tech oligarchs may be pushing this narrative for their own marketing, economic gain, and to try and prevent new competitors, but we should also use this opportunity to take more ownership over our future.
“Setting up an AI institute, like we see in Australia and elsewhere, would make us much more prepared to tackle AI harms — both existing and emerging or existential ones. Such an institute would have a mandate to oversee and govern the use of AI in New Zealand to prioritise the benefits to us rather than to Big Tech. We can’t directly control what happens overseas, but we can show much stronger leadership and governance at home, which will also get us involved in conversations on the global stage. At the moment, we’re looking like the odd duck out!”
Conflict of interest statement: “Co-director of LensenMcGavin AI (AI consultancy).”
Dr Kevin Shedlock (Ngapuhi, Ngati Porou, Whakatohea), Lecturer, School of Engineering and Computer Science, Victoria University of Wellington, comments:
Is a global agreement to slow AI development achievable and how would this work? What kinds of precedents can we draw from?
“International communities are mobilizing their efforts to push back on artificial intelligence however, I think the ‘cat is out of the bag’ and turning back the clock is unachievable. As a society we must be pro-active in confronting this technology in a way that aids our community in positive ways as opposed to mining community or national knowledge-based systems.
“This will require becoming proactive, paying attention during the development of any legislation, new standards, new technologies moving forward – as trusted experts, our nation’s academics will provide important directions for this to occur.”
How does NZ compare to the rest of the world in terms of AI adoption and concerns around harm?
“New Zealand continues to be lagging during the adoption of AI infrastructure which will create further environmental challenges for Aotearoa as a country.
“The idea that explainable AI systems will continue to be highly debated given the level of impact this technology will have upon our nation is an area of expertise where New Zealand can be future leaders. This speaks directly to levels of trust within the AI eco-system during the development of these technologies, opening questions specific to New Zealand.
“For example, what are our Tiriti o Waitangi commitments to our nation, how will Māori become impacted or benefactors of this technology given the current and past grievances (trust) that exist between the settlement government and Māori communities?”
Amidst all of the noise, what should we be focusing on?
“As a nation, be diligent during the ongoing growth of this technology. Artificial Intelligence is now in our communities and in our homes… we interact with this technology on a daily basis – we can’t just turn the tap off and remove ourselves from our responsibilities to our whanau or our community to be proactively challenging new and arriving AI technologies.
“AI will change the way we live and interact, the question we should be asking ourselves is ‘what do our digital selves look like as a nation, community and whanau’. If we fail to confront these types of questions, we risk endangering our current way of life and allowing a new form of colonization… one that weaponizes the way we think and interact and take into our community.”
Conflict of interest statement: “No conflicts.”
Dr Cassandra Mudgway, Senior Lecturer Above the Bar, Faculty of Law, University of Canterbury, comments:
Is a global agreement to slow AI development achievable and how would this work? What kinds of precedents can we draw from?
“I think there is a useful distinction between slowing the development of AI models and regulating the harms that AI systems are already enabling.
“A global agreement to slow AI development would be difficult to achieve. There are strong commercial and geopolitical incentives for companies and states to keep developing more capable systems, particularly where AI capability is being treated as a strategic advantage. But we have managed to make agreements within similarly charged climates before, including international agreements to limit nuclear technology.
“I would also be cautious about making frontier model development the centre of the AI safety conversation. People are already experiencing serious harms from generative AI. Sexualised deepfakes are one example. We are seeing technology being used to create sexualised material depicting people without their consent, with women and girls disproportionately targeted. Those harms do not depend on AI reaching some future level of capability. There are already here.”
How does NZ compare to the rest of the world in terms of AI adoption and concerns around harm?
“New Zealand is part of the same AI ecosystem as the rest of the world. The products are available here, people are using them here, and harms arising from those systems can occur here regardless of where the company behind the technology is based.
“From my perspective, one of the concerns is that AI regulation can still be discussed quite abstractly. We should be paying much closer attention to what these systems are already enabling.”
Amidst all of the noise, what should we be focusing on?
“I think we need to pay more attention to the harms people are experiencing from AI now.
“A lot of the public discussion is being driven by concerns about increasingly autonomous systems, very advanced models and the possibility of much more serious risks in the future. While those conversations have value, but they can also pull attention away from problems that are already here. Right now.
“Again, AI-enabled gendered harm is one of those problems. We know that AI is being used to produce sexualised images of people without their consent. We know that these systems can reproduce existing patterns of discrimination and abuse. We can therefore ask quite practical questions about what developers and deployers should be required to do, when platforms should have to intervene, and what remedies should be available to someone whose image has been used in this way.
“I would like to see more attention paid to those questions. Regulation should be concerned with the conditions that allow harmful uses of AI to develop and spread, rather than leaving the response until after someone has been harmed.
“The question is ultimately about responsibility. Who is making these systems? Who is deploying them? What do they know about the ways their products are being used? And what should the law require them to do about foreseeable harms? Those are questions we can answer now, without waiting to see what the next generation of AI looks like.”
Conflict of interest statement: “No conflicts.”
Professor Alistair Knott, Centre for Data Science and AI, Victoria University of Wellington, comments:
“To participate properly in the global conversation about AI capability and AI safety, the New Zealand government needs an Office for AI.
“AI technologies are obviously advancing at a rapid pace. As AI systems improve to human level and beyond, it’s increasingly important that New Zealand participates in the international conversation about how best to assess and govern these technologies. This is certainly something that NZ needs to do in discussions with other countries. The tech companies pushing AI products are multinational; the discussion about how to oversee their products must also be multinational.
“The discussions which must take place must be organised and resourced. Other countries have created institutional support to help with the required work, by establishing dedicated offices, that can coordinate efforts from relevant groups around the country. The UK has an AI Security Institute, and the Alan Turing Institute, which performs a similar role outside government). Australia recently set up an Office of AI. Canada has an AI Safety Institute; Singapore has one too; so does Japan. New Zealand does not yet have one, despite increasingly urgent ca
“A nationally resourced AI office in New Zealand will achieve two related things. Firstly it will coordinate work happening within the country, in universities, industry, the public sector, and civil society, so that relevant voices can be heard as the situation moves forward. Secondly, it can serve as an interface to international conversations, so that New Zealand’s position can be heard internationally. Increasingly, these conversations take place through the AI institutes of different countries. Without a national institute of our own, our voices won’t be heard.
“To make the right choices as AI advances, we need efficient conversations, both nationally in New Zealand and internationally. Setting up a national AI office in New Zealand will help create both kinds of efficiency.”
Conflict of interest statement: Not yet received. This will be updated on our website.
Dr Olivia Erdélyi, Senior Lecturer, Faculty of Law, University of Canterbury, comments:
Is a global agreement to slow AI development achievable and how would this work? What kinds of precedents can we draw from?
“Given political leaders’ increasingly blatant disregard for the public good and the rule of law altogether, as well as explicit declarations about intending to win the AI race, I do not realistically believe that a global agreement is achievable. Nor do I necessarily think AI developers’ declaration of intent to slow down development is genuine. Other problems I see with this proposition include defining what they mean by ‘slowing down’ and, if such an agreement were made, detecting violations of it.”
How does NZ compare to the rest of the world in terms of AI adoption and concerns around harm?
“Unfortunately, NZ is lagging behind significantly, maybe not so much in adoption, but definitely in terms of regulating AI.”
Amidst all of the noise, what should we be focusing on?
“Debunking myths, misinformation and disinformation — like the ‘rogue AI’ narratives around the recent Open AI-Hugging Face incident — educating people about the technical aspects of AI technologies to enable them to assess their risks and benefits and make informed decisions about if, when, and how to use them.”
Conflict of interest statement: “I am not aware of any conflicts of interest.”
Professor Te Taka Keegan, AI Institute, Computer Science Department, University of Waikato, comments:
“Given the geo-political state of the world right now and the current AI race I do not think these calls will slow down the AI momentum at the moment.
“I think the steps that the Canadian and Australian governments are taking at the moment are wholly appropriate, but I am not sure if the current NZ government has that capacity or indeed that will.
“We need to be focusing on NZ sovereign infrastructure, NZ sovereign AI, ethics and practises that are guided by indigenous and Māori values and working towards breaking the vendor lock in that we have right now with big tech.
“And given that this is te wiki o te reo Māori, wouldn’t it be great if someone was actively working on a sovereign reo Māori LLM?”
Conflict of interest statement: Not yet received.