HEALTH SERVICES
Calls for a regulator to oversee AI use in healthcare
Citizens' jury recommends transparency and patient choice around AI use
February 28, 2025
-
A Citizens’ jury on the use of artificial intelligence (AI) in healthcare has published its verdict, setting out a series of 25 recommendations for health policy-makers on the safe, ethical and inclusive use of AI in Ireland’s healthcare system.
Organised by the Irish Platform for Patient Organisations, Science and Industry (IPPOSI), the jury of 24 representatives of the population of Ireland, convened from September to December 2024 to offer the public’s perspective on this complex topic. Loretto Grogan, HSE chief nursing/midwifery information officer served among the jury oversight panel.
While jurors endorsed the early, low-risk deployment of high-quality, human-monitored AI tools which can enhance care in many ways, they emphasised the need for strong regulation, transparent oversight and robust data security.
One of the key recommendations agreed by the jury to build public trust in AI is for the establishment of a statutory regulator to oversee the use of AI in healthcare. This body would be responsible for developing and enforcing standards for those using AI, including its licensing, data governance and monitoring, as well as imposing penalties for breaches. It would also publish compliance reports to advance the development of AI technologies in healthcare that are secure, transparent and accountable.
To complement the work of the regulator, the jury also called for the creation of a separate independent commissioner for AI in healthcare to serve as a public interest watchdog and protect patient rights. Their role would be to increase public awareness about how AI is being used in healthcare, make recommendations on the use of individuals’ health data by AI, and independently identify opportunities to strengthen compliance with regulatory standards. The commissioner would also operate a complaints process for members of the public.
With AI potentially involved in everything from managing waiting lists, to analysing x-rays, to undertaking robotic surgery, the jury emphasised the importance of transparency, patient autonomy and informed consent. While supporting the automatic enrolment of individual health data for training AI, jurors agreed that people must be clearly informed and given the option to opt out.
Among other recommendations, the jury:
- Emphasised the importance of human oversight and control over AI-enabled care. While AI can augment and support clinicians, it cannot replace them
- Called for a national initiative to communicate to the public about the use of AI across the health system, what it is, what role it plays in care and how it involves their health data
- Suggested that a national strategy detailing the role of AI in healthcare be published by quarter one of 2026, building on the National Artificial Intelligence Strategy published in 2021. This would provide renewed focus on specific ethical and regulatory considerations within the health sector, as well as on stakeholder rights and responsibilities, timelines for implementation, etc.
Speaking about the Jury’s recommendations, Ms Grogan said that their view that AI in healthcare should always work alongside humans and never replace them was fundamental.
“When AI supports humans, with guidance and oversight rather than taking control, it fosters trust and leads to better outcomes. Everyone, regardless of where they are treated, should have access to AI-enabled care – there should be no gaps between public and private healthcare systems. Safeguarding health data is paramount. Strong legislation and robust regulation must be enacted to prevent misuse and ensure the ethical application of AI across the health sector,” she added.