AI pilots to public service transformation
Salesforce hosts experts from the public and private sectors for a round table discussion on scaling AI pilots to transform public services.
What does trustworthy AI look like in citizen-facing services, and what builds public confidence in
practice?
Louise McKeever
For citizens to trust AI, they must know what problem it is solving, why we are using it, and what value it brings. We must also ensure that recourse is in place in the event that something goes wrong. It is critical that AI is used transparently, that there is accountability for its application, and humans are kept in the loop. Public confidence will come from experience. If AI makes a service faster, more accessible, and more personalised while citizens retain meaningful human oversight and control, people will see the benefit.
Mihai Bilauca
Delivering trustworthy AI is an internal challenge as well as an external one. It is critical to understand the fundamentals of AI and what trustworthy AI means. You must build the correct capabilities so staff understand what AI entails. Embedding good governance structures, providing staff with training, and dealing with staff concerns around AI use is vital to achieving this.
Damien McCallion
Citizens must be engaged in the process when any new technology is being introduced so you can build trust. We have just finished our AI for Care strategy and patients, industry, and clinicians were involved in that process. It is critical to be outcome-focused to demonstrate the benefit of a new technology to reassure citizens of the value of its application. Organisations must also embed frameworks for governance, clarity, and accountability to reassure the public.
Richard Boyd
People are understandably worried about AI, so you must have a strong ethics framework that drives its use. Decision-making processes must have accountability embedded in them, not just for the service user, but also for the internal workforce. This is so they understand how decisions are being made on their behalf and workers can feel confident with the output. Large language models (LLMs) are probabilistic. They do not always come up with the same answer. Services designed around AI should capitalise on an LLMs’ ability to understand intent and then blend in deterministic logic throughout the interaction, the first gives a sense of natural interaction and the latter consistency. It is important any outcome ensures there are recourse procedures so the ‘why’ can be checked and verified.
Vincent Kennedy
Trust in AI starts with transparency. You must be clear about when and how AI is being used. From a Department of Social Protection (DSP) perspective, when dealing with entitlement to benefit services, we would always have to ensure there is a human in the loop if AI is being used. One phrase that should guide our AI use is, ‘do not take the human out of humanity’. To enable public confidence, staff must be comfortable with using AI. They need to be able to explain, justify, and stand over anything AI-enabled in citizen-facing services. We must also ensure the right safeguards are in place. Building public confidence is less about technology and more about ensuring people feel in control. If customers can see the clear benefits, that will build trust.
“It is critical to utilise language that resonates with people when communicating why AI is being deployed.”
Richard Smith
Richard Smith
Transparency provides people with a level of assurance that the decisions we make are safe and compliant with our main goals. The introduction of AI-enabled services is a big change for people, so we must clearly articulate why we are using AI and how it benefits citizens. They will not engage with these services if you do not communicate that. It is critical to utilise language that resonates with people when communicating why AI is being deployed to address people’s concerns regarding the impact it will have on their jobs.
How is your organisation embedding EU AI Act requirements into AI delivery without slowing down innovation?
Louise McKeever
There tends to be quite a negative narrative around regulation. However, the EU AI Act is our friend. It is critical that the correct governance structures, guardrails, and policies are in place, and that people are sufficiently trained. All of that reinforces using an AI solution. When you have all of this in place and start with a low-risk use case, you can build on that. That helps to accelerate AI adoption in a very positive and controlled way. Governance and human oversight must be embedded from the outset, rather than adding them as a compliance step at the end.
Richard Boyd
We welcome the EU AI Act because it provides a foundation to build on and creates a safe environment to work in. Now, it is a question of how the Act can be operationalised so organisations can monitor AI use; for example, manage hallucinations, LLM drift, and monitor toxicity, all of which needs to persist in an immutable log. The Act gives clarity on what is expected. In the future, agents will be able to interact with each other, and we must be able to monitor these interactions. There is a lot to be considered, and organisations will need to balance how internal resources are deployed; should they build and maintain internal solutions or use platforms with pre-built capabilities that help accelerate the AI strategy of their organisation.
Mihai Bilauca
Despite popular belief, regulations and policies are not mechanisms to slow down innovation. They are the only mechanisms that speed up AI adoption. All of my colleagues have been requesting guidance and guardrails so they can operate AI in a clear policy environment. Governance does not hinder AI adoption; it helps AI adoption.
“We are building our AI delivery models and processes around the EU AI Act rather than building services and checking compliance with the Act at the end.”
Vincent Kennedy
Vincent Kennedy
Building in governance is an enabler to progress. People want assurances that they are doing the right thing. We are building our AI delivery models and processes around the EU AI Act rather than building services and checking compliance with the Act at the end. I would recommend adopting a low-risk case first so you can navigate teething problems and then you are set up to adopt the higher-risk cases down the line. For high-risk cases especially, you need the right level of governance like that provided for in the EU AI Act.
Richard Smith
We have operationalised a medical cert reader that matches doctors’ handwritten certs to what is entered into our HR system. It has over 80 per cent-plus efficiency. In the middle of 2026, we had a backlog of 12,000 of these cases. We will have it cleared by Christmas 2026. Prior to the EU AI Act, there was minimal guidance to enable organisations to engage with AI. The Act gives us a framework informing a risk-based approach to the development of initiatives. As the Act stipulates, for high-risk initiatives, more layers of governance should be embedded while less are required for low-risk initiatives.
Damien McCallion
Many medical devices incorporate AI, and we analysed how they would be impacted by the EU AI Act. We developed a strategy built on four pillars: clinical care, operations, research and innovation, and public health. Regarding AI use in clinical terms, we had a tender for a stroke detection system that could detect strokes 22 minutes earlier. That reduces the likelihood of long-term disability and death arising from a stroke. It also reduces lifetime costs of healthcare. While AI can deliver better outcomes across our four strategic areas, it is paramount that the clinician controls the system. The legislation provides clinicians with the reassurance they need to make decisions.
What data, legacy system, or institutional knowledge challenges are holding back AI adoption in your organisation?
Vincent Kennedy
We are working across a number of legacy systems. Data and our data governance present a challenge in trying to implement AI services in the Department. We have decades of valuable data that is dispersed across various systems. Establishing an effective data platform is fundamental for adopting AI. Our key focus is ensuring the underlying data is in a consistent format and of high standard.
Damien McCallion
Our foundational systems are poor, and a lot of our data is paper-based. However, we are investing in our foundational systems to address this. One clinical area in which we have been able to scale AI use at pace is radiology. That is because we invested in a national imaging system over a decade ago. Looking internationally, a hospital in Chicago adopted AI and senior staff said they did not need it and left its use to more junior staff. The AI learnt from junior staff who are less experienced and more prone to error, which eventually led to the AI model having to be retrained. It is critical that mistakes like this are not repeated.
Louise McKeever
One of our biggest barriers is interoperability. Trustworthy data-sharing across government is crucial for this. Traditionally, data-sharing is done between organisations on a case-by-case basis, but the ‘life events’ approach requires information to move smoothly between organisations. This requires the removal of boundaries, while digital building blocks can provide the appropriate data ecosystem. Each organisation will still have to provide good quality data, the technical data architecture, and the skills. AI does not remove the need to fix our foundations; it makes the case for fixing them much more compelling.
“Start with the outcomes you are trying to achieve. AI may help achieve that outcome in a way that was not previously possible, but sprinkling AI on broken processes would not magically fix them.”
Richard Boyd
Mihai Bilauca
It is not just the quality of the data that is important, the quality of the description of the data is also vital. I am glad that the new data strategy highlights the need for better metadata. I am concerned that the need to invest in metadata is not sufficiently resonating with organisations at the moment. This is so AI understands the meaning of the data more comprehensively. Furthermore, we need to keep investing in effective file management. The introduction of computer systems has created a gap in proper file management which makes document navigation more difficult. This is critical for AI adoption.
Richard Boyd
When dealing with legacy systems, data quality and consistency of a common meaning is a challenge. Metadata is data that describes data. It also helps AI establish context and meaning; a semantic layer is a method of structuring metadata to help AI, BI tools, and people achieve a common interpretation of data. Without this, AI is left to do the interpretation, increasing the likelihood of hallucination. When sitting above legacy data, the consistency provided by a semantic layer underpins trusted AI. In addition, new AI tools have the ability to analyse legacy data systems, assess quality issues, and create strategies to improve quality and consistency. This enables organisations to unlock data at a scale and pace that was simply not possible in the past.
Richard Smith
A unique citizen identifier is required to ensure effective data-sharing across departments. This is not being prevented by a technical impediment, but a legislative one. We recently tried to use MyGovID to enable people to log in to our systems and we could not do it because we did not have legislative cover for it. The Government and public sector bodies are all doing the one thing. From that perspective, the introduction of a unique citizen identifier can only be a good thing for public service delivery.
“We must ensure the workforce does not become complacent in decision-making processes as AI matures.”
Damien McCallion
What does it take to move AI from successful pilots to scaled solutions that deliver meaningful service transformation?
Richard Smith
IT can sometimes have an inclination to engage in behaviour where solutions go around hunting for problems. There is a tendency to unnecessarily attempt to find problems to which you can apply the latest technological fad. There is a lot of push to use AI, but you must first determine what problem you are trying to solve by using it. Otherwise, it is a science project that is not focused on any particular goal. When you define the problem you are trying to solve and what success looks like, you can work back from there.
Louise McKeever
AI transformation is not about deploying more technology, it is about delivering better government. The measure of success is simple: simpler services, faster outcomes, and greater trust for citizens. That is how we turn AI pilots into real public value.
Richard Boyd
Start with the outcomes you are trying to achieve. AI may help achieve that outcome in a way that was not previously possible, but sprinkling AI on broken processes would not magically fix them. Just freeing up time or completing a task faster is only a partial measure; freeing up time to do what? For clinicians that may be a reduction in pyjama time which is linked to burnout, if so, measure the reduction in burnout not just the time saved. Without specific measurable outcomes, something else will fill the void. The message is to be intentional and specific and know the outcome you want.
Vincent Kennedy
There is a common perception that AI is going to be a panacea. Successful pilots prove that AI technology can work, but scaling to enterprise level is a different type of problem. Scaling must be focused on the delivery of better outcomes. To move from pilot to a scaled solution, you need to focus on three things: the correct foundations, including appropriate data structures, effective governance, and a relentless focus on the outcomes you intend to achieve.
“Despite popular belief, regulations and policies are not mechanisms to slow down innovation. They are the only mechanisms that speed up AI adoption.” Mihai Bilauca
Mihai Bilauca
We very often run into pilot fatigue which impedes AI adoption. Organisations can get tired from running too many pilots that do not result in a viable product. Pilots should not be started until a plan to scale it out is considered. Otherwise, you are just tinkering with a problem and not serious about solving it in full. To look at a historical example, creating the light bulb only answered part of the problem. Edison also had to think about a distribution network to scale that to entire cities. It is also critical to consider if you have the capacity, capability, and resources to scale out pilots, if successful.
Damien McCallion
We should not consider AI differently than any other innovation. There are myriad innovative solutions being devised all the time, but the real question is how many of these can be scaled. If you do not consider this at the beginning of a pilot, you will run into problems with procurement and buy-in. You cannot do everything, so you must have a clear prioritisation process, and you must embed a governance framework to ensure you deliver on your priorities. It is also important to conduct a cost-benefit assessment to ensure you deliver the best value for money when scaling innovations.
As AI takes on a greater role in decision-making, where should accountability sit, and how do we bring the workforce along?
Richard Boyd
There is no doubt AI has had a huge impact on how people view their place in the workforce. Where we have seen the greatest success is where AI is treated and managed as a digital work colleague working alongside the human team. The same leader is able to make the necessary changes to the agent when there are developments in policy or advice. That approach has completely changed people’s attitudes. They do not see AI as a technology; they see it as someone who enables them to do their job better. In terms of accountability, that sits with people for important decisions and the final say is ultimately a human decision
“AI does not remove the need to fix our foundations; it makes the case for fixing them much more compelling.”
Louise McKeever
Louise McKeever
Accountability cannot be delegated, it will always sit with people, not AI. We need clear ownership and the confidence to question and challenge decisions. That is the human-in-the-loop part. We must equip people around data, digital, and AI; it is essential. We need people to be data literate, digitally confident, and AI capable. If you are going to interact with models you must understand the data, you must be able to assess the outputs, and you must be able to intervene. We need a different approach to skills and training so people are equipped appropriately.
Richard Smith
Accountability will always rest with the business owner, but they must be appropriately informed. You want accountability to remain with business owners so it can breed appropriate behaviours. AI is not deterministic, it is probabilistic. At the end of the day, it is simply making its best guess. Being accountable is not enough, you must be informed so you understand your responsibilities.
Vincent Kennedy
Accountability must always sit with the organisation. The technology is there to assist you, not to drive. In DSP, the types of decisions we make can involve very vulnerable people. That means those decisions need to remain with people. AI can support them to make those decisions, but the accountability must sit with a person and the organisation. To bring staff along, training is crucial. People need to thoroughly understand what they are working with. Transparency is also critical to bring people on the journey.
Mihai Bilauca
Humans must always have accountability for decisions. When software is installed, it comes with a message that the company that manufactured the product is not responsible for the way it is used. Software companies do not take responsibility for the outputs of a particular software system. The challenge is how do you assist the system owners in taking on that role. Most people do not want to do this without support. It is important that AI systems are built with feedback loops and error-correction mechanisms. There must be a channel for errors to be identified and rectified by humans that own these systems and make decisions.
Damien McCallion
I would change the term ‘human in the loop’ to ‘AI in the loop’ because the human should be there by default. Some processes will be highly automated, but we do not want AI bots to replace jobs that involve human interaction. We must also protect against complacency and that is done through effective education and training. If people rely too heavily on automation, it could lead to errors which will negatively impact outcomes. We must ensure the workforce does not become complacent in decision-making processes as AI matures.
