The Ethics of AI — Could We, Should We, Would We? with James Poulter

Most of us already know what an AI model is. The harder questions are how we should use it, and why. In this talk, James Poulter (JP) breaks the ethics of AI into three simple questions: Could we? Should we? Would we?

JP is head of AI and innovation at House 337 and a longtime voice in the Missional AI community. He gave this talk at CV's Global Digital Day in London, and we're sharing it as part of a series of talks and conversations from the event. Our thanks to CV for partnering with us on these episodes and on the Missional AI Summit.

Could we? We already are

AI is already part of daily life for a huge share of the world, and it's moving quickly from conversation to action.

JP expects ChatGPT to reach around a billion daily users soon, roughly one in seven people on the planet. And we're moving through distinct phases:

• Conversational AI: you talk to an AI, then do the work yourself.
• Agentic AI: you talk to an AI, and it does the work for you.
• Organizational AI: AIs talk to each other and do the work together, a phase JP expects by late 2027 at the latest.

OpenAI's Sam Altman predicts people will ask AI agents to do in an hour what used to take a month. JP notes that Altman also chairs World, the company behind the Orb, which scans people to issue a digital identity that could one day support digital currency or universal basic income. One company builds the intelligence; the other works out who's human and how to pay them when there's nothing left for them to do.

Others go further. The AI 2027 report, by former OpenAI researchers including Daniel Kokotajlo, suggests AI could begin training itself as soon as 2027. Today only about a thousand people can train frontier models, which limits how fast they grow. Remove that ceiling, and superintelligence starts to look less like science fiction.

Robots, risk and one shot to get it right

Science fiction is quickly becoming science fact. JP points to Neo, a humanoid home robot from 1X expected to ship next year for about $20,000, or $500 a month. Who wants a robot to do the dishes? Fine, but do you want it handling knives? Strong enough to lift a laundry basket, but not to throw it? With capability comes opportunity, and opportunity for misuse.

Then there's Eliezer Yudkowsky's book If Anyone Builds It, Everyone Dies, which argues superintelligence built with current techniques could wipe us out. As Christians, JP quips, we already believe everybody dies, bar one guy. He doesn't think doom is guaranteed, but the warning matters: if we pursue superintelligence, we may get only one shot. And something all-knowing, all-powerful, everywhere and eternal "sounds like something I've heard of before."

He closes this section with C. S. Lewis, writing about the atomic bomb: if we're going to be destroyed, let it find us doing sensible, human things.

Should we? AI is grown, not made

AI isn't just a tool. It's grown and shaped by the people who train it, which means its values matter, and the church has something to offer.

People often tell JP that AI is just a tool. In some ways it is. But most tools, like a chair or a spade, are built from components. AI is grown: we train it and see what happens, often without fully understanding how it does what it does. And like children or students, what's grown is shaped by those who raise it.

One example is the system prompt, hidden instructions sent to a model alongside every question you ask, guiding how it behaves. JP points to a line in one major model's system prompt asking it to treat religious texts and other symbolic works as non-literal while still discussing them critically. Models used by a seventh of the planet are, in effect, helping decide what counts as literal truth.

AI offers huge opportunity, JP stresses, but it also raises real threats to human dignity: job displacement, surveillance, deepfakes, distorted truth and bias. He believes the church must rise up and be part of the answer. The core values in most AI ethics frameworks, such as transparency, inclusion, fairness and privacy, have deep roots in Scripture. As historian Tom Holland argues in Dominion, Western ethics rest on Christian foundations.

So should we use AI? Yes, JP says, but well, and aligned with Scripture's vision for the world.

Would we? Beware the trust transfer

People already rely on AI for support and companionship, so the real danger is trusting it in high-stakes areas just because it was right in easy ones.

According to a Harvard Business Review analysis of how people use generative AI, therapy and companionship have become the top personal use, and personal and professional support the top use at work. Meanwhile, Collins Dictionary has named "vibe coding" a word of the year, a term coined by OpenAI co-founder Andrej Karpathy. JP sees that spreading into "vibe working," where AI companions do much of the work alongside us.

That's where he warns of the trust transfer. You ask an AI for a recipe or holiday ideas, it gives a confident, helpful answer, and you build trust. Then you carry that trust into a very different domain, like asking it to read a mammogram for early signs of cancer. It answers just as confidently, and you trust it, with no real evidence it's reliable there.

We create professions such as doctors, lawyers and theologians precisely so trusted answers can be verified through training, qualifications and community. With AI, we often skip that and transfer trust from the trivial to the consequential. JP's charge: use AI, but with caution, aligned with Scripture and ethics, knowing what and whom you're trusting, "because these things are being grown, not made."

A weird time, and three responses

As debates about AI rights emerge, the church needs research, better guidelines and education.

We live in a weird time, JP says, and it will get weirder. Anthropic now has a model welfare team exploring how to protect AI models from potential harm. JP doesn't believe AI is conscious. But if the wider world comes to call some models sentient, ministries building on them could face unexpected questions.

He proposes three responses:

• Better governance and guidelines. The Missional AI community, working with partners including Biblica, Biola University, RightNow Media and Gloo, has developed seven Christian AI principles. Use them as gut checks on whether your AI use aligns with Scripture and theological values.
• Research. House 337's second annual AI Impact Report studies how 6,000 people across the US, UK and Germany use AI.
• Education. JP's book, AI at Work, comes out next August and unpacks how to use AI well at work.

His closing: Could we use AI? We almost certainly will. Should we? The church should be the best users of it in the kingdom. Would you? That's up to you. "But use it, please use it well."

Watch more from JP

JP has spoken at several Missional AI gatherings. Find his talks in the Missional AI Video Library:

• Orlando 2024: "Talking Gods: How the Church Should Ready Itself for Ethical AI Adoption."
• Dallas 2025: "How to Become a 'Full Stack Professional.'"
• London 2025: the Standards Panel he led.

Listen and join the conversation

Listen to JP's full talk on the Missional AI Podcast, and subscribe for more talks and conversations from CV's Global Digital Day.

The Missional AI 2026 Global Summit is just weeks away: April 7–9, 2026, in Silicon Valley. Find out more and register at missional.ai.

This episode was made possible in partnership with CV.

Sources

• Missional AI Video Library