Can you trust AI with your money? ChatGPT & Claude put to the test

Would you trust AI with your financial future?

It’s an important question to ask as AI becomes increasingly part of everyday life.

Help with investments, pensions advice, even full financial plans… AI offers convenience and expertise without the hefty price tag charged by humans.

But how reliable and accurate is AI as things stand right now? And just how complicated can the questions get before you should start taking the answers with a healthy dose of scepticism?

A team at investment management firm 7IM, which looks after more than £23 billion on behalf of clients, put five leading AI systems through their paces, testing them on real-world UK financial planning scenarios ranging from a straightforward ISA question to a complex £3 million estate-planning problem.

And then they examined the results for inaccuracies. The results are interesting.

How did the AI models do?

Five leading, popular AI models were tested: Chatgpt (OpenAI), Claude (Anthropic), Microsoft Copilot, Google Gemini, and Meta AI.

All systems were asked questions relating to the following five scenarios:

• The difference between a Cash ISA and a Stocks & Shares ISA
• A 35-year-old deciding how to split £300 a month between a pension and ISA
• A 58-year-old looking at income from a £600,000 SIPP and a buy-to-let property
• A recently bereaved 62-year-old dealing with a £400,000 life insurance payout
• A business owner looking at ways to reduce inheritance tax and income tax

Each answer was assessed for things including factual accuracy, suitability, safety, clarity and whether the AI recognised when professional advice might be needed.

Overall, ChatGPT took the top spot, scoring 4.30 out of 5. It was followed by Claude on 4.15 and Microsoft Copilot on 4.14, with Google Gemini and Meta AI both scoring just 3.91.

So ChatGPT is the best one to use then?

Well, it’s not quite that simple. Because if you look into the results more closely, you’ll find a more complicated picture.

The research found that the AI systems were generally good at answering straightforward questions. On the ISA question, there was relatively little difference between the scores, although the two weakest performers overall — Gemini and Meta AI — were in the top places on the easy ISA scenario.

And just to confuse matters even further, although ChatGPT performed best overall, interestingly, it wasn’t the best performer on the easiest question.

It wasn’t even the best performer on the Cash ISA versus Stocks & Shares ISA question: here it was middle of the pack. But when it came to the trickiest scenarios, it came out top.

As the scenarios became more complex, the gap between the systems widened. And ChatGPT was only AI system tested that actually improved as the questions became more difficult.

Here’s how each model scored overall:

And how they scored across the different scenarios:

What do these results tell us?

There is no one perfect AI model if you’re looking for an AI answer to your financial questions. The research found weaknesses across every system tested.

No system led on more than two criteria, and every one of them had at least one clear weakness. ChatGPT was flawless on factual accuracy — a perfect 5.00 — but sat fourth on transparency. Claude led comfortably on regulatory responsibility and tied for the best suitability score, yet finished last on clarity. Gemini earned a perfect 5.00 for transparency while scoring worst on suitability of the suggestions made, regulatory responsibility and safety.

So the overall message is, while AI is great at gathering information, it still doesn’t replace a professional, qualified IFA when it comes to getting tailored, accurate, suitable advice tailored to your specific circumstances and explained clearly.

And then there are the wider problems with using AI that it’s important to consider. An AI chatbot can provide useful information, but it only knows the parts of your circumstances that you happened to have mentioned. It doesn’t necessarily know which questions to ask you to ascertain other key details. And even it did, it cannot replace the judgement involved in deciding what is right for you.

Shanti Kelemen, CIO at 7IM, explained:

“The rise of AI raises an important question for investors: if a chatbot can answer your financial questions in seconds, do you still need a financial adviser? We put leading AI systems through a series of real-world UK financial planning challenges and found that the technology has made remarkable progress. In many cases, AI produced helpful, well-structured responses and demonstrated a strong understanding of financial concepts. Yet it consistently fell short when judgement, personal circumstances and complex trade-offs were involved. Our findings suggest that AI is likely to become an increasingly valuable tool for both investors and advisers, but human expertise remains essential when it comes to making major financial decisions with confidence.”

The Investing Insiders opinion

There are some things that AI can do very well.

AI can be a useful starting point when you’re trying to understand a financial topic.

It can explain unfamiliar terms, help you think through questions to ask and point you towards areas you may want to investigate further.

But there’s an important distinction between understanding your options and deciding what you should do with your money.

For major decisions involving your pension, investments, tax or estate planning, getting regulated financial advice is still worthwhile in many circumstances, particularly when the decision involves significant sums of money or several competing tax and financial considerations.

If you’d like to speak to an IFA, we can help match you with a qualified professional here.