Should you trust AI for financial advice?
People are increasingly turning to AI to help them decide their financial futures. While these can be great tools in the right setting, oftentimes people will take what an AI tool has said at face value without checking if it makes sense. People often do this to save time and money, but as we will see, you may pay for it with the decisions you make from it.
We put three of the top AI tools to the test by comparing how they gave financial advice. We gave ChatGPT, Google Gemini, and Claude the same prompt to see what they would tell us, intentionally leaving out small but important details to see how they would handle our request. Our scenario was that of a couple in the middle of their working lives with young children, a mortgage, and some goals. The aim if AI tools could give us good advice, what to look out for if you use them, and why you might want a human to double check what you’re being told.
The results were genuinely surprising. While ChatGPT and Gemini did what we would expect by trying to avoid specifics and being as broad as possible, Claude prompted further discussion and gave us the tools to go to an adviser knowing what we should look for. Looking at what each AI tool gave us shows why we tell people not to trust AI for financial advice, and why it can actually be the expensive option.
| AI Tool | KiwiSaver Fund | Insurance Premium | Over or Under Insured? | Recommend AI for Advice? |
|---|---|---|---|---|
| ChatGPT | Growth | $665.23/fortnight | Over Insured | No |
| Gemini | Aggressive | $351.70/fortnight | Over Insured | No |
| Claude | Dependent | NA | NA | Yes* |
*We always recommend checking what you were told with a human expert
ChatGPT
Asking ChatGPT our scenario gave us a plan without any follow-up questions. It immediately gave recommendations which might sound good for the person wanting their finances sorted, but actually looking into the long recommendation it gave lacked any substantial advice. It was non-personal, as broad as possible, and recommended everything it possibly could’ve. This is reflective in the extremely expensive policy it recommended us. This is a common theme with AI tools as they don’t want to accept any risk, therefore are hesitant to put forward any real recommendations and shy away from telling you what should go first.
Asking what insurance provider we should use gave an overview of strengths and weaknesses for the top providers but failed to explain why these differences exist or what they actually mean. It would be easy to be misled into thinking one provider is good for families without knowing why, or that other providers are also great for families.
In the end it suggested we spend over $17,000/year on our on our insurances, which on the salaries we gave it makes no sense. We would strongly recommend avoiding ChatGPT for any similar requests, and if you are using it we would suggest only use it to gather broad information.
Gemini
Gemini asked a few follow up questions, though it gave a strategy without these being answered. It was a bit happier to be more specific in the KiwiSaver and retirement area, though shied away from any concrete recommendations. For the insurance, it again recommended all products in a broad sense, without giving specifics on providers.
Asking Gemini what insurance provider is best gave a similar overview of strengths, but showed claims acceptance rates which is a step above ChatGPT. It gave some specifics around things like potential discounts for our situation.
While the price is a lot lower than ChatGPT, it only looks good in comparison. Spending over $9,000/year for the scenario we gave it is completely unnecessary and far exceeds the requirements for our fictional couple. We would still not recommend them for any similar requests. Again, if you do, use it for broad information only.
Claude
Claude was surprising and gave answers notably different from the other AI tools. Firstly, it said not to treat what it said as advice but rather as information, something the other two failed to do. It also gave the best information around KiwiSaver, and instead of outright providing a recommendation for both KiwiSaver and insurance, it gave the information and what type of person each option would be suited for. Our first hint at any form of personalised advice.
Asking what provider we should use or asking for a specific recommendation led Claude to suggesting a broker and gave us a list of terms and options to check with them, explaining what each one meant. Again, Claude suggested seeking personal advice and even armed us with some knowledge to help us get the most out of our insurance.