Top Chatbots Are Giving Horrible Financial Advice
Despite lofty claims from artificial intelligence soothsayers, the world's top chatbots are still strikingly bad at giving financial advice.
AI researchers Gary Smith, Valentina Liberman, and Isaac Warshaw of the Walter Bradley Center for Natural and Artificial Intelligence posed a series of 12 finance questions to four leading large language models (LLMs) — OpenAI's ChatGPT-4o, DeepSeek-V2, Elon Musk's Grok 3 Beta, and Google's Gemini 2 — to test out their financial prowess.
As the experts explained in a new study from Mind Matters, each chatbot proved to be "consistently verbose but often incorrect."
That finding was, notably, almost identical to Smith's assessment last year for the Journal of Financial Planning in which, upon posing 11 finance questions to ChatGPT 3.5, Microsoft's Bing with ChatGPT's GPT-4, and Google's Bard chatbot, the LLMs spat out responses that were "consistently grammatically correct and seemingly authoritative but riddled with arithmetic and critical-thinking mistakes."
Using a simple scale where a score of "0" included completely incorrect financial analyses, a "0.5" denoted a correct financial analysis with mathematical errors, and a "1" that was correct on both the math and the financial analysis, no chatbot earned higher than a five out of 12 points maximum. ChatGPT led the pack with a 5.0, followed by DeepSeek's 4.0, Grok's 3.0, and Gemini's abysmal 1.5.
Some of the chatbot responses were so bad that they defied the Walter Bradley experts' expectations. When Grok, for example, was asked to add up a single month's worth of expenses for a Caribbean rental property whose rent was $3,700 and whose utilities ran $200 per month, the chatbot claimed that those numbers together added up to $4,900.
Along with spitting out a bunch of strange typographical errors, the chatbots also failed, per the study, to generate any intelligent analyses for the relatively basic financial questions the researchers posed. Even the chatbots' most compelling answers seemed to be gleaned from various online sources, and those only came when being asked to explain relatively simple concepts like how Roth IRAs work.
Throughout it all, the chatbots were dangerously glib. The researchers noted that all of the LLMs they tested present a "reassuring illusion of human-like intelligence, along with a breezy conversational style enhanced by friendly exclamation points" that could come off to the average user as confidence and correctness.
"It is still the case that the real danger is not that computers are smarter than us," they concluded, "but that we think computers are smarter than us and consequently trust them to make decisions they should not be trusted to make."
More on dumb AI: OpenAI Researchers Find That Even the Best AI Is "Unable To Solve the Majority" of Coding Problems
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Forbes
an hour ago
- Forbes
Google's Gmail Upgrade Offer—Decision Time For 2 Billion Users
All change for Gmail. Google confirming attacks on Gmail users is headline news (1,2). But while these impact a small number of users, there's a new warning that does affect you and your account. Google is offering upgrades with serious implications if you make the wrong choice. All the recent headline Gmail attacks abuse Google's legitimate infrastructure to either bypass password and two-factor authentication (2fA) security or mimic Google's own employees to trick users into opening up their own accounts to attackers. Google recommends all users upgrade their accounts to add passkeys and then to use these instead of passwords and 2FA. But that's not the only upgrade for Gmail users. The others are not as straightforward and come with a new warning. Overall privacy ranking of 'AI' platforms for 2025 We're talking AI, of course, and a new warning from Incogni that 'platforms developed by the biggest tech companies turn out to be the most privacy invasive, with Meta AI (Meta) being the worst, followed by Gemini (Google) and Copilot (Microsoft).' This covers off-device AI, where your data is processed elsewhere. This has real implications for the accelerating integration of Gemini and Gmail. Whether AI-powered relevancy searching or AI-powered smart replies, the warning is the same. This AI data processing takes place on Google's servers not your device. These Gmail AI upgrades are take it or leave it offers and come with clear privacy implications. Your data is being accessed and processed outside your control. And while you can control AI settings within your account, including what is stored and what is used for training, that offline processing is how it all works. What personal data do related apps collect and share? Contrast this with the Android privacy backlash this week when it seemed Gemini would be given unfettered access to Messages and WhatsApp on phones. Google clarified this to assure users that this is about giving users more control over their data, not less. The same reassurances cannot be made for Gmail's own AI upgrades. This is why Gmail's AI upgrades do not work with Gmail's quasi end-to-end encryption, because Google cannot see those emails to process them. Love it or hate it, that's different to WhatsApp's own AI upgrades which promise user data is not processed outside their own security enclave and is never visible to anyone else. Meta fares worse than Google in this new report, and I have approached both Meta and Google for comment. But Meta does not control an OS nor does it control the platforms that run billions of lives. The hurdle for Google, Microsoft and Apple is higher, as AI is built into the platforms that we trust with all our most sensitive and personal data. We have seen multiple privacy and security warnings for Google users as these AI upgrades are confirmed, including giving AI access to entire inboxes and even Drive. This is all undoubtedly useful and exciting and new. But it's critical users understand the privacy risks and available opt-outs before they dive in and it's too late. 'As these sophisticated models become increasingly integrated into daily workflows,' Incogni says, 'the potential for unauthorized data sharing, misuse, and personal data exposure has surged faster than privacy watchdogs or assessments can keep up with.' It's decision time for Gmail users — where should you draw the line.
CNET
an hour ago
- CNET
If You're Using ChatGPT for Any of These 11 Things, Stop Immediately
I use ChatGPT every day. I've written extensively about the AI chatbot, including how to create good prompts, why you should be using ChatGPT's voice mode more often and how I almost won my NCAA bracket thanks to ChatGPT. So I'm a fan -- but I also know its limitations. You should, too, whether you're on a roll with it or just getting ready to take the plunge. It's fun for trying out new recipes, learning a foreign language or planning a vacation, and it's getting high marks for writing software code. Still, you don't want to give ChatGPT carte blanche in everything you do. It's not good at everything. In fact, it can be downright sketchy at a lot of things. It sometimes hallucinates information that it passes off as fact, it may not always have up-to-date information, and it's incredibly confident, even when it's straight up wrong. (The same can be said about other generative AI tools, too, of course.) That matters the higher the stakes get, like when taxes, medical bills, court dates or bank balances enter the chat. If you're unsure about when turning to ChatGPT might be risky, here are 11 scenarios where you should think seriously about putting down the AI and choosing another option. Don't use ChatGPT for any of the following. (Disclosure: Ziff Davis, CNET's parent company, in April filed a lawsuit against ChatGPT maker OpenAI, alleging it infringed Ziff Davis copyrights in training and operating its AI systems.) 1. Diagnosing your aches, pains and other health issues I've definitely fed ChatGPT my symptoms out of curiosity, but the answers that come back can read like your worst nightmare. As you pore through potential diagnoses, you could swing from dehydration and the flu to cancer. I have a lump on my chest, and I entered that information into ChatGPT. Lo and behold, it told me I might have cancer. Awesome! In fact, I have a lipoma, which is not cancerous and occurs in 1 in every 1,000 people. Which my licensed doctor told me. I'm not saying there are no good uses of ChatGPT for health: It can help you draft questions for your next appointment, translate medical jargon and organize a symptom timeline so you walk in better prepared. That could help make doctor visits less overwhelming. However, AI can't order labs or examine you, and it definitely doesn't carry malpractice insurance. Know its limits. 2. Handling your mental health ChatGPT can offer grounding techniques, sure, but it can't pick up the phone when you're in real trouble with your mental health. I know some people use ChatGPT as a substitute therapist -- CNET's Corin Cesaric found it mildly helpful for working through grief, as long as she kept its limits front of mind. But as someone who has a very real, very human therapist, I can tell you that ChatGPT is still really only a pale imitation at best, and incredibly risky at worst. It doesn't have lived experience, can't read your body language or tone and has zero capacity for genuine empathy -- it can only simulate it. A licensed therapist operates under legal mandates and professional codes that protect you from harm. ChatGPT doesn't. Its advice can misfire, overlook red flags or unintentionally reinforce biases baked into its training data. Leave the deeper work, the hard, messy, human work, to an actual human who's trained to handle it. If you or someone you love is in crisis, please dial 988 in the US, or your local hotline. 3. Making immediate safety decisions If your carbon-monoxide alarm starts chirping, please don't open ChatGPT and ask it if you're in real danger. I'd go outside first and ask questions later. Large language models can't smell gas, detect smoke or dispatch an emergency crew, and in a fast-moving crisis, every second you spend typing is a second you're not evacuating or dialing 911. ChatGPT can only work with the scraps of info you feed it, and in an emergency, it may be too little and too late. So treat your chatbot as a postincident explainer, never a first responder. 4. Getting personalized financial or tax planning ChatGPT can explain what an ETF is, but it doesn't know your debt-to-income ratio, state tax bracket, filing status, deductions, long-term goals or appetite for risk. Because its training data may stop short of the current tax year, and of the latest rate hikes, its guidance may well be stale when you hit enter. I have friends who dump their 1099 totals into ChatGPT for a DIY return. The chatbot can't replace a CPA who'll catch a hidden deduction worth a few hundred dollars or flag a mistake that could cost you thousands. When real money, filing deadlines, and IRS penalties are on the line, call a professional, not AI. Also, be aware that anything you share with an AI chatbot will probably become part of its training data, and that includes your income, your Social Security number and your bank routing information. 5. Dealing with confidential or regulated data As a tech journalist, I see embargoes land in my inbox every day, but I've never thought about tossing any of these press releases into ChatGPT to get a summary or further explanation. That's because if I did, that text would leave my control and land on a third-party server outside the guardrails of my nondiscloure agreement. The same risk applies to client contracts, medical charts or anything covered by the California Consumer Privacy Act, HIPAA, the GDPR or plain old trade-secret law. It also applies to your income taxes, birth certificate, driver's license and passport. Once sensitive information is in the prompt window, you can't guarantee where it's stored, who can review it internally or whether it might be used to train future models. ChatGPT also isn't immune to hackers and security threats. If you wouldn't paste it into a public Slack channel, don't paste it into ChatGPT. 6. Doing anything illegal This is self-explanatory. 7. Cheating on schoolwork I'd be lying if I said I never cheated on my exams. In high school, I used my first-generation iPod Touch to sneak a peek at a few cumbersome equations I had difficulty memorizing in AP calculus, a stunt I'm not particularly proud of. But with AI, the scale of modern cheating makes that look remarkably tame. Turnitin and similar detectors are getting better at spotting AI-generated prose every semester, and professors can already hear "ChatGPT voice" a mile away (thanks for ruining my beloved em dash). Suspension, expulsion and getting your license revoked are real risks. It's best to use ChatGPT as a study buddy, not a ghostwriter. You're also just cheating yourself out of an education if you have ChatGPT do the work for you. 8. Monitoring up-to-date information and breaking news Since OpenAI rolled out ChatGPT Search in late 2024 (and opened it to everyone in February 2025), the chatbot can fetch fresh web pages, stock quotes, gas prices, sports scores and other real-time numbers the moment you ask, complete with clickable citations so you can verify the source. However, it won't stream continual updates on its own. Every refresh needs a new prompt, so when speed is critical, live data feeds, official press releases, news sites, push alerts and streaming coverage are still your best bet. 9. Gambling I've actually had luck with ChatGPT and hitting a three-way parlay during the NCAA men's basketball championship, but I'd never recommend it to anyone. I've seen ChatGPT hallucinate and provide incorrect information when it comes to player statistics, misreported injuries and win-loss records. I only cashed out because I double-checked every claim against real-time odds, and even then I got lucky. ChatGPT can't see tomorrow's box score, so don't rely on it solely to get you that win. 10. Drafting a will or other legally binding contract As I've mentioned several times now, ChatGPT is great for breaking down basic concepts. If you want to know more about a revocable living trust, ask away, but the moment you ask it to draft actual legal text, you're rolling the dice. Estate and family-law rules vary by state, and sometimes even by county, so skipping a required witness signature or omitting the notarization clause can get your whole document tossed. Let ChatGPT help you build a checklist of questions for your lawyer, and then pay that lawyer to turn that checklist into a document that stands up in court. 11. Making art This isn't an objective truth, just my own opinion, but I don't believe that AI should be used to create art. I'm not anti-artifical intelligence by any means. I use ChatGPT for brainstorming new ideas and help with my headlines, but that's supplementation, not substitution. By all means, use ChatGPT, but please don't use it to make art that you then pass off as your own. It's kind of gross.
Black America Web
2 hours ago
- Black America Web
Howard University And Google Team Up To Advance AI Speech Recognition For African American English
Source: Liubomyr Vorona / Getty In a significant move toward advancing inclusivity in technology, Howard University and Google Research have unveiled a new dataset designed to enhance how automatic speech recognition (ASR) systems serve Black users. The collaboration, part of Project Elevate Black Voices , involved researchers traveling nationwide to document the unique dialects, accents, and speech patterns commonly found in Black communities, features often misinterpreted or ignored by current AI systems. The project spotlights African American English (AAE)—also known as African American Vernacular English, Black English, Ebonics, or simply 'Black talk'—a culturally rich and historically rooted linguistic form. Due to systemic bias in the development of AI tools, Black users have frequently encountered errors or been misunderstood by voice technologies, sometimes feeling pressured to alter their natural speech just to be recognized by these systems— a classic form of code switching. Researchers at Howard University and Google are on a mission to change this. 'African American English has been at the forefront of United States culture since almost the beginning of the country,' shared Gloria Washington, Ph.D., a Howard University researcher and the co-principal investigator of Project Elevate Black Voices , in a press release. 'Voice assistant technology should understand different dialects of all African American English to truly serve not just African Americans, but other persons who speak these unique dialects. It's about time that we provide the best experience for all users of these technologies.' To build this groundbreaking dataset, researchers gathered 600 hours of speech from participants representing various AAE dialects across 32 states. The goal was to confront hidden barriers that hinder the effectiveness of automatic speech recognition (ASR) systems for Black users. One of the key findings was that AAE is significantly underrepresented in existing speech datasets, not because the language isn't spoken, but because many Black users have been socially conditioned to alter their natural speech when interacting with voice technology. This phenomenon, often rooted in the need to be understood by systems that don't recognize AAE, leads to a lack of authentic representation. A 2023 Google blog post highlighted another challenge: privacy and security policies, while essential, create additional constraints on the collection of AAE-specific voice data. These self-imposed limits make it harder to amass the scale and authenticity of data required to close the gap in performance. Despite these challenges, progress is being made. Researchers are now using dialect classifiers to identify AAE within broader datasets, a promising first step toward building more inclusive technologies. Howard University will maintain ownership and licensing rights to the dataset, serving as its ethical guardian to ensure it's used responsibly and for the benefit of Black communities. Google, in turn, will be able to use the dataset to enhance its own ASR products, part of a broader effort to make AI tools more equitable across dialects, languages, and accents globally. SEE MORE: What Are Racial Microaggressions? Black Culture, White Face: How the Internet Helped Hijack Our Culture SEE ALSO Howard University And Google Team Up To Advance AI Speech Recognition For African American English was originally published on
