The Rise Of False AI Insights: When More Data Means More Problems
Apple found itself in hot water when news broke that the Apple Card was offering lower credit limits to women, regardless of their financial profiles. The fault lies with the AI algorithm used to analyze creditworthiness; the training data for the algorithm included historical lending data that reflected gendered biases.
The lesson here is straightforward: AI makes mistakes. Sometimes AI hallucinates, filling in gaps in data to provide a 'made up' answer—a phenomenon where the model generates plausible but incorrect information. But sometimes AI also generates misinformation and false insights. It provides an answer based on real data, but data that is outdated, incomplete or inaccurate.
For example, say you ask AI to pull average salaries for a role at your company. For some reason, the 2020 salary document was opened more recently than the document for 2024. The AI tool thinks the 2020 document is the most relevant, so it uses that data to provide an answer. It's a false AI insight—AI provided an answer based on real data, just not the correct data.
As leaders increasingly turn to AI for help with data-driven decision-making, it's important to understand how false insights occur, how to prevent them and what this means for data governance.
How Does Too Much Data Lead To False AI Insights?
It's no secret that many organizations are drowning in data. On average, organizations leverage nearly 900 apps, with each modern business application collecting and running on a staggering amount of data.
This creates a critical challenge: AI requires masses of clean data to produce trustworthy, valuable results. Companies certainly have the volume of necessary data. But where most organizations fall short is in the quality of their data. It's impossible to clean up all their data and properly govern each data point—there's simply too much of it.
Consider the possibilities for fake insights from AI when employees start using a tool like Microsoft Copilot. Copilot scans all the documents across the entire system for answers to questions, crafting a response based on what seems relevant. It's not always right. Copilot could pull data from an outdated document from a long-gone employee—not exactly a relevant or trusted source of information.
What's more, with new tools such as Microsoft Fabric, a cloud-based, end-to-end data analytics platform, employees are more empowered than ever before to access and act on data. While this creates massive opportunities for organizations, it also multiplies the potential for exposing AI to ungoverned, unmanaged and inaccurate data.
It's a catch-22. Governing every piece of data isn't feasible but letting AI access ungoverned data leads to unreliable results. And restricting AI to only well-governed data may limit its usefulness.
So what's the solution? How can leaders harness the power of AI and ensure AI doesn't just produce misleading insights? What's needed is a new mindset around governance.
Prevent AI Misinformation With A New Output Governance Mindset
The age of AI requires a new governance mindset. What's out: governing all the individual data points. What's in: Governing the outputs of AI tools through end-to-end testing strategies. This change in approach will allow organizations to encourage innovation and take advantage of AI while also mitigating the risks of fake insights leading to poor data-driven decision making.
Big picture, this new governance framework allows teams to access a broad array of data—including raw or ungoverned data—to build automation tools. But before the tool is brought to production, it must go through a governance checkpoint to evaluate the model and its outputs using standard test cases. The scale and speed with which these innovations occur requires that the testing framework leverage automation to keep up. Skipping this governance checkpoint essentially means letting people create powerful and untested tools for decision making, which could be disastrous to an organization's future success.
In addition to a governance checkpoint, each AI tool should be closely monitored during its first 90 days of deployment. This period requires proactive monitoring, with a plan to transition to reactive monitoring once the team gains confidence in the tool's performance.
Proactive monitoring involves direct human oversight—reviewing logs, evaluating test cases and using AI-based guardrails to observe the tool's behavior in real time. Once the tool has demonstrated reliability, the team can shift to reactive monitoring, which relies on other AI systems to detect anomalies and trigger alerts when potentially unacceptable behavior occurs.
Good output governance means using AI to help govern AI. Think of it like this: the AI doing the actual work—like analytics—is the adult in the room, capable of complex reasoning. The AIs that monitor it are more like kids: they don't always get the big picture, but they're great at shouting, 'Hey! That's not okay!' when something clearly breaks the rules.
Another tactic to prevent AI misinformation and inspire confidence in the output from AI is to require AI tools to include annotations in their responses. With every factual question an employee asks of an AI tool, it should list where it's pulling the data from. Employees can quickly scan the annotations and decide if the data sources are trustworthy and make sense. (Needless to say, annotations are most appropriate for AI tools intended for internal use.)
AI requires masses of data to work correctly. Organizations have no shortage of data, but most struggle applying data governance to their thousands upon thousands of data points. The solution isn't to lock data away or just let AI loose on ungoverned data.
Rather, leaders need to reconsider their governance mindset, putting in place a robust end-to-end testing strategy for any new AI tools to ensure the outputs are accurate and decrease the likelihood of AI producing false insights, leading to poor decision making.
By shifting their mindset from data governance to output governance, organizations can unlock AI's potential—without falling victim to AI misinformation.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Bloomberg
20 minutes ago
- Bloomberg
US Core CPI Rises Less Than Expected for Fifth Month
00:00 We've got an upside surprise here that kind of makes a little hash of perhaps the treasury secretary's claim that inflation hasn't hit yet. CPI comes it up 3/10 of a percent. The downside surprise, though, is in the core, up just 2/10 of a percent. Both were up a 10th of a percent last month. Year over year we go to 2.7%, which is higher than the anticipated 2.6%. It's a 3/10 percent move up from last month. And core is at 2.9%. That is what was expected last month. It was 2.8%. Let me just quickly mentioned empire manufacturing. It goes up 22 points to a positive 5.5 important because that's July number one of the first July numbers we get. And it is the first time it's been positive since February. Prices paid, though, went up.
Forbes
23 minutes ago
- Forbes
How To Structure Cross-Department Collaboration
Dustin Lemick, founder and CEO of BriteCo, an innovative insurance technology company transforming the retail jewelry insurance experience. We've all heard the sayings 'There's strength in numbers' and 'There's no 'I' in team' repeatedly. But there's a reason these statements continue to resonate and appear on motivational posters that decorate so many offices: because they're true. There are few (if any) businesses that have gotten off the ground with just one person. The fact that a business is referred to as a 'company' is telling. We need partners, collaborators, investors and workers to help bring life to an idea. But beyond just having people at your side, it is important to build a strong team and ensure they all work well together. In my experience as a founder and CEO, establishing and encouraging cross-department collaboration is especially important for fast-growing, growth-phase companies when everything is moving fast and many initiatives are launching simultaneously. Key opportunities and important details can get missed if every department is not on the same page and continually communicating and sharing knowledge. With that in mind, here are three valuable lessons I've learned over the years that you can use to create a solid cross-department team culture. 1. Full-team meetings are important. Consider having 'standup' meetings once a week where the entire staff (leadership and upper management included) meet and share updates, both big and small. This creates a regular opportunity for teams to come together to applaud everyone's efforts and learn what everyone is working on, what developments are happening and where they can support each other. My company amplifies this further with an annual 'all hands' gathering, during which different departments share presentations about achievements, new projects and ideas they may want to get off the ground in the near future. I've found that this route can also grow team members' confidence that everyone is moving in the same direction and that their role is critical to the effort. 2. Teams can grow each others' knowledge. The more knowledge and understanding each employee has about the organization, its current and upcoming projects and its future goals, the more ammunition they have to do their job well. For example, if someone on your marketing team is trying to navigate the intricacies of AI to help them come up with new content ideas, it may be helpful for them to ask the tech team about what they are doing to input AI into the company's processes. This could also lead to conversations with other departments about how best to leverage the growing technology in their own roles. Encouraging this kind of cooperation can help translate team members' best-in-class knowledge across departments. 3. Everyone's role is everyone's business. One great program we have at my company is "job shadowing": Employees are assigned to watch a colleague while they work and ask them questions so they can gain perspective on the different roles in the organization. Consider having your employees log a certain number of job shadowing hours every year with as many departments as possible. Also, encourage team members to attend any open meetings they see on the calendar if they want to learn more about the related topic or project. I've found that fostering this type of positive, open culture results in everybody talking to each other and keeping lines of communication open, both of which are important for the success of a healthy organization. 4. Keep your door open. A key part of successfully creating a cross-department collaborative culture is ensuring the management team is invested. This means being open and communicating effectively with team members at all levels of the organization. For example, every time you hire someone new, tell them that you have an open-door policy—and truly mean it. Your team should know they can approach you anytime to ask questions, share ideas or get feedback, and trust that you'll provide open-book responses. I also recommend ensuring that your onboarding process is very comprehensive so new employees can access all the information they need from day one. Consider scheduling one-on-one meetings with new hires' coworkers in and out of their department, and provide a robust library of educational tools that present a detailed roadmap of your company. 5. See where there's room to improve. To keep fostering cross-department collaboration, it's equally important that you continually evaluate the strategies and procedures you've put in place and look for areas where there is room for improvement. For example, while we have had a number of these processes in place for years, our company recently decided that we should launch an employee recognition program. The program incentivizes everyone to know the intricacies of their colleagues' roles and to provide acknowledgment when a job is well done. Employees get a monthly balance of 'points' that they can give to different coworkers or departments, who can then redeem the points for rewards. Final Thoughts When it comes to fostering a collaborative culture, I have found that quality is more important than quantity. Consider starting with something like reducing the number of less-productive single-department meetings in favor of fewer, more imperative group discussions where key players from different departments can be part of the conversation. When everyone is able to work together without the restriction of departmental boundaries, you can create true harmony between your teams. Forbes Business Council is the foremost growth and networking organization for business owners and leaders. Do I qualify?
Motor Trend
23 minutes ago
- Motor Trend
Alternative History: What if AMC Had Survived the '80s?
American Motors Corporation was an absolute mess by the mid-1980s, and its financial problems in the U.S. market were compounded by infighting at its European corporate parent, Renault, where executives went back and forth about how much money they were willing to pour into their trans-Atlantic subsidiary. Writer Benjamin Hunting imagines an alternate history where AMC survives the 1980s by leveraging government contracts, launching popular models like the "Space Van" and Grand Cherokee, and thriving under Renault's support—ultimately leading to huge industry shifts. This summary was generated by AI using content from this MotorTrend article Read Next The assassination of Renault's chairman in 1986 by French terrorists caused AMC to lose its most powerful supporter, and a hasty sale to Chrysler ultimately condemned it to the dustbin of automotive history. Chrysler hoovered up the tastiest bits of the American Motors portfolio—namely, Jeep—and slowly phased out the rest of the AMC's offerings over the course of the next decade. In retrospect, however, AMC was holding not one, not two, but three aces up its sleeve that could have seen it weather the financial storm throughout the '80s. It's entirely possible that had a few key moments in the company's timeline gone a different way, it would have been American Motors and not Chrysler enjoying the fruits of Jeep's mainstream renaissance in the early 1990s—a rebirth that AMC in fact already had in development when it was scooped up by the suits in Auburn Hills. How different would the car industry have looked at the turn of the millennium if AMC had never changed hands? It turns out that this ripple in the chronological pond had the potential to upset big chunks of established history, not just in America, but in nearly every corner of the established automotive hegemony. Here's our alternative timeline in which AMC not only survives but thrives—and what the resulting fallout would have likely meant for one of Detroit's longtime stalwarts. 1983 After intense lobbying by American Motors, the U.S. government carves out an exception to a law forbidding foreign ownership of defense contractors, contingent on Renault spinning off AM General (the builder of the Humvee owned at the time by AMC) as an independently managed concern under the AMC umbrella. The continued, steady flow of government contracts acts as a lifeline for American Motors, and it cancels plans to take out a significant loan from its corporate parent to fund operations. 1985 The Renault Espace arrives in AMC showrooms, where it is rebadged as the 'Space Van,' an Americanized take on the literal translation of the French vehicle's European identity. Surprisingly, the funky badge helps give the people-mover some personality, which, combined with its genuine practicality and roomier interior versus rivals from Ford (the Aerostar) and Chevrolet (the Astro), helps put it alongside the Dodge Caravan and Plymouth Voyager as a popular and affordable family ride. Following the introduction of the redesigned Jeep Cherokee the year before, this gives AMC a presence in two of the highest-growth segments in the American auto industry, and for the first time in years the company is cash-flow positive. This convinces American Motors to accelerate investment in a larger sport utility vehicle that would complement the Cherokee, called the Grand Cherokee, the design of which is already well underway. Renault chairman Georges Besse's chauffeur is surprised to see two armed women in front of the home of his boss while driving back from the office on a cold November evening. He immediately hits the gas, slamming the rear door shut before Besse can exit the vehicle, and the pair escapes with just a few bullet holes in the rear quarter panel. After surviving the assassination attempt, Besse is given carte blanche at Renault to move forward with his plans for focusing on Jeep as the automaker's piggybank to fund not only AMC, but to also further the expansion of the French brand onto American shores. 1987 Chrysler, on a brash spending spree that includes buying a controlling stake in Lamborghini and an expansion of its partnership with Mitsubishi to form Diamond Star Motors, sees exactly the same potential in Jeep as Georges Besse. An offer is made to Renault not just for the off-road brand but for all of AMC, with Chrysler trying to cloak its true intentions about what it considers the real prize of the transaction. Besse won't be bought, however, and Chrysler returns hat-in-hand to Auburn Hills. 1988 Ambitious planning begins for the upcoming decade in the American market. With Jeep as its crown jewel, both Eagle and AMC are slated for repositioning beneath Renault. The French badge is no longer interested in its role as an entry-level brand hawking low-spec Le Cars and lays the groundwork for leveraging existing dealerships to form a stronger toehold for the revitalized company. The Jeep Grand Cherokee breaks cover as a 'concept' at the North American International Auto Show in Detroit. The response from both the media and show attendees is overwhelmingly positive, leading to a brief spike in compact Cherokee sales from customers too impatient for what they assume will be a lengthy wait for the production version. No one realizes that Besse's pressure to keep pumping cash into Jeep has dramatically accelerated the Grand Cherokee's timeline. 1990 The Grand Cherokee makes its debut in showrooms to universal acclaim. On top of offering a choice of either AMC's old faithful inline six-cylinder engine or a newly developed, 5.9-liter fuel-injected V-8, it also provides a turbodiesel option borrowed from the Renault parts bin. The latter choice positions the Jeep in its higher trim levels as something more than an off-roader, pushing it onto the radar of Europhiles who have become used to parking Range Rovers in their driveways. This opens a second front of European sales for Jeep in the luxury sphere. 1991 The Ford Explorer joins the midsize SUV scene, splitting the difference between the Grand Cherokee's off-road chops and the practical character prized by family buyers now tempted to abandon their minivans. SUV sales are soaring, leaving General Motors and Chrysler playing catch-up. Chevrolet and GMC are at least able to soak up some sales thanks to the four-door compact Blazer/Jimmy and full-size four-door Suburbans sitting on full-size truck platforms, but the two-door Dodge Ramcharger remains in a distant fourth place as it plays out the string on a dated pickup chassis. 1992 Buoyed by strong Jeep sales, Renault launches the initial phase of its ambitious American strategy. First, it spins off AMC as a value-focused brand selling cars on a 'no-haggle' model: What you see advertised is what you pay at the dealership. Along with a redesigned Espace, an entirely new lineup of hatchbacks, small wagons, sedans, and budget coupes are gradually deployed over the course of the next few years, some sharing components with Renault's European offerings while others benefit from AMC's next-generation four-cylinder engine program. This puts AMC in direct competition with GM's Saturn brand, which arrived on the scene in mid-1990. Next, a revitalized Eagle steps out of the AMC shadow and becomes its own brand. The focus remains on what are now being called 'crossovers,' automobiles that sit between a wagon and a sport utility vehicle. Eagle also benefits from Renault's technical prowess in the form of unibody models that feature sophisticated all-wheel-drive systems in place of their earlier, low-range four-wheel drive setups. The new Eagles are an immediate hit in regions like Colorado and New England. 1993 Chrysler, facing considerable financial strain as sales of the Grand Caravan and Voyager slow in the face of the SUV onslaught, are forced to sell Lamborghini to MegaTech, an Indonesian company owned by Tommy Suharto, the son of that country's president-for-life. The automaker takes a loss on the deal, but it helps stem some of the financial bleeding that's beginning to concern both executives and Wall Street alike. 1994 Dodge introduces a new Ram pickup that instantly makes it a player in the full-size segment after years of disappointing sales. Unfortunately, that same success doesn't translate to its revised version of the SUV, which updates the two-door Ramcharger with the new pickup's underpinnings. As the market continues to move toward family-friendly four-door haulers, many of them taking their cues from Eagle's crossovers, the Ramcharger is out of step with what customers are actually looking for in a sport utility. 1995 Renault implements the next stage of its U.S. transformation by introducing the second generation of what had originally been planned as the Eagle Premier sedan. Originally kept exclusive to the European market, where it was sold as the Medallion, the new Renault Premier pushes the automaker into a higher class than it had previously enjoyed among American buyers, leading some to compare the car to offerings from Oldsmobile and even Audi. 1996 After a fraught development process, the Dodge Viper concept car makes a late debut at the Detroit auto show. Although it was originally hoped that Lamborghini's engineers could be more involved in the design of the vehicle's drivetrain, the early sale forced Dodge to move on from its planned V-10 and instead supercharge the company's long-standing 5.9-liter V-8. Heart-stopping styling doesn't make up for the lack of an exotic engine, making it harder for the public to stomach the no windows/no roof inconveniences of its cabin. Production plans for the Viper are quietly scuttled. The Viper team is diverted to focus on the Dodge Durango, a four-door, Grand Cherokee–sized SUV that the company hopes will turn its fortunes around. 1997 Subaru, in the face of strong sales from Eagle eating into its core customer base, makes a product cancellation of its own. The Outback, a tall-riding version of its Legacy wagon, is deemed too derivative of the Eagle lineup to make a dent in the market, and its development is halted. Facing dwindling revenues, and unable to finance new product development, Subaru's leadership initiates back-channel talks with Toyota about a possible merger. 1998 Renault, emboldened by the money pouring into its coffers from the success of AMC, Jeep, and Eagle, makes the surprise move of purchasing Volvo, scooping Ford who had planned on making overtures for the Swedish brand to join its nascent Premier Automotive Group. After decades of working together on various shared projects, Renault hopes to leverage Volvo's dealer network and customer base to continue its colonization of the near-luxury space in the United States. Talks also begin with Nissan about a potential alliance. Two new premium models emerge on American roads bearing the Renault badge: the Megane sport hatch and the Laguna hatchback sedan, with the latter praised for its near-crossover utility and excellent handling. 1999 Concerned by Renault's burgeoning acquisition portfolio, Toyota signs a deal to bring Subaru in-house. At the same time, executives announce a new subbrand called Scion that's intended to take on both AMC and Saturn, which have split much of the entry-level market between them in the United States. Chrysler, looking for a savior of its own, begins talks with Daimler about a potential 'merger of equals.' The German automaker's boardroom doesn't see much of value in Chrysler's mishmash of cheap cars, fading minivans, and almost-luxury sedans, and while the Dodge Ram is appealing, it's too far outside the Daimler playbook to integrate properly into its American operations. Discussions never advance past the initial stages. 2000 Emboldened by its newfound partnership with Nissan (which involved a stock share and co-investment in each other's companies), Renault has the cash to add the missing piece to its U.S. portfolio: Dodge, which it plucks from a flailing Chrysler as part of a general takeover bid. While the Ram pickup fills an important void, the Ramcharger is quietly put out of its misery, along with any plans to bring the stillborn Durango to market. The Chrysler brand is relegated to special trim levels on several Renault models, specifically those sold to livery companies for use as limousines. The Walter P. Chrysler package becomes a popular choice in the black car business over the course of the next decade.
