SMI CEO claims Nvidia wants SSDs with 100 million IOPS — up to 33X performance uplift could eliminate AI GPU bottlenecks
Now that the AI industry has exceptionally high-performance GPUs with high-bandwidth memory (HBM), one of the bottlenecks that AI training and inference systems face is storage performance. To that end, Nvidia is working with partners to build SSDs that can hit random read performance of 100 million input/output operations per second (IOPS) in small-block workloads, according to Wallace C. Kuo, who spoke with Tom's Hardware in an exclusive interview.
"Right now, they are aiming for 100 million IOPS — which is huge," Kuo told Tom's Hardware.
Modern AI accelerators, such as Nvidia's B200, feature HBM3E memory bandwidth of around 8 TB/s, which significantly exceeds the capabilities of modern storage subsystems in both overall throughput and latency. Modern PCIe 5.0 x4 SSDs top at around 14.5 GB/s and deliver 2 – 3 million IOPS for both 4K and 512B random reads. Although 4K blocks are better suited for bandwidth, AI models typically perform small, random fetches, which makes 512B blocks a better fit for their latency-sensitive patterns. However, increasing the number of I/O operations per second by 33 times is hard, given the limitations of both SSD controllers and NAND memory.
In fact, Kioxia is already working on an 'AI SSD' based on its XL-Flash memory designed to surpass 10 million 512K IOPS. The company currently plans to release this drive during the second half of next year, possibly to coincide with the rollout of Nvidia's Vera Rubin platform. To get to 100 million IOPS, one might use multiple 'AI SSDs.'
However, the head of SMI believes that achieving 100 million IOPS on a single drive featuring conventional NAND with decent cost and power consumption will be extremely hard, so a new type of memory might be needed.
"I believe they are looking for a media change," said Kuo. "Optane was supposed to be the ideal solution, but it is gone now. Kioxia is trying to bring XL-NAND and improve its performance. SanDisk is trying to introduce High Bandwidth Flash (HBF), but honestly, I don't really believe in it. Right now, everyone is promoting their own technology, but the industry really needs something fundamentally new. Otherwise, it will be very hard to achieve 100 million IOPS and still be cost-effective."
Currently, many companies, including Micron and SanDisk, are developing new types of non-volatile memory. However, when these new types of memory will be commercially viable is something that even the head of Silicon Motion is not sure about.
Follow Tom's Hardware on Google News to get our up-to-date news, analysis, and reviews in your feeds. Make sure to click the Follow button.
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Yahoo
25 minutes ago
- Yahoo
ESPN, NFL Media deal raises real revenue questions for NFLPA
The massive deal between the NFL and ESPN regarding various NFL Media assets raises a wide range of questions. For NFL players, it raises one very important one. What's in it for us? The league reportedly will get equity in the deal, receiving up to 10 percent of ESPN. How will that translate into payment for players? Long-time agent Peter Schaffer flagged the issue, in a Saturday text message to PFT. "Agents and players see major revenue potential in the ESPN–NFL deal, but there's real concern the league structured it to conceal money from the player pool," as it relates to, for example, the "value of game packages." Among other things, ESPN will get access to the seven regular-season games that the NFL had retained and televised via NFL Network. With the league reportedly getting ESPN ownership in lieu of cash, what does that mean for the broader revenue split? The players currently receive roughly half of the cash that flows through the overall cash register. With the league getting paper ownership and not paper money from the ESPN deal, what will the players receive via a potential bump to their collective pay? "This could artificially depress the salary cap, directly cutting into player contracts and earnings," Schaffer said. "The NFLPA needs to stay on top of this and demand full transparency." Because the deal has yet to be finalized, it's premature for the union to do anything. When the deal is done, the union will have the right to fully explore the details of the arrangement and, if necessary, to challenge it. Of course, the first order of business for the NFLPA is to appoint a new leader. And the new leader will have plenty of work to do. The ESPN-NFL deal adds another important item to the list: Figuring out how the players will get their fair share of whatever tangible benefits the NFL will receive through its unprecedented partnership with a media conglomerate.
Yahoo
25 minutes ago
- Yahoo
Shopping lists out, tax savings on: Florida's Back-to-School Holiday draws crowds
For the first time, Florida's Back-to-School Sales Tax Holiday runs all August, helping families save on supplies, clothes, and technology.
The Verge
27 minutes ago
- The Verge
Today I'm toying with
Skip to main content The simple joy of gadgets — that's what 'Today I'm Toying With' is all about. It's our video series where we try to encapsulate that joy of playing with technology, sharing what it's like to experience gadgets that not all of us get to touch! Like a 3D printer that prints delicious chocolate, or the realized dream of a Transformers toy that actually transforms into a walking, talking robot, or a dual-screen handheld game system like nothing the world has ever seen. Sometimes even simple, handy stuff you might not be aware of, like USB-C cables with tricks up their sleeves, or delightful novelties like tiny TVs. We're always looking for new gadgets to toy with... particularly exclusive first-looks! If you've got a worthy product, hit me up at sean@ with 'Today I'm toying with' in your subject line.
