Latest news with #Mellanox
Forbes
11-07-2025
- Business
- Forbes
New Fabrics Enable Efficient AI Acceleration
Photo by Marijan Murat/picture alliance via Getty Images While GPU performance has been the focus in data centers over the last few years, the performance of fabrics has become a key enabler or bottleneck in achieving the throughput and latency required to create and deliver artificial intelligence at scale. Nvidia's prescient acquisition of Mellanox has been a critical component of its success over the last few years, enabling scalable AI and HPC performance. However, it's not just scale-up (in-rack) performance and scale-out (rack-to-rack) connectivity; latencies in scale-within network-on-chip (NoC) have also become essential for achieving high AI throughput and improved response times. The Importance of Advanced Fabric Solutions The computing landscape has undergone significant changes with the advent of artificial intelligence, evolving from a loosely coupled network of independent computers to a highly integrated fabric of collaborating, accelerated computing nodes. Three levels of scale require such interconnects: the chip/chiplet, rack, and data center. Each compute element must share data with its neighbors and beyond over a low-latency, high-bandwidth communication channel to maximize performance and minimize latency. Scale-Within Fabrics On-chip fabrics connect processor cores, accelerators, and cache memory within a single or multi-chip module. As SoCs become more complex, integrating tens or even hundreds of cores or IP blocks, a single NoC often cannot provide the required bandwidth and scalability. Multiple NoCs, or subnetworks, are used to manage traffic between chiplets, each potentially optimized for specific data types or communication patterns. For example, one NoC might handle high-bandwidth data transfers between compute chiplets, while another manages control signals or memory access. As chiplet-based designs gain wider adoption, these NoCs become the bottleneck of chiplet-to-chiplet communication and data sharing. A unified fabric significantly enhances latency and bandwidth in chiplet-based systems by streamlining communication across fragmented networks-on-chip (NoCs) and optimizing physical interconnects. Such a fabric can help minimize hops, improve routing, enable a higher degree of scaling, and manage congestion. More importantly, it can provide improvement in performance and reduce footprint and power through reuse of wires and logic, in a segment where every saving or every extra ounce of performance is treasured. At the chip level, the networks on chips (SoCs) tend to be isolated; they are designed to connect a specific domain on the chip, which works great until you need to move data to another domain, creating a latency-inducing "hop" or carry overheads of working across different protocols. A unified network on chip (NoC), such as that provided by Baya Systems (a client of Cambrian-AI Research), provides a single transport mechanism for the various protocols for each fabric. Transport is separate from protocol layers, minimizing wires and logic in building a unified fabric that supports coherent, non-coherent, and custom protocols for maximum efficiency, lowest cost, and reduced power consumption. The various on-chip networks tend to be distinct, but could be unified with the right technologies. On-chip fabrics connect processor cores, accelerators, and cache memory within a single or multi-chip module. As SoCs become more complex, integrating tens or even hundreds of cores or IP blocks, a single NoC often cannot provide the required bandwidth and scalability. Multiple NoCs, or subnetworks, are used to manage traffic between chiplets, each potentially optimized for specific data types or communication patterns. For example, one NoC might handle high-bandwidth data transfers between compute chiplets, while another manages control signals or memory access. As chiplet-based designs gain wider adoption, these NoCs become the bottleneck of chiplet-to-chiplet communication and data sharing. Scale-Up Fabrics Scale-up fabrics connect accelerators (GPUs, AI processors) within a single rack or AI pod, prioritizing ultra-low latency and high bandwidth communication. Scaling up with NVLink has been the go-to standard, but the industry needs an open alternative, such as UALink, to interconnect accelerators from other vendors. UALink and Ultra Ethernet solve different problems in the data center. UALink is a memory-semantic interconnect standard led by the UALink Consortium, enabling accelerators to share memory directly. Its four-layer protocol stack supports single-stage switching, reducing latency and congestion. UALink will deliver up to 200 Gbps per lane and memory-sharing capabilities to scale (up) accelerator connectivity. The Consortium recently approved the V1.0 Specification of UALink in April 2025, and the first silicon is expected to be available later this year, with volume production scheduled for 2026. Scale-Out Fabrics Scale-out fabrics interconnect multiple racks or pods, enabling the distribution of workloads across larger clusters, or more often running a lot more 'copies' of the workloads thereby increasing services to more clients. Nvidia offers both Ethernet and InfiniBand networking to connect racks for east-west traffic. As for scale-out alternatives, the industry is standardizing a high-bandwidth open networking protocol called Ultra Ethernet tailored for AI workloads across as many as 1 million heterogeneous nodes. Ultra Ethernet IP solution will enable 1.6 Tbps of bandwidth for scaling (out) massive AI networks. UALink will deliver up to 200 Gbps per lane and memory-sharing capabilities to scale (up) accelerator connectivity. Companies in the Fabric IP Business Historically, fabrics have been proprietary and come from companies like Nvidia, AMD, and Intel. For Arm provides the CoreLink NIC-301 and related interconnect IP, widely used in Arm-based SoCs for scalable, configurable on-chip interconects. While Arm's fabric is really designed for Arm CPU SoCs, Baya Systems and Arteris provide fabric IP for many implementations, including RISC-V and custom accelerators. And Baya is unique in its chiplet-first focus and the ability to scale out and scale up, while Arteris is recognized as a leader in providing what we have been referring to as Scale-Within fabric NoCs and SoC integration automation software to speed the development of complex SoCs. Arteris went public in October 2021 (Nasdaq: AIP), with a market cap of approximately $300 million as of mid-2025. Arteris has over 200 customers such as Samsung, AMD, Qualcomm, Baidu, Mobileye, and NXP, with an installed base of nearly four billion devices. Arteris IP is broadly deployed across the automotive segment (notably ADAS, with >70% market share), communications, consumer electronics, enterprise computing, and industrial markets. Arteris' products include the FlexNoC Interconnect with its integrated physical awareness technology, gives place and route teams a much better starting point while simultaneously reducing interconnect area and power consumption. Arteris claims that the FlexNoC delivers up to 5X shorter turn-around-time versus manual physical iterations. Ncore IP is similar, but is designed for multi-core cache-coherent designs. As we have noted, the AI transformation has driven the need for scale-up and scale out, and has also put a lot of demands on scale-within. In addition, the market perceived a gap emerging that wasn't readily solved by off the shelf, scale-within IP. The market transition to chiplets which offer a promise of greater scale and cost effectiveness has different demands on a more agile data-driven design philosophy to handle the complexity of the new systems. This is exactly what Baya Systems, a relatively new entrant aims to solve and it has been gaining a great deal of traction since it came out of stealth a year ago. Baya Systems (a client of Cambrian-AI Research) is a Silicon Valley startup with strong backing and leadership that has architected a semiconductor IP and software portfolio to enable designers of SoCs, systems, and data center scale infrastructure to build high-performance AI technology quickly and efficiently. Baya Systems chiplet-first fabrics are designed to address both on-chip, scale-up, and cross-system (scale-out) networking challenges. Its flexibility and modularity position it for broader applications, potentially integrating various processing units and accelerating communication in diverse, high-performance environments. The Baya Systems fabric supports multiple protocols, including AMBA, UCIe, UALink, and UltraEthernet. Baya Systems has created a comprehensive fabric that supports popular protocols for scale-within, ... More scale-out and scale-up. Tenstorrent, an AI chipmaker considered an emerging challenger to Nvidia, recently released a white paper demonstrating how Baya's fabric substantially boosts performance by up to 66% while reducing footprint by 50% compared to their home-grown state-of-the-art custom fabric. Tenstorrent is led by legendary computer architect Jim Keller, who is also a backer of Baya Systems, and sits on their board. Beyond NoCs, Baya NeuraScale offers a scalable fabric solution based on the company's WeaveIP technology, providing a non-blocking cross-bar replacement fabric that is designed to power switches for UALink or UltraEthernet standards in emerging scale-up and scale-out systems. The unique approach of using a "mesh"-based, tileable architecture that simplifies chiplet-based scaling, opens the path to much larger accelerator node counts compared to traditional crossbar switches, which are hitting reticle limits. This could enable 144-port or even 288-port racks, compared to today's 72-port ones, substantially expanding scale. Interestingly the company claims that the technology could enable much larger node counts beyond this once the industry adopts this. But what makes this additionally disruptive is that NeuraScale can substantially reduce the resources, time, and cost required to build these high-performance switches, thereby enabling smaller, nimble entrants to broaden and scale the market. The WeaveIP NeuraScale Fabric Fabrics Will Enable The Future Of AI The modern data center is evolving rapidly, both in its compute elements (chiplets, chips, CPUs, GPUs) and in fabrics, to enable these systems to scale to hundreds of thousands of nodes and support AI. While Nvidia's new NVLink Fusion will allow non-Nvidia CPUs and GPUs to participate in the Nvidia rack-scale architecture, hardware vendors and hyperscalers will continue to seek an open fabric alternative to an ecosystem controlled by a single firm. Consequently, we envision a significant increase in these heterogeneous fabric technologies as AMD, Intel, and hyperscalers adopt them to build out their own AI Factories, both with and without Nvidia hardware. Fabrics like that of Baya Systems represent a key enabler in that evolution. We have a more in-depth report on Baya Systems here. And more information about Arteris can be found on their website.
CNA
09-07-2025
- Business
- CNA
Planned Nvidia expansion in Israel prompts multiple offers of sites
JERUSALEM :Nvidia has received a high number of offers of potential sites to help it carry out a plan to greatly expand its operations in Israel to meet growing demand for artificial intelligence data centres, two sources told Reuters. The Santa Clara-based Nvidia, which has become the most valuable company in history at $4 trillion, earlier this week issued a request for information, or RFI, to buy land to build a new campus near its facility in northern Israel that industry sources estimated would cost billions of dollars and create thousands of jobs. Nvidia, a leading designer of high-end AI chips, entered Israel in 2020 after buying Mellanox Technologies for nearly $7 billion. It is located in Yokne'am, where many tech companies are based, near the northern port city of Haifa. Nvidia in Israel declined to comment beyond its RFI. A third source, speaking on condition of anonymity because they were not authorised to speak to the press, said the company received "dozens and dozens and dozens" of offers from municipalities and others, not all near Haifa. Nvidia has set a July 23 deadline for offers to build its campus of up to 180,000 square metres. For its part, the Haifa municipality said it was "currently busy preparing an attractive offer for the company. We think we are the city with the best potential for them." A race among Microsoft,, Meta Platforms, Alphabet, and Tesla, to build AI data centres and dominate the emerging technology has led to a surge in demand for Nvidia's high-end processors. One of the sources said Israel's expertise was "extremely important to the AI era" and Nvidia needed to expand rapidly. The company has already nearly tripled in size in Israel since its acquisition of Mellanox, which a source said contributed to $13 billion in revenue to Nvidia last year. The company has not confirmed the figure. Nvidia has also made a number of other acquisitions over recent years in the country where it has 5,000 employees. It has also built Israel's most powerful AI supercomputer that was a blueprint for Elon Musk's Colossus supercomputer. Dror Bin, CEO of the Israel Innovation Authority, said the new Nvidia campus will be massive and could house "a few thousand employees". "Nvidia sees its operation in Israel as something which is going to stay here for a very long time and to expand here," he told Reuters. "This declaration is a sign of confidence in Israel." Nvidia's planned expansion in Israel comes as rival Intel - in Israel since 1974, and one of the country's largest employers at 9,350 - has begun to trim its workforce globally. Israel media said a few hundred workers in Israel are being made redundant.
Yahoo
09-07-2025
- Business
- Yahoo
Planned Nvidia expansion in Israel prompts multiple offers of sites
By Steven Scheer JERUSALEM (Reuters) -Nvidia has received a high number of offers of potential sites to help it carry out a plan to greatly expand its operations in Israel to meet growing demand for artificial intelligence data centres, two sources told Reuters. The Santa Clara-based Nvidia, which has become the most valuable company in history at $4 trillion, earlier this week issued a request for information, or RFI, to buy land to build a new campus near its facility in northern Israel that industry sources estimated would cost billions of dollars and create thousands of jobs. Nvidia, a leading designer of high-end AI chips, entered Israel in 2020 after buying Mellanox Technologies for nearly $7 billion. It is located in Yokne'am, where many tech companies are based, near the northern port city of Haifa. Nvidia in Israel declined to comment beyond its RFI. A third source, speaking on condition of anonymity because they were not authorised to speak to the press, said the company received "dozens and dozens and dozens" of offers from municipalities and others, not all near Haifa. Nvidia has set a July 23 deadline for offers to build its campus of up to 180,000 square metres. For its part, the Haifa municipality said it was "currently busy preparing an attractive offer for the company. We think we are the city with the best potential for them." A race among Microsoft,, Meta Platforms, Alphabet, and Tesla, to build AI data centres and dominate the emerging technology has led to a surge in demand for Nvidia's high-end processors. One of the sources said Israel's expertise was "extremely important to the AI era" and Nvidia needed to expand rapidly. The company has already nearly tripled in size in Israel since its acquisition of Mellanox, which a source said contributed to $13 billion in revenue to Nvidia last year. The company has not confirmed the figure. Nvidia has also made a number of other acquisitions over recent years in the country where it has 5,000 employees. It has also built Israel's most powerful AI supercomputer that was a blueprint for Elon Musk's Colossus supercomputer. Dror Bin, CEO of the Israel Innovation Authority, said the new Nvidia campus will be massive and could house "a few thousand employees". "Nvidia sees its operation in Israel as something which is going to stay here for a very long time and to expand here," he told Reuters. "This declaration is a sign of confidence in Israel." Nvidia's planned expansion in Israel comes as rival Intel - in Israel since 1974, and one of the country's largest employers at 9,350 - has begun to trim its workforce globally. Israel media said a few hundred workers in Israel are being made redundant. A local spokesperson would not comment on numbers, only pointing to Intel CEO Lip-Bu Tan's comments in April that the company was "taking steps to become a leaner, faster and more efficient company".
Forbes
30-06-2025
- Business
- Forbes
The AI Network Is The Computer, Says Nvidia
Nvidia co-founder and CEO Jensen Huang delivers the first keynote speech of Computex 2025 at the ... More Taipei Music Center in Taipei on May 19, 2025. (Photo by I-Hwa Cheng / AFP) Nvidia just reclaimed its title as the world's most valuable company. Whether it retains this top position and for how long depends on its success in defining and developing a worldwide network of AI processing units. Nvidia is pursuing a vision of a future where 'part of the application runs in the data center, another part in a data center at the edge, and another part in an autonomous machine roaming around the world.' This is how Jensen Huang, Nvidia's co-founder and CEO, described the future of computer applications in a conversation with Bob Metcalfe, inventor of the Ethernet. Huang and Metcalfe are prime examples of the remarkable marriage of engineering ingenuity and marketing creativity that has made many American entrepreneurs successful. Already five years ago, Huang saw the data center as 'a composable disaggregated infrastructure,' where the critical path is the interaction of one 'computing node' with another 'computing node' over the Ethernet network. In response, Metcalfe asked, 'Is this why you bought Mellanox?' and Huang answered, 'It is exactly the reason why I bought Mellanox,' adding a great insight: 'Understanding the direction of software inspires you about what's the best way to design and evolve hardware.' In other words, anticipating how applications will be developed and run in the future, Nvidia has added to its portfolio (developing in-house or acquiring) new hardware elements so that it can offer its customers faster, more efficient, more resilient, and less expensive shuttling of data inside and outside the data center. Founded in Israel in 1999, Mellanox initially focused on developing computer networking products based on the then-new InfiniBand standard. These products featured high throughput and low latency, ensuring fast data movement between one 'computing node' and another. Mellanox later added networking products based on the Ethernet standard and was acquired by Nvidia for $6.9 billion in 2019. Kevin Deierling, the first Mellanox employee in the U.S., is now Nvidia's senior vice president of networking. Nvidia's networking division develops and sells the Spectrum-X networking platform, which the company calls 'the world's first Ethernet networking platform for AI.' Deierling explains that the unique nature of data processing for AI makes the capabilities of the network critical. Cloud computing serves millions of users, each transferring a small amount of data, and that data is completely unsynchronized. In contrast, AI—and Nvidia's processing units, or GPUs—do things in parallel. 'With AI workloads,' says Deierling, 'we have enormous, what we call elephant [data] flows, that are synchronized.' Each of the vast number of AI computing nodes operates on its part of the data and then shares all the data it's processed with the other nodes. 'That ends up being extremely bursty traffic,' observes Deierling. The second trend driving the need for Spectrum-X's capabilities is the shift in the focus of AI projects. Until recently, AI work mainly involved 'training,' feeding an AI model vast amounts of data to learn patterns and relationships. Enterprises are now moving to 'inference,' or using the trained model to process new data, make predictions, or take action. With inferencing, many customers share the same network infrastructure, increasing performance expectations and requirements. The Spectrum-X platform answers these, bringing InfiniBand's high-performance bandwidth and latency specifications to Ethernet. The significant benefit of using Ethernet for connecting all the components of the AI infrastructure—the data storage unit, the network moving the data, and the data processing units or GPUs—is that it is a widely deployed standard familiar to the many customers now investing in AI. Spectrum-X 'uses standard Ethernet protocols,' says Deierling, 'but it does things under the hood that make it extremely high performance. The largest AI supercomputer in the world today is based on our Spectrum-X platform.' The faster and more efficient data movement in the data center implies increased profits for the service provider. 'If you're offering an AI service, you're extremely interested in the performance per dollar and the performance per watt of the data center,' says Deierling. In addition, Spectrum-X allows the data center to offer a customized service, adjusting the network's performance based on the varying needs of different end-users and, of course, on what they pay. Deierling reports that enterprises are rapidly adopting AI agents, adapting them by adding their proprietary data to a model trained on what's found on the internet. Especially in the context of AI research agents, that's a sure way to reduce AI 'hallucinations' and comply with regulations. 'The next wave we're starting to see is physical AI, edge applications, and robotics,' says Deierling, with the Ethernet connecting everything from the cloud to enterprise data centers to mobile and stationary sensors. 'The Network is the Computer' was the 1984 tag line for Sun Microsystems, a maker of 'workstations,' or networked desktop computers. Nvidia's founders played together flight simulator and 'theorized that the killer app would be virtual reality, video games, and 3D games,' Huang told Metcalfe, and that 'everybody would want to be a gamer.' Four decades later, with AI constituting 'a new way of writing software,' everybody would want to be a coder, writing applications for the composable disaggregated infrastructure developed and maintained by Nvidia and its partners. 'We found ourselves at the right place at the right time. Part insight, part strategy, part serendipity,' said Huang.
