0G Labs Achieves Breakthrough in Decentralized AI Training With 100 Billion+ Parameters
'DiLoCoX: A Low-Communication Large-Scale Training Framework for Decentralized Cluster' proposes training on slow networks using a low-communication, large-scale decentralized framework. By chaining a number of complementary technologies together, 0G has been able to overcome the shortfalls of incumbent decentralized models.
0G's researchers have demonstrated that DiLoCoX is capable of pre-training a 107B foundation model for the first time over a 1Gbps network. Its solution achieves 357x greater speeds in distributed training versus AllReduce while maintaining negligible degradation in model convergence. This achievement is believed to be the first decentralized training framework successfully applied to models with over 100 billion parameters.
In a field where centralized data centers dominate AI training due to their high-speed connectivity, DiLoCoX breaks new ground by enabling efficient, verifiable training on slower networks. The framework ingeniously combines Pipeline Parallelism, a Dual Optimizer Policy, One-Step-Delay Overlap of Communication and Local Training, and an Adaptive Gradient Compression Scheme. These innovations not only scale up model sizes dramatically but deliver significant speed improvements while preserving model convergence with minimal degradation.
In experiments detailed in the paper, 0G pre-trained a 107 billion parameter model – a scale roughly 10x larger than the recently released Intellect-1 model from PrimeIntellect – demonstrating its potential to democratize access to advanced AI infrastructure. This solution addresses key challenges in decentralized AI, including bandwidth constraints and the need for verifiable processing, setting the stage for more powerful yet decentralized AI development.
'DiLoCoX is both a proof of concept and a statement of intent,' said Michael Heinrich, CEO of 0G Labs. 'By making it possible to train enormous models in truly decentralized settings, we're not just pushing technical boundaries, but are unlocking a future where AI serves as a public good. This is about building an open ecosystem where anyone can contribute to and benefit from intelligent systems.'
The release of the paper supports 0G's commitment to advancing verifiable, democratized AI. As the foundation for high-performance infrastructure that unifies storage, compute, and data availability, 0G is on a mission to empower developers and researchers to create the next wave of AI-native applications.
The research paper can be read in full here.
About 0G
0G is the first decentralized AI protocol (AIP). A modular, infinitely scalable layer 1, 0G makes possible decentralized AI apps to bring about a truly democratized future of intelligence. Designed for AI execution at scale, 0G unifies decentralized storage, compute, and data availability (DA) to support the next generation of AI-native applications. With high-performance infrastructure, verifiable AI processing, and a permissionless agent ecosystem, 0G is building the foundation for an open, unstoppable AI economy.
Contact
CMO
0G Labs
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Yahoo
16 hours ago
- Yahoo
1&1's (ETR:1U1) Returns On Capital Not Reflecting Well On The Business
There are a few key trends to look for if we want to identify the next multi-bagger. One common approach is to try and find a company with returns on capital employed (ROCE) that are increasing, in conjunction with a growing amount of capital employed. This shows us that it's a compounding machine, able to continually reinvest its earnings back into the business and generate higher returns. Although, when we looked at 1&1 (ETR:1U1), it didn't seem to tick all of these boxes. AI is about to change healthcare. These 20 stocks are working on everything from early diagnostics to drug discovery. The best part - they are all under $10bn in marketcap - there is still time to get in early. Understanding Return On Capital Employed (ROCE) For those who don't know, ROCE is a measure of a company's yearly pre-tax profit (its return), relative to the capital employed in the business. The formula for this calculation on 1&1 is: Return on Capital Employed = Earnings Before Interest and Tax (EBIT) ÷ (Total Assets - Current Liabilities) 0.07 = €546m ÷ (€8.4b - €680m) (Based on the trailing twelve months to March 2025). Therefore, 1&1 has an ROCE of 7.0%. In absolute terms, that's a low return and it also under-performs the Wireless Telecom industry average of 9.0%. Check out our latest analysis for 1&1 Above you can see how the current ROCE for 1&1 compares to its prior returns on capital, but there's only so much you can tell from the past. If you're interested, you can view the analysts predictions in our free analyst report for 1&1 . The Trend Of ROCE On the surface, the trend of ROCE at 1&1 doesn't inspire confidence. To be more specific, ROCE has fallen from 13% over the last five years. Meanwhile, the business is utilizing more capital but this hasn't moved the needle much in terms of sales in the past 12 months, so this could reflect longer term investments. It's worth keeping an eye on the company's earnings from here on to see if these investments do end up contributing to the bottom line. The Bottom Line On 1&1's ROCE To conclude, we've found that 1&1 is reinvesting in the business, but returns have been falling. Since the stock has declined 16% over the last five years, investors may not be too optimistic on this trend improving either. All in all, the inherent trends aren't typical of multi-baggers, so if that's what you're after, we think you might have more luck elsewhere. One more thing to note, we've identified 2 warning signs with 1&1 and understanding these should be part of your investment process. While 1&1 isn't earning the highest return, check out this free list of companies that are earning high returns on equity with solid balance sheets. Have feedback on this article? Concerned about the content? Get in touch with us directly. Alternatively, email editorial-team (at) article by Simply Wall St is general in nature. We provide commentary based on historical data and analyst forecasts only using an unbiased methodology and our articles are not intended to be financial advice. It does not constitute a recommendation to buy or sell any stock, and does not take account of your objectives, or your financial situation. We aim to bring you long-term focused analysis driven by fundamental data. Note that our analysis may not factor in the latest price-sensitive company announcements or qualitative material. Simply Wall St has no position in any stocks mentioned. Error in retrieving data Sign in to access your portfolio Error in retrieving data Error in retrieving data Error in retrieving data Error in retrieving data
Yahoo
16 hours ago
- Yahoo
1&1's (ETR:1U1) Returns On Capital Not Reflecting Well On The Business
There are a few key trends to look for if we want to identify the next multi-bagger. One common approach is to try and find a company with returns on capital employed (ROCE) that are increasing, in conjunction with a growing amount of capital employed. This shows us that it's a compounding machine, able to continually reinvest its earnings back into the business and generate higher returns. Although, when we looked at 1&1 (ETR:1U1), it didn't seem to tick all of these boxes. AI is about to change healthcare. These 20 stocks are working on everything from early diagnostics to drug discovery. The best part - they are all under $10bn in marketcap - there is still time to get in early. Understanding Return On Capital Employed (ROCE) For those who don't know, ROCE is a measure of a company's yearly pre-tax profit (its return), relative to the capital employed in the business. The formula for this calculation on 1&1 is: Return on Capital Employed = Earnings Before Interest and Tax (EBIT) ÷ (Total Assets - Current Liabilities) 0.07 = €546m ÷ (€8.4b - €680m) (Based on the trailing twelve months to March 2025). Therefore, 1&1 has an ROCE of 7.0%. In absolute terms, that's a low return and it also under-performs the Wireless Telecom industry average of 9.0%. Check out our latest analysis for 1&1 Above you can see how the current ROCE for 1&1 compares to its prior returns on capital, but there's only so much you can tell from the past. If you're interested, you can view the analysts predictions in our free analyst report for 1&1 . The Trend Of ROCE On the surface, the trend of ROCE at 1&1 doesn't inspire confidence. To be more specific, ROCE has fallen from 13% over the last five years. Meanwhile, the business is utilizing more capital but this hasn't moved the needle much in terms of sales in the past 12 months, so this could reflect longer term investments. It's worth keeping an eye on the company's earnings from here on to see if these investments do end up contributing to the bottom line. The Bottom Line On 1&1's ROCE To conclude, we've found that 1&1 is reinvesting in the business, but returns have been falling. Since the stock has declined 16% over the last five years, investors may not be too optimistic on this trend improving either. All in all, the inherent trends aren't typical of multi-baggers, so if that's what you're after, we think you might have more luck elsewhere. One more thing to note, we've identified 2 warning signs with 1&1 and understanding these should be part of your investment process. While 1&1 isn't earning the highest return, check out this free list of companies that are earning high returns on equity with solid balance sheets. Have feedback on this article? Concerned about the content? Get in touch with us directly. Alternatively, email editorial-team (at) article by Simply Wall St is general in nature. We provide commentary based on historical data and analyst forecasts only using an unbiased methodology and our articles are not intended to be financial advice. It does not constitute a recommendation to buy or sell any stock, and does not take account of your objectives, or your financial situation. We aim to bring you long-term focused analysis driven by fundamental data. Note that our analysis may not factor in the latest price-sensitive company announcements or qualitative material. Simply Wall St has no position in any stocks mentioned.
Business Wire
2 days ago
- Business Wire
Caliber Announces Confidential Submission of Draft Registration Statement for Proposed Initial Public Offering
LEWISVILLE, Texas--(BUSINESS WIRE)--Caliber Holdings Inc. ('Caliber') today announced that it has confidentially submitted a draft registration statement on Form S-1 with the Securities and Exchange Commission (the 'SEC') relating to the proposed initial public offering of its common stock. The number of shares to be offered and the price range for the proposed offering have not yet been determined. Caliber expects to use the proceeds from the offering for general corporate purposes, which may include the repayment of indebtedness. This news release is being made pursuant to and in accordance with Rule 135 under the Securities Act of 1933, as amended, and does not constitute an offer to sell or the solicitation of an offer to buy securities, and shall not constitute an offer, solicitation or sale in any jurisdiction in which such offer, solicitation or sale would be unlawful prior to registration or qualification under the securities laws of that jurisdiction.
