Best printers for Home

Best Mixer Grinder

Best wired Earphones

Best 43 Inch TV in India

Best Wi Fi Routers

Best Home Theatre in India

Smart Watch under 5000

Best Laptops for Education

Best Laptop for Students

<ul><li><keyword id="8010226" type="UNKNOWN" weightage="20" keywordseo="cerebras-systems">Cerebras Systems</keyword> has launched a new transistor chip with 1.2 trillion transistors, 400,000 processing cores, and 18GB of on chip memory.</li><li>The California-based start up fit all of that on a single wafer measuring 46,225 square millimeters.</li><li>The main purpose of the chip is to remove the bottleneck of limited data processing in the training of <keyword id="107950" type="TECHNOLOGY" weightage="20" keywordseo="artificial-intelligence">artificial intelligence</keyword> (AI) algorithms.</li></ul><keyword id="8010227" type="UNKNOWN" weightage="20" keywordseo="transistor-chips">Transistor chips</keyword> are key to any computerised process, be it playing a game on your laptop or running <a href="https://www.businessinsider.in/accenture-number-one-on-idc-ai-rankings/articleshow/69986095.cms">artificial intelligence</a> (AI) algorithms. The biggest, baddest and largest of them all has finally made it into the market after three years. It took 173 engineers and $112 million venture capital funding from Benchmark, Foundation Capital, Eclipse and other companies, to make it. Cerebras Systems’ ‘<a href="https://www.cerebras.net/wp-content/uploads/2019/08/Cerebras-Wafer-Scale-Engine-Whitepaper.pdf">Wafer Scale Engine</a>’ as it is called has more cores, more memory and more power than any chip till date. The California-based startup is trying to pace the evolution of chips to the increasing needs of the tech industry. <div type="image" title="Angled_Chip_1400_edited" msid="70780720" hostid="261" caption="&amp;lt;p&amp;gt;&amp;lt;sub&amp;gt;&amp;lt;/sub&amp;gt;Wafer Scale Engine&amp;lt;/p&amp;gt;"></div>

 The <keyword id="8010225" type="UNKNOWN" weightage="20" keywordseo="wafer-scale-engine">Wafer Scale Engine</keyword> has the potential to solve one of the primary bottlenecks in the tech industry. Tech giants like <a href="https://www.businessinsider.in/yougov-report-says-google-is-the-top-brand-in-india-followed-by-whatsapp-and-youtube/articleshow/70752208.cms">Google</a>, <a href="https://www.businessinsider.in/facebook-india-partners-with-vc-funds-to-help-businesses/articleshow/70055762.cms">Facebook</a> and <a href="https://www.businessinsider.in/chinese-giant-tencent-is-in-talks-to-buy-a-3-4-billion-stake-in-ariana-grandes-music-label-universal-music-group/articleshow/70552858.cms">Tencent</a> have expressed that the biggest limitation with AI in today’s age is the fact that it takes too long to train AI models. A larger chip could change just that. Over a trillion transistors The Wafer Scale Engine has 1.2 trillion transistors — the highest number of transistors on any transistor chip so far. To put that in perspective, any laptop with <a href="https://www.businessinsider.in/miscellaneous/heres-what-all-those-numbers-and-letters-in-an-intel-chip-mean-when-youre-buying-a-new-computer/slidelist/65626149.cms">Intel Core i3, i5 or i7 processors</a> have about 1.5 million transistors. Even Advanced Micro Devices’ (AMD) 7 nanometer <a href="https://www.businessinsider.in/amds-latest-chip-reveal-is-one-of-the-biggest-turning-points-in-the-history-of-silicon-valley-one-analyst-says/articleshow/70593102.cms">Epyc “Rome” CPU </a>with 32 billion transistors, which was announced earlier this month, doesn’t even come close to matching this. In addition to the massive amount of transistors, the Wafer Scale Engine has 400,000 processing cores. Total recall The Wafer Scale Engine also has more memory than other chip in the market with 18GB — and it’s one of the key components of a computer’s architecture. More memory on-chip means faster calculation, lower latency, and better power efficiency as data moves from one place to the other. More impressively, all of this is on a single wafer that functions as one single chip instead of dividing a single wafers into individual chips — what is normal every chip until now. The reason it hasn’t been done till now is because there are multiple issues in trying to make it happen. One, the software to handle these chips has to be written from scratch. And two, since all of it is one wafer, there had to be a fail safe in place in case even a single core went down — rendering the entire flow inoperative. Cerebras overcame the laws of physics by simply building in redundancies that could backup in the event that one of the core’s stopped working. It’s not a cheap fix but it works, according to the company. Size does matter The Wafer Scale Engine has been specifically designed to process AI applications. And, when it comes to AI, chip size is important. Bigger chips can handle more information and produce answers in a shorter time space. The ‘training time’ that machine learning algorithms is reduced substantially. Researchers, in turn, can test a lot more ideas and feed in bigger chucks on data to solve their problems. The Wafer Scale Engine, in this regard, measures 46,225 square millimeters. 
<div type="image" title="chip-comparison_edited" msid="70780760" hostid="261"></div>

 Nvidia — known as a market leader when it comes to graphic processing units (GPUs) — only offers 815 square millimeters and 21.1 billion transistors on its largest processor. That means the Wafer Scale Engine is 56.7 times larger, 3,000 times faster and has 10,000 times more memory bandwidth. See also: <a href="https://www.businessinsider.in/amds-latest-chip-reveal-is-one-of-the-biggest-turning-points-in-the-history-of-silicon-valley-one-analyst-says/articleshow/70593102.cms">AMD's latest chip reveal is 'one of the biggest turning points in the history of Silicon Valley,' one analyst says</a> <a href="https://www.businessinsider.in/qualcomms-new-chipset-will-change-the-way-you-drive/articleshow/28955186.cms">Qualcomm’s New Chipset Will Change The Way You Drive</a>

The biggest, baddest and largest of all transistor chips found a loophole to overcome the laws of physics

<ul class="summary-list"><li>All kinds of startups are rushing into the AI inference market.</li><li>Inference market competition may lower the price of AI, benefiting builders but challenging clouds.</li></ul>Jared Quincy Davis and his AI computing startup Foundry sell <a target="_blank" class="" href="https://www.businessinsider.com/nvidia-competition-sambanova-groq-cerebras-show-faster-inference-speed-2024-9">inference.</a> They don't make chips or build <a target="_blank" class="" href="https://www.businessinsider.com/openai-anthropic-ai-bots-havoc-raise-cloud-costs-websites-2024-9">large language models</a>. Foundry has a unique method of making cloud computing more efficient. Instead of selling its technology to cloud providers, the Foundry team decided to become one and use its tech to operate a more efficient cloud.Once companies looking to leverage and sell an AI product have trained their models and know that they perform, they're looking for ease, speed, and value whenever generating outputs. Inference-as-a-service providers like Foundry, <a target="_blank" class="" href="https://www.businessinsider.com/foundry-ceo-hack-improving-ai-models-gpt-5-2024-8">aim to simplify</a> the process of generating those outputs.Foundry offers training and fine-tuning, too, as many cloud providers do, but these days, it seems like anyone with an AI compute-boosting technology is attempting to monetize by selling inference — or more specifically, tokens, the base unit of data in AI.<a target="_blank" class="" href="https://www.businessinsider.com/cerebras-nvidia-challenger-ai-chip-battle-2024-7">Cerebras</a> sells inference too. The company's core expertise is designing chips for training and inference, but it recently started selling the latter as a service. So does Groq, a chip company formed by two former Googlers, who recognized early that inference was going to get the bigger share of computing. <a target="_blank" class="" href="https://www.businessinsider.com/nvidia-competition-sambanova-groq-cerebras-show-faster-inference-speed-2024-9">SambaNova Systems</a>, another hardware platform, also sells inference as a service.Companies like Lambda, CoreWeave, Together AI, and Crusoe, all close partners of Nvidia, run data centers suited specifically to AI workloads and offer inference services. And then there are the hyperscalers like AWS and Microsoft Azure.With so many companies specializing in inference, suspicion is rising that the cost of inference is about to drop off a cliff."Part of the reason inference is a little commoditizable is customers are kind of paying for tokens at the end of the day," Davis told Business Insider.The current market for inference is kind of like the electricity market, Davis said. There are a ton of niche sources you can access if you actually shop around, but not everyone does. Most people just want to flip the light switch and have it work.But there is a lot of nuance to sift through for those willing. For some customers, <a target="_blank" class="" href="https://www.businessinsider.com/nvidia-competition-sambanova-groq-cerebras-show-faster-inference-speed-2024-9">speed is of the utmost importance</a>. Speed has distinctions too, like time to the first token and tokens per second. There's total job completion time and there are different kinds of inference workloads that lend themselves to different computing setups.Energy efficiency of the underlying hardware and networking is a big determinant of cost. And cost in inference computing is even more important than in training, Groq cofounder Jonathan Ross recently told BI. Training is an overhead cost, while inference is an operating cost.Zoom out from all of the intricacies, and inference is becoming the commodity of the AI age."Some companies just want output and they don't care about infrastructure," Mitesh Agrawal, head of cloud for Lambda, told BI.<h2 id="dff58147-0a83-484f-8489-a31c08d0a473">Commoditizing AI</h2>Lambda is in the early stages of an inference-as-a-service offering, but Agrawal said the company is going about it carefully, focusing on providing holistic computing services, and not just tokens.Inference profit margins can vary widely, Agrawal said. With general compute — where the customer rents fixed capacity — the margins are easier to manage. When you're charging for usage or input and output of a model, the return is less predictable.Organizing multiple users across a finite number of servers takes finesse. Whether or not the cost of operating the hardware is actually covered with room for profit comes down to how well that organization is done, Agrawal explained.So why would neoclouds offer the riskier service?Agrawal said it's about getting potential customers in the door. Inference-as-a-service customers can turn into more traditional compute customers, and as the slate of competitors grows, relationships, and history grow in importance.Lambda's financial models assume that price cuts are coming soon as more players enter the inference space and chips become more efficient.<h2 id="59b576b3-4703-4b08-b315-2c7f48e230fb">A race to the bottom?</h2>How fast the demand for inference is growing is up for debate, but in recent public statements, Nvidia CEO Jensen Huang has said on multiple occasions that new models, like OpenAI's o1, require more compute to generate the same number of responses because they run multiple models to check their own work or "reason." Accuracy, it turns out, requires more compute.Inference loads are poised to grow, but service providers still anticipate a drop in price from the influx of new players. Davis isn't worried though.He recalled Jevon's paradox — an economic principle in which a drop in price or an increase in efficiency leads to more total consumption — like when you widen a highway and traffic gets worse."If I make something 10 times cheaper, people won't spend 10 times less, nor will they even hold their budgets the same. They'll spend more," Davis said. "That makes sense because what are you doing when you make something 10 times cheaper, you're making the ROI better."In other words, "it turns out, when you make inference cheaper, people decide to do a lot more inference," Davis said.The ride ahead could be "bumpy" though, and not all players are likely to survive the moments of mismatch between supply and demand."As my old boss at Intel Andy Grove used to say, 'Let chaos reign, and then reign in the chaos'," said Sriram Viswanathan, founding managing partner at Celesta Capital and investor in SambaNova Systems.He agrees the next few years will be wildly competitive for inference providers, but he believes the winners will be decided on merit."The core innovation can't be in the go-to-market, but in the performance and power of the underlying architecture," Viswanathan said.Many of the companies selling tokens to break into the AI market aspire to more. The chip designers eventually want to sell chips to hyperscalers rather than inference to AI startups. The ultimate version of Foundry's tech is bigger too."If we do our job, right, you know, we will be a core part of how every GPU runs," Davis said. All roads, it seems, run through inference.Hugh Langley contributed reporting.Got a tip or an insight to share? Contact Senior Reporter Emma Cosgrove at <a target="_blank" href="mailto:ecosgrove@businessinsider.com">ecosgrove@businessinsider.com</a> or use the secure messaging app Signal: 443-333-9088

'Let chaos reign': AI inference costs are about to plummet  

<ul class="summary-list"><li>NEX creates "controllable and IP-consistent" image-generation models for creative tasks.</li><li>Foundation Capital led the company's $3.5 million seed round.</li></ul>Creating an advertising campaign is no easy feat. "It's a whole operation," said NEX CEO Darius Lam. To pull off a campaign, major brands fly celebrities and athletes to a studio, hire professional photographers, and shoot images with the product they're marketing, whether it's jewelry or a handbag, said Lam. "Gathering a set of images might cost thousands or tens of thousands of dollars."Lam aims to change that with NEX, which develops AI image tools for entertainment, gaming, and marketing. Founded in late 2023, the company recently raised a $3.5 million seed round led by Foundation Capital, with participation from Point72 Ventures.NEX creates "controllable and IP-consistent" image-generation models for creative tasks, according to its website. Instead of time-intensive and costly photoshoots, customers can automatically generate images for a fraction of the cost, Lam said.NEX does this by enabling users to generate images inspired by other images, not just text inputs, in a "real-time canvas." The platform gives users a high degree of control by enabling enhancements to image quality and adjustments to color, pose, or depth in the images.<div type="image" title="Set of three images generated by NEX model" height="1211" width="3431" msid="112834896" hostid="261" sizebytes="434608"></div>Additionally, NEX can integrate with customers' data to generate content consistent with their IP. The startup's tech can also tailor the AI models based on a customer's existing data, a process called finetuning, or even develop models from scratch. For example, "if you are a film studio that's been operating for decades and you have access to huge amounts of data," said Lam. "Then we could effectively build the model from scratch and it would be completely IP safe."NEX initially took a consumer-focused approach, attracting nearly 100,000 users within a few months of launching. But the company has since pivoted to targeting business users, selling to small businesses and enterprise customers. The startup now generates revenue by charging up to $50 a month per user for access to its platform and through fees for model development and maintenance.While models like OpenAI's Dall-E and consumer-focused Midjourney are popular, many don't offer the level of control and customization that design professionals require, said Lam. Some customers may seek to develop in-house models on top of open-sourced ones but it ends up being very expensive, he added.Setting up and onboarding machine learning engineers, GPUs, and infrastructure is very difficult and costly. "It's much easier to work with a provider like NEX that not only is building the models but can integrate the models directly with applications that their creative teams can use."Other visual AI companies include BRIA, which raised a $24 million Series A in March and targets enterprise users, and Cuebric, an early-stage company focused on the film production process.Lam leads a small team of four employees at the young startup. Before founding the company, he was the head of product for computer vision at Cerebras Systems, which develops AI chips and confidentially filed for an IPO earlier this year. While companies like Adobe and Autodesk thrived in the internet age, Lam expects AI to completely upend the software landscape for creative tasks. This transformation will extend to areas like 3D design for game development, product design for software development, and visual creation for entertainment. Lam's vision for NEX is to be the "visual media tool for the AI era."

Visual AI startup NEX raises $3.5 million seed round led by Foundation Capital  

Chhath Puja 2024 Wishes

Bank holidays in India

ChatGPT will let you create images for free

Should the government give freebies to the poor?

World’s most powerful passports

Jio vs Airtel vs Vi Prepaid Plans

You Searched For "cerebras systems" and got 3 results

The biggest, baddest and largest of all transistor chips found a loophole to overcome the laws of physics

Prabhjote Gill

'Let chaos reign': AI inference costs are about to plummet

Emma Cosgrove

Visual AI startup NEX raises $3.5 million seed round led by Foundation Capital

Sri Muppidi

Popular Keywords

Buying Guides