Google increases its internal chip efforts
Google is expanding its chip production, introducing new hardware capable of managing various tasks such as YouTube advertising and big data analysis. This move is aimed at addressing the increasing costs associated with artificial intelligence.
Google’s latest chip, Axion, is a testament to their long-standing commitment to advancing computing resources. This dedication began over a decade ago with the development of specialized chips for AI applications. Google has embraced that strategy ever since the late 2022 launch of ChatGPT sparked a competition that has posed a challenge to its dominant role as an internet gateway.
According to analysts, the chip efforts have the potential to decrease Google’s dependence on external vendors and put it in direct competition with its long-standing partners like Intel and Nvidia. Google officials stated that they did not perceive it as a competition.
In the opinion of Amin Vahdat, the Google vice president in charge of the company’s in-house chip operations, this could serve as a foundation for expanding the size of the pie.
Google’s major rivals in the cloud industry, Amazon.com and Microsoft, have also invested heavily in developing their own chips to meet the growing demand for computing resources driven by the AI boom.
Google’s early success was greatly attributed to its investment in the chips required to power the company’s web search algorithm. That frequently involved assembling inexpensive, readily available hardware in innovative ways.
Due to the surge in AI and the increased demand for computing resources, Google has been compelled to lean more towards tailored solutions. It attributes the cost-saving benefits of its specialized AI chips, called tensor processing units (TPUs), to services that heavily rely on AI.
Google has collaborated extensively with semiconductor company Broadcom since 2016 to develop custom hardware.
During a March internal presentation, Chief Executive Hock Tan mentioned that Broadcom’s custom-chip division experienced a significant boost in business. This was a result of Google’s rapid increase in production of TPUs. According to him, the rise was partially due to Microsoft’s integration of AI capabilities into its Bing search engine, directly competing with Google’s main business.
According to Tan, the majority of the Broadcom division’s operating profit in a recent quarter was derived from Google’s business, amounting to over $1 billion. No response was received from Broadcom when asked for comment.
In January, Google Chief Financial Officer Ruth Porat informed investors that there would be a significant increase in spending on technical infrastructure, particularly on AI chips, this year. Alphabet’s parent company saw a significant increase in fourth-quarter capital expenditures, reaching $11 billion, which is almost 50% higher compared to the previous year.
Axion chips are highly versatile and can be used for a variety of tasks, such as powering Google’s search engine and handling AI-related work. According to Google officials, economists can play a crucial role in AI by efficiently managing vast amounts of data and ensuring the smooth deployment of services to billions of users.
Google has joined the ranks of Amazon and Microsoft as the third major tech company to utilize circuitry from the renowned British chip-design firm Arm for their data center CPU. This move showcases Google’s commitment to staying at the forefront of technological advancements. There has been a shift that has replaced the previous status quo, in which large operators of server farms primarily purchased their CPUs from Intel and Advanced Micro Devices.
Google has refrained from directly selling chips to customers for installation in their own data centers. With this move, the company will be entering into direct competition with Intel and Nvidia. Nvidia has been the dominant player in the AI market, currently holding over 80% of the market share for chips used in developing and implementing AI technology.
According to Google’s Vahdat, the process of becoming a top-tier hardware company is distinct from achieving greatness as a cloud company or as an organizer of the world’s information.
Google has opted to offer custom chip rentals to its cloud customers. The Axion chips are set to be made available to external customers later this year, while the latest generation of TPUs is now widely accessible.
In November, Google announced that it had achieved a significant milestone by connecting over 50,000 TPUs to develop AI systems. This endeavor was described as the most extensive undertaking of its nature. Google developed Gemini using TPUs and will solely utilize the chips for handling user queries.
Managing the company’s expanding cloud business has involved juggling the needs of different internal teams and the increasing demand from AI startups like Anthropic. This task has been further complicated by the widespread supply constraints.
According to insiders, certain teams within Google have been informed that they will not receive any additional computing resources this year. This decision is partly due to the increasing demand for AI services.
Vahdat explained that Google’s decision-making process for allocating computing resources is based on prioritizing the fastest-growing products and services.
Google’s in-house chip efforts started with a significant advancement in voice-recognition technology in 2013.
According to Jeff Dean, an experienced engineering leader, he informed the systems infrastructure division at Google that a significant increase in the number of chips in their data centers would be necessary if the technology gained widespread adoption. “That was the initial indication of this forthcoming problem,” he stated.
When Google developed the initial version of the TPU a few years later, Dean advocated for executives to acquire more than the company initially planned for. Researchers subsequently utilized them to develop a software system known as Transformers, which served as the foundation for generative AI products like ChatGPT.
Google’s track record in granting access to external customers has been a bit inconsistent. Although Google has managed to attract well-known startups like Character and Midjourney, certain developers have faced challenges when it comes to creating software for the chips.
Google has collaborated with Nvidia and other tech companies on a software project called OpenXLA. The goal of this project is to simplify the development of AI systems across various chip types.
Anthropic, a major user of TPUs, started transitioning some of its AI requirements to custom chips created by Amazon following a substantial investment agreement of up to $4 billion with the cloud giant. Google later announced a $2 billion funding commitment to Anthropic and revealed an expanded partnership with the startup.
AssemblyAI, a startup specializing in speech-to-text products, developed its latest technology using TPUs due to challenges in acquiring GPUs in the past, according to CEO Dylan Fox. “From a perspective of availability, we have been quite pleased,” he remarked.
According to Google’s internal data, the new Axion processors offer a significant performance boost of up to 30% when compared to the fastest Arm-based chips available in the cloud. According to reports, several customers, including Snap, are considering testing the new hardware.
According to Forrester principal analyst Mike Gualtieri, Google’s investment in Axion would be worthwhile even if the company only achieved half of its claimed performance improvements. According to him, there is still fierce competition from other major cloud companies when it comes to acquiring new business.
This will resemble the typical range of web services provided by hyperscalers, according to Gualtieri. There will be a reciprocal exchange of actions.
