First Round News Hubb
Advertisement Banner
  • Home
  • First Round News
  • Funding News
  • Contact
No Result
View All Result
  • Home
  • First Round News
  • Funding News
  • Contact
No Result
View All Result
First Round News Hubb
No Result
View All Result
Home First Round News

The big challenge for Generative AI is GPU capacity and server costs

admin by admin
April 19, 2023
in First Round News


“The unanticipated demand to create AI software has outstripped the cloud service capacities of Amazon Web Services (AWS), Microsoft, Google Cloud, and Oracle”

The Information

The big challenge in generative AI is GPU shortage and costs. This affects both big players like OpenAI /Stability AI as well as small startups. That’s partly why the big cloud platforms will continue to be active investors in the space. They get their investment dollars back in the cloud bill. The recent $10 billion investment by Microsoft to acquire 51% of OpenAI (much of it in the form of cloud credits) or Google’s $300M investment in Anthropic are good examples of this.

The costs should come down with time, but for now, there’s a mild GPU shortage and the costs are significant. ChatGPT costs approximately $100,000 per day or $3 million per month to run on Microsoft’s Azure Cloud, with each word generated costing $0.0003. Assuming the continued exponential growth of ChatGPT, which has become one of the fastest growing consumer products, the company needs deep pockets to continue to providing the service.

In comparison, Google spends about $100 billion a year on infrastructure costs. It translates roughly to a cost of 5 cents for every query we type into the search box. The difference is that Google knows how to make $2 for every $1 they spend, using Adwords. When it comes to monetisation, OpenAI is expected to make $200M in 2023 and $1Billion in revenue in 2024.

The big winner from this rapid increase in demand is Nvidia, the leader in the market for chips designed to excel at AI computations. Nvidia GPU demand has seen its stock nearly double in the first 3.5 months of 2023. But even Nvidia is two to three months behind on new order fulfilment for cloud server chips. Intel, once the leader in semiconductors, has a lot of catching up to do, especially as the US chips act is trying to bring manufacturing home to the US.

Regardless of cost, it’s clear that generative AI and ChatGPT are rapidly changing consumer behaviour, and the big tech generative AI race is on as previously covered on VC Cafe. Generally speaking, Google has little to worry about when it comes to search marketshare as you can see in the chart below. However small, Bing is growing given their Bing AI integration and Samsung’s consideration of possibly replacing Google search with Bing has put Sundar Pichai in panic mode.

Google’s response in AI so far has been perceived to be more of a follower trying to catch up with BARD and by slowly making AI features available in GSuite, compared with Microsoft’s break neck pace of AI integrations in Bing, Office 365, Microsoft Team’s etc. Google is hoping to change this perception with “Project Magi”, a new search experience would offer users a far more personalised experience than the company’s current service, attempting to anticipate users’ needs as you can see in the video below.

Amazon also positioned itself in the generative AI space with “Project Bedrock” which provides a way to build generative AI-powered apps via pre-trained models from startups including AI21 Labs, Anthropic and Stability AI. Available in a “limited preview,” Bedrock also offers access to Titan FMs (foundation models), a family of models trained in-house by AWS.

The latest entrant to the generative AI space is Elon Musk, who after signing a letter requesting to slow down the training of new AI models is rumoured to be starting a company to rival OpenAI, under the newly purchased X.ai domain. Even at this stage in the game, the opportunity is huge. According to Ori Goshen, co-founder and CEO of AI21 Labs, an Israeli startup developing LLMs that also rival OpenAI:

“Every business will enter the world of generative AI, because every other business in its field will implement solutions from this world“.

For now, the main implication of this for generative startups is that they should either have a clear business model, deep pockets, or a deal with one of the leading cloud providers to be able to serve customers despite limited cloud resources and costs.

Eze Vidra
Eze is managing partner of Remagine Ventures, a seed fund investing in ambitious founders at the intersection of tech, entertainment, gaming and commerce with a spotlight on Israel.

I’m a former general partner at google ventures, head of Google for Entrepreneurs in Europe and founding head of Campus London, Google’s first physical hub for startups.

I’m also the founder of Techbikers, a non-profit bringing together the startup ecosystem on cycling challenges in support of Room to Read. Since inception in 2012 we’ve built 11 schools and 50 libraries in the developing world.

Eze Vidra
Latest posts by Eze Vidra (see all)

Related





Source link

Previous Post

Rally bags $12M for its next-gen e-commerce checkout platform

Next Post

Say goodbye to reusable laparoscopes: Xenocor brings in $10M to launch world’s First HD, fog-free laparoscope

Next Post

Say goodbye to reusable laparoscopes: Xenocor brings in $10M to launch world’s First HD, fog-free laparoscope

Recommended

Cyware Grows Marketing Leadership in North America

8 months ago

Bits raises €1M to reshape the way global fintechs build financial services

9 months ago

Magnax closes €20M to develop next-gen, power efficient for electric vehicles

9 months ago

AiAdvertising Campaign Performance Platform Helps Drive Results

7 months ago

Danish startup raises €3M to disrupt customer onboarding for financial services

4 months ago

Qualtrics Reveals Full Agenda for X4

5 months ago
first-round-white

© First Round News Hubb All rights reserved.

Use of these names, logos, and brands does not imply endorsement unless specified. By using this site, you agree to the Privacy Policy and Terms & Conditions.

Navigate Site

  • Home
  • First Round News
  • Funding News
  • Contact

Newsletter Sign Up.

No Result
View All Result
  • Home
  • First Round News
  • Funding News
  • Contact

© 2022 First Round News Hubb All rights reserved.