Nvidia Data Center GPUs

In early March 2024, Nvidia replaced Netflix in the circle of the most powerful technology companies, which is often abbreviated as FAANG. Nvidia has reported their best quarter yet with income more than doubled to the previous year. Their stock raised over $100 in a single day, which translates to around $273 billion in market cap, the highest single-day raise in history. As of writing, Nvidia’s market cap is around $2.2 trillion which is in hot pursuit of Apple ($2.6 trillion and falling) and Microsoft ($3 trillion).

But what exactly does Nvidia sell that causes such a monumental shift of fortune to Nvidia? Nvidia is well-known in the gaming community for producing highly performant graphic cards to bring out the best performance in playing high-paced, graphically intense games like Call of Duty and Diablo 4. However, a quick look at their income statement shows that income from selling graphic cards does not contribute much to Nvidia’s bottom line. In Nvidia’s statement, the segment that contributes the most is simply labeled ‘Datacenter’.

Graphics in the Datacenter

For a very long time, Nvidia is the company you look for if you want better performance while gaming. This is what Nvidia is mostly famous for. But for some professionals, they are the ones you look for if you want your 3D simulation/model to run a lot smoother. Their Quadro line of professional cards looks like their gaming RTX line of cards that went to the gym. And they are not wrong, the professional Quadro line and gaming RTX are similar but optimized for different apps.

But more recently, there has been a resurgence. A gold rush. There was a breakthrough in Artificial Intelligence (AI) because of the convergence of a few events. Chips get faster but more importantly, the data we collected get a lot bigger, and a slight tweak of decades-old algorithms makes a big impact in the field of AI.

In short, we found the best way to classify things and later generate things based on that classification. All we need to do is the right hardware to process the algorithm faster and this is how the tensor processing chip is born.

Nvidia family of Cloud & Data center Graphics

Enter the datacenter GPU or how Nvidia got very rich, very quickly. A large server room that most people won’t see is called a data center. Some take up a floor of a building, some companies have dedicated buildings to store all the servers, and big tech companies like Google, Amazon, and Facebook, have warehouses of this.

For a long time, there was no sense in putting powerful graphic chips inside the data center since they are not good at processing general-purpose data. The graphic is just to drive the interface, nothing more. Then the revolution came: the AI revolution.

Suddenly, we have a lot of data that can be put in matrices (the math kind) and the general-purpose CPU just does not cut it. This is where the opportunity for Nvidia comes in. People find out that GPUs are much better suited to crunch matrices that make up AI data into meaningful output. With a little modification, these chips can perform AI jobs like object recognition in images or large language models with ease.

The differences

These are some of the differences between Nvidia GPUs that are made for data centers and the ones made for gaming

• Price: The most obvious difference is price. While top-of-the-line RTX 4090 can be found on Amazon for $2,000, the H200, the latest Nvidia data center GPU is priced at around $40,000. If you can get it, which brings us to the next point
• Availablity: You can easily buy the RTX 4090 from Amazon and ship it to you the next day, some of the cards are restricted because of national security.
• Ports: The RTX is engineered to have a display connected to it because of its expected use of the card: to play games. Meanwhile, the H200 that Nvidia sells does not have any output to display. This is because they are not optimized for gaming, but crunching AI workloads.
• Workloads: Both cards are optimized for different workloads. While gaming cards like the RTX 4090 goal is to render as many frames as possible in a second, the H200 is to crunch as much AI data as fast as possible.

Who’s using them?

Now the bigger question is who is giving Nvidia billions of dollars to put GPUs in their server room?

The obvious ones are the cloud providers like AWS, Google, and Microsoft that have AI services. They need to crunch a lot of customer data and to be competitive, they will need powerful AI chips to process them.

Another are companies that are providing advanced AI services like OpenAI, Midjourney, and also Google. To help crunch a lot of their data, they would need chips like the H200 from Nvidia.

Finally, the people who have a treasure trove of data and would like to use Artificial Intelligence to help organize their data better. Imagine having petabytes of pictures, user-generated content, and ideas. Those are social networks like Twitter, Facebook, and Google which can read those data and generate AI models of their own.

Conclusion

Right now there is an AI gold rush. For a long time, the field of AI has stagnated with very little progress to show for it. Now we have breakthrough after breakthrough and now we need custom hardware to crush all those data.

There’s a saying: “When in a gold rush, start selling pickaxes”. And Nvidia has the best pickaxes at the moment. Until the AI mania ends, Nvidia is in a secure position for its immediate future.

Plug

Support this free website by visiting my Amazon affiliate links. Any purchase you make will give me a cut without any extra cost to you

Other Ecosystem Items

• Apple TV (Amazon) / Apple TV 4K (Amazon)
• AirTags (Amazon)
• HomePod Mini (Amazon)