NVIDIA Redefines Moore's Law, Ushers in the Era of Accelerated Computing and Generative AI

Post by Otoniel Trejo - Published June 4th, 2023

For decades, Moore's law—the idea that transistor density on chips doubles roughly every two years—has pushed a steady, predictable rate of technological progress. However, within the last decade, we’ve begun to encounter the limitations to this law, with CPU technology reaching the limits of what is physically possible.

While the last decade has begun to expose the limits of the CPU, it's also begun to show the promise of GPUs. Using the power of GPUs, researchers have made major breakthroughs in machine learning (ML) and artificial intelligence (AI), leading to some of the most important technological advancements in human history. Finding new ways to merge GPU technologies with ML and AI may be our best chance to not only extend the progress of Moore’s law, but to take it further than we ever could have thought to be possible.

As the world’s leading producer of GPUs over the last several years, NVIDIA has played a massive role in the development of AI technologies. And this year, in a recent keynote speech given by founder and CEO Jensen Huang, NVIDIA demonstrated the first real preview of the incredible potential these AI technologies have to offer. By developing the twin models of Accelerated Computing (AC) and Generative AI (GAI), NVIDIA has positioned itself at the center of one of the biggest shifts in technological history.

The Era of CPU Scaling is Over

The age of simple CPU scaling is gradually coming to an end. On the micro level, transistors are reaching near-atomic levels of size, with companies like TSMC already moving to a 4nm generation of processors. On a macro level, datacenters worldwide are beginning to face bottlenecks over power consumption, land use, and utilization levels.

In short, we are running against the physical limits of silicon-based CPUs. Obtaining more computational power by simply shrinking transistor sizes, or increasing the number of cores, is no longer a feasible solution to the scaling problems presented by new technologies like ML and AI.

The GPU Solution

Machine learning, deep learning, neural networks, and large language models—what people generally consider “AI”—rely on a wide range of highly sophisticated algorithms and mathematical techniques. Methods involving linear regression, gradient descent, back propagation, and transformers can all require massive levels of repetitive computational power. Unfortunately, CPUs are ill-suited for handling this level of demand.

GPUs, on the other hand, are perfect for exactly that kind of task. GPUs have a unique combination of power, scalability, and parallelization that allow them to outperform CPUs on all these fronts, often by several orders of magnitude. In fact, according to recent benchmarks from NVIDIA’s keynote, GPUs can cost less per compute unit, require less power, and require less space, all for total speed increases that can exceed 100 times the performance of a comparably configured CPU in many use cases.

The transition from CPU to GPU-accelerated computing represents a major shift. NVIDIA’s GPU-based architectures can completely our approach to computing—one that moves away from focusing on individual processors, and instead considers the datacenter as a whole.

From “Direct and Perform” to “Interpret and Generate”

It’s hard to understand how transformative this change is without recognizing the real change at the core of it all. We are not merely looking at enhancing existing models: we're fundamentally redefining them. Traditional computing, largely based on the CPU, has focused on a sequential Direct and Perform model: instructions are fed into a system, and the CPU performs the specific task. The low-core count architecture of CPUs lent itself naturally to this model of data processing. But now, adding powerful GPU-based processing and artificial intelligence enables a completely new model: one of Interpret and Generate.

In NVIDIA’s new 'Interpret and Generate’ model, computer’s don’t merely execute predefined instructions. They interpret data, learn from it, and generate results—often in the form of new, never-before-seen data. This is a far more dynamic, flexible, and intelligent model that positions AI not just as a tool for automation, but as a sort of "partner" in creativity, exploration, and discovery.

With this concept in mind, let's delve deeper into how NVIDIA is pioneering this shift with its twin technologies: Accelerated Computing (AC) and Generative AI (GAI).

Accelerated Computing - A Datacenter-Centric Problem

One of the key points highlighted by NVIDIA was the focus on Accelerated Computing (AC). AC leverages the power of GPUs, alongside CPUs, to perform computation-heavy tasks more efficiently. The advent of this AC model takes the focus of computational power off of individual processors and moves to the datacenter as a whole. NVIDIA CEO Jensen Huang emphasized that "the datacenter is the new computer," referring to the increasing importance of system-wide optimization and the evolution of computing from the local hardware scale to a more resilient, distributed, and networked system.

In order to make Accelerated Computing a reality, NVIDIA had to view it as a full-stack challenge. Their approach required a complete, years-long reengineering of chips, system software, and algorithms, all specially adapted to many different domains of operation.

After years of hard work, it appears they’ve developed the exact solution they were searching for. From their cutting-edge graphics cards to their innovative platforms and software, NVIDIA has reshaped the technology landscape, making Accelerated Computing not only possible, but practical and cost-effective. Meanwhile, in the process of refining AC, NVIDIA managed to make transformative developments in another field: Generative AI.

Generative AI - The Future of Computing

The other key point highlighted in NVIDIA's keynote was the breath-taking advances they've made in the field of Generative AI (GAI). At a high level, GAI is a type of artificial intelligence in which a GAI model generates data that emulates the information it was trained on. This can be applied to any data type: text, images, sounds, and more. This makes GAI an extremely versatile tool with various applications across many industries. NVIDIA's breakthroughs in this domain, powered by their highly efficient GPUs and expertise in AI, have broken open the doors of what these GAI models can do. When merged with the ideas behind Accelerated Computing, the possibilities are endless.

Take NVIDIA's RTX Hardware Acceleration technology, for example. It uses NVIDIA Tensor Cores to run AI that can predict proximate pixels, significantly reducing the computation load while also improving the efficiency and quality of the image rendering processes. In fact, this GAI based approach is so effective that their new Ada architecture enables real-time raytracing (a long sought after gold standard in consumer rendering applications). Another noteworthy application is NVIDIA's ACE, an AI model used to generate game characters' animations, sounds, speech, and art styles, which offers an exciting glimpse into the potential of GAI in the realm of entertainment and media.

Generative AI can already create highly realistic, nuanced outputs, making it a game changer for creative fields like graphic design, animation, and video game development. However, the potential uses go far beyond these industries. Whether it's creating realistic training scenarios for autonomous vehicles, generating personalized responses in customer service and advertisement, or even predicting future climate patterns, GAI is poised to transform the world in profound and exciting ways.

Generative AI goes far beyond simple creation: it’s enabling unprecedented levels of understanding. By learning and emulating patterns within data, GAI can provide insights into complex systems and phenomena on a scale we’ve never seen before. By turning AI into a tool for hardware acceleration and data creation, NVIDIA has taken the old computing model of “Direct and Perform” and turned it to the radically new “interpret and generate” model. As NVIDIA continues to innovate and expand the capabilities of GAI, we should all be looking forward to a future where AI is not just a tool for automation, but a partner in creativity, exploration, and discovery.

NVIDIA's Great Hop Forward

As the future shifts towards Accelerated Computing and Generative AI, NVIDIA’s revolutionary Grace Hopper Superchip continues NVIDIA’s long march toward the new era of computing.

Designed for data centers, Grace Hopper is the world's first Accelerated Computing processor that, according to NVIDIA, also offers a large, CPU-to-GPU mutually coherent memory (576GB). And it's not just about the sheer power and memory capacity of Grace Hopper; the real game changer lies in the intricate design and engineering considerations that facilitate efficient data processing and computational acceleration.

In the world of data processing, features such as mutual coherency and efficient memory management are critical for minimizing redundancy and optimizing performance. It allows data to be shared more seamlessly and greatly enhances the performance of memory-intensive applications. This is especially beneficial for the burgeoning applications in GAI and other data-intensive tasks. Coupled with its use of low power DDR memory, which has been optimized for data centers, the Grace Hopper chip seems poised to push the boundaries of AC and GAI further than ever.

Grace Hopper is more than just a processor; it represents a culmination of NVIDIA's strides in the domains of Accelerated Computing and Generative AI. As NVIDIA continues to pioneer advances in hardware and AI technologies, they are simultaneously reshaping the technological landscape, making AC and GAI not only possible, but practical and more efficient. It's a future where our data centers become more powerful, more intelligent, and more creative—and it's a future that NVIDIA is actively bringing to life.

NVIDIA's Network Infrastructure and Software Ecosystem

Lastly, as part of NVIDIA’s full-stack solutions for Accelerated Computing and Generative AI, NVIDIA's key note addressed some additional aspects of computing infrastructure. Their technology reaches all the way across from network infrastructure to software ecosystems, creating an interlinked, optimized data processing pipeline that leverages the full potential of their innovations. This holistic approach not only transforms the very core of computation and data processing, but also ensures a robust synergy between hardware and software, elevating the capabilities of AI and computing as a whole.

As part of the next generation of infrastructure, NVIDIA has introduced products like the BlueField 3 DPU and the NVIDIA Aerial framework. The BlueField 3 DPU is an advanced data processing unit that offloads, accelerates, and isolates data center workloads, enabling applications to run more efficiently. Meanwhile, the Aerial framework represents a significant innovation in telecommunication networks that allows service providers to run 5G networks as part of the software stack, transforming a traditionally hardware-based functionality into a flexible software solution. Both the Bluefield 3 DPU and the Aerial Framework rely heavily on accelerated computing infrastructure concepts.

Complementing this is the introduction of the Spectrum-X Ethernet platform. This a new platform combines the astounding 51.2 Tb/s bandwidth of the Spectrum-4 network switch with the processing power of the BlueField DPU. This development underscores NVIDIA's commitment to minimizing data center network losses, significantly improving network performance, efficiency, and reliability, and further bolstering the capabilities of Accelerated Computing.

NVIDIA recognizes that hardware innovations are only as powerful as the software that harnesses them. To that end, they have built a robust software ecosystem to leverage the full potential of their advanced hardware solutions. The NVIDIA AI Foundations suite of cloud services and tools allows users to build proprietary AI models tailored to their specific needs. These AI models can then be seamlessly deployed via NVIDIA AI Enterprise, whether on-premise or in a cloud service provider environment. Coupled with a clear commitment to maintaining the software stack necessary for robust GPU support, it’s easy to see how NVIDIA’s symbiosis of software and hardware enables a more efficient, powerful, and scalable AI infrastructure, solidifying their position at the forefront of the AI revolution.

Final Thoughts

In the face of the declining utility of Moore's Law and the inherent limitations of CPU scaling, the world is on the cusp of a new computing era—an era that will be shaped not by the number of transistors on a chip, but by our ability to blend sophisticated hardware architectures with intelligent software algorithms. NVIDIA stands at the forefront of this paradigm shift, leveraging their prowess in GPU technology to redefine the very principles of computation.

With their dual-focus approach on Accelerated Computing and Generative AI, NVIDIA is not only circumventing the limitations of traditional computing but also paving the way for a future where technology is more than just a tool. In this future, AI becomes an intelligent and creative partner, capable of generating new insights, ideas, and solutions.

This isn’t just an evolution of technology: it’s a revolution of capabilities. By reimagining the very essence of computing, NVIDIA is helping us to break free from preprogrammed, linear thought, and open the doors to a world of limitless potential and creativity. As we move forward into this exciting new era, our success will depend not just on our ability to develop more powerful processors, but also on our ability to conceive more intelligent, generative systems. NVIDIA’s work in Accelerated Computing and Generative AI serves as a huge point of innovation, leading us to a future where AI and humans work in synergy, enabling us to solve problems, create experiences, and explore domains that we have barely begun to imagine.

These advancements are set to revolutionize every single industry. From healthcare and education to entertainment and logistics, they hold the potential to greatly enhance our ability to tackle pressing global challenges such as climate change, disease, and resource scarcity. Economically, this new era of accelerated computing and generative AI is set to drive robust growth, fueling innovation, productivity, and job creation.

With its powerful combination of hardware and software innovations, NVIDIA is not just redefining what is technologically possible but also shaping our collective digital future. It's a genuinely exciting time to be in the field of computing, and I personally can't wait to see what NVIDIA will do next.

Watch the key note

Check out the key note that inspired this post. It goes into a lot more detail on NVIDIA's newest technologies, and demos some exciting capabilties.