In the year 2000, Ian Buck harbored an ambitious dream: to play Quake III in stunning 8K resolution. As a computer science student at Stanford specializing in computer graphics, Buck embarked on a bold experiment by assembling 32 GeForce graphics cards. His aim was to render Quake III on eight strategically positioned projectors, a task that seemed impossible at the time.
Years later, Buck reminisced, saying, “That was beautiful.” This anecdote is recounted in ‘The Thinning Machine,’ an essay by Stephen Witt published in 2025, which chronicles the intriguing history of NVIDIA. Central to this narrative is the origin of CUDA, an architecture that has evolved into a vital tool for AI developers, propelling NVIDIA to the forefront of the tech industry by market capitalization.
The GPU as a Home Supercomputer
What began as an experimental pastime became a pivotal moment for Buck. He discovered that specialized graphics processing units (GPUs) could accomplish far more than merely rendering graphics. To explore this potential, Buck immersed himself in the technical aspects of NVIDIA graphics processors, guiding his research throughout his PhD at Stanford. With support from DARPA (Defense Advanced Research Projects Agency), Buck and a small research group embarked on creating a groundbreaking open-source programming language named Brook.
This language transformed GPUs into home supercomputers. Buck demonstrated that these chips, initially confined to rendering graphics, could solve general problems while leveraging the inherent parallelism of their design. One chip could illuminate triangle A while another rasterized triangle B, showcasing computing power that significantly exceeded that of any CPU available at the time.
In 2006, the GeForce 8800 GTS (and its higher version, the GTX) initiated the CUDA era.
Buck’s groundbreaking language eventually culminated in a paper titled ‘Brook for GPUs: Stream Computing on Graphics Hardware’. While it received minimal attention publicly, one person recognized its significance: Jensen Huang, NVIDIA’s founder. After the paper’s publication, Huang swiftly brought Buck on board, understanding the immense potential of GPUs beyond traditional graphics rendering.
CUDA is Born
Following the collapse of Silicon Graphics in 2005, a significant influx of talent joined NVIDIA’s R&D division, contributing to the growth of GPU capabilities. Buck and his collaborator, John Nickolls, who had previously explored parallel computing, initiated a project dubbed Compute Unified Domain Architecture.
Thus, CUDA was born. The rapid progress led to the first version of CUDA being launched in November 2006, designed exclusively for NVIDIA hardware. Like many revolutions, CUDA’s adoption was initially slow; the technology struggled to resonate with the majority of NVIDIA users, who primarily sought gaming capabilities. Only about 13,000 downloads occurred in 2007, highlighting the challenges faced in integrating CUDA into mainstream computing.
John Nickolls and Ian Buck played critical roles in CUDA’s development.
The early applications of CUDA were unrelated to artificial intelligence, which was hardly a buzzword at the time. Instead, scientific communities began to utilize CUDA for various computational tasks. This slow adoption was a challenging period for the project, consuming significant resources without immediate payout.
A Late (But Deserved) Success
In a 2012 interview, Buck reflected on CUDA’s potential future applications. He envisioned using CUDA for tasks like sorting and searching photos—a calculation-heavy process. At that moment, the true transformative power of CUDA was yet to materialize.
However, two doctoral students, Alex Krizhevsky and Ilya Sutskever, under the guidance of Geoffrey Hinton, were about to change everything with a project known as AlexNet. This breakthrough allowed for automatic image classification, a challenge previously hindered by high computational costs. They leveraged NVIDIA graphics cards and CUDA software to train their neural networks effectively.
Suddenly, AI and CUDA found their synergy, marking the beginning of a revolution that would define not only CUDA’s evolution but also the broader field of artificial intelligence. The rest, as they say, is history.