EP79:The Four Pillars of AI Processing Analyzing CPUs, GPUs, FPGAs, and ASICs

Summary:

In this episode, we discuss the various types of chips used in Artificial Intelligence (AI), focusing on GPUs, CPUs, FPGAs, and ASICs. Each chip type has its own strengths and weaknesses: GPUs excel at processing power for complex tasks, CPUs are becoming less relevant as AI advances, FPGAs offer versatility and efficiency for custom applications, and ASICs specialize in specific AI tasks with high performance and energy efficiency. We conclude by highlighting the geopolitical implications of AI chips, including their impact on economic competitiveness, tech sovereignty, defense and intelligence sectors, innovation, and energy efficiency.

Questions to consider as you read/listen:

1. What are the different types of chips used in AI and how do they compare in terms of their strengths and weaknesses?
2. How do AI chips impact global economic competitiveness, tech sovereignty, and defense capabilities?
3. What are the broader implications of AI chips for innovation, alliances, and environmental sustainability?

Long format:

Four types of chips used in AI

There are essentially four different silicon options that can be used for the training and development of artificial intelligence technology: GPUs, CPUs, FPGAs, and ASICs.

GPUs

A GPU chip, or Graphics Processing Unit chip, is an electronic circuit that performs mathematical calculations at high speeds to handle graphics-related work. These are the type of “gammer chips” that PZ mentioned. GPUs have many smaller, more specialized cores that work together to divide processing tasks across many cores simultaneously. This design allows GPUs to perform the same operation on multiple data values in parallel, increasing their processing efficiency. Within GPUs, there are three subsets generally framed as:

Integrated GPUs: Built into a PC's motherboard, allowing laptops to be thin, lightweight, and power-efficient

Mobile GPUs: Found in larger laptops, these chips are less bulky than a full desktop-style GPU but offer better graphics performance than a CPU's built-in graphics power

High-end GPUs: Can sell for tens of thousands of dollars

Importantly, there are other types of chips used in AI these days.

CPUs

A central processing unit (CPU) is a general-purpose chip that can be used for some AI tasks, but it's becoming less useful as AI advances. CPUs are designed to handle sequential tasks, like running operating systems and managing applications. They can also be used for pre-processing data, initial data analysis, and orchestrating overall system operations. However, CPUs are less specialized than other chips, and their processing power tends to decline quickly compared to more specialized chips.

FPGAs

A Field Programmable Gate Array (FPGA) chip is a reprogrammable integrated circuit that can be used to accelerate artificial intelligence (AI) applications. An FPGA is a flexible computer component that can be reconfigured to meet specific needs after manufacturing. The circuitry inside an FPGA is not hard etched, unlike in graphics processing units (GPUs) or application-specific integrated circuits (ASICs). FPGAs are valued in AI for their versatility, power efficiency, and adaptability. They can be used to build neural networks from scratch and optimize applications for specific needs. FPGAs are particularly useful for real-time processing and prototyping new projects. FPGAs offer a number of benefits, including: reduced latency (FPGAs can provide low latency and deterministic latency (DL)); speed to market (FPGAs can help speed time to market by reducing the need to develop and release new hardware) and cost savings (FPGAs can help reduce costs by reducing the complexities of developing application-specific integrated circuits (ASICs)).

ASICs

ASIC stands for Application-Specific Integrated Circuit, and it's a type of computer chip that's custom-designed to perform a specific task, such as artificial intelligence (AI). ASICs are digital or analog circuits that are custom-built to perform a specific function. They're not reconfigurable and can't contain additional instructions. ASICs are often used in industry, intelligence agencies, space programs, and defense systems. They're also used in AI to create accelerator chips that are designed to support specific applications. ASICs offer similar computing ability to FPGAs, but they're not reprogrammable. Because their circuitry is optimized for a specific task, they often outperform general-purpose processors or other AI chips. ASICs are designed early in the process to address specific needs. The two primary ASIC design methods are gate-array and full-custom.

November 2024 Inventory of major AI chip designers and fabricators

Which one is better?

It depends.

FPGAs are best used for custom, low-latency applications that require customization for specific deep learning tasks, such as bespoke AI applications. FPGAs are also well suited for tasks that value energy efficiency over processing speeds.

Higher-powered GPUs, on the other hand, are generally preferred for heavier tasks like training and running large, complex models. The GPUs superior processing power makes it better suited for effectively managing larger datasets.

CPUs are fading off into the sunset. CPUs do offer some initial pricing advantages. When training small neural networks with a limited dataset, a CPU can be used, but the trade-off will be time. The CPU-based system will run much more slowly than an FPGA or GPU-based system. Another benefit of the CPU-based application will be power consumption. Compared to a GPU configuration, the CPU will deliver better energy efficiency.

The primary advantage of using ASICs (Application-Specific Integrated Circuits) in AI chips is their superior performance and energy efficiency for specific AI tasks, as they are custom-designed to excel at a particular workload, like neural network processing, resulting in significantly faster execution compared to general-purpose processors while consuming less power; making them ideal for applications with well-defined AI requirements, particularly in edge computing scenarios where power consumption is critical.

CONCLUSION

Understanding what these chips are, how they are best used and the related issues are important in geopolitics.

AI chips impact economic competitiveness. Advanced chip design and fabrication are crucial for countries aiming to lead in AI and tech innovation. Nations with robust AI chip industries can boost their economic power and maintain competitive advantages. For instance, the U.S., China, Taiwan, Japan and South Korea are major players in chip manufacturing, and access to AI chip technologies has become a priority to maintain economic influence.

AI chips impact tech sovereignty and trade relations. Given the strategic importance of AI chips, countries have become more protective of their semiconductor industries, as seen with the U.S.-China trade tensions. Export controls, such as those restricting high-end GPUs and ASICs, are designed to prevent rival nations from acquiring advanced AI capabilities that could threaten national security or economic interests.

AI chips impact defense and intelligence sectors. High-performance AI chips, especially ASICs and FPGAs, are increasingly critical in military applications for their processing power, low latency, and energy efficiency. Countries investing in custom AI chips for intelligence gathering, autonomous systems, and real-time decision-making gain substantial defense advantages. This has led to competition to secure supply chains, develop local industries, and ensure that military applications do not rely on foreign chip manufacturers.

AI chips impact innovation and alliances. Nations are fostering alliances to secure the resources and intellectual property needed for chip innovation. For instance, alliances like the Quad (involving the U.S., Japan, Australia, and India) focus on semiconductor supply chains as part of their strategy to counterbalance China's influence. Additionally, governments are investing in research and development to ensure that their tech ecosystems can support sustainable chip innovation, essential for long-term leadership in AI.

AI chips impact energy efficiency and environmental concerns. As countries work to balance energy policies and meet sustainability targets, the energy efficiency of AI chips, like those of FPGAs and ASICs, plays into larger energy strategies. Countries with a lead in energy-efficient AI chips may position themselves as leaders in sustainable technology, reinforcing their influence in international environmental and technology forums.

Sources:

https://aws.amazon.com/what-is/gpu/#:~:text=A%20graphics%20processing%20unit%20(GPU,for%20many%20compute%2Dintensive%20tasks

https://semiengineering.com/knowledge_centers/integrated-circuit/ic-types/processors/graphics-processing-unit-gpu/#:~:text=An%20electronic%20circuit%20designed%20to%20handle%20graphics%20and%20video

https://support.microsoft.com/en-us/windows/all-about-graphics-processing-units-gpus-e159bedb-80b7-4738-a0c1-76d2a05beab4#:~:text=The%20graphics%20processing%20unit%20(GPU,lightweight%2C%20and%20power%2Defficient

https://www.intel.com/content/www/us/en/products/docs/processors/cpu-vs-gpu.html#:~:text=The%20graphics%20processing%20unit%20(GPU,are%20built%20for%20different%20purposes

https://singularityhub.com/2024/03/13/what-is-a-gpu-the-chips-powering-the-ai-boom-and-why-theyre-worth-trillions/#:~:text=Is%20a%20GPU?-,The%20Chips%20Powering%20the%20AI%20Boom%2C%20and%20Why%20They're,what%20makes%20them%20so%20special?

https://www.itpro.com/hardware/30399/what-is-a-gpu#:~:text=These%20cards%20can%20often%20be,when%20Ether%20mining%20is%20detected

https://support.microsoft.com/en-us/windows/all-about-graphics-processing-units-gpus-e159bedb-80b7-4738-a0c1-76d2a05beab4#:~:text=The%20graphics%20processing%20unit%20(GPU,lightweight%2C%20and%20power%2Defficient

https://builtin.com/articles/ai-chip#:~:text=What%20is%20the%20difference%20between,games%2C%20and%20training%20AI%20models

https://cset.georgetown.edu/publication/ai-chips-what-they-are-and-why-they-matter/#:~:text=AI%20chips%20include%20graphics%20processing,calculations%20required%20by%20AI%20algorithms

https://www.einfochips.com/blog/how-ai-enabled-chipsets-are-different-from-general-purpose-chipsets/#:~:text=Sneak%20Peek%20into%20Different%20Types,can%20handle%20simpler%20AI%20workloads

https://www.linkedin.com/pulse/cpus-gpus-components-systems-ai-mahendra-vasant-date-fpvrf#:~:text=Central%20Processing%20Unit%20(CPU)&text=CPUs%20manage%20the%20general%20control,and%20orchestrating%20overall%20system%20operations

https://www.intel.com/content/www/us/en/learn/fpga-for-ai.html#:~:text=Field%2Dprogrammable%20gate%20arrays%20(FPGAs,support%20new%20AI%20use%20cases

https://www.intel.com/content/www/us/en/fpga-solutions/artificial-intelligence/fpga-gpu.html#:~:text=Field%20programmable%20gate%20arrays%20(FPGAs,investment%E2%80%94to%20design%20and%20fabricate

https://www.arm.com/glossary/fpga#:~:text=Field%20Programmable%20Gate%20Arrays%20(FPGAs,requirements%20after%20the%20manufacturing%20process

https://www.ibm.com/blog/fpga-vs-gpu/#:~:text=Understanding%20field%20programmable%20gate%20arrays,applications%20and%20prototyping%20new%20projects

https://my.avnet.com/silica/resources/article/fpga-vs-gpu-vs-cpu-hardware-options-for-ai-applications/#:~:text=Field%20programmable%20gate%20arrays%20(FPGAs)%20are%20types%20of%20integrated%20circuits,as%20deterministic%20latency%20(DL)

https://builtin.com/articles/ai-chip#:~:text=ASICs%20are%20accelerator%20chips%2C%20designed,to%20boost%20machine%20learning%20performance

https://csrc.nist.gov/glossary/term/application_specific_integrated_circuits#:~:text=Definitions:,and%20cannot%20contain%20additional%20instructions

https://www.baesystems.com/en-us/definition/what-are-asics

https://www.arm.com/glossary/asic#:~:text=An%20application%2Dspecific%20integrated%20circuit,ability%20to%20process%20greater%20workloads