Greg Martin is Strategic Marketing Director at Xilinx
In dynamic and evolving markets – such as 5G, data centers, automotive and industry – applications demand ever-increasing computational acceleration while remaining within tight power envelopes. Artificial intelligence (AI) is a major driver of the demand for higher compute density, the adoption of which is rapidly accelerating.
AI inference requires high processing performance with tight power budgets, whether deployed in the cloud, edge, or endpoint. Dedicated AI hardware is often required to speed up AI inference workloads.
At the same time, AI algorithms are evolving much faster than the speed of traditional silicon development cycles. Fixed silicon chips, like ASIC implementations of AI networks, risk becoming obsolete very quickly due to rapid innovation in advanced AI models.
Adaptive computing is the answer to these challenges.
Adaptive computing is unique in that it includes silicon hardware that can be optimized for specific applications after manufacturing. Since the optimization happens after the hardware is manufactured, it can be configured with the latest AI models, the ones that did not exist when competing ASICs were designed. This optimization can also be done and redone an almost infinite number of times, providing unique flexibility. Hardware changes can even be made after the device has been fully deployed in a production environment. Just as a production processor may be given a new program to run, an adaptive platform may be given a new hardware configuration to accommodate, even in a live production environment.
Adaptive hardware vs alternatives
CPUs and GPUs have unique capabilities and are well suited for certain tasks. Processors are optimal for decision-making functions where complex logic needs to be evaluated. GPUs are best for offline data processing when high throughput is needed, but latency is not an issue. Adaptive computing is best when high throughput is needed with low latency, such as real-time video streaming, 5G communications, and automotive sensor fusion.
The reason adaptive computing can deliver high performance with low latency is the ability to enable Domain Specific Architectures (DSAs), which optimally implement specific applications in specific domains. In contrast, CPUs and GPUs have fixed von-Neumann-based architectures that do not allow domain optimization of their underlying architecture.
DSAs can also be constructed using a dedicated (fixed) silicon device, commonly referred to as an Application Specific Standard Product, or ASSP. While there are advantages to implementing a DSA in a fixed ASSP, there are also disadvantages.
The first is the pace of innovation. To keep pace, manufacturers must create and deliver new services in shorter time frames than ever before. Specifically, the lead times are shorter than the time required to design and build a new fixed silicon DSA. This creates a fundamental market misalignment between market demands for innovation and the time it takes for companies to design and manufacture ASSPs. Changes to industry standards or other fluctuating requirements can quickly render these devices obsolete.
The second consideration is the cost of custom silicon. The one-time cost of designing and manufacturing a single silicon design, such as a complex 7nm ASIC, can cost several hundred million dollars in one-time engineering costs (NRE). Costs are expected to increase further as device geometries shrink to 5nm and below. Rising costs are slowing the adoption of advanced nodes for ASSPs, which can leave their users with outdated and less efficient technology.
Presentation of adaptive platforms
Adaptive platforms are all built on the same fundamental adaptive hardware foundation; however, they include much more than just silicon hardware or peripheral. Adaptive platforms encompass a full set of runtime software. In combination, hardware and software provide a unique capability from which highly flexible yet efficient applications can be built.
These devices make adaptive computing accessible to a wide variety of software and systems developers. These platforms can be used as the basis for many products, with advantages such as:
Short time to market. An application built using a platform such as the Alveo Data Center Accelerator Card can take advantage of hardware accelerated for a specific application without requiring hardware customization. A PCIe card is added to the server and the accelerated libraries are called directly from an existing software application.
Reduced operating costs. Optimized applications based on an adaptive platform can provide significantly higher efficiency per node than solutions using only the processor, due to the increase in compute density.
Flexible and dynamic workloads. Adaptive platforms can be reconfigured according to current needs. Developers can easily change applications deployed within an adaptive platform, using the same equipment to meet changing workload needs.
Future-proof designs. Adaptive platforms can be continuously adapted. If new functionality is required in an existing application, the hardware can be reprogrammed to optimally implement these functionality, thereby reducing the need for hardware upgrades and thus extending the life of the system.
Acceleration of the entire application. AI inference rarely exists in isolation. It is part of a larger chain of data analysis and processing, often with several pre- and post-steps that use a traditional (non-IA) implementation. The integrated AI parts of these systems benefit from AI acceleration. Non-AI parties also benefit from the acceleration. The flexible nature of adaptive computing is suited to speeding up both AI and non-AI processing tasks. It’s called “whole application acceleration” and has become increasingly important as compute-intensive AI inference permeates more applications.
Adaptive platform accessibility
In the past, to take advantage of FPGA technology, developers had to create their own hardware boards and use a hardware description language (HDL) to configure the FPGA. In contrast, adaptive platforms allow developers to benefit from adaptive computing directly from their familiar software frameworks and languages such as C ++, Python, TensorFlow, etc. Software and AI developers can now use adaptive computing without having to build a board or be hardware experts.
Different types of adaptive platforms
There are many types of adaptive platforms depending on the application and needs, including standardized data center acceleration cards and edge modules. Multiple platforms exist to give the best possible starting point for the desired application. Applications vary widely, from latency sensitive applications such as autonomous driving and real-time video streaming, to the high complexity of 5G signal processing and unstructured database data processing.
Adaptive computing can be deployed in the cloud, network, edge, and even at the endpoint, bringing the latest architectural innovations to discrete and end-to-end applications. The range of deployment locations is possible through a variety of adaptive platforms – from large capacity devices on PCIe accelerator cards in the data center, to small, low power devices suitable for endpoint processing required by IoT devices.
Introducing the AI Engine
One of the biggest recent innovations in adaptive computing has been the introduction of the AI engine by Xilinx. The AI engine is still basically a configurable block, but it is also programmable as a processor. Instead of being formed from standard FPGA processing hardware, an AI engine contains high performance scalar and multiple data single instruction (SIMD) processors. These processors are optimized to efficiently implement math rich functions typically found in AI inference and wireless communications.
AI engine networks are always connected to adaptable FPGA-style data interconnects that help create efficient and optimized data paths for the target application. This combination of computation-dense (math-rich) CPU-like processing elements connected to an FPGA-like interconnect ushers in a new generation of AI and communication products.
Prepare for a more connected and smarter world
Fundamentally, adaptive computing builds on existing FPGA technology while making it more accessible than ever to a wider range of developers and applications. Software and AI developers can now build optimized applications using adaptive computing, a technology previously inaccessible to them.
The ability to tailor hardware to a specific application is a unique differentiator from CPUs, GPUs, and ASSPs, which have fixed hardware architectures. Adaptive computing allows hardware to be tailored to an application, providing high efficiency, while allowing future adaptation as workloads or standards change.
As the world becomes increasingly connected and intelligent, adaptive computing will continue to be at the forefront of optimized and accelerated applications, enabling all developers to build a better future.