The AI Race Expands Qualcomm Reveals Cloud AI 100 Family of Datacenter AI Inference Accelerators for 2020 AnandTech

The impact that advances in convolutional neural networking and different synthetic intelligence technologies have made to the processor landscape within the final decade is unescapable. AI has end up the buzzword, the catalyst, the issue that every one processor makers want a bit of, and that every one software providers are eager to invest in to increase new capabilities and new functionality. A marketplace that outright didn’t exist at the begin of this decade has over the previous few years come to be a center of studies and revenue, and already a few processor providers have built small empires out of it.

But this modern technology of AI is still in its early days and the marketplace has yet to find a ceiling; datacenters preserve to shop for AI accelerators in bulk, and deployment of the tech is more and more ratcheting up in customer processors as nicely. In a market that many consider continues to be up for grabs, processor markers across the globe are trying to discern out how they are able to end up the dominant pressure in one of the greatest new processor markets in a technology. In short, the AI gold rush is in complete swing, and proper now anybody is lining as much as sell the pickaxes.

In phrases of the underlying generation and the producers in the back of them, the AI gold rush has attracted interest from every nook of the era international. This has ranged from GPU and CPU agencies to FPGA firms, custom ASIC markers, and extra. There is a want for inference at the brink, inference on the cloud, schooling within the cloud – AI processing at each stage, served by way of a ramification of processors. But amongst all of these sides of AI, the maximum moneymaking market of all is the marketplace on the pinnacle of this hierarchy: the datacenter. Expansive, costly, and nonetheless growing by means of leaps and bounds, the datacenter marketplace is the last ceremonial dinner or famine setup, as operators are looking to shop for nothing brief of big portions of discrete processors. And now, one of the ultimate juggernauts to sit at the sidelines of the datacenter AI market is finally making its flow: Qualcomm

This morning at their first Qualcomm AI Day, the 800lb gorilla of the cell world announced that they are moving into the AI accelerator market, and in an aggressive way. At their occasion, Qualcomm introduced their first discrete committed AI processors, the Qualcomm Cloud AI 100 circle of relatives. Designed from the ground up for the AI market and sponsored with the aid of what Qualcomm is promising to be an in depth software program stack, the agency is throwing their hat into the hoop for 2020, trying to establish themselves as a prime supplier of AI inference accelerators for a hungry marketplace.

But earlier than we too a ways into things right here, it’s possibly high-quality to start with some context for nowadays’s assertion. What Qualcomm is announcing today is nearly extra of a teaser than a right monitor – and actually far from a generation disclosure. The Cloud AI 100 circle of relatives of accelerators are merchandise that Qualcomm is putting collectively for the 2020 timeframe, with samples going out later this 12 months. In brief, we’re in all likelihood nevertheless a terrific 12 months out from commercial products delivery, so Qualcomm is playing matters cool, announcing their efforts and their purpose at the back of them, however no longer the underlying generation. For now it’s approximately making their intentions acknowledged nicely earlier, especially to the large customers they're going to try and woo. But nevertheless, nowadays’s declaration is an essential one, as Qualcomm has made it clean that they're going in a distinctive route than the 2 juggernauts they’ll be competing with: NVIDIA and Intel.

The Qualcomm Cloud AI one hundred Architecture: Dedicated Inference ASIC

So what exactly is Qualcomm doing? In a nutshell, the corporation is growing a circle of relatives of AI inference accelerators for the datacenter market. Though not pretty a top-to-backside initiative, those accelerators will are available a selection of shape elements and TDPs to match datacenter operator wishes. And inside this market Qualcomm expects to win via distinctive feature of presenting the maximum efficient inference accelerators on the market, offering overall performance properly above cutting-edge GPU and FPGA frontrunners.

The actual architectural details on the Cloud AI one hundred family are narrow, but Qualcomm has given us just enough to paintings with. To start with, those new elements will be manufactured on a 7nm process – presumably TSMC’s performance-orientated 7nm HPC process. The employer will provide a selection of cards, however it’s no longer clean presently if they're sincerely designing multiple processor. And, we’re instructed, this is a wholly new layout constructed from the ground up; so it’s no longer say a Snapdragon 855 with all of the AI bits scaled up.

In reality it’s this final point that’s possibly the most vital. While Qualcomm isn’t imparting architectural information for the accelerator nowadays, the corporation is making it very clear that this is an AI inference accelerator and nothing greater. It’s now not being called an AI training accelerator, it’s no longer being known as a GPU, and many others. It’s simplest being pitched for AI inference – effectively executing pre-educated neural networks.

This is an vital distinction due to the fact, at the same time as the satan is in the details, Qualcomm’s assertion very strongly factors to the underlying structure being an AI inference ASIC – ala some thing like Google’s TPU circle of relatives – rather than being a extra bendy processor. Qualcomm is of course some distance from the primary dealer to construct an ASIC in particular for AI processing, however at the same time as other AI ASICs have either been focused at the low-quit of the marketplace or reserved for inner use (Google’s TPUs once more being the prime instance), Qualcomm is speaking about an AI accelerator to be bought to customers for datacenter use. And, relative to the opposition, what they are talking approximately is plenty greater ASIC-like than the GPU-like designs each person is awaiting in 2020 out of the front-runner NVIDIA and competitive newcomer Intel.

That Qualcomm’s Cloud AI 100 processor design is so narrowly targeted on AI inference is critical to its performance ability. In the processor design spectrum, architects balance flexibility with efficiency; the toward a set-function ASIC a chip is, the extra efficient it is able to be. Just as how GPUs provided a massive jump in AI performance over CPUs, Qualcomm desires to do the identical element over GPUs.

The seize, of path, is that a extra fixed-feature AI ASIC is giving up flexibility. Whether that’s the ability to address new frameworks, new processing flows, or completely new neural networking models remains to be seen. But Qualcomm can be making some vast tradeoffs right here, and the huge query goes to be whether or not these are the right tradeoffs, and whether or not the market as an entire is ready for a datacenter-scale AI ASIC.

Meanwhile, the other technical difficulty that Qualcomm will have to address with the Cloud AI 100 series is the reality that that is their first committed AI processor. Admittedly, all people has to begin somewhere, and in Qualcomm’s case they are trying to translate their know-how in AI at the threshold with SoCs into AI at the datacenter. The corporation’s flagship Snapdragon SoCs have come to be a force to be reckoned with, and Qualcomm thinks that their enjoy in green designs and sign processing in wellknown will give the agency a extensive leg up right here.

It doesn’t harm both that with the enterprise’s sheer length, they have the potential to ramp up production very quickly. And even as this doesn’t assist them towards the likes of NVIDIA and Intel – each of which could scale up at TSMC and their inner fabs respectively – it gives Qualcomm a particular advantage over the myriad of smaller Silicon Valley startups which are also pursuing AI ASICs.

Why Chase the Datacenter Inferencing Market?

Technical concerns apart, the opposite essential component in these days’s announcement is why Qualcomm goes after the AI inference accelerator marketplace. And the answer, in quick, is cash.

Projections for the eventual length of the AI inferencing marketplace range extensively, however Qualcomm buys in to the idea that datacenter inference accelerators on my own can be a $17 billion marketplace with the aid of 2025. And if this proves to be genuine, then it would constitute a widespread marketplace that Qualcomm could otherwise be lacking out on. One that might rival the completely of their modern chipmaking enterprise.

It’s additionally really worth noting here that that is explicitly the inference market, and now not the general datacenter inference + education marketplace. This is an important difference due to the fact whilst schooling is vital as well, the computational requirements for training are very distinction from inferencing. While correct inferencing can be performed with enormously low-precision datatypes like INT8 (and occasionally lower), presently maximum education calls for FP16 or more. Which calls for a very exceptional form of chip, especially whilst we’re talking about ASICs in place of some thing a piece greater wellknown purpose like a GPU.

This also leans into scale: at the same time as schooling a neural community can take quite a few assets, it simplest needs to be performed as soon as. Then it can be replicated out usually over to farms of inference accelerators. So as crucial as training is, capacity clients will absolutely need many more inference accelerators than they will training-capable processors.

Meanwhile, though now not explicitly stated by the organization, it’s clear that Qualcomm is trying to take down market chief NVIDIA, who has constructed a small empire out of AI processors even in these early days. Currently, NVIDIA’s Tesla T4, P4, and P40 accelerators make up the backbone of datacenter AI inference processors, with datacenter revenues as an entire proving to be quite worthwhile for NVIDIA. So even supposing the total datacenter market doesn’t develop pretty as projected, it might still be pretty lucrative.

Qualcomm additionally has to maintain in mind the hazard from Intel, who has very publicly telegraphed their personal plans for the AI marketplace. The organization has several unique AI initiatives, ranging from low-electricity Movidius accelerators to their brand new Cascade Lake Xeon Scalable CPUs. However for the particular market Qualcomm is chasing, the largest hazard might be Intel’s coming near near Xe GPUs, which might be coming out of the organisation’s lately rebuilt GPU department. Like Qualcomm, Intel is gunning for NVIDIA here, so there is a race for the AI inference marketplace that none of the titans desire to lose.

Making It to the Finish Line

Qualcomm’s goals aside, for the following one year or so, the company’s recognition is going to be on lining up its first customers. And to do that, the corporation has to show that it’s extreme about what it’s doing with the Cloud AI one hundred circle of relatives, that it can supply at the hardware, and that it could suit the benefit of use of rivals’ software program ecosystems. None of this may be clean, which is why Qualcomm has needed to begin now, to date ahead of while industrial shipments start.

While Qualcomm has had numerous dreams of servers and the datacenter market for many years now, perhaps the most well mannered way to describe the ones efforts are “overambitious.” Case in point might be Qualcomm’s Centriq family of ARM-based server CPUs, which the agency released with extremely good fanfare back in2019, most effective for the whole task to fall apart within a year. The deserves of Centriq aside, Qualcomm remains a corporation that is largely locked to cellular processors and modems at the chipmaking side. So to get datacenter operators to invest in the Cloud AI circle of relatives, Qualcomm now not only needs a wonderful plan for the primary technology, however a plan for the subsequent couple of generations beyond that.

The upshot here is that in the younger, growing market for inference accelerators, datacenter operators are extra inclined to experiment with new processors than they're, say, CPUs. So there’s no motive to accept as true with that the Cloud AI 100 series can’t be as a minimum fairly a hit proper off the bat. But it will likely be as much as Qualcomm to persuade the in any other case still-careful datacenter operators that Qualcomm’s wares are well worth making an investment so many sources into.

Parallel to this is the software program aspect of the equation. A big a part of NVIDIA’s success thus far has been of their AI software program ecosystem – itself is an expansion of their decade-old CUDA ecosystem – which has vexed GPU rival AMD for some time now. The right news for Qualcomm is that the maximum famous frameworks, runtimes, and gear have already been hooked up; TensorFlow, Caffe2, and ONNX are the massive objectives, and Qualcomm knows it. Which is why Qualcomm is promising an extensive software stack right off the bat, due to the fact not anything less than so as to do. But Qualcomm does ought to rise up to speed very quickly right here, as how well their software stack clearly works can make or break the complete task. Qualcomm wishes to supply good hardware and right software to be triumphant here.

But for the instant as a minimum, Qualcomm's declaration today is a teaser – a proclamation of what’s to come back. The employer has developed a very ambitious plan to interrupt into the growing AI inference accelerator market, and to deliver a processor drastically unlike whatever else at the open marketplace. And whilst getting from here to there may be going to be a challenge, as one of the titans of the processor international Qualcomm is a number of the maximum succesful available, each in funding and engineering sources. So it’s as tons a question of how badly Qualcomm wishes the inference accelerator marketplace as it's miles their capability to broaden processors for it; and the way well they are able to avoid the type of missteps that have sunk their preceding server processor plans.

Above all else, but, Qualcomm gained’t honestly take the inference accelerator marketplace: they’re going to should combat for it. This is NVIDIA’s marketplace to lose and Intel has eyes on it as well, in no way mind all of the smaller gamers from GPU companies, FPGA vendors, and other ASIC players. Any and all of which could quickly upward thrust and fall in what’s nonetheless a young market for an emerging era. So while it’s nevertheless almost a year off, 2020 is fast shaping up to be the first massive struggle for the AI accelerator market.

Gallery: Qualcomm Cloud AI one hundred Press Deck

• 
• 
• 
• 
• 
• 

Let's block advertisements! (Why?)

//www.anandtech.com/display/14187/qualcomm-famous-cloud-ai-one hundred-family-of-datacenter-ai-inference-accelerators-for-2020
2019-04-09 17:30:00Z
52780265129342

Share this post