The AI Race Expands Qualcomm Reveals Cloud AI 100 Family of Datacenter AI Inference Accelerators for 2020 AnandTech

The effect that advances in convolutional neural networking and different synthetic intelligence technologies have made to the processor landscape inside the final decade is unescapable. AI has become the buzzword, the catalyst, the thing that every one processor makers need a bit of, and that all software carriers are eager to spend money on to develop new capabilities and new capability. A market that outright didn’t exist at the start of this decade has over the last few years emerge as a middle of studies and sales, and already a few processor companies have built small empires out of it.

But this modern generation of AI is still in its early days and the marketplace has but to discover a ceiling; datacenters keep to buy AI accelerators in bulk, and deployment of the tech is an increasing number of ratcheting up in customer processors as well. In a market that many agree with remains up for grabs, processor markers throughout the globe are trying to discern out how they are able to turn out to be the dominant pressure in one of the best new processor markets in a technology. In quick, the AI gold rush is in full swing, and proper now all people is lining up to sell the pickaxes.

In terms of the underlying era and the producers behind them, the AI gold rush has attracted interest from each corner of the generation international. This has ranged from GPU and CPU companies to FPGA firms, custom ASIC markers, and more. There is a need for inference at the brink, inference on the cloud, education inside the cloud – AI processing at every degree, served via a diffusion of processors. But amongst all of these sides of AI, the most lucrative marketplace of all is the market at the top of this hierarchy: the datacenter. Expansive, costly, and still growing by using leaps and limits, the datacenter marketplace is the last dinner party or famine setup, as operators are searching to buy not anything brief of large portions of discrete processors. And now, one of the ultimate juggernauts to sit at the sidelines of the datacenter AI marketplace is ultimately making its flow: Qualcomm

This morning at their first Qualcomm AI Day, the 800lb gorilla of the cell global announced that they may be moving into the AI accelerator market, and in an competitive manner. At their occasion, Qualcomm introduced their first discrete committed AI processors, the Qualcomm Cloud AI 100 circle of relatives. Designed from the floor up for the AI market and backed by means of what Qualcomm is promising to be an extensive software stack, the business enterprise is throwing their hat into the hoop for 2020, seeking to establish themselves as a first-rate supplier of AI inference accelerators for a hungry market.

But before we too a long way into things right here, it’s probable nice first of all a few context for nowadays’s announcement. What Qualcomm is saying these days is almost more of a teaser than a right display – and definitely a ways from a technology disclosure. The Cloud AI 100 circle of relatives of accelerators are merchandise that Qualcomm is putting together for the 2020 time frame, with samples going out later this yr. In quick, we’re probably nonetheless an awesome yr out from business products delivery, so Qualcomm is gambling things cool, saying their efforts and their reason behind them, however now not the underlying technology. For now it’s approximately making their intentions known properly earlier, specifically to the massive clients they are going to attempt to woo. But nonetheless, these days’s announcement is an vital one, as Qualcomm has made it clean that they are going in a exclusive path than the two juggernauts they’ll be competing with: NVIDIA and Intel.

The Qualcomm Cloud AI one hundred Architecture: Dedicated Inference ASIC

So what precisely is Qualcomm doing? In a nutshell, the corporation is developing a own family of AI inference accelerators for the datacenter market. Though now not pretty a top-to-backside initiative, those accelerators will are available a selection of form factors and TDPs to suit datacenter operator desires. And inside this market Qualcomm expects to win through virtue of offering the maximum efficient inference accelerators in the marketplace, offering overall performance nicely above contemporary GPU and FPGA frontrunners.

The real architectural details at the Cloud AI one hundred own family are slender, however Qualcomm has given us just sufficient to paintings with. To start with, those new elements may be synthetic on a 7nm procedure – presumably TSMC’s overall performance-oriented 7nm HPC process. The agency will provide a ramification of playing cards, but it’s no longer clean at the moment if they may be virtually designing a couple of processor. And, we’re told, this is an entirely new layout constructed from the floor up; so it’s not say a Snapdragon 855 with all the AI bits scaled up.

In truth it’s this final point that’s probably the maximum critical. While Qualcomm isn’t imparting architectural information for the accelerator today, the company is making it very clean that that is an AI inference accelerator and not anything extra. It’s no longer being referred to as an AI education accelerator, it’s now not being called a GPU, and so forth. It’s simplest being pitched for AI inference – efficiently executing pre-trained neural networks.

This is an critical distinction because, whilst the devil is within the info, Qualcomm’s announcement very strongly factors to the underlying architecture being an AI inference ASIC – ala some thing like Google’s TPU own family – rather than being a greater flexible processor. Qualcomm is of route a ways from the primary vendor to build an ASIC specially for AI processing, however whilst other AI ASICs have either been centered on the low-quit of the market or reserved for inner use (Google’s TPUs once more being the high example), Qualcomm is speaking about an AI accelerator to be sold to customers for datacenter use. And, relative to the opposition, what they're talking about is a whole lot extra ASIC-like than the GPU-like designs absolutely everyone is looking ahead to in 2020 out of the front-runner NVIDIA and aggressive newcomer Intel.

That Qualcomm’s Cloud AI one hundred processor layout is so narrowly focused on AI inference is important to its performance capacity. In the processor layout spectrum, architects balance flexibility with performance; the toward a fixed-feature ASIC a chip is, the greater green it may be. Just as how GPUs provided a large jump in AI overall performance over CPUs, Qualcomm wants to do the equal aspect over GPUs.

The catch, of path, is that a extra fixed-characteristic AI ASIC is giving up flexibility. Whether that’s the potential to address new frameworks, new processing flows, or entirely new neural networking fashions stays to be seen. But Qualcomm can be making a few extensive tradeoffs here, and the large query goes to be whether or not these are the proper tradeoffs, and whether or not the marketplace as an entire is prepared for a datacenter-scale AI ASIC.

Meanwhile, the opposite technical problem that Qualcomm will must tackle with the Cloud AI one hundred series is the fact that this is their first devoted AI processor. Admittedly, everybody has to start somewhere, and in Qualcomm’s case they're trying to translate their understanding in AI at the brink with SoCs into AI at the datacenter. The business enterprise’s flagship Snapdragon SoCs have turn out to be a pressure to be reckoned with, and Qualcomm thinks that their experience in efficient designs and signal processing in popular will provide the company a big leg up here.

It doesn’t hurt either that with the corporation’s sheer size, they have the ability to ramp up production right away. And whilst this doesn’t help them against the likes of NVIDIA and Intel – both of which could scale up at TSMC and their internal fabs respectively – it gives Qualcomm a particular advantage over the myriad of smaller Silicon Valley startups which can be additionally pursuing AI ASICs.

Why Chase the Datacenter Inferencing Market?

Technical concerns apart, the alternative critical factor in today’s declaration is why Qualcomm is going after the AI inference accelerator market. And the answer, in short, is money.

Projections for the eventual size of the AI inferencing marketplace range broadly, however Qualcomm buys in to the concept that datacenter inference accelerators on my own may be a $17 billion market via 2025. And if this proves to be true, then it'd constitute a vast market that Qualcomm would in any other case be lacking out on. One that would rival the totally of their present day chipmaking enterprise.

It’s also worth noting here that this is explicitly the inference marketplace, and not the overall datacenter inference + schooling marketplace. This is an vital difference because even as training is critical as well, the computational necessities for schooling are very difference from inferencing. While accurate inferencing can be accomplished with tremendously low-precision datatypes like INT8 (and once in a while decrease), presently most training calls for FP16 or extra. Which calls for a completely one-of-a-kind type of chip, specifically when we’re speakme approximately ASICs in preference to something a bit extra preferred cause like a GPU.

This additionally leans into scale: whilst training a neural community can take a whole lot of resources, it only wishes to be completed once. Then it could be replicated out oftentimes over to farms of inference accelerators. So as important as education is, ability customers will certainly need many more inference accelerators than they will education-succesful processors.

Meanwhile, though not explicitly stated with the aid of the organization, it’s clear that Qualcomm is trying to take down marketplace chief NVIDIA, who has constructed a small empire out of AI processors even in these early days. Currently, NVIDIA’s Tesla T4, P4, and P40 accelerators make up the backbone of datacenter AI inference processors, with datacenter sales as a whole proving to be quite worthwhile for NVIDIA. So despite the fact that the full datacenter market doesn’t develop pretty as projected, it'd nonetheless be pretty rewarding.

Qualcomm additionally has to keep in thoughts the chance from Intel, who has very publicly telegraphed their personal plans for the AI market. The organisation has numerous extraordinary AI initiatives, ranging from low-energy Movidius accelerators to their modern day Cascade Lake Xeon Scalable CPUs. However for the precise marketplace Qualcomm is chasing, the largest danger might be Intel’s forthcoming Xe GPUs, which are coming out of the enterprise’s currently rebuilt GPU division. Like Qualcomm, Intel is gunning for NVIDIA here, so there may be a race for the AI inference marketplace that not one of the titans want to lose.

Making It to the Finish Line

Qualcomm’s aims apart, for the subsequent twelve months or so, the organisation’s focus is going to be on lining up its first clients. And to do this, the agency has to expose that it’s critical about what it’s doing with the Cloud AI 100 own family, that it could supply on the hardware, and that it can healthy the convenience of use of competitors’ software program ecosystems. None of this will be smooth, which is why Qualcomm has had to begin now, up to now beforehand of when business shipments begin.

While Qualcomm has had diverse goals of servers and the datacenter market for many years now, perhaps the maximum polite way to explain the ones efforts are “overambitious.” Case in point might be Qualcomm’s Centriq circle of relatives of ARM-primarily based server CPUs, which the corporation launched with first-rate fanfare lower back in2019, simplest for the complete mission to collapse within a year. The deserves of Centriq apart, Qualcomm remains a employer that is essentially locked to cell processors and modems at the chipmaking side. So to get datacenter operators to invest within the Cloud AI own family, Qualcomm no longer best desires a splendid plan for the primary generation, however a plan for the following couple of generations past that.

The upshot here is that inside the young, developing market for inference accelerators, datacenter operators are more willing to experiment with new processors than they're, say, CPUs. So there’s no motive to believe that the Cloud AI 100 series can’t be at least moderately a hit right off the bat. But it'll be as much as Qualcomm to convince the otherwise nonetheless-cautious datacenter operators that Qualcomm’s wares are well worth investing such a lot of assets into.

Parallel to this is the software program facet of the equation. A massive a part of NVIDIA’s achievement so far has been of their AI software environment – itself is a ramification in their decade-antique CUDA atmosphere – which has vexed GPU rival AMD for a while now. The precise news for Qualcomm is that the most famous frameworks, runtimes, and tools have already been mounted; TensorFlow, Caffe2, and ONNX are the huge targets, and Qualcomm is aware of it. Which is why Qualcomm is promising an extensive software stack proper off the bat, because not anything less than with a purpose to do. But Qualcomm does need to rise up to hurry very quickly right here, as how properly their software program stack clearly works can make or spoil the entire task. Qualcomm wishes to deliver properly hardware and correct software to prevail here.

But for the instant at the least, Qualcomm's assertion today is a teaser – a proclamation of what’s to return. The business enterprise has advanced a very bold plan to interrupt into the growing AI inference accelerator market, and to supply a processor substantially unlike some thing else at the open market. And even as getting from here to there may be going to be a mission, as one of the titans of the processor international Qualcomm is some of the maximum succesful obtainable, both in funding and engineering assets. So it’s as a whole lot a question of how badly Qualcomm desires the inference accelerator marketplace as it's far their ability to develop processors for it; and the way nicely they could keep away from the sort of missteps which have sunk their preceding server processor plans.

Above all else, however, Qualcomm received’t definitely take the inference accelerator market: they’re going to must fight for it. This is NVIDIA’s marketplace to lose and Intel has eyes on it as well, by no means thoughts all of the smaller gamers from GPU carriers, FPGA companies, and other ASIC players. Any and all of that can fast rise and fall in what’s still a young market for an emerging technology. So while it’s still almost a yr off, 2020 is speedy shaping up to be the first big struggle for the AI accelerator marketplace.

Gallery: Qualcomm Cloud AI 100 Press Deck

• 
• 
• 
• 
• 
• 

Let's block commercials! (Why?)

//www.anandtech.com/show/14187/qualcomm-famous-cloud-ai-one hundred-circle of relatives-of-datacenter-ai-inference-accelerators-for-2020
2019-04-09 16:50:22Z
52780265129342

Share this post