As the global shortage of semiconductors looms around the enterprise world, organizations will have to embrace edge AI to make the chips efficient and run for a longer stint.
The discussion around the latest software and digital transformation often seems to eclipse the importance of semiconductors. However, the shortage of chips to be persisted at least till the second quarter of 2022 left enterprises with no choice but to take on the challenges of low-power neural networks.
There are multiple startups that have already taken the plunge to move neural network-based machine learning from the cloud data center to embedded systems in the field, also known as “the edge”. To make the semiconductor chips work in a world where their shortage persists will require IT leaders to identify novel ways to set up neural, design memory paths as well as compile to hardware. Building this formula will require the brightest minds in the electrical engineering space to embrace the challenge. However, the push has already been set for edge AI. Startups such as Axelera.AI, Deep Vision, EdgeQ, have taken the plunge.
Edge AI startups opportunities
The push towards the edge is mainly driven by the necessity for local data processing, low latency as well as avoidance of repeated calls to AI chips on the cloud, as per the ABI Research firm. The firm also emphasizes better data privacy as of prime importance. Additionally, according to ABI’s ‘Artificial Intelligence and Machine Learning’ 2021 report, the edge AI chipset market will grow to US $28 billion in 2026, for a compound annual growth rate (CAGR) of 28.4% from 2021 to 2026.
Such growth will need the designs to move beyond bellwether AI apps such as the ones recognizing the images of cats and dogs, often built in power-rich cloud data centers. This quest to expand use cases will bring pause to optimists.
While making semiconductor chips is one thing, getting them to work across a range of neural networks is another for which the industry is not prepared yet. A few things that are necessary while designing new chips are:
- Acknowledge that the design of neural network matters
Even though re-using the data points can help to conserve energy in neural processing, a different neural scheme will eventually lead to a different design tradeoff. IT leaders should understand that the specific data elements depend strongly on the specific topology of various neural network layers. Since there is a single architecture that has the capability to handle various forms of neural networks efficiently, it is crucial that IT leaders decide how flexible and software-programmable they want their infrastructure to be, this will affect the power area performance.
- Knowing memory path hierarchy matters
One of the objectives of designing a memory path for neural processing is to keep the processor fed with data. While organizations can opt for using Moore’s law to put a lot of multipliers on a chip, the challenge is to provide them all with the necessary data every clock cycle. They will need a memory hierarchy that should have sufficient bandwidth, where organizations can utilize data at different levels based on how they will require the data again.
- Knowing why algorithms are mapping is crucial
Compiling code that can run efficiently on the underlying hardware is a constant venture. It is still in progress for Edge AI chips. Organizations should understand while compiler chains are not really mature. They should standardize compilation flow where people are constantly trying to develop it with initiatives such as EVM and Glow. Also, as every accelerator is different, organizations will need to make their own low-level kernel functions for specific accelerators, which will be painful.
To address this issue, startups such as Axelera AI, are preparing themselves to go to the market with an accelerator chip that is centered around analog in-memory processing, data flow architecture, and transformer neural network while consuming less than 10 watts of data. While the competition to build a solution in edge AI is tough, startups are enthusiastic about dealing with them.