A Preliminary Study on Edge AI

Edge AI has received increasing attention recently. Here we share some recent research insights:

1. A new phone replacement cycle?

Last year, Samsung launched the S24, the first AI smartphone to become a hit. The Apple WWDC in June this year further confirmed that AI smartphones are the next direction for progress. Although current AI applications on smartphones are still primitive and not very useful, it's essential to recognize two key points:

First, considering the capabilities and pace of development of current large and small models, the emergence of a killer app on smartphones—be it advances in operating systems, voice assistants, games, or other applications—is highly likely; it just needs more time. This allows phone companies to prepare the necessary hardware and software and entrepreneurs to turn their ideas into reality. For instance, nearly two years passed after the launch of the first-generation iPhone in January 2007 before classic touchscreen applications like Angry Birds (December 2009) and Fruit Ninja (April 2010) appeared. In terms of AI hardware, Qualcomm launched the Snapdragon 8gen3 last October, achieving 45 TOPS of AI computing power, while Apple's A17 Pro last year reached 35 TOPS (with supported memory also increased from 6GB to 8GB for the first time). In the software ecosystem, Apple introduced the AI development tool APP Intents at this year's WWDC, facilitating over 36 million developers to create AI applications, a significant leap compared to Samsung's S24, which mainly relies on the phone manufacturer Samsung and system manufacturer Google for AI applications.

Second, even if it takes a long time or even never for a killer app to emerge due to various reasons (such as inadequate edge AI capabilities, unsolvable illusion issues, etc.), the competition among smartphone manufacturers will lead to an increasing proportion of AI phones, much like the situation with 5G phones. Despite no particularly significant applications to date, nearly all phones available in the market, especially high-end models, are 5G phones. The scenario for AI phones is significantly better because both manufacturers and consumers are very optimistic about the future experiences AI can offer. This situation has already been previewed in cloud AI, where, despite low revenues from AI applications (OpenAI's annual revenue is only 3.4 billion USD), major companies have invested over a hundred billion dollars (with NVIDIA's data center revenue expected to exceed 100 billion USD this year) because industry giants like Microsoft, Google, Meta, and Amazon fear falling behind in this AI wave. On the mobile front, we will see a similar scenario: various manufacturers competing to launch AI phones and gradually extending from high-end to low-end models, with virtually no non-AI phone models within a year or two. AI PCs are progressing slower than AI phones; the first AI PC was Microsoft's Surface, launched just this June, and Windows Copilot will only become practically usable with the Windows update in September-October this year. However, the overall logic for AI PCs is similar to that for phones, though PCs will be more office-oriented.

2. Increased value

Therefore, even without considering the long replacement cycle brought by killer apps, the incremental value brought by switching from ordinary mobile phones and PCs to AI mobile phones and AI PCs is hard to ignore:

Among them, the two parts with the largest increase in value are:

Chip:

Looking at the data in recent years, generally speaking, mobile phone chips are updated every year, PC chips are updated every two years, and the cost of each generation increases by ~10%: if the process remains unchanged, the chip area will increase by 10-20%; if the process is upgraded, the chip area will decrease by 10-20%, but because the process upgrade causes the wafer cost to increase by 30-40%, the actual chip price will still increase by 10-20%. Under the wave of Edge AI, the computing power demand of NPU has increased rapidly, requiring more chip area, and it is even possible that the chip area will not decrease while the process is upgraded. For example, TSM said at the earnings conference that the area of ​​AI chips has increased by 5-10%, although this year Qualcomm and MediaTek upgraded from 4nm to 3nm.

Storage:

In order to run large models on the edge, the capacity of DRAM must also be increased. Apple's AI function can only run on mobile phones with more than 8G (the 23-year iPhone 15 Pro is 8G, and the regular version is 6G). Samsung's S24 has two versions, 8G and 12G. It is said that subsequent Android AI phones will be upgraded to 16G. The situation is similar for PCs. The current basic configuration of PCs is 8G memory, but Windows Copilot requires at least 16G memory. It can be said that the value increment of Edge AI storage is higher than that of chips. In 2023, DRAM used for mobile phones and PCs will account for ~45% of the total DRAM usage. If this part of the usage doubles in the next few years, the overall DRAM market size will increase by about 50% in quantity. If AI mobile phones and AI PCs penetrate rapidly, resulting in insufficient DRAM production capacity (HBM usage is also growing rapidly, and HBM consumes twice as much wafer as ordinary DRAM), it is very likely that storage will be in short supply and will increase prices significantly in the next few years.

3. Competition landscape

In the smartphone industry, Apple, Qualcomm, and MediaTek are the main competitors, with particularly intense competition between Qualcomm and MediaTek. Last year, both Qualcomm and MediaTek utilized Arm's standard architecture, resulting in MediaTek's flagship performance even surpassing Qualcomm's. This year, Qualcomm's new chip will use the Oryon architecture developed by Nuvia, a company founded by former Apple chip architects and acquired by Qualcomm in 2021, which might give Qualcomm an edge in the competition.

The competition in PC chips is even fiercer. Intel has fully switched to TSMC, with its new chip, Lunar Lake, utilizing TSMC's 3nm process and chiplet technology, freeing it from previous manufacturing constraints. AMD has also launched a new generation of AI chips, manufactured using TSMC's 4nm process, with a significant increase in NPU power to 50 TOPS, placing it at the forefront of current products. Qualcomm has also entered the PC CPU race with the X Elite, using TSMC's 4nm process and an NPU power of 45 TOPS. Its performance is comparable to x86 chips of the same process, but with significantly better battery life, and it has received strong support from Microsoft.

Although changes in market shares between AMD and Intel will depend on the new chip performances, the battle between Arm and x86 chips has already begun. Currently, the advantages and disadvantages of Arm chips are clear: the advantage is longer battery life, and the disadvantage is a weaker software ecosystem. However, the battery life advantage of Arm architecture over x86 is permanent, and the software ecosystem is expected to improve over time. In the long run, Arm is certain to take market share from x86, especially in the laptop sector.

Reflecting on AMD's history of taking market share from Intel, AMD launched the Zen architecture in 2017, manufactured by GlobalFoundries; Zen2 was launched in 2019 using TSMC's 7nm process, and AMD's market share increased from 9% to 20%. In recent years, due to the significant battery life advantage of Macs, Microsoft faces pressure from Windows users switching to other platforms, urgently needing an Arm PC chip to address this weakness. Additionally, the new workload brought by AI PCs has further increased the demand for low power consumption, and it is expected that the penetration rate of Arm PCs will be significantly faster than AMD's rate of gaining market share. We anticipate that Arm PCs (including those from Qualcomm, MediaTek, etc.) will achieve 30% of the notebook market share by 2027, with a long-term goal of 60% (Notebooks constitute about three-quarters of total PC shipments).

Of course, what the future will be like? Everything has just begun and is still full of variables. Let us wait and see.