Edge AI Reaches a Critical Point: Computing-in-Memory Moves from Lab to Industrial Mainstream
As large models accelerate their transition from the cloud to the edge, edge AI is moving from "being able to run models" to "being needed by users." Over the past year or so, the size of models with equivalent intelligence levels has continued to shrink, the scale of models that edge chips can support has jumped from the 7B level to over 100B, and the iteration speed of core elements such as models and chips has significantly accelerated, thereby speeding up the commercialization process of edge AI.
Multiple industry insiders told reporters from China Business Journal that the ecosystem closed-loop for edge AI is gradually forming. Chip manufacturers are continuously improving edge computing power, model manufacturers are constantly compressing model parameters, and terminal manufacturers are accelerating the integration of edge AI capabilities into various devices. This synchronized advancement across different segments of the industrial chain has transformed edge AI from a single-point technological breakthrough into a complete, deployable product. "Being needed" has become the industry consensus for edge AI, and it will next rapidly penetrate into consumer electronics, intelligent manufacturing, automotive travel, and other fields, opening up new growth space.
The Window for Edge AI Has Opened
To judge whether edge AI is mature, the industry is accustomed to looking at two underlying variables first: models and chips. Xin Xiaoxu, co-founder and vice president of product at Houmo AI, provided a comparison: taking Alibaba's Qwen as an example, over the past 17 months, the size of Dense models with equivalent intelligence has been compressed to 1/8 of the original; while the more closely watched MoE (Mixture of Experts) models have been compressed even faster, reaching 1/6 of the original in just 6 months. Changes on the chip side have been equally dramatic. Around 2024, edge AI chips could only run 7B parameter models, with single-token performance stuck in the single digits; by October 2026, publicly demonstrated solutions were already capable of running 120B-level models locally.
"Users never need the most expensive model; they only need a model that can solve their needs," said Wang Guanbo, founder of Wange Zhiyuan, whose judgment represents the views of many practitioners. He believes that as model density laws improve, models with tens to over a hundred billion parameters can already cover the vast majority of daily office and programming scenarios. "In the future, only two types of tokens will have value: the most expensive cloud tokens and edge tokens that are approaching free."
Xu Chao, industry director of intelligent hardware for the Qwen model at Alibaba Cloud Intelligence Group, revealed that the latest generation of 125B parameter Qwen models can already cover 80%–85% of Agent-class application scenarios, while the 27B model can handle 50%–60% of programming tasks. This means that edge models no longer need to compromise between "good enough" and "easy to use."
The rapid miniaturization on the model side has directly transferred pressure to the chip side. When a terminal needs to run a 10-billion-parameter model within a power budget of a dozen or even a few watts, the traditional "separation of computing and storage" architecture hits a wall. "In the past, computing was always the protagonist, and storage was just a supporting role; but in the AI era, the entire script has been rewritten," Xin Xiaoxu said. Taking HBM (High Bandwidth Memory) as an example, this niche category, which barely existed outside of professional fields before AI, has now become a battleground for major storage manufacturers. This essentially reflects the same contradiction: the repeated movement of data between computing units and storage units is becoming the biggest bottleneck for performance and power consumption.
Computing-in-memory (CIM) attempts to solve this problem at the architectural level, allowing data to be calculated directly within the storage array rather than being moved back and forth.
However, computing-in-memory is not without controversy. Against the backdrop of slowing Moore's Law, HBM capacity shortages, and the concentration of advanced packaging capacity, this architecture is seen by many entrepreneurs as a "lane-changing" opportunity to bypass the process race. However, some observers point out that computing-in-memory still needs to catch up in terms of precision, programmability, and ecosystem toolchains. A chip industry analyst attending the conference told reporters that the real test for computing-in-memory is not the energy efficiency ratio in the lab, but whether it can be continuously delivered on mass production lines and whether it can be accepted by mainstream software ecosystems.
Why is the technology needed right now? A more urgent demand than "saving tokens" comes from compliance and privacy. Lin Jun, deputy general manager of the terminal product department at China Great Wall Technology, stated that in data-sensitive industries such as government, finance, justice, and healthcare, customers are most eager to use AI, but the compliance red lines are also the strictest. "In market bidding, keeping data on the device is not a bonus item, but an entry ticket," he said.
If the demand on the industrial side is driven by compliance, then the discussion from a governance perspective pushes the issue even further. Jiang Changjian, deputy director of the Global AI Innovation Governance Center and an associate professor at Fudan University, believes that the sinking of computing power is not just a technical choice, but a redistribution of power, risk, and responsibility. "Robot walking, car obstacle avoidance, and industrial control must complete perception and decision-making within an extremely short time and cannot rely on cloud instructions; once the network is disconnected or the cloud service fails, the consequences could endanger personal safety." He advocates for a hybrid architecture: the edge is responsible for real-time control, privacy-sensitive tasks, and basic safety, while the cloud is responsible for complex reasoning, large-scale training, and necessary coordination.
From "Able to Run" to "Being Needed"
Behind the implementation of the technology is the support of the regional industrial ecosystem. Li Yantao, deputy director of the Beijing Economic-Technological Development Area (BDA) Management Committee and director of the Beijing Integrated Circuit Major Office, provided more specific figures: Yizhuang has gathered nearly 400 integrated circuit companies, covering the entire industrial chain of design, manufacturing, advanced packaging, equipment, and components. From January to September 2026, the total output value of integrated circuits in the district reached 166.7 billion yuan, a year-on-year doubling. The new-generation information technology industry with integrated circuits as the core has become the largest leading industry in Yizhuang. In August 2026, Yizhuang released the country's first "AI for Chip" special policy.
Houmo AI is a representative company in this ecosystem. According to Xin Xiaoxu, the company currently has two generations of chips in mass production, covering four major application scenarios. From the time the M50 chip was sent for testing by third-party customers to the end of August 2026, it had already obtained more than 150 product design wins. It has also completed adaptation with mainstream platforms such as X86, ARM, RISC-V, Windows, Ubuntu, OpenHarmony, Android, Kylin, UnionTech, and OpenEuler, alongside CPU, operating system, and AI Infra partners.
Echoes from the terminal side have already appeared: Li Weichang, vice president of Lenovo Group, revealed that the AI workstation P7, a collaboration between Lenovo and Houmo, will be launched on October 24, positioned as an "exclusive AI computing center placed by the user's side."
Behind the excitement, practitioners still have a clear understanding of the "critical point." Chen Zhongming, chief engineer of AI systems at Houmo AI, summarized the engineering constraints of current edge chips with an "impossible triangle"—it is difficult to maximize computing power, bandwidth, and power consumption simultaneously. Houmo AI's strategy is to address this with two product lines: the M series maximizes computing power, while the A series maximizes bandwidth through 3D CIM. Chen Zhanling, CMO of Yinpu Computing, pointed out that the current real bottleneck is not chip parameters, but scenarios and ecosystems. "We see that the actual situation is that scenarios are not yet sufficient. If we want to expand on a larger scale, the speed will still be slow."
At the same time, cost is another unavoidable topic. Huang Junshi, founder of Liaoqu Intelligent, admitted that in implementation scenarios such as elderly care and government affairs, corporate budgets vary greatly, and most customers cannot afford the computing power expenditures of leading companies. "Cost is a very critical part." And with storage prices currently high, even chip manufacturers have to admit that "it's good enough if you can get the memory."
In the view of the aforementioned analyst, the competition for edge AI has just begun. When the miniaturization curve on the model side, architectural innovation on the chip side, industrial guidance on the policy side, and compliance-driven demand on the user side form a resonance in 2026, whether computing-in-memory, as one of the technical routes, can truly grow from a "lab star" to an "industrial mainstream" will depend on the delivery on mass production lines over the next one to two years, the maturity of the software ecosystem, and those terminal scenarios that have not yet been defined.
