Apple's AI architecture bypasses memory limits

Apple has developed a novel on-device AI architecture, AFM 3 Core Advanced, that circumvents traditional memory constraints. By storing model weights in NAND flash instead of DRAM and routing decisions per prompt rather than per token, the 20-billion-parameter model can operate locally. This breakthrough allows for more capable AI agents on consumer devices, addressing a significant limitation previously faced by developers. The architecture dynamically adjusts active parameters based on task complexity, offering a flexible solution for on-device AI workloads. While Apple has detailed the memory design, further information on practical deployment constraints like energy and thermal performance is expected later this summer.