Server/Cloud Computing/AI Forum - Korea

Coming soon

Coming soon

The rapid growth of machine learning is accelerating the adoption of AI applications across a range of platforms, including cloud-based servers, edge devices, and endpoint devices. However, deploying AI at the edge and on the end-point devices introduces unique challenges, particularly related to system memory management. These challenges include limited memory resources, power consumption restrictions, bandwidth constraints, and the need for real-time processing capabilities. This presentation will begin by outlining the expected trends in edge AI platforms. It will then offer an overview of the key components essential for edge AI computing, highlighting their relative importance. Also, it will highlight the memory usage demand of the Key Value Cache layer in Large Language Model (LLM). Finally, recent advancements in system memory technology from JEDEC will be discussed and evaluated in the context of addressing the specific challenges and demands of edge AI computing.

Coming soon.

Introduction to MediaTek - MediaTek datacenter products - Challenges to innovation in the memory space - Summary and Call to action.

Coming soon.

The coming generation of memory solutions pose unique challenges for JEDEC. The module solutions are smaller than previous generations, yet device density is higher and power consumption continues to go up. Quantum computing is on the horizon, making the next generation of systems potentially vulnerable to a new variety of malicious attacks. This talk will holistically examine how these challenges tie together from voltage regulation (ala PMIC), registers, and serial presence detect (SPD) with an expansion of the Hub features introduced in DDR5 generation.

As the Wearables (AI Glasses, AR Glasses, and Smart Glasses) market experiences rapid and robust growth, we are confronted with a significant challenge: the absence of smaller capacity NAND (eMMC/UFS) options in the market. The trend is towards increasing capacity, driven by the demands of infrastructure, mobile, and automotive sectors. In this presentation, my objective is to emphasize the necessity of creating wearables-specific NAND standards that not only accommodate smaller capacities but also achieve substantial reductions in peak power consumption and footprint. Establishing these new market requirements is crucial.

Coming soon

In the post-Moore era, the shrinkage of 2D DRAM technology is approaching the limit, and 3D DRAM has not yet arrived. AI intelligent computing is booming, which has led to explosive growth in the demand for memory bandwidth and capacity. AI training, inference, central inference, edge inference, and different AI computing architectures have very diverse requirements for memory bandwidth & capacity. Existing memory interface standards can no longer meet the requirements of AI computing. Whether future memory solutions are standardized or customized requires rapid response from the memory industry and JEDEC standards organizations, based on advanced packaging and chiplet technology to provide the optimal solution.

Server memory solutions had faced challenges at every step forward. 4400Mbps DDR5 SDRAM had been adopted as the 1st generation after 3200Mbs DDR4, and has reached to 6400 today with expectation of expanding to 9200Mbps in coming years. This presentation tries to review brief historical steps of evolutions up to today, and to look ahead to the future.

As the industry continues to push the boundaries of memory system performance, the challenges in system characterization and qualification are increasing in complexity. The rapid evolution of technologies, such as AI, high-performance computing, and mobile devices has placed new demands on our testing capabilities. The ability to probe and measure high-speed signals in modern systems is becoming more difficult, as the bandwidth of our testing equipment struggles to keep up with the speed of signaling. In this presentation we will explore these growing challenges in the context of memory subsystems. We will look into the unique issues faced in qualifying memory modules across different industry segments, focusing on high end server-class memory. Additionally, we will examine the physical limitations of current equipment, specifically the difficulty of mid-bus probing. We will show how emerging advanced testing solutions are helping to address these issues in leading new memory technologies such as RDIMM, MRDIMM and CAMM2 module families.

HBM, LPDDR, GDDR, and DDR DRAM; on-die SRAM; novel nonvolatile memories; and CXL(TM)-based storage devices have all been proposed as the main memory in different AI processors. How should a chip architect choose the correct memory for their AI application? This presentation will focus on the non-Von-Neumann GPU/TPU/NPU architectures used in AI, how they interact with memory, and how that drives the choice of memory for edge, client and datacenter AI chips.

AI and HPC performance is being limited by the memory and IO walls. The performance of HPC & AI architectures can be improved by utilizing HBM4 DRAM (a new specification recently published by JEDEC) along with custom HBM4 solutions.

Over time, as memory interface speeds have increased, the fundamental approach used to move data has had to change. Traditional high speed digital timing and noise with min/typ/max specifications has given way to high speed serial approaches based on eye masks with jitter specifications. Next generation memory must go a step further to deal with distorted eyes using tunable equalization. At each point the need to characterize and measure what’s defined in the spec has made measurement science and design for test (DFT) increasingly important in defining a standard. This session will focus on the Measurement Science as used for both device and system validation.