Efficient Large Foundation Models Design: A Perspective From Model and System Co-Design

This paper focuses on modern efficient training and inference technologies on foundation models and illustrates them from two perspectives: model and system design. Model and System Design optimize LLM training and inference from different aspects to save computational resources, making LLMs more efficient, affordable, and more accessible. The paper list repository is available at \url{https://github.com/NoakLiu/Efficient-Foundation-Models-Survey}