A Systematic Survey on Large Language Models for Algorithm Design

Algorithm Design (AD) is crucial for effective problem-solving across various domains. The advent of Large Language Models (LLMs) has notably enhanced the automation and innovation within this field, offering new perspectives and promising solutions. Over the past three years, the integration of LLMs into AD (LLM4AD) has progressed significantly, finding applications in diverse areas such as optimization, machine learning, mathematical reasoning, and scientific discovery. Given the rapid development and broadening scope of this field, a systematic review is both timely and essential. This paper provides a systematic review of the works on LLM4AD. First, we present an overview and summary of existing studies. Then, we present a systematic taxonomy and a review of existing works along four dimensions, including the role of LLMs, search techniques, prompt strategies, and applications, with a discussion of the potential and achievements of using LLMs. Finally, we explore current challenges and propose several open questions and promising directions for future research.