Awesome-LLM-Survey

This repo aims to record survey of LLM, including instruction tuning, human alignment, LLM agent, hallucination, multi-modal, etc.

We strongly encourage the researchers that want to promote their fantastic work to the LLM survey community to make pull request to update their paper's information!

• Awesome-LLM-Survey
  • General Survey
  • Training of LLM
    • Instruction Tuning
    • Human Alignment for LLM
  • Prompt of LLM
    • Chain of Thought for LLM
    • Prompt Engineering for LLM
    • Retrieval-Augmented LLM
  • Challenge of LLM
    • Hallucination in LLM
    • Compression for LLM
    • Evaluation of LLM
    • Reasoning with LLM
    • Explainability for LLM
    • Fairness in LLM
    • Graph for LLM
    • Long-Context for LLM
    • Factuality in LLM
    • Knowledge for LLM
    • Self-Correction for LLM
    • Attributions for LLM
    • Tool Using of LLM
    • Calibration of LLM
    • Agent of LLM
    • Role-play of LLM
    • Vulnerabilities of LLM
    • Efficiency of LLM
    • Data of LLM
    • Security and Privacy of LLM
    • Continual Learning of LLM
  • Mulitmodal of LLM
    • Visual LLM
    • Audio LLM
    • Code LLM
  • LLM for Domain Application
    • LLM for Health
    • LLM for Finance
    • LLM for Education
    • LLM for Law
    • LLM for Mental Health
    • LLM for Robotics
  • LLM for Downstream Tasks
    • LLM for Recommendation
    • LLM for Information Retrieval
    • LLM for Software Engineering
    • LLM for Autonomous Driving
    • LLM for Time Series
    • Detection of LLMs-Generated Content
    • LLM for Society
    • LLM for Citation
    • LLM for Text Watermarking
    • LLM for Math
    • LLM for Environmental Disciplines
    • LLM for Information Extraction
    • LLM for Data Annotation
    • LLM for Game
  • Star History

---

General Survey

• A Survey of Large Language Models, 2023.11 [paper][project]

• A Survey of GPT-3 Family Large Language Models Including ChatGPT and GPT-4, 2023.10 [paper]

• Challenges and Applications of Large Language Models, 2023.07 [paper]

• Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond, 2023.04 [paper][project]

• A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT, 2023.02 [paper]

• Large language models: a comprehensive survey of its applications, challenges, limitations, and future prospects, 2023.12 [paper] [project]

• The future of gpt: A taxonomy of existing chatgpt research, current challenges, and possible future directions, 2023.04 [paper]

• A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges, 2023.10 [paper]

• Understanding LLMs: A Comprehensive Overview from Training to Inference, 2024.01 [paper]

---

Training of LLM

Instruction Tuning

• Are Prompts All the Story? No. A Comprehensive and Broader View of Instruction Learning, 2023.03 [paper] [project]

• Vision-Language Instruction Tuning: A Review and Analysis, 2023,11 [paper][project]

• Instruction Tuning for Large Language Models: A Survey, 2023.08 [paper]

• A Survey on Data Selection for LLM Instruction Tuning, 2024.02 [paper]

Human Alignment for LLM

• Towards a Unified View of Preference Learning for Large Language Models: A Survey, 2024.09 [paper][project]

• AI Alignment: A Comprehensive Survey, 2023.10 [paper][project]

• Large Language Model Alignment: A Survey, 2023.09 [paper]

• From Instructions to Intrinsic Human Values -- A Survey of Alignment Goals for Big Model, 2023.08 [paper][project]

• Aligning Large Language Models with Human: A Survey, 2023.07 [paper][project]

---

Prompt of LLM

Chain of Thought for LLM

• Towards Better Chain-of-Thought Prompting Strategies: A Survey, 2023.10 [paper]

• A Survey of Chain of Thought Reasoning: Advances, Frontiers and Future, 2023.09 [paper][project]

• Igniting Language Intelligence: The Hitchhiker's Guide From Chain-of-Thought Reasoning to Language Agents, 2023.11 [paper] [project]

Prompt Engineering for LLM

• Prompting Frameworks for Large Language Models: A Survey, 2023.11 [paper][project]

• Unleashing the potential of prompt engineering in Large Language Models: a comprehensive review, 2023.10 [paper]

• Towards Better Chain-of-Thought Prompting Strategies: A Survey, 2023.10 [paper]

Retrieval-Augmented LLM

• A Survey on Retrieval-Augmented Text Generation, 2022.02 [paper]
• Retrieval-Augmented Generation for Large Language Models: A Survey, 2023.12 [paper] [project]
• RAG and RAU: A Survey on Retrieval-Augmented Language Model in Natural Language Processing, 2024.04 [paper]

---

Challenge of LLM

Hallucination in LLM

• Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey, 2023.11 [paper]

• A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions, 2023.11 [paper][project]

• A Survey of Hallucination in “Large” Foundation Models, 2023.09 [paper][project]

• Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models, 2023.09 [paper][project]

• Cognitive Mirage: A Review of Hallucinations in Large Language Models, 2023.09 [paper][project]

• Augmenting LLMs with Knowledge: A survey on hallucination prevention, 2023.09 [paper]

• A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models, 2024.01 [paper]

• Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignment, 2023.08 [paper]

• Hallucination of Multimodal Large Language Models: A Survey, 2024.04 [paper]

Compression for LLM

• A Survey on Model Compression for Large Language Models, 2023.08 [paper]
• A Comprehensive Survey of Compression Algorithms for Language Models, 2024.01 [paper]

Evaluation of LLM

• Evaluating Large Language Models: A Comprehensive Survey, 2023.10 [paper][project]

• A Survey on Evaluation of Large Language Models, 2023.07 [paper][project]

Reasoning with LLM

• Reasoning with Language Model Prompting: A Survey, 2022.12 [paper][project]

• A Survey of Reasoning with Foundation Models, 2023.12 [papaer][project]

Explainability for LLM

• Explainability for Large Language Models: A Survey, 2023.09 [paper]
• The Mystery and Fascination of LLMs: A Comprehensive Survey on the Interpretation and Analysis of Emergent Abilitie, 2023.11 [paper]
• If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents, 2024.01 [paper]
• From Understanding to Utilization: A Survey on Explainability for Large Language Models, 2024.01 [paper]
• Awesome Attention Heads, 2024.07 [project]
• Internal Consistency and Self-Feedback in Large Language Models: A Survey, 2024.07 [paper]
• Attention Heads of Large Language Models: A Survey, 2024.09 [paper]

Fairness in LLM

• A Survey on Fairness in Large Language Models, 2023.08 [paper]

Graph for LLM

• A Survey of Graph Meets Large Language Model: Progress and Future Directions, 2023.11 [paper]
• Large Language Models on Graphs: A Comprehensive Survey, 2023.12 [paper] [project]

Long-Context for LLM

• Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey, 2023.11 [paper]
• Length Extrapolation of Transformers: A Survey from the Perspective of Position Encoding, 2023.12 [paper]

Factuality in LLM

• A Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity, 2023.10 [paper][project]

• Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models, 2023.10 [paper]

Knowledge for LLM

• A Survey on Knowledge Distillation of Large Language Models, 2024.02 [paper]

• Knowledge Unlearning for LLMs: Tasks, Methods, and Challenges, 2023.11 [paper]

• Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applications, 2023.11 [paper]

• Knowledge Editing for Large Language Models: A Survey, 2023.10 [paper]

• Editing Large Language Models: Problems, Methods, and Opportunities, 2023.05 [paper][project]

• Building trust in conversational ai: A comprehensive review and solution architecture for explainable, privacy-aware systems using llms and knowledge graph, 2023.08 [paper]

Self-Correction for LLM

• Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies, 2023.08 [paper][project]

Attributions for LLM

• A Survey of Large Language Models Attribution, 2023.11 [paper][project]

Tool Using of LLM

• Foundation Models for Decision Making: Problems, Methods, and Opportunities, 2023.03 [paper]

• Augmented Language Models: a Survey, 2023.02 [paper]

Calibration of LLM

• A Survey of Language Model Confidence Estimation and Calibration, 2023.11 [paper]

Agent of LLM

• A Survey on Large Language Model based Autonomous Agents, 2023.08 [paper][project]
• The Rise and Potential of Large Language Model Based Agents: A Survey, 2023.09 [paper][project]
• Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives, 2023.12 [paper]
• Large Multimodal Agents: A Survey, 2024.02 [paper][project]

Role-play of LLM

• The Oscars of AI Theater: A Survey on Role-Playing with Language Models, 2024.09 [paper][project]
• Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization, 2024.06 [paper][project]
• From Persona to Personalization: A Survey on Role-Playing Language Agents, 2024.04 [paper]

Vulnerabilities of LLM

• Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks, 2023.10 [paper]

Efficiency of LLM

• The Efficiency Spectrum of Large Language Models: An Algorithmic Survey, 2023.12 [paper][project]

• Efficient Large Language Models: A Survey, 2023.12 [paper][project]

• Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment, 2023.12 [paper]

• A Survey on Hardware Accelerators for Large Language Models, 2024.01 [paper]

• Model Compression and Efficient Inference for Large Language Models: A Survey, 2024.02 [paper]

Data of LLM

• Data Management For Large Language Models: A Survey, 2023.12 [paper][project]
• A Survey on Data Selection for Language Models, 2024.02 [paper]
• Datasets for Large Language Models: A Comprehensive Survey, 2024.02 [paper][project]

Security and Privacy of LLM

• A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly, 2023.12 [paper]

Continual Learning of LLM

• Continual Learning with Pre-Trained Models: A Survey, 2024.01 [paper] [project]
• Continual Learning of Large Language Models: A Comprehensive Survey, 2024.04 [paper]

---

Mulitmodal of LLM

Visual LLM

• A Survey on Benchmarks of Multimodal Large Language Models. 2024,08 [paper][project]

• The (R)Evolution of Multimodal Large Language Models: A Survey, 2024,02 [paper]

• Vision-Language Instruction Tuning: A Review and Analysis, 2023,11 [paper][project]

• How to Bridge the Gap between Modalities: A Comprehensive Survey on Multimodal Large Language Model, 2023.11 [paper]

• A Survey on Multimodal Large Language Models, 2023.06 [paper][project]

• Multimodal Large Language Models: A Survey, 2023.11 [paper]

• Large Language Models Meet Computer Vision: A Brief Survey, 2023.11 [paper]

• Foundational Models Defining a New Era in Vision: A Survey and Outlook, 2023.07 [paper][project]

• Video Understanding with Large Language Models: A Survey, 2023.12 [paper] [project]

Audio LLM

• Sparks of large audio models: A survey and outlook, 2023.08 [paper] [project]

Code LLM

• A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond, 2024.03 [paper][project]
• A Survey on Language Models for Code, 2023.11 [paper][project]
• Pitfalls in Language Models for Code Intelligence: A Taxonomy and Survey, 2023.10 [paper][project]
• Large Language Models Meet NL2Code: A Survey, 2022.12 [paper]
• A Prompt Learning Framework for Source Code Summarization, 2023.12 [paper]
• If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents, 2024.01 [paper]

---

LLM for Domain Application

LLM for Health

• A Survey of Large Language Models in Medicine: Progress, Application, and Challenge, 2023.11 [paper][project]

• Large Language Models Illuminate a Progressive Pathway to Artificial Healthcare Assistant: A Review, 2023.10 [paper][project]

• Large AI Models in Health Informatics: Applications, Challenges, and the Future, 2023.03 [paper][project]

• A SWOT (Strengths, Weaknesses, Opportunities, and Threats) Analysis of ChatGPT in the Medical Literature: Concise Review, 2023.11 [paper]

• ChatGPT in Healthcare: A Taxonomy and Systematic Review, 2023.03 [paper]

LLM for Finance

• Large Language Models in Finance: A Survey, 2023.09 [paper]

LLM for Education

• ChatGPT and Beyond: The Generative AI Revolution in Education, 2023.11 [paper]

LLM for Law

• Large Language Models in Law: A Survey, 2023.12 [paper]

LLM for Mental Health

• A review of the explainability and safety of conversational agents for mental health to identify avenues for improvement, 2023.10 [paper]
• Towards a Psychological Generalist AI: A Survey of Current Applications of Large Language Models and Future Prospects, 2023.12 [paper]
• Large Language Models in Mental Health Care: a Scoping Review, 2024.01 [paper]

LLM for Robotics

• Large Language Models for Robotics: A Survey, 2023.11 [paper]

---

LLM for Downstream Tasks

LLM for Recommendation

• Foundation Models for Recommender Systems: A Survey and New Perspectives, 2024.02 [paper]
• User Modeling in the Era of Large Language Models: Current Research and Future Directions, 2023.12 [paper][project]
• A Survey on Large Language Models for Personalized and Explainable Recommendations, 2023.11 [paper]
• Large Language Models for Generative Recommendation: A Survey and Visionary Discussions, 2023.09 [paper]
• A Survey on Large Language Models for Recommendation, 2023.08 [paper][project]
• How Can Recommender Systems Benefit from Large Language Models: A Survey, 2023.06 [paper][project]

LLM for Information Retrieval

• Large Language Models for Information Retrieval: A Survey, 2023.08 [paper][project]

LLM for Software Engineering

• Large Language Models for Software Engineering: Survey and Open Problems, 2023.10 [paper]
• Large Language Models for Software Engineering: A Systematic Literature Review, 2023.08 [paper]

LLM for Autonomous Driving

• A Survey on Multimodal Large Language Models for Autonomous Driving, 2023.11 [paper]
• A Survey of Large Language Models for Autonomous Driving, 2023.11 [paper][project]

LLM for Time Series

• Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook, 2023.10 [paper][project]

Detection of LLMs-Generated Content

• A Survey on Detection of LLMs-Generated Content, 2023.10 [paper][project]
• A Survey on LLM-generated Text Detection: Necessity, Methods, and Future Directions, 2023.10 [paper] [project]
• Detecting ChatGPT: A Survey of the State of Detecting ChatGPT-Generated Text, 2023.09 [paper]

LLM for Society

• Large Language Models as Subpopulation Representative Models: A Review, 2023.10 [paper]

LLM for Citation

• When Large Language Models Meet Citation: A Survey, 2023.09 [paper]

LLM for Text Watermarking

• A Survey of Text Watermarking in the Era of Large Language Models, 2023.12 [paper]

LLM for Math

• Mathematical Language Models: A Survey, 2023.12 [paper]

LLM for Environmental Disciplines

• Recent applications of AI to environmental disciplines: A review, 2023.10 [paper]
• Opportunities and Challenges of Applying Large Language Models in Building Energy Efficiency and Decarbonization Studies: An Exploratory Overview, 2023.12 [paper]

LLM for Information Extraction

• Large Language Models for Generative Information Extraction: A Survey, 2023.12 [paper] [project]

LLM for Data Annotation

• Large Language Models for Data Annotation: A Survey, 2024.02 [paper] [project]

LLM for Game

• Large Language Models and Games: A Survey and Roadmap, 2024.02 [paper]
• A Survey on Game Playing Agents and Large Models: Methods, Applications, and Challenges, 2024.03 [paper] [project]

Star History