A systematic collection of reading notes for LLMs top conference papers, covering motivation and method analysis in core areas like PEFT (LoRA/QLoRA), RAG, and Agents (RoleLLM), providing structured learning paths for algorithm engineers.
Project Overview#
LLMs Paper Reading Club is a knowledge base project maintained by NLP developer 杨夕 (Yang Xi), designed to address the challenge of literature explosion in the Large Language Models field. The project provides structured paper reading notes to help algorithm engineers and researchers quickly grasp the research motivation, core methods, and code implementations of top conference papers.
Use Cases#
- Technology stack building and interview preparation for LLMs algorithm engineers
- Quick understanding of SOTA work in Prompt, LoRA, RAG, Agent and other sub-fields
- Internal team learning sharing and technology selection reference
Core Content Matrix#
PEFT Series (Parameter-Efficient Fine-Tuning)#
| Technique | Key Points |
|---|---|
| Prompt | Prompt learning methods for pre-trained language models |
| Instruction | Instruction tuning methods |
| Self-Instruct | Self-generated instruction distillation |
| LoRA | Low-Rank Adaptation for LLM fine-tuning with zero inference latency |
| DyLoRA | Dynamic search-free low-rank adaptation |
| LOMO | Full parameter fine-tuning with limited resources |
| QLoRA | 4-bit quantized fine-tuning strategy |
| VeRA | Vector-based Random Matrix Adaptation |
LoRA Method Details:
- Motivation: Adapters increase inference latency; Prefix Tuning is hard to optimize with non-monotonic performance
- Method: Add a bypass to the original model, simulate parameter updates through low-rank decomposition (down-projection matrix A + up-projection matrix B), freeze original model during training, add BA to original parameters during inference with zero additional latency
RAG Series (Retrieval-Augmented Generation)#
| Technique | Key Points |
|---|---|
| Self-RAG | Self-reflective retrieval-augmented generation |
| Active RAG (FLARE) | Active retrieval-augmented generation |
| MemSum-DQA | Long document question answering system |
| PDFTriage | Long structured document QA |
Multi-domain Applications: Medical, religious, commonsense, legal, knowledge graphs, task-oriented dialogue, automotive
LLMs Agents Series#
| Project | Description |
|---|---|
| RoleLLM | Role-playing capability benchmark |
| Character-LLM | Trainable role-playing agents |
| ChatHaruhi | Anime character revival technology |
| Role-Play with LLMs | Large language model role-playing research |
Other Topics#
- GPT Series: Table parsing, few-shot QA
- Prompt Series: Few-shot data augmentation methods
- LMMs Interpretability: LLM factuality survey, self-explanation studies
- LLMs4KG: ChatKBQA knowledge base QA framework
Knowledge Architecture#
llms_paper/
├── PEFT Series/
│ ├── Prompt / Instruction / Self-Instruct
│ └── LoRA / DyLoRA / LOMO / QLoRA / VeRA
├── GPT Series/
│ └── Table Parsing
├── RAG Series/
│ ├── RAG Tricks
│ └── RAG Applications
├── Prompt Series/
├── LMMs Interpretability/
├── LLMs4KG/
└── LLMs Agents/
└── Role-Playing
Note Structure Standard#
Each reading note contains:
- Paper Title and Conference/Source
- arXiv Link and GitHub Code Link (if available)
- Core Motivation analysis
- Methodology diagrams/steps
Usage Guide#
Online Reading#
Visit GitHub repository or Gitee mirror directly
Local Access#
git clone https://github.com/km1994/llms_paper.git
Use Markdown editors (Obsidian, Typora) for local search and reading
Related Project Ecosystem#
Projects maintained by the same author:
- LLMsNineStoryDemonTower: Systematic LLMs learning path
- LLMs_interview_notes: LLMs interview question bank
- NLP-Interview-Notes: Traditional NLP interview notes
- nlp_paper_study: Early NLP paper notes