Lecture 5 Training & Scaling LLM

In-class Material

1. Slides

2. Notebook

Extra Reading

1. LLM Scaling Law

2. Scaling Laws for Neural Language Models

3. Instruct GPT

4. New LLM Pre-training and Post-training

5. Building a LLM (from Scratch) in PyTorch

6. HuggingFace Course: FineTune LLM