Lecture 4 LLM and Its Practices I

In-class Material

1. Slides

2. Notebook

Extra Reading

1. Large Language Models: A Survey

2. GPT1: Improving Language Understanding by Generative Pre-Training

3. GPT2: Language Models are Unsupervised Multitask Learners

4. GPT3: Language Models are Few-Shot Learners

5. InstructGPT: Training language models to follow instructions with human feedback

6. Llama2

7. Ilya Sutskever: “pretraining is done. we are now in the post training era”

8. How Scaling Laws Will Determine AI’s Future