Lecture 3 From Word2Vec to Transfomers

In-class Material

1. Slides

2. Notebook

Extra Reading

1. RNN vs Autoregressive Models: Transformer

2. Awesome BERT & Transfer Learning in NLP

3. Why BERT Fails In Commercial Environments

4. When Recurrent Models Don’t Need to be Recurrent

5. The Time Series Transformer

6. Applying massive language models in the real world with Cohere

7. Understanding and Coding the Self-Attention Mechanism of Large Language Models From Scratch