SFU NLP class: Syllabus

Georges Artsrouni's mechanical brain, a translation device patented in 1933 in France.

Schedule

The schedule is preliminary and subject to change. Slides will be updated as the term progresses.

Course Recordings (CMPT 713 X1)

Week	Date	Topic	Assignments	Readings and Resources
1
	9/9	Lecture: Introduction (NLP history and basics of language modeling) [L1-intro]		(optional) How Language Shapes the Way We Think. Lera Boroditsky. (optional) Meaning and Intelligence in Language Models (KDD 2025 Keynote). Chris Manning.
	9/11	Lecture: Introduction 2 and Language Modeling [L1a-intro-2] [L1b-language-models (Updated Sept 15)]		N-Gram Language Models (J+M chapter 3) (optional) Language models. Michael Collins.
	9/11	Tutorial (optional): Review of a) probability, linear algebra and calculus and b) useful python/unix commands
2
	9/16	Lecture: Text classification - Naive Bayes and evaluation [L2-classification (Updated Sept 15)]		Logistic Regression and Text Classification (J+M chapter 4)
	9/18	Lecture: Text classification - Logistic Regression [L3-classification-LR-zp (Updated Sept 15)]	Due: HW0
	9/18	Tutorial (optional): Building language models
3
	9/23	Lecture: NN review and word representations [L4-neural-nets-zp (Updated Sept 28)]		Neural Networks (J+M chapter 6) (optional) Feedforward neural networks. Michael Collins. (optional) A Primer on Neural Network Models for Natural Language Processing. Yoav Goldberg.
	9/25	Lecture: Word Representations and neural language models [L5-wordvectors-zp (Updated Oct 2nd)]		Embeddings (J+M chapter 5) (optional) Improving Distributional Similarity with Lessons Learned from Word Embeddings. Omer Levy, Yoav Goldberg, and Ido Dagan.
	9/25	Tutorial (optional): Learning pytorch
4
	9/30	No class - No class - National Day for Truth and Reconciliation
	10/2	Lecture: Word2Vec [L6-word2vec (Updated Oct 8th)]		Embeddings (J+M chapter 5) (optional) Improving Distributional Similarity with Lessons Learned from Word Embeddings. Omer Levy, Yoav Goldberg, and Ido Dagan.
	10/2	Tutorial (optional): Building classifiers with pytorch
5
	10/7	Lecture: Word2Vec [L6-word2vec (Updated Oct 8th)]	Due: HW1	Embeddings (J+M chapter 5) (optional) Improving Distributional Similarity with Lessons Learned from Word Embeddings. Omer Levy, Yoav Goldberg, and Ido Dagan.
	10/9	Lecture: Word2Vec [L6-word2vec (Updated Oct 8th)]		Embeddings (J+M chapter 5) (optional) Improving Distributional Similarity with Lessons Learned from Word Embeddings. Omer Levy, Yoav Goldberg, and Ido Dagan.
	10/9	Tutorial (optional): Building text classifiers with word embeddings
6
	10/14	Lecture: Sequence Models (HMMs) [L9-sequence-modeling -zp (Updated Oct 16th)]		Hidden Markov Models (J+M Appendix A.1-A.4)
	10/16	Lecture: Neural Sequence Modeling (RNNs) [L10-RNN-zp (Updated Oct 23rd)]		RNNs and LSTMs (J+M chapter 13) (optional) Understanding LSTM Networks. Christopher Olah.
	10/16	Tutorial (optional): Building text classifiers with pretrained word embeddings
7
	10/21	Lecture: Midterm Exam
	10/23	Lecture: Neural Sequence Modeling (LSTM/GRU), Sequence-to-Sequence Models, and Project Information [L11a-LSTM-GRU-zp (Updated Oct 23rd)] [L11b-sequence-generation-zp (Updated Oct 27th)] [Project related P53 done (Updated Oct 28th)] [benchmarks]	Due: HW2
	10/23	Tutorial (optional): Neural language models with BoW/Fixed Window FFN
8
	10/28	Lecture: Transformers and Self-Attention [L12-transformers-zp (Updated Oct 30th)]	Due: Project proposal	Transformers (J+M chapter 8) (optional) Attention Is All You Need. Vaswani et al. 2017. (optional) The Annotated Transformer. Alexander Rush.
	10/30	Lecture: Modern LLM architecture [L17-modern-llms]
	10/30	Tutorial (optional): Using RNNs in pytorch
9
	11/4	Lecture: Posttraining - instruction tuning and preference alignment [L18-instruct_tuning] [L19-preference-alignment]
	11/6	Lecture: Posttraining - preference alignment and reasoning [L19-reasoning]	Due: HW3
	11/6	Tutorial (optional): Training/debugging networks and running experiments
10
	11/11	No class - No class - Remembrance Day
	11/13	Lecture: Scaling laws and data [L20-scaling]
	11/13	Tutorial (optional): Using Transformers in pytorch
11
	11/18	Lecture: Final project tips, model debugging and analysis [L21-project-tips-analysis]	Due: HW4
	11/20	Lecture: Information retrieval and RAG [L23-retrieval]
	11/20	Tutorial (optional): Prompting large language models
12
	11/25	Lecture: Spare class	Due: Project milestone
	11/27	Lecture: Spare class
13
	12/2	Lecture: Grounding and multimodal models [L24-grounding]
	12/4	Lecture: Final project presentations & Conclusion [L26-conclusion]
Exam
Exam	TBD	Lecture: Final Exam	Due: Final project report