SFU NLP class: Syllabus

Georges Artsrouni's mechanical brain, a translation device patented in 1933 in France.

Schedule

The schedule is preliminary and subject to change. Slides will be updated as the term progresses.

Course Recordings (CMPT 713 X1)

SFU Stream Channel
Recordings for future lectures will also be made available on the course website (via YouTube and Bilibili links).

Week	Date	Topic	Assignments	Readings and Resources
1
	9/9	Lecture: Introduction (NLP history and basics of language modeling) [slides]		(optional) How Language Shapes the Way We Think. Lera Boroditsky. (optional) Meaning and Intelligence in Language Models (KDD 2025 Keynote). Chris Manning.
	9/11	Lecture: Introduction 2 and Language Modeling [Intro 2 slides] [LM slides]		N-Gram Language Models (J+M chapter 3) (optional) Language models. Michael Collins.
	9/11	Tutorial (optional): Review of a) probability, linear algebra and calculus and b) useful python/unix commands
2
	9/16	Lecture: Text classification - Naive Bayes and evaluation [NB slides] [Evaluation slides]		Logistic Regression and Text Classification (J+M chapter 4)
	9/18	Lecture: Text classification - Logistic Regression [Logistic regression slides]	Due: HW0
	9/18	Tutorial (optional): Building language models
3
	9/23	Lecture: NN review and word representations [Neural network and classification review] [WV slides 1]		Neural Networks (J+M chapter 6) (optional) Feedforward neural networks. Michael Collins. (optional) A Primer on Neural Network Models for Natural Language Processing. Yoav Goldberg.
	9/25	Lecture: Word Representations and neural language models [WV slides 2]		Embeddings (J+M chapter 4) (optional) Improving Distributional Similarity with Lessons Learned from Word Embeddings. Omer Levy, Yoav Goldberg, and Ido Dagan.
	9/25	Tutorial (optional): Learning pytorch
4
	9/30	No class - No class - National Day for Truth and Reconciliation
	10/2	Lecture: Sequence modeling (HMMs and RNNs) [slides] [slides]		Hidden Markov Models (J+M Appendix A.1-A.4)
	10/2	Tutorial (optional): Building classifiers with pytorch
5
	10/7	Lecture: Neural Sequence Modeling (LSTM/GRU) [slides]	Due: HW1	RNNs and LSTMs (J+M chapter 13) (optional) Understanding LSTM Networks. Christopher Olah.
	10/9	Lecture: Sequence generation, attention, and intro to transformers [sequence generation slides] [transformer slides]
	10/9	Tutorial (optional): Building text classifiers with word embeddings
6
	10/14	Lecture: Transformers [transformer slides]		Transformers (J+M chapter 8) (optional) Attention Is All You Need. Vaswani et al. 2017. (optional) The Annotated Transformer. Alexander Rush.
	10/16	Lecture: Pretraining LLMs [pretraining slides]		N-Gram Language Models (J+M chapter 3) (optional) Language models. Michael Collins. N-Gram Language Models (J+M chapter 3) (optional) Language models. Michael Collins.
	10/16	Tutorial (optional): Building text classifiers with pretrained word embeddings
7
	10/21	Lecture: Midterm Exam
	10/23	Lecture: Tasks, benchmarks and project information [project info slides] [benchmark slides]	Due: HW2
	10/23	Tutorial (optional): Neural language models with BoW/Fixed Window FFN
8
	10/28	Lecture: Using LLMs: Prompting and parameter efficient fine-tuning [prompting] [fine-tuning]	Due: Project proposal
	10/30	Lecture: Modern LLM architecture [slides]	Due: HW3
	10/30	Tutorial (optional): Using RNNs in pytorch
9
	11/4	Lecture: Posttraining - instruction tuning and preference alignment [instruction tuning] [preference alignment]
	11/6	Lecture: Posttraining - preference alignment and reasoning [slides]
	11/6	Tutorial (optional): Training/debugging networks and running experiments
10
	11/11	No class - No class - Remembrance Day
	11/13	Lecture: Scaling laws and data [slides]
	11/13	Tutorial (optional): Using Transformers in pytorch
11
	11/18	Lecture: Final project tips, model debugging and analysis [slides]	Due: HW4
	11/20	Lecture: Information retrieval and RAG [slides]
	11/20	Tutorial (optional): Prompting large language models
12
	11/25	Guest Lecture by Issam Laradji: Emerging directions in Large Language Models and AI agents	Due: Project milestone
	11/27	Lecture: LLMs for math and jailbreaking LLMs Jailbreaking LLMs (Jialin Song) AI4Math (Mike Lu)
13
	12/2	Lecture: Grounding and multimodal models. Talks from TAs [slides] Text and 3D Representation Learning (Qirui Wu) When Language Meets the Physical World: Semantic Representation for Robots (Sonia Raychaudhuri) Text-to-Scene Generation (Xiaohao Sun) LLMs for modeling DNA (Austin Wang) Visual grounding in 3D (from 2025) (Austin Wang)
	12/4	Lecture: Final project presentations & Conclusion [slides]
Exam
Exam	12/9	Lecture: Final Exam	Due: Final project report