ECE 467: Natural Language Processing

• Instructor Info
• Textbook
• Grading
• Schedule

Instructor Info

Instructor: Carl Sable
Email: carl.sable@cooper.edu
Office: Room 614

Textbook

"Speech and Language Processing, 3rd Edition" (draft, in-progress, on-line only)
by Daniel Jurafsky and James H. Martin

We will also rely on content from various published papers, as cited throughout the course.

Grading

Assignments will be posted here when they are assigned.

• Text categorization project: 25%
  Click here for the corpora for the text categorization project
• First deep learning project: 25%
• Final project (open-ended deep learning project): 25%
• Three quizzes: 25% (total)

Schedule

Note 1: All dates and topics which have not yet occurred are tentative.

Note 2: All below references to chapters and sections of the Jurafsky and Martin textbook are based on a draft version of the 3rd edition dated August 24, 2025, unless otherwise noted.

• Wednesday, September 3
  Topic 1: Course Introduction (Slides)
  See the syllabus as a single document
• Thursday, September 4
  Topic 2: Words, Morphology, Subwords, and Tokenization (Slides)
  Suggested reading: Chapter 2 of Jurafsky and Martin
• Wednesday, September 10
  Topic 2 (continued)
  Topic 3: Vector Space Models, Information Retrieval, and Text Categorization (Slides)
  Suggested reading: Sections 5.2 - 5.4, Section 11.1, Sections B.1 - B.8 (apppendixes are provided as separate files on the textbook's website) of Jurafsky and Martin
• Thursday, September 11
  Topic 3 (continued)
• Wednesday, September 17
  Topic 3 (continued)
  Project #1 assigned
• Thursday, September 18
  Topic 4: Linguistic Concepts: Parts of Speech, Syntax, Semantics, Pragmatics, Discourse, Coreference (Slides)
  Suggested reading: Section 17.1, Sections 18.1 - 18.6, Section 19.1, Section 23.1, Section 23.9, Section 24.1, Appendix F, Sections H.1 - H.4 of Jurafsky and Martin
• Wednesday, September 24
  Topic 4 (continued)
• Thursday, September 25
  Quiz #1
• Wednesday, October 1 (first half)
  Topic 4 (continued)

• Wednesday, October 1 (second half)
  Topic 5: Feedforward Neural Networks (units, layers, activation functions, backpropagations, hyperparameters, training/validation/testing, etc.)
  Suggested reading: Sections 6.1 - 6.4. Section 6.6 of Jurafsky and Martin
• Thursday, October 2
  Topic 5 (continued)
• Wednesday, October 8
  Topic 5 (continued)
  Topic 6: Word Embeddings; Neural Language Models; Word2vec
  Suggested reading: Sections 5.5 - 5.7, Section 6.5 of Jurafsky and Martin, original word2vec paper, negative sampling paper
• Thursday, October 9
  Topic 6 (continued)
• Wednesday, October 15
  Topic 6 (continued)
  Topic 7: Recurrent Neural Networks (RNNs); Long Short-Term Memory Networks (LSTMs)
  Suggested reading: Sections 13.1 - 13.6 of Jurafsky and Martin, popular LSTM blog
• Thursday, October 16
  Topic 7 (continued)
  Topic 8: Sequence-to-Sequence Tasks and Encoder-Decoder Models; Cross-Attention; Machine Translation
  Suggested reading: Sections 13.7 - 13.8, Sections 12.1 - 12.4, Section 12.6 of Jurafsky and Martin
• Wednesday, October 22
  Topic 8 (continued)
  Introduction to Jupyter Notebooks, Google Colab, and PyTorch
  Project #2 assigned
• Thursday, October 23
  Topic 9: Question Answering, Reading Comprehension, and SQuAD; Contextual Embeddings and ELMo
  Suggested reading: Sections 14.3 - 14.4 of Jurafsky and Martin (February 3, 2024 draft), ELMo paper
• Wednesday, October 29
  Topic 9 (continued)
• Thursday, October 30
  Quiz #2

• Wednesday, November 5
  Topic 10: Transformers
  Suggested Reading: Chapter 8 of Jurafsky and Martin, original transformer paper
• Thursday, November 6
  Topic 11: BERT and BERT Variations; Pretraining, Fine-tuning and Transfer Learning; the GLUE Benchmark
  Suggested Reading: Chapter 9 of Jurafsky and Martin, original BERT paper
• Wednesday, November 12
  Topic 11 (continued)
  Topic 12: Large Language Models (LLMs); GPT (several variations); Reinforcement Learning from Human Feedback (RLHF); Retrieval-Augmented Generation (RAG)
  Suggested Reading: Sections 7.1 - 7.6, Section 11.3 of Jurafsky and Martin, GPT-1 paper, GPT-3 paper, InstructGPT paper
• Thursday, November 13
  Topic 12 (continued)
• Wednesday, November 19
  Topic 13: Ethics of NLP; Relevant Philosophical Issues
  Suggested Readings: Section B.10, Section 5.8, Section 7.7, Section 12.7 of Jurafsky and Martin, Stochastic Parrots paper
• Thursday, November 20
  Topic 13 (continued)
  Project #3 assigned
• Tuesday, November 25 (Thursday schedule)
  Quiz #3

• Wednesday, December 3
  Class time to work on final projects
• Thursday, December 4
  Class time to work on final projects
• Wednesday, December 10
  Class time to work on final projects
• Wednesday, December 17
  Final Project Presentations
• Thursday, December 18
  Final Project Presentations