ECE 467: Natural Language Processing

• Instructor Info
• Textbook
• Grading
• Schedule

Instructor Info

Instructor: Carl Sable
e-mail: carl.sable@cooper.edu (but please submit assignments to CarlSable.Cooper@gmail.com)
Office: Room 614

Textbook

"Speech and Language Processing, 3rd Edition" (draft, in-progress, on-line only)
by Daniel Jurafsky and James H. Martin

Grading

Assignments will be posted here when they are assigned.

• Text categorization project: 25%
  Click here for the corpora for the text categorization project
  Project #1 is due before midnight the night of Tuesday, March 21
• Parsing project: 25%
  Click here for a sample grammar expressed as a general CFG
  Click here for a weakly equivalent grammar in CNF
  Click here for my Python 3 program that converts a CFG to CNF
  (NOTE: I changed the file extension to .txt to avoid server issues, you should change it back to .py)
  Project #2 is due before midnight the night of Tuesday, April 11
• Individual or group open-ended deep learning project: 25%
  Project ideas and group members should be sent to be before midnight the night of Tuesday, May 2
  Project #3 is due before midnight the night of Tuesday, May 9
• Three quizzes: 25% (total)

Schedule

Note 1: All dates and topics which have not yet occurred are tentative!

Note 2: All below references to chapters and sections of the Jurafsky and Martin textbook are based on a draft version of the 3rd edition dated January 7, 2023.

• Wednesday, January 18
  Topic 1: Course Introduction (Slides)
  See the syllabus as a single document
• Thursday, January 19
  Topic 2: Tokenization, Words, and Morphology (Slides)
  Suggested reading: Chapter 2 of Jurafsky and Martin
• Wednesday, January 25
  Topic 2 (continued)
• Thursday, January 26
  Topic 3: N-grams and Conventional Language Models (Slides)
  Suggested reading: Chapter 3 of Jurafsky and Martin
• Wednesday, February 1
  Topic 3 (continued)
  Topic 4: Part-of-Speech Tagging (Slides)
  Suggested reading: Chapter 8 of Jurafsky and Martin
• Thursday, February 2
  Topic 4 (continued)
• Wednesday, February 8
  Topic 4 (continued)
  Topic 5: Vector Space Models, Information Retrieval, and Text Categorization (Slides)
  Suggested reading: Sections 4.1 - 4.9, Sections 6.3 - 6.5, Section 14.1 of Jurafsky and Martin
• Thursday, February 9
  Topic 5 (continued)
• Thursday, February 16
  Topic 5 (continued)
• Wednesday, February 22
  Topic 5 (continued)
  Quiz #1
• Thursday, February 23
  Project #1 assigned

• Wednesday, March 1
  Topic 6: Phrase Structure Grammars and Dependency Grammars (Slides)
  Suggested reading: Sections 17.1 - 17.3, Appendix D, and Section 18.1 of Jurafsky and Martin
• Thursday, March 2
  Topic 6 (continued)
• Wednesday, March 8
  Topic 7: Natural Languages and Psycholinguistics (Slides)
  Topic 8: Parsing (Slides)
  Suggested reading: Sections 17.4 - 17.6 and Appendix C of Jurafsky and Martin
• Thursday, March 9
  Topic 8 (continued)
• Wednesday, March 22
  Topic 8 (continued)
  Project #2 assigned
  Topic 9: First-Order Logic and Semantics (Slides)
  Suggested reading: Chapter 19 of Jurafsky and Martin
• Thursday, March 23
  Topic 9 (continued)
• Wednesday, March 29
  Topic 9 (continued)
  Quiz #2

• Thursday, March 30
  Topic 10: Feedforward Neural Networks (Slides)
  Suggested reading: Sections 7.1 - 7.4 and Section 7.6 of Jurafsky and Martin
• Wednesday, April 5
  Topic 10 (continued)
  Topic 11: Word Embeddings, Neural Language Models, and Word2vec (Slides)
  Suggested reading: Sections 6.8 - 6.10, Section 7.5, and Section 7.7 of Jurafsky and Martin
• Thursday, April 6
  Topic 11 (continued)
• Wednesday, April 12
  Topic 11 (continued)
  Topic 12: Recurrent Neural Networks and LSTMs (Slides)
  Suggested reading: Sections 9.1 - 9.6 of Jurafsky and Martin
• Thursday, April 13
  Topic 12 (continued)
• Wednesday, April 19
  Topic 12 (continued)
• Thursday, April 20
  Project #3 assigned
  Topic 13: Encoder-Decoder Models, Attention, and Machine Translation (Slides)
  Suggested reading: Sections 9.7 - 9.8, Sections 13.1 - 13.3, Section 13.5
• Wednesday, April 26
  Topic 13 (continued)
• Thursday, April 27
  Topic 14: Advanced Topics: Subword Embeddings; Question Answering; Transformers; Contextual Embeddings and Large Language Models (e.g., ELMo, BERT, GPT); Ethics and NLP (Slides)
  Suggested reading: Section 2.4.3, Section 6.8.3, Sections 14.2 - 14.7, Sectiond 10.1 - 10.2, Section 10.4, Sections 11.1 - 11.3, Section 4.10, Section 6.11, Section 13.6, Section 10.7, Section 15.6.1 of Jurafsky and Martin
• Wednesday, May 3
  Topic 14 (continued)
• Wednesday, May 10
  Topic 14 (continued)
• Thursday, May 11
  Topic 14 (continued)
  Quiz #3