CISC 889: Statistical Natural Language Processing -- Spring 2008
T/Th 5:00-6:15 pm, Smith 201
Instructor: Vijay K. Shanker
Office: Room 442, Smith Hall
Hours: Wednesdays 10:00am-12:00pm
Phone: 831-1952
Email: vijay@cis.udel.edu
Textbook
Foundations of Statistical Natural Language Processing , by Christopher
Manning and Hinrich Schuetze. MIT Press, 2000.
Course Overview
The intent of this course is to give a general introduction to statistical
methods and more generally emperical methods in natural language processing.
The primary focus of the course is to provide an understanding of the underlying
methods used in various NLP tasks. Class lectures will cover the general issues
and related algorithms. Homeworks will cover these aspects. Programming
exercises are intended to provide a feel for how the systems discussed in
class actually work. Class projects and/or presentations should be chosen to further the understanding of these systems,
study alternate
methods or to use statistical techniques to solve an interesting NLP task.
(More details on projects will be provided soon.)
Grading
Homework and Programming Exercises -- 50%
Project -- 50%
All homework assignments and programming exercises must be completed individually.
Late submissions will not be accepted without prior permission.
Course Content
The following topics from the text will be covered:
Basic probability and information theory.
Chapter 2.Conditional probability and independence. Bayes' theorem. Bayesian
statistics. Standard distributions. Information Theory: entropy, mutual information,
Kullback-Liebler distance or relative entropy, perplexity.
Statistical Estimation and Inference
Chapter 6. N-gram models and reliability of n-grams. Statistical estimators.
Collocations
Chapter 5. Frequency methods, mean and variance methods, hypothesis testing
and mutual information-based methods.
Word Sense Disambiguation
Chapter 7. Supervised, unsupervised and dictionary-based methods.
Markov Models
Chapter 9. Markov models, Hidden Markov Models. Probability of an observation,
Viterbi algorithm, parameter estimation.
Part of Speech Tagging
Chapter 10. Markov Model taggers, Transformation-based Learning.
Probabilistic Grammars and Parsers
Chapters 11 and 12.Probabilistic context-free grammars, inside-outside algorithm.
Parsing models vs. language models. Search methods. Lexicalized models, dependency
models.
Clustering
Chapter 14. Clustering as unsupervised learning. Hierarchical clustering:
single link vs. complete link. Agglomerative clustering. Non-hierarchical
clustering: k-means and EM methods.
Information Retrieval
Chapter 15. Design of IR systems. Vector Space Methods.
TF-IDF schemes. Latent Semantic Indexing.
Text Categorization
Chapter 16.
Decision Trees and Naive Bayes metods. Nearest neighbor methods.