Course Description
Probabilistic models for data are ubiquitous in many areas of science and engineering, and specific domains such as visual and language understanding, finance, healthcare, biology, climate informatics, etc. This course will be an advanced introduction to probabilistic models of data (often through case studies from these domains) and a deep-dive into advanced inference and optimization methods used to learn such probabilistic models. This is an advanced course and ideally suited for students who are doing research in this area or are interested in doing research in this area.
Course Content
The course will tentatively cover the following topics:
- Basics of probabilistic modeling and parameter estimation
- Common probability distributions and their properties
- Conjugate priors and closed-form Bayesian posterior updates
- Exponential family and its role in probabilistic inference
- Generative models and latent variable models
- Sampling based approximate Bayesian inference (MCMC methods)
- Optimization based approximate Bayesian inference (variational inference)
- Online approximate Bayesian inference for large-scale learning
- Model comparison and model selection
- Bayesian Deep Learning: Supervised (classification/regression) and unsupervised (deep generative models, such as variational autoencoders and generative adversarial networks), amortized inference
- Nonparametric Bayesian modeling
- Case-studies/running-examples: Bayesian linear regression and classification, Gaussian Process (GP) regression and classification, sparse linear models, finite and infinite mixture and latent factor models, matrix factorization of real/discrete/count data, linear Gaussian models, linear dynamical systems and time-series models, topic models for text data, deep generative models for image/text data