Bayesian Statistics, Simulation and Software

Information

1. Organisers and lecturers:
   Kasper K. Berthelsen, Associate Professor (e-mail: kkb (snabel-a) math.aau.dk) and
   Søren L. Buhl, Associate Professor (e-mail: slb (snabel-a) math.aau.dk),
   Department of Mathematical Sciences, Aalborg University.
2. Place: Fredrik Bajers Vej 7G, Room G5-108 Aalborg University
3. You are expected to bring a laptop. If possible, before the course starts install the free statistics software R. You can wait until the second course day before installing OpenBUGS. You can find a guide here on how to install these packages.
4. There is wireless access in the lecture room (at least for people with a AAU e-mail account or using a VPN client). If everything fails there is also wired internet access.
5. Course material:
   1. Basics of probability theory.
   2. Exercises to Basics of probability theory.
   3. Basic methods for simulation of random variables: 1. Inversion.
   4. A brief introduction to (simulation based) Bayesian inference.
   5. Basic methods for simulation of random variables: 2. Accept/rejection algorithm.
   6. Simulation from specific distributions - especially the normal distribution.
   7. Monte Carlo methods.
   8. Model checking based on p-values.
   9. Some elementary Markov chain theory.
   10. A short diversion into the theory of Markov chains, with a view to Markov chain Monte Carlo methods.
   11. Importance sampling for unnormalized densities
   12. Use of WinBUGS, Coda and DoodleBUGS
   13. The beta-binomial distribution
   14. Bother with the Notation
6. Some useful textbooks:
   • Peter M. Lee (2004). Bayesian Statistics: an introduction, 3rd ed. Arnold.
   • Andrew Gelman, et al. (2003). Bayesian Data Analysis, 2nd ed. Chapman & Hall/CRC.
   • Olle Häggström (2002). Finite Markov Chains and Algorithmic Applications. Cambridge University Press.
   • Christian P. Robert and George Casella (2004). Monte Carlo Statistical Methods, 2nd ed. Springer.
   • Maria Rizzo (2007). Statistical Computing with R. Chapman & Hall/CRC.
   • Ioannis Ntzoufras (2009). Bayesian Modeling Using WinBUGS. Wiley.

Programme

This is an intensive PhD course on which you should expect to spend most of your working effort during the period Moreover, you should prepare yourself before joining each day's programme.

A tentative programme is given below; it may very well be adjusted as the course goes on!

1) Friday 27^th May 9:00-16:00

• Morning
  1. It is important that you have R installed on your computer!
  2. Introduction to R. Using this demo and this dataset.
  3. Exercises in R. Use this document and this dataset.
  4. Solutions to the exercises.
• Afternoon
  1. In the afternoon we will brush-up some basics about probability (events, conditional probablity, stochatic variables etc), discrete and continuous distribution. For this you could read "Basics of probability theory" or corresponding chapters in most books on basic statistics.
  2. After the lecture you should have a go at the exercises in "Exercises to Basics of probability theory."

2) Monday 30^th May 9:00-16:00

• Morning
  1. Basic concepts in Bayesian statistics. Read A brief introduction to (simulation based) Bayesian inference and Examples of Bayesian inference.
  2. Exercises in Examples of Bayesian inference
  3. Solutions to the exercises.
• Afternoon
  1. It is important that you have OpenBUGS (or WinBUGS) installed on your computer! When I refer the WinBUGS in the sequel, it could just as well (or even better) be OpenBUGS.
  2. More on R. Use this demo and this dataset in Open Document Format. If you only know Microsoft Office, try this.
  3. The WinBUGS part of Use of WinBUGS, CODA and DoodleBUGS.
  4. Exercises in R and WinBUGS. Use this document.
  5. Solutions to the exercises.

3) Wednesday 1^st June 9:00-16:00

• Morning
  1. Semi-conjugate priors, as touch upon in Examples of Bayesian inference.
  2. The Gibbs sampler, read Section 8 in A short diversion into.... Notice that the gamma distribution is parameterised differently from we have done so far in the course
  3. Hierarchical and Graphical models, read the sections 'Graphical Models' and 'Graphs as a formal language' in the WinBUGS manual.
  4. Exercises from this document.
  5. Solutions to the exercises.
• Afternoon
  1. It will be a good idea to have installed the R library coda.
  2. Things we didn't get through Wednesday.
  3. The beta-binomial distribution, except "An Urn Model".
  4. The WinBUGS and "doodle" parts of Use of WinBUGS, CODA and DoodleBUGS.
  5. Exercise in BUGS. Use this document.
  6. Solution to the exercise.

4) Friday 3^rd June 9:00-16:00

• Morning
  1. Basic Markov chain theory, read at least Section 2 in "Some elementary Markov chain theory" which deals with Markov chains with a discrete state space.
  2. Markov chains on (more) general state spaces, read Sections 1-4 in "A Short diversion...".
  3. Exercises from this document.
  4. Solutions to the exercises.
• Afternoon
  1. During the lecture we shall be using the demo and the exercises for today.
  2. A simple linear regression using DoodleBUGS (see my demo).
  3. Programming in R illustrated by the Fibonacci numbers (see my demo). You could check Sec. 10.1 in the manual "R Language Definition" which can be found using help.start() in R.
  4. Exercise 1 for today.
  5. A bit of Gibbs samling and Exercise 2 for today.
  6. Section 2.1 in "A short diversion..." and Exercise 3 for today.
  7. If time permits, my demo illustrating how to make a presentation plot.
  8. Solutions to the exercises.

5) Friday 10^th June 9:00-16:00

• Morning
  1. We continue with Markov chain theory, read Sections 9-10 in "A Short diversion...". In Section 9 you can skip the example if you like.
  2. Exercises from this document. These are mainly R exercises. Start with exercise 0. The other exercises can be done according to what you find most interesting.
  3. Here are the slides from the lecture.
  4. Solution to the mining exercise when a and b are fixed.
  5. Solution to the mining exercise when a and b are not fixed.
  6. Solution to the beetles exercise.
  7. Solution to the banana exercise.
• Afternoon
  1. Second half of Exercise 3 from Lecture 4.
  2. A bit about graphical models.
  3. Exercises with Pump failure data. Use this document and this dataset.
  4. Filerne pump.R og pump-fig.R.
  5. The Metropolis-Hastings algorithm.
  6. Laird Breyer has made a Java-based M-H simulator.
  7. Solutions to the exercises.

6) Tuesday 14^th June 9:00-16:00

• Morning
  1. First we will look at prior and posterior predictive distributions as a tool for model checking. Read Model checking based on p-values.
  2. Next we will look at Metropolis within Gibbs, Section 9 in "A short diversion...".
  3. After this you are meant to work on a small project. The project consists in analysing the rat tumour data, as described in Exercise 10 and pages 20 to 26 in "A short diversion...".
• Afternoon
  1. Functions to the rat tumour analysis.
  2. Same without logarithms in the density.
  3. Finish the project.
  4. At 15:45 we meet to evaluate the course.