Bayesian Modelling

Lecturers: Antonietta Mira & Francesco Denti

Modality: In-presence

Week 1: 12-16 August 2024

Workshop contents and objectives

Bayesian statistics has experienced a surge in popularity over the last few decades, primarily due to the computational advancements that have mitigated its traditionally perceived complexity. The progressive expansion of the Bayesian method has allowed practitioners to embrace its intuitive, probabilistic reasoning and leverage its flexibility, transparency and coherence in formulating elaborate models for real-world data.

This course aims to give participants a simple but rigorous foundation of Bayesian Statistics. Our program is designed to start from the fundamental concepts and progress to developing simple and advanced models explicitly tailored for applications in the social sciences.

The course will cover essential topics, starting with the basics of Bayesian inference, including posterior distribution, point estimation, credible intervals, and hypothesis testing. Moving forward, we will explore specific areas such as:

• Regression Models and Variable Selection: We will discuss the basic regression models and then discuss the use of priors for variable selection.
• Models for Network Data: We will delve into the application of Bayesian statistics for modeling and interpreting network data, providing insights into the dynamics of interconnected systems with specific applications to social networks.
• Approximate Bayesian Inference: models for network type data and, more generally, models for complex systems, often lack an analytic expression of the likelihood function implied by the modeling framework. In this context, to perform Bayesian inference we need to resort to likelihood free methodologies among which we will introduce Approximate Bayesian Computation.
• Model-Based Clustering: This section will cover model-based clustering, a technique crucial for segmenting complex datasets into homogeneous groups. This approach facilitates a nuanced understanding of patterns within diverse datasets.

Workshop design

The course is carefully structured to maintain a balanced approach, incorporating both theoretical classes and hands-on practical laboratories. This dual strategy aims to provide participants with a comprehensive understanding of the reliability and practical applications of Bayesian statistics. Engaging in both theoretical concepts and practical applications will enable attendees to gain valuable insights into the theory and the real-world applicability of Bayesian statistical techniques with a focus to social science.

More specifically, during the theoretical classes, the basics of Bayesian modeling will be covered, and essential methods will be introduced and described.

The laboratories will focus on the R software and their utility is twofold. On the one hand, they consolidate the understanding of the theoretical topics. On the other hand, they provide guidance on using the R software and dedicated packages to implement, fit, and interpret the Bayesian models applied to data from social sciences.

Detailed lecture plan (daily schedule)

Day 1.
All day: Introduction to the Bayesian modeling framework and comparison with the frequentist maximum likelihood approach from a methodological and philosophical point of view. The concepts of priors and posterior distributions. Some notable examples of conjugate priors.

Day 2.
Morning: Methods for posterior simulation: Monte Carlo and Monte Carlo Markov Chains.
Afternoon: LAB 1 - R basics, conjugacy, basic model estimation, MCMC foundations, Stan

Day 3.
Morning: Bayesian linear regression, Bayesian logistic regression, Applications to social science
Afternoon: LAB 3 - practical implementation. Shrinkage priors for variable selection: the Bayesian Lasso and the Horseshoe prior

Day 4. 
Morning: Advanced Bayesian modeling 1 - Bayes for network data and Approximate Bayesian Computation (theory)
Afternoon: LAB 4 - Application of Approximate Bayesian Computation to estimate social networks

Day 5.
Morning: Bayesian model-based clustering via mixture models, Challenges and estimation strategies
Afternoon: LAB 5 - Implementing a Gibbs sampler for clustering

Prerequisites

The course assumes a basic familiarity with probability theory and with linear regression analysis. A good knowledge of R is essential for the successful completion of the course.

Recommended readings or preliminary materials

Alicia A. Johnson, Miles Q. Ott, Mine Dogucu, “Bayes Rules! An Introduction to Applied Bayesian Modeling”, ISBN 9780367255398 by Chapman & Hall https://www.bayesrulesbook.com