Summary of Regression Modeling Concepts

1 We use different probability models for different data types

• Binary outcomes: Bernoulli models
• Event rate outcomes: Poisson/Negative binomial models
• Time-to-event outcomes: Survival models
• Catch-all: Gaussian models

2 We use different link functions to connect these models with covariates

• Bernoulli models: logit link
• Count models: log link + offset
• Survival models: log link
• Gaussian models: identity link

Figure 1 sketches how the various models we have studied have analogous structures. To do: convert this sketch into a nicely formatted figure.

Figure 1: Parallel Model Structures

3 We use maximum likelihood estimation to fit models to data

• likelihood
• log-likelihood
• score function
• hessian

4 We use asymptotic normality of MLEs to quantify uncertainty about models

• observed information matrix
• expected information matrix
• standard error
• confidence intervals
• p-values

5 We use (log) likelihood ratios to compare models

Sometimes we adjust these comparisons for model size (AIC, BIC)

References