08-moderation-categorical.Rmd

# Regression Analysis And A Categorical Moderator {#moderationcat}

> Key concepts: regression equation, dummy variables, normally distributed residuals, linearity, homoscedasticity, independent observations, statistical diagram, interaction variable, covariate, common support, simple slope, conditional effect.

Watch this micro lecture on regression analysis with a categorical moderator for an overview of the chapter.

```{r, echo=FALSE, out.width="640px", fig.pos='H', fig.align='center', dev="png", screenshot.opts = list(delay = 5)}
knitr::include_url("https://www.youtube.com/embed/lDkGyTvPzOY", height = "360px")
```

### Summary {.unnumbered}

```{block2, type='rmdimportant'}
My dependent variable is numerical but at least one predictor is also numerical, so I cannot apply analysis of variance. How can I investigate moderation with regression analysis?
```

The linear regression model is a powerful and very popular tool for predicting a numerical dependent variable from one or more independent variables. In this chapter, we use regression analysis to evaluate the effects of an anti-smoking campaign. We predict attitude towards smoking from exposure to the anti-smoking campaign (numerical), time spent with smokers (numerical), and the respondent's smoking status (categorical).

Regression coefficients, that is, the slopes of regression lines, are the effects in a regression model. They show the predicted difference in the dependent variable for a one unit difference in the independent variable (exposure, time spent with smokers) or the predicted mean difference for two categories (smokers versus non-smokers).

But what if the predictive effect is not the same in all contexts? For example, exposure to an anti-smoking campaign may generally generate a more negative attitude towards smoking. The effect, however, is probably different for people who smoke than for people who do not smoke. In this case, the effect of campaign exposure on attitude towards smoking is moderated by context: Whether or not the person exposed to the campaign is a smoker.

Different effect sizes for different contexts are different regression coefficients for different contexts. We need different regression lines for different groups of people. We can use an interaction variable as an independent variable in a regression model to accommodate for moderation as different effects. An interaction variable is just the product of the predictor variable and the moderator variable.

As an independent variable in the model, the regression coefficient of an interaction variable (interaction effect for short) has a confidence interval and a *p* value. The confidence interval tells us the plausible values for the size of the interaction effect in the population. The *p* value tests the null hypothesis that there is no interaction effect at all in the population.

To interpret the interaction effect, we must determine the size of the effect of the predictor on the dependent variable for each group of the moderator. For example, the effect of campaign exposure on smoking attitude for smokers and the effect for non-smokers.

An interaction effect in a regression model closely resembles an interaction effect in analysis of variance. The effect of a single predictor that is involved in an interaction effect in a regression model, however, is not a main effect as in analysis of variance. It is a conditional effect, namely the effect for one particular value of the moderator, that is, the effect within one particular context. To understand this, we must pay close attention to the regression equation.

#### Essential Analytics {.unnumbered}

In SPSS, we use the *Linear* option in the *Regression* submenu for regression analysis. For a moderation model, we first use the *Compute Variable* option in the *Transform* menu to calculate an interaction variable: we multiply (using `*`) the predictor variable by the moderator variable. The interaction variable is included in the regression model as an independent variable, just like the predictor, moderator, and any other independent variables (covariates).

A categorical predictor variable such as a participant's residential area (urban, suburban, rural) is included in the regression model as dummy variables, which have the values 0 or 1, for example: dummy variables *suburban* and *rural*, each with values yes (1) and no (0). The category without dummy variable is the reference group. 

The regression coefficient of a dummy variable gives us the difference between the average score on the dependent variable of the group scoring 1 on the dummy variable and the reference group. In the example presented in Figure \@ref(fig:regressiontable), the attitude towards smoking for participants living in a suburban environment (red box) is on average 0.33 more positive than among participants in an urban environment (the reference group).

```{r regressiontable, echo=FALSE, out.width="100%", fig.pos='H', fig.align='center', fig.cap="SPSS table of regression effects for a model in which the effect of exposure is moderated by participant's smoking status (reference group: people who never smoked)."}
knitr::include_graphics("figures/S8_AE1.png")
```

The predictor variable *Exposure* is included in the interaction effect. As a consequence, the regression coefficient for this variable (green box in Figure \@ref(fig:regressiontable)) expresses the effect of exposure on attitude towards smoking for the reference group on the other variable included in the interaction effect, namely, people who never smoked. A one unit increase in exposure predicts a 0.20 more negative (-0.197) attitude towards smoking **for people who never smoked**.

If we would like to know the effect of exposure on attitude towards smoking for former smokers, we must add the regression coefficient for the interaction of exposure with former smokers (blue box) to the regression coefficient of exposure (green box): A one unit increase in exposure predicts a 0.47 more negative (-0.465 = -0.197 + -0.268) attitude towards smoking for former smokers.

An interaction effect such as -0.27 for *Former smoker \* exposure* tells us the difference between the exposure effect for former smokers and the exposure effect for people who never smoked (reference group). A plot of the regression lines shows the different exposure effects (Figure \@ref(fig:regressionplot)). The red line (effect of exposure for former smokers) has a stronger downward tendency than the blue line (exposure effect for people who never smoked). 

```{r regressionplot, echo=FALSE, out.width="75%", fig.pos='H', fig.align='center', fig.cap="Simple regression lines showing the effect of exposure on attitude towards smoking for former smokers and people who never smoked (in an urban environment)."}
knitr::include_graphics("figures/S8_AE2.png")
```

```{r SPSS-PROCESS, eval=FALSE, echo=FALSE}
# TERMINOLOGY: predictor (X, cause), moderator (M, represents context), covariate (C, control), dependent variable (Y).

# SPSS versus PROCESS:
# + SPSS: visual checks on residuals: normal distribution and zpred by zresid (linearity, homoscedasticity)
# + SPSS: scatterplot (X, Y) with regression lines per group (Moderator) with original variable and value labels, showing common support ; add reference line for each group manually specifying the regression equation, setting covariates to their mean values (with categorical moderator and no covariates or covariates that are not correlated with the predictor, Regression Variable Plots can be used -not in SPSS V25? - or a regression line per subgroup with equation as label can be added in the Chart Editor)
# - SPSS: manual entering of regression equation with selected values for covariates (and a numerical moderator; lines can only be labeled with the equation text)
# - SPSS: interaction predictors have to be created by hand (also multiple interaction variables for a categorical predictor; Transform>Create Dummy Variables, taught in RMCS?)
# - SPSS: mean-centering must be done by hand
# - SPSS: statistical inference for non-zero moderator values requires separate regression models where the low category requires ADDING one SD instead of subtracting.
# + PROCESS: must be used anyway for mediation models
# - PROCESS: no visual checks on assumptions
# - PROCESS: no visual impression of common support of predictor for different values of the moderator (requires additional work with numerical moderator also in SPSS)
# - PROCESS: data list for visualization of results must be copied from output to syntax file, variable and value labels must be added, lines must be added (and this requires that the moderator has no decimal places in SPSS?) in chart editor
# - PROCESS: model number must be remembered
# - PROCESS: because the student need not create the interaction variables, mean-center or "re-center" for probing the interaction, PROCESS output is more mysterious (but the estimated slopes for different moderator values are directly linked to the graph)
# - PROCESS: dichotomies are automatically treated as indicator variables but categorical predictors/moderators are treated as numerical ; it is not possible to use more than one moderator variable, so PROCESS cannot handle a categorical moderator. (Fixed in V3?)
# DECISION: Use SPSS for results, interpretation, and assumption checks (interaction variables and, possibly, dummies must be created but no need for mean-centering).
```

## The Regression Equation {#regression-equation}

In the social sciences, we usually expect that a particular outcome has several causes. Investigating the effects of an anti-smoking campaign, for instance, we would not assume that a person's attitude towards smoking depends only on exposure to a particular anti-smoking campaign. It is easy to think of other and perhaps more influential causes such as personal smoking status, contact with people who do or do not smoke, susceptibility to addiction, and so on.

```{r concept-smoke, echo=FALSE, fig.asp=0.4, fig.pos='H', fig.align='center', fig.cap="A conceptual model with some hypothesized causes of attitude towards smoking."}
# Draw conceptual diagram: Attitude towards smoking predicted by Exposure, Smoking status, and Contact with smokers.
library(ggplot2)
# Create coordinates for the variable names.
variables <- data.frame(x = c(0.3, 0.3, 0.3, 0.7), 
                        y = c(.1, .3, .5, .3),
                        label = c("Exposure", "Smoking Status", "Contact with Smokers", "Attitude"))
ggplot(variables, aes(x, y)) + 
  geom_segment(aes(x = x[1], y = y[1], xend = x[4] - 0.04, yend = y[4] - 0.02), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  geom_segment(aes(x = x[2], y = y[2], xend = x[4] - 0.04, yend = y[4]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  geom_segment(aes(x = x[3], y = y[3], xend = x[4] - 0.04, yend = y[4] + 0.02), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  geom_label(aes(label=label)) + 
  coord_cartesian(xlim = c(0.2, 0.8), ylim = c(0.05, 0.55)) +
  theme_void()
# Cleanup.
rm(variables)
```

Figure \@ref(fig:concept-smoke) summarizes some hypothesized causes of the attitude towards smoking. Attitude towards smoking is measured as a scale, so it is a numerical variable. In linear regression, the dependent variable ($y$) must be numerical and in principle continuous. There are regression models for other types of dependent variables, for instance, logistic regression for a dichotomous (0/1) dependent variable and Poisson regression for a count dependent variable, but we will not discuss these models.

A regression model translates this conceptual diagram into a statistical model. The statistical regression model is a mathematical function with the dependent variable (also known as the outcome variable, usually referred to with the letter $y$) as the sum of a constant, the effects ($b$) of independent variables or predictors ($x$), which are *predictive effects*, and an error term ($e$), which is also called the *residuals*, see Equation \@ref(eq:regression).

```{=tex}
\begin{equation} 
\small
  y = constant + b_1*x_1 + b_2*x_2 + b_3*x_3 + e 
  (\#eq:regression) 
\normalsize
\end{equation}
```
If we want to predict the dependent variable ($y$), we ignore the error term ($e$) in the equation. The equation without the error term [Eq. \@ref(eq:regressionpred)] represents the regression line that we visualize and interpret in the following subsections. We use the error term only when we discuss the assumptions for statistical inference on a regression model in Section \@ref(regr-inference).

```{=tex}
\begin{equation} 
\small
  y = constant + b_1*x_1 + b_2*x_2 + b_3*x_3 
  (\#eq:regressionpred) 
\normalsize
\end{equation}
```
### A numerical predictor

Let us first have a close look at a *simple regression equation*, that is, a regression equation with just one predictor ($x$). Let us try to predict attitude towards smoking from exposure to an anti-smoking campaign.

```{r regression-continuous, fig.pos='H', fig.align='center', fig.cap="Predicting attitude towards smoking from exposure to an anti-smoking campaign. The orange dot represents the predicted attitude for the selected value of exposure.", echo=FALSE, out.width="775px", screenshot.opts = list(delay = 5), dev="png"}
# Goal: Understand the meaning of the constant and the regression coefficient
# with a single numerical predictor.
# Generate a data set with attitude towards smoking as Y and exposure as X with
# a negative (b = -0.6) more or less linear relation.
# Display the scatterplot.
# Add the line of a simple regression of attitude on exposure for the generated data.
# Allow the user to change the value of exposure.
# A change to the exposure value triggers the app to add/reposition the predicted value as a dot (on the regression line), and show the contribution of b*x to the predicted value as a 'triangle' anchored on (0, constant). In addition, the values between parentheses of x and y are updated.
knitr::include_app("http://82.196.4.233:3838/apps/regression-continuous/", height="408px")
```

<A name="question8.1.1"></A>

```{block2, type='rmdquestion'}
1. What is the predicted attitude for a person with zero exposure to the campaign? Explain why this value equals the constant of the regression equation. [<img src="icons/2answer.png" width=115px align="right">](#answer8.1.1)
```

<A name="question8.1.2"></A>

```{block2, type='rmdquestion'}
2. What does the vertical orange line in Figure \@ref(fig:regression-continuous) mean if exposure is set to 1? [<img src="icons/2answer.png" width=115px align="right">](#answer8.1.2)
```

<A name="question8.1.3"></A>

```{block2, type='rmdquestion'}
3. Use the equation to calculate the predicted attitude if exposure is 10. Check your answer using the exposure slider. What is troublesome about this predicted value? [<img src="icons/2answer.png" width=115px align="right">](#answer8.1.3)
```

Good understanding of the regression equation is necessary for understanding moderation in regression models. So let us have a close look at an example equation [Eq. \@ref(eq:regrexample)]. In this example, the dependent variable attitude towards smoking is predicted from a constant and one independent variable, namely exposure to an anti-smoking campaign.

```{=tex}
\begin{equation}
\small
  attitude = constant + b*exposure 
  (\#eq:regrexample) 
\normalsize
\end{equation}
```
The constant is the predicted attitude if a person scores zero on all independent variables. To see this, plug in (replace) zero for the predictor in the equation (Eq. \@ref(eq:regsmokedummy)) and remember that zero times something yields zero. This reduces the equation to the constant.

```{=tex}
\begin{equation}
\small
\begin{split}
  attitude &= constant + b*0 \\ 
  attitude &= constant + 0 \\
  attitude &= constant
\end{split}
  (\#eq:regsmokedummy) 
\normalsize
\end{equation}
```
For all persons scoring zero on exposure, the predicted attitude equals the value of the regression constant. This interpretation only makes sense if the predictor can be zero. If, for example, exposure had been measured on a scale ranging from one to seven, nobody can have zero exposure, so the constant has no straightforward meaning.

The unstandardized regression coefficient $b$ represents the predicted difference in the dependent variable for a difference of one unit in the independent variable. For example, plug in the values 1 and 0 for the *exposure* variable in the equation. If we take the difference of the two equations, we are left with $b$. Other terms in the two equations cancel out.

```{=tex}
\begin{equation}
\small
\begin{split}
  attitude = constant + b*1 \\ 
  \underline{- \mspace{20mu} attitude = constant + b*0} \\
  attitude \mspace{4mu} difference = b*1 - b*0 = b - 0 = b
\end{split}
(\#eq:regweight) 
\normalsize
\end{equation}
```
```{block2, type='rmdimportant'}
The unstandardized regression coefficient $b$ represents the predicted difference in the dependent variable for a difference of one unit in the independent variable.

It is the slope of the regression line.
```

Whether this predicted difference is small or large depends on the practical context. Is the predicted decrease in attitude towards smoking worth the effort of the campaign? In the example shown in Figure \@ref(fig:regression-continuous), one additional unit of exposure decreases the predicted attitude by 0.6. This seems to be quite a substantial change on a scale from -5 to 5.

In the data, the smallest exposure score is (about) zero, predicting a positive attitude of 1.6. The largest observed exposure score is around eight, predicting a negative attitude of -3.2. If exposure causes the predicted differences in attitude, the campaign would have interesting effects. It may change a positive attitude into a rather strong negative attitude.

If we want to apply a rule of thumb for the strength of the effect, we usually look at the standardized regression coefficient ($b^*$ according to APA, *Beta* in SPSS output). See Section \@ref(assoc-size) for some rules of thumb for effect size interpretation.

Note that the regression coefficient is calculated for predictor values that occur within the data set. For example, if the observed exposure scores are within the range zero to eight, these values are used to predict attitude towards smoking.

We cannot see this in the regression equation, which allows us to plug in -10, 10, or 100 as exposure values. But the values for attitude that we predict from these exposure values are probably nonsensical (if possible at all: -10 exposure?) Our data do not tell us anything about the relation between exposure and anti-smoking attitude for predictor values outside the observed zero to eight range. We should not pretend to know the effects of exposure levels outside this range. It is good practice to check the actual range of predictor values.

### Dichotomous predictors {#dichpredictor}

Instead of a numerical independent variable, we can use a dichotomy as an independent variable in a regression model. The dichotomy is preferably coded as 1 versus 0, for example, 1 for smokers and 0 for non-smokers among our respondents.

```{r regression-dichotomy, eval=TRUE, echo=FALSE, fig.pos='H', fig.align='center', fig.cap="What is the difference in attitude between non-smokers and smokers?", screenshot.opts = list(delay = 5), dev="png", out.width="775px"}
# Goal: Refresh interpretation of unstandardized regression weight for a
# dichotomous independent variable by manipulating group averages.
# Generate attitude scores  with average values -0.6 for non-smokers and 1.0 for
# smokers (N = 20 per group).
# Draw horizontal and vertical lines from the axes to the group means and add a
# regression line (line through the two group means).
# Display the current regression equation beneath or in the plot. 
# Allow the user to change the average score per group. Update the scatterplot,
# regression line and equation, and the horizontal/vertical lines.
knitr::include_app("http://82.196.4.233:3838/apps/regression-dichotomy/", height="330px")
```

<A name="question8.1.4"></A>

```{block2, type='rmdquestion'}
4. What is the relation between the constant of the regression line in Figure \@ref(fig:regression-dichotomy) and group averages? Motivate your answer by changing the average attitude towards smoking for non-smokers in Figure \@ref(fig:regression-dichotomy). [<img src="icons/2answer.png" width=115px align="right">](#answer8.1.4)
```

<A name="question8.1.5"></A>

```{block2, type='rmdquestion'}
5. Change the slider for smokers to detect the relation between group means and the unstandardized regression coefficient ($b$). How can we calculate the unstandardized regression coefficient ($b$) from the group averages? [<img src="icons/2answer.png" width=115px align="right">](#answer8.1.5)
```

The interpretation of the effect of a dichotomous independent variable in a regression model is quite different from the interpretation of a numerical independent variable's effect.

It does not make sense to interpret the unstandardized regression coefficient of, for example, smoking status as predicted difference in attitude for a difference of one 'more' smoking status. After all, the 0 and 1 scores do not mean that there is one unit 'more' smoking. Instead, the coefficient indicates that we are dealing with different groups: smokers versus non-smokers.

If smoking status is coded as smoker (1) versus non-smoker (0), we effectively have two versions of the regression equation. The first equation \@ref(eq:regdicho1) represents all smokers, so their smoking status score is 1. The smoking status of this group has a fixed contribution to the predicted average attitude, namely $b$.

```{=tex}
\begin{equation}
\small
\begin{split}
  attitude &= constant + b*status \\
  attitude_{smokers} &= constant + b*1 \\
  attitude_{smokers} &= constant + b
\end{split}
(\#eq:regdicho1) 
\normalsize
\end{equation}
```
Regression equation \@ref(eq:regdicho0) represents all non-smokers. Their smoking status score is 0, so the smoking status effect drops from the model.

```{=tex}
\begin{equation}
\small
\begin{split}
  attitude &= constant + b*status \\
  attitude_{non-smokers} &= constant + b*0 \\
  attitude_{non-smokers} &= constant + 0 
\end{split}
  (\#eq:regdicho0) 
\normalsize
\end{equation}
```
If you compare the final equations for smokers [Eq. \@ref(eq:regdicho1)] and non-smokers [Eq. \@ref(eq:regdicho0)], the only difference is $b$, which is present for smokers but absent for non-smokers. It is the difference between the average score on the dependent variable (attitude) for smokers and the average score for non-smokers. We are testing a mean difference. Actually, this is exactly the same as an independent-samples *t* test!

```{block2, type='rmdimportant'}
The unstandardized regression coefficient for a dummy (0/1) variable represents the difference between the average outcome score of the group coded as '1' and the average outcome score of the group coded as '0'.
```

Imagine that $b$ equals 1.6. This indicates that the average attitude towards smoking among smokers (coded '1') is 1.6 units above the average attitude among non-smokers (coded '0'). Is this a small or large effect? In the case of a dichotomous independent variable, we should **not** use the standardized regression coefficient to evaluate effect size. The standardized coefficient depends on the distribution of 1s and 0s, that is, which part of the respondents are smokers. But this should be irrelevant to the size of the effect.

Therefore, it is recommended to interpret only the unstandardized regression coefficient for a dichotomous independent variable. Interpret it as the difference in average scores for two groups.

### A categorical independent variable and dummy variables {#categorical-predictor}

How about a categorical variable containing three or more groups, for example, the distinction between respondents who smoke (smokers), stopped smoking (former smokers), and respondents who never smoked (non-smokers)? Can we include a categorical variable as an independent variable in a regression model? Yes, we can but we need a trick.

```{r regression-categorical, fig.pos='H', fig.align='center', fig.cap="What are the predictive effects of smoking status?", echo = FALSE, screenshot.opts = list(delay = 5), dev="png", out.width="775px"}
# Goal: Understanding effects of dummy variables by manipulating the reference
# group.
# Generate numerical dependent variable data (attitude, range [-5, 5]) for a categorical
# independent variable with 3 categories: (1) non-smoker, (2) former smokers, (3) smoker,
# and a random choice of one of the following scenarios:
#* (1 M = -1.7, SD = 1) < (2) = (3 M = 0.75, SD = 1) {initial situation},
#* (1 M = -1.7, SD = 1) = (2) < (3 M = 0.75, SD = 1), 
#* (2  M = -1.7, SD = 1) < (1) = (3 M = 0.75, SD = 1),
#* (1 M = -1.7, SD = 1) < (2 M = 0.75, SD = 1) < (3 M = 1.8, SD = 1). 
# Display jittered scatterplot containing the three groups with group means
# indicated by a line segment and value, regression lines through the reference
# group mean and each of the other group means. Display b and p value of
# regression weights with the regression lines.
# Add input to select the reference group, initially set to group (1). Update
# regression lines and their p values on selection of a new reference group.
# Add button to generate a new plot (with a new scenario).
knitr::include_app("http://82.196.4.233:3838/apps/regression-categorical/", height="325px")
```

<A name="question8.1.6"></A>

```{block2, type='rmdquestion'}
6. Interpret the effects of smoking status (grey and black *b*'s) in Figure \@ref(fig:regression-categorical). [<img src="icons/2answer.png" width=115px align="right">](#answer8.1.6)
```

<A name="question8.1.7"></A>

```{block2, type='rmdquestion'}
7. In the initial state of Figure \@ref(fig:regression-categorical), can you tell whether the attitude of smokers is significantly different from the attitude of former smokers? If not, how can you get the _p_ value that you need? [<img src="icons/2answer.png" width=115px align="right">](#answer8.1.7)
```

<A name="question8.1.8"></A>

```{block2, type='rmdquestion'}
8. Select some new plots. For each plot, determine which reference group you think is most convenient for summarizing the results. [<img src="icons/2answer.png" width=115px align="right">](#answer8.1.8)
```

In this example, smoking status is measured with three categories: (1) non-smokers, (2) former smokers, and (3) smokers. Let us use the term *categorical variable* only for variables containing three or more categories or groups. This makes it easy to distinguish them from dichotomous variables. This distinction is important because we can include a dichotomous variable straight away as a predictor in a regression model but we cannot do so for a variable with more than two categories. We can only include such a categorical independent variable if we change it into a set of dichotomies.

We can create a new dichotomous variable for each group, indicating whether (score 1) or not (score 0) the respondent belongs to this group. In the example, we could create the variables *neversmoked*, *smokesnomore*, and *smoking*. Every respondent would score 1 on one of the three variables and 0 on the other two variables (Table \@ref(tab:dummytable)). These variables are called *dummy variables* or *indicator variables*.

```{r dummytable, echo=FALSE, screenshot.opts=list(delay = 2)}
knitr::kable(rbind(c("1 - Non-smoker", "1", "0", "0"), 
                   c("2 - Former smoker", "0", "1", "0"),
                   c("3 - Smoker", "0", "0", "1")), 
             col.names = c("Original categorical variable:", "neversmoked", "smokesnomore", "smoking"), caption = "Dummy variables for a categorical independent variable: One dummy variable is superfluous.", align = c("l", "c", "c", "c"), booktabs = TRUE) %>%
  kable_styling(font_size = 12, full_width = F, position = "float_right",
                latex_options = c("scale_down", "HOLD_position"))
```

If we want to include a categorical independent variable in a regression model, we must use all dummy variables as independent variables **except one**. In the example, we must include two out of the three dummy variables. Equation \@ref(eq:regcat) includes dummy variables for former smokers ($smokesnomore$) and smokers ($smoking$).

```{block2, type='rmdimportant'}
Include dummy variables as independent variables for *all except one* categories of a categorical variable.

The category without dummy variable is the *reference group*.
```

```{=tex}
\begin{equation}
\small
\begin{split}
  attitude &= constant + b_1*smokesnomore + b_2*smoking
\end{split}
(\#eq:regcat) 
\normalsize
\end{equation}
```
The two dummy variables give us three different regression equations: one for each smoking status category. Just plug in the correct 0 or 1 values for respondents with a particular smoking status.

Let us first create the equation for non-smokers. To this end, we replace both $smokesnomore$ and $smoking$ by 0. As a result, both dummy variables drop from the equation [Eq. \@ref(eq:regcat1)], so the constant is the predicted attitude for non-smokers. The non-smokers are our *reference group* because they are not represented by a dummy variable in the equation.

```{=tex}
\begin{equation}
\small
\begin{split}
  attitude &= constant + b_1*smokesnomore + b_2*smoking \\
  attitude_{non-smokers} &= constant + b_1*0 + b_2*0 \\
  attitude_{non-smokers} &= constant
\end{split}
(\#eq:regcat1) 
\normalsize
\end{equation}
```
For former smokers, we plug in 1 for $smokesnomore$ and 0 for $smoking$. The predicted attitude for former smokers equals the constant plus the unstandardized regression coefficient for the $smokesnomore$ dummy variable ($b_1$), see Equation \@ref(eq:regcat2). Remember that the constant represents the non-smokers (reference group), so the unstandardized regression coefficient $b_1$ for the $smokesnomore$ dummy variable shows us the difference between former smokers and non-smokers: How much more positive or more negative the average attitude towards smoking is among former smokers than among non-smokers.

```{=tex}
\begin{equation}
\small
\begin{split}
  attitude &= constant + b_1*smokesnomore + b_2*smoking \\
  attitude_{former smokers} &= constant + b_1*1 + b_2*0 \\
  attitude_{former smokers} &= constant + b_1
\end{split}
(\#eq:regcat2) 
\normalsize
\end{equation}
```
Finally, for smokers, we plug in 0 for $smokesnomore$ and 1 for $smoking$ [Eq. \@ref(eq:regcat3)]. The predicted attitude for smokers equals the constant plus the unstandardized regression coefficient for the $smoking$ dummy variable ($b_2$). This regression coefficient, then, represents the difference in average attitude between smokers and non-smokers (reference group).

```{=tex}
\begin{equation}
\small
\begin{split}
  attitude &= constant + b_1*smokesnomore + b_2*smoking \\
  attitude_{smokers} &= constant + b_1*0 + b_2*1 \\
  attitude_{smokers} &= constant + b_2
\end{split}
(\#eq:regcat3) 
\normalsize
\end{equation}
```
The interpretation of the effects (regression coefficients) for the included dummies is similar to the interpretation for a single dichotomous independent variable such as smoker versus non-smoker. It is the difference between the average score of the group coded 1 on the dummy variable and the average score of the reference group on the dependent variable. The reference group is the group scoring 0 on all dummy variables that represent the categorical independent variable.

If we exclude the dummy variable for the respondents who never smoked, as in the above example, the regression weight of the dummy variable $smokesnomore$ gives the average difference between former smokers and non-smokers. If the regression weight is negative, for instance -0.8, former smokers have on average a more negative attitude towards smoking than non-smokers. If the difference is positive, former smokers have on average a more positive attitude towards smoking.

Which group should we use as reference category, that is, which group should not be represented by a dummy variable in the regression model? This is hard to say in general. If one group is of greatest interest to us, we could use this as the reference group, so all dummy variable effects express differences with this group. Alternatively, if we expect a particular ranking of the average scores, we may pick the group at the highest, lowest or middle rank as the reference group. If you can't decide, run the regression model several times with a different reference group.

Finally, note that we should not include all three dummy variables in the regression model [Eq. \@ref(eq:regcat)]. We can already identify the non-smokers, because they score 0 on both the $smokesnomore$ and $smoking$ dummy variables. Adding the $neversmoked$ dummy variable to the regression model is like including the same independent variable twice. How can the estimation process decide which of the two identical independent variable is responsible for the effect? It can't decide, so the estimation process fails or it drops one of the dummy variables. If this happens, the independent variables are said to be perfectly *multicollinear*.

### Sampling distributions and assumptions {#regr-inference}

```{r regression-sampling, eval=FALSE, echo=FALSE, fig.pos='H', fig.align='center', fig.cap="What happens to regression lines from sample to sample?"}
# Goal: Understand that regression constant and coefficient(s) have sampling
# distributions.
# Generate a population with a weak negative effect (-0.6) of exposure on
# attitude and exposure, with a sizable error term (so a lot of variation in
# sample regression lines).
# Generate a sample (N = 10) and display it in a scatterplot with regression
# line, labelled with it's unstandardized regression coefficient value. Also
# plot the sampling distribution for the regression coefficient.
# Add a button to allow drawing a new sample; display the new sample and new
# regression line but retain the existing regression lines.
# Add button (or change sampling button) to draw 1,000 samples: don't display
# samples, just update sampling distribution with normal (or t) distribution as
# superimposed curve.

1. Which estimates can change from sample to sample: the regression constant, the regression coefficient, or both? Check your answer by drawing new samples.

2. What is the shape of the sampling distribution if you draw a lot of samples?

3. What happens if you draw samples of larger size? Think of what you learned in preceding chapters. Formulate your answer before you change sample size in Figure \@ref(fig:regression-sampling).
```

If we are working with a random sample or we have other reasons to believe that our data could have been different due to chance (Section \@ref(no-random-sample)), we should not just interpret the results for the data set that we collected. We should apply statistical inference---confidence intervals and significance tests---to our results. The confidence interval gives us bounds for plausible population values of the unstandardized regression coefficient. The *p* value is used to test the *null hypothesis that the unstandardized regression coefficient is zero in the population*.

Each regression coefficient as well as the constant may vary from sample to sample drawn from the same population, so we should devise a sampling distribution for each of them. These sampling distributions happen to have a *t* distribution under particular assumptions.

Chapters \@ref(param-estim) and \@ref(hypothesis) have extensively discussed how confidence intervals and *p* values are constructed and how they must be interpreted. So we focus now on the assumptions under which the *t* distribution is a good approximation of the sampling distribution of a regression coefficient.

#### Independent observations

The two most important assumptions require that the observations are *independent and identically distributed*. These requirements arise from probability theory. If they are violated, the statistical results should not be trusted.

Each observation, for instance, a measurement on a respondent, must be independent of all other observations. A respondent's dependent variable score is not allowed to depend on scores of other respondents.

It is hardly possible to check that our observations are independent. We usually have to assume that this is the case. But there are situations in which we should not make this assumption. In time series data, for example, the daily amount of political news, we usually have trends, cyclic movements, or issues that affect the amount of news over a period of time. As a consequence, the amount and contents of political news on one day may depend on the amount and contents of political news on the preceding days.

Clustered data should also not be considered as independent observations. Think, for instance, of student evaluations of statistics tutorials. Students in the same tutorial group are likely to give similar evaluations because they had the same tutor and because of group processes: Both enthusiasm and dissatisfaction can be contagious.

#### Identically distributed observations

To check the assumption of identically distributed observations, we inspect the residuals. Remember, the residuals are represented by the error term ($e$) in the regression equation. They are the difference between the scores that we observe for our respondents and the scores that we predict for them with our regression model.

```{r resid-normal, fig.pos='H', fig.align='center', fig.cap="What are the residuals and how are they distributed?", echo=FALSE, screenshot.opts = list(delay = 5), dev="png", out.width="775px"}
# Goal: Understand the meaning of residuals by linking residuals in a
# scatterplot to the x values in a histogram.
# Generate a sample (N = 20?) with a weak negative effect (-0.6) of exposure on
# attitude and exposure, with a sizable error term to have residuals that are
# clearly visible. Generate a sample with uniformly distributed residuals.
# Display the sample as a scatterplot with the regression line and (red) line
# segments linking the dots vertically to the regression line. Display the
# residuals also as a histogram with normal curve. Hovering over/clicking a line
# segment (residual) in the scatterplot should highlight the corresponding bar
# in the histogram. Add a button to draw a new sample.
knitr::include_app("http://82.196.4.233:3838/apps/resid-normal/", height="264px")
```

<A name="question8.1.9"></A>

```{block2, type='rmdquestion'}
9. What do the lines between the dots and the (blue) regression line represent in the scatter plot at the left of Figure \@ref(fig:resid-normal)? [<img src="icons/2answer.png" width=115px align="right">](#answer8.1.9)
```

<A name="question8.1.10"></A>

```{block2, type='rmdquestion'}
10. What is the relation between the scatter plot and the histogram? Can you point out the dot in the scatter plot that belongs to the leftmost bar in the histogram? Tip: Drag your mouse around a dot while pressing the left mouse button to see its residual in the histogram. [<img src="icons/2answer.png" width=115px align="right">](#answer8.1.10)
```

<A name="question8.1.11"></A>

```{block2, type='rmdquestion'}
11. Draw some new samples. Are the residuals always normally distributed: Does the top of the histogram coincide with the center of the normal distribution and are the left and right tails equally "fat"?  [<img src="icons/2answer.png" width=115px align="right">](#answer8.1.11)
```

If we sample from a population where attitude towards smoking depends on exposure, smoking status, and contact with smokers, we will be able to predict attitude from the independent variables in our sample. Our predictions will not be perfect, sometimes too high and sometimes too low. The differences between predicted and observed attitude scores are the residuals.

If our sample is truly a random sample with independent and identically distributed observations, the sizes of our errors (residuals) should be normally distributed for each value of the dependent variable, that is, attitude in our example. The residuals should result from chance (see Section \@ref(datageneratingprocess) for the relation between chance and a normal distribution).

So for all possible values of the dependent variable, we must collect the residuals for the observations that have this score on the dependent variable. For example, we should select all respondents who score 4.5 on the attitude towards smoking scale. Then, we select the residuals for these respondents and see whether they are approximately normally distributed.

Usually, we do not have more than one observation (if any) for a single dependent variable score, so we cannot apply this check. Instead, we use a simple and coarse approach: Are all residuals normally distributed?

A histogram with an added normal curve (like the right-hand plot in Figure \@ref(fig:resid-normal)) helps us to evaluate the distribution of the residuals. If the curve more or less follows the histogram, we conclude that the assumption of identically distributed observations is plausible. If not, we conclude that the assumption is not plausible and we warn the reader that the results can be biased.

#### Linearity and prediction errors

The other two assumptions that we use tell us about problems in our model rather than problems in our statistical inferences. Our regression model assumes a linear effect of the independent variables on the dependent variable (*linearity*) and it assumes that we can predict the dependent variable equally well or equally badly for all levels of the dependent variable (*homoscedasticity*, next section).

The regression models that we estimate assume a linear model. This means that an additional unit of the independent variable always increases or decreases the predicted value by the same amount. If our regression coefficient for the effect of exposure on attitude is -0.25, an exposure score of one predicts a 0.25 more negative attitude towards smoking than zero exposure. Exposure score five predicts the same difference in comparison to score four as exposure score ten in comparison to exposure score nine, and so on. Because of the linearity assumption, we can draw a regression model as a straight line. Residuals of the regression model help us to see whether the assumption of a linear effect is plausible.

```{r pred-linearity, fig.pos='H', fig.align='center', fig.cap="How do residuals tell us whether the relation is linear?", echo=FALSE, screenshot.opts = list(delay = 5), dev="png", out.width="775px"}
# Goal: Understand the relation between linear model (scatterplot) and residuals plot by manipulating the shape of the association.
# Generate a sample (N = 20?) with a weak negative effect (-0.6) of exposure on attitude, with a sizable error term to have residuals that are clearly visible. Generate either a sample with (1) linear, (2) curved, (3) U-shaped association. Display the sample as a scatterplot with the regression line and (red) line segments linking the dots vertically to the regression line. Display the residuals also in a residuals (Y) by predicted values (X) plot. Hovering over/clicking a dot in the scatterplot should highlight the corresponding dot in the residuals plot. Add a button to select a different association shape. Upon selection of a shape, generate & display new sample data.
knitr::include_app("http://82.196.4.233:3838/apps/pred-linearity/", height="460px")

#ADD OPTION: LINEAR POSITIVE?
```

<A name="question8.1.12"></A>

```{block2, type='rmdquestion'}
12. Which dot in the plot of residuals (Figure \@ref(fig:pred-linearity) bottom) corresponds with the left-most observation (dot) in the scatter plot of attitude by exposure (Figure \@ref(fig:pred-linearity) top)? Drag your mouse around the left-most dot while pressing the left mouse button to check your choice. Repeat for more dots until you understand the relation between the two plots. [<img src="icons/2answer.png" width=115px align="right">](#answer8.1.12)
```

<A name="question8.1.13"></A>

```{block2, type='rmdquestion'}
13. Select a U-shaped curve in Figure \@ref(fig:pred-linearity). Explain how the plot of residuals tells you that the association is not linear. Do the same for a curved association. [<img src="icons/2answer.png" width=115px align="right">](#answer8.1.13)
```

The relation between an independent and dependent variable, for example, exposure and attitude towards smoking, does not have to be linear. It can be curved or have some other fancy shape. Then, the linearity assumption is not met. A straight regression line does not nicely fit such data.

We can see this in a graph showing the (standardized) residuals (vertical axis) against the (standardized) predicted values of the dependent variable (on the horizontal axis), as exemplified by the lower plot in Figure \@ref(fig:pred-linearity). Note that the residuals represent prediction errors. If our regression predictions are systematically too low at some levels of the dependent variable and too high at other levels, the residuals are not nicely distributed around zero for all predicted levels of the dependent variable. This is what you see if the association is curved or U-shaped.

This indicates that our linear model does not fit the data. If it would fit, the average prediction error is zero for all predicted levels of the dependent variable. Graphically speaking, our linear model matches the data if positive prediction errors (residuals) are more or less balanced by negative prediction errors everywhere along the regression line.

#### Homoscedasticity and prediction errors

The plot of residuals by predicted values of the dependent variable tells us more than whether a linear model fits the data.

```{r pred-homoscedasticity, fig.pos='H', fig.align='center', fig.cap="How do residuals tell us that we predict all values equally well?", echo=FALSE, screenshot.opts = list(delay = 5), dev="png", out.width="775px"}
# Goal: Understand the relation between linear model (scatterplot) and residuals plot by manipulating homoscedasticity.
# Generate a sample (N = 20) with a weak negative effect (-0.6) of exposure on attitude and exposure, with a sizable error term to have residuals that are clearly visible. Generate error terms with a dependency on the independent variable ranging from -1 to +1. Display the sample as a scatterplot with the regression line and (red) line segments linking the dots vertically to the regression line. Display the residuals also in a residuals (Y) by predicted values (X) plot. Hovering over/clicking a dot in the scatterplot should highlight the corresponding dot in the residuals plot. Add a slider (range [-@, 0], initial value 0) to set the levl of heteroscedasticity. Upon slider change, generate & display new sample data.
knitr::include_app("http://82.196.4.233:3838/apps/pred-homoscedasticity/", height="460px")
```

<A name="question8.1.14"></A>

```{block2, type='rmdquestion'}
14. What strikes you about the residuals in Figure \@ref(fig:pred-homoscedasticity)? Remember that residuals represent prediction errors. [<img src="icons/2answer.png" width=115px align="right">](#answer8.1.14)
```

<A name="question8.1.15"></A>

```{block2, type='rmdquestion'}
15. What happens if you move the slider to the far left? [<img src="icons/2answer.png" width=115px align="right">](#answer8.1.15)
```

<A name="question8.1.16"></A>

```{block2, type='rmdquestion'}
16. At which slider position are all attitude levels predicted more or less equally well or equally badly? [<img src="icons/2answer.png" width=115px align="right">](#answer8.1.16)
```

The other assumption states that we can predict the dependent variable equally well at all dependent variable levels. In other words, the prediction errors (residuals) are more or less the same at all levels of the dependent variable. This is called *homoscedasticity*. If we have large prediction errors at some levels of the dependent variable, we should also have large prediction errors at other levels. As a result, the vertical width of the residuals by predictions scatter plot should be more or less the same from left to right. The dots representing residuals resemble a more or less rectangular band.

If the prediction errors are not more or less equal for all levels of the predicted scores, our model is better at predicting some values than other values. For example, low values can be predicted better than high values of the dependent variable. The dots representing residuals resemble a cone. This may signal, among other things, that we need to include moderation in the model.

```{r eval=FALSE, echo=FALSE}
#DROPPED

Why do we use the residuals and predicted values instead of a scatterplot for each dependent-independent variable pair to assess linearity and homoscedasticity? The reason is that some independent variables may predict low values and other independent variables may predict high values. This is perfectly OK if together they predict low and high values equally well.
```

### Answers {.unnumbered}

<A name="answer8.1.1"></A>

```{block2, type='rmdanswer'}
Answer to Question 1. 

* The predicted attitude is 1.6.
* This equals the constant of a regression equation because we plug in zero (0) as the value of exposure, so the regression equation simplifies to:
  
> _attitude_ = 1.6 + _b_ \* 0 = 1.6 + 0.

* In other words, the constant is the predicted value of the dependent variable if all predictors are zero. [<img src="icons/2question.png" width=161px align="right">](#question8.1.1)
```

<A name="answer8.1.2"></A>

```{block2, type='rmdanswer'}
Answer to Question 2. 

* The vertical orange line shows the difference between the predicted attitude if exposure is one and the predicted attitude if exposure is zero.

![](figures/S8_1Q2.png)

* This difference is captured by the unstandardized regression coefficient, denoted by the symbol $b$.
* More generally, this coefficient tells us the predicted difference in the dependent variable for a difference of one unit in the independent variable.
* According to the equation, the predicted attitude decreases by 0.6 for a one unit difference (0 to 1) in exposure.
* This is the decrease from 1.6 to 1.0 signalled by the vertical orange line. [<img src="icons/2question.png" width=161px align="right">](#question8.1.2)
```

<A name="answer8.1.3"></A>

```{block2, type='rmdanswer'}
Answer to Question 3. 

* Just replace exposure by 10 in the equation and calculate the result: 

> _attitude_ = 1.6 + (-0.6) \* 10 = 1.6 + -6.0 = -4.4 

* It is troublesome that we do not have any respondents with exposure scores near 10. The highest exposure scores are below 8. We cannot check that the regression line still fits the observations. We should not trust the regression line outside the range of values that we have observed for the predictor. [<img src="icons/2question.png" width=161px align="right">](#question8.1.3)
```

<A name="answer8.1.4"></A>

```{block2, type='rmdanswer'}
Answer to Question 4. 

* Here, the constant is equal to the average attitude of non-smokers.
* Use the slider to change the average attitude towards smoking for
non-smokers.
* This will change the value of the constant in the regression equation. Why?
* Non-smokers score 0 on the (smoking) status variable. The regression
equation for non-smokers, then, is:
  
> _attitude_ = _constant_ + _b_ * _status_ = _constant_ + _b_ * 0 = _constant_

* Thus, we see that the predicted attitude for non-smokers equals the
constant. In addition, we know that the predicted value for a group
equals the average score of the group. As a result, the constant equals
the average score of non-smokers in this example.
* Note that this is true only if the group is coded zero and if the regression
model contains only one independent variable (simple regression model). [<img src="icons/2question.png" width=161px align="right">](#question8.1.4)
```

<A name="answer8.1.5"></A>

```{block2, type='rmdanswer'}
Answer to Question 5. 

* In this example, the unstandardized regression coefficient (_b_) is equal to
the average attitude score of smokers minus the average attitude score of
non-smokers.
* This is so because smokers are coded as ones and non-smokers are coded as
zeros. The difference between smokers and non-smokers on the smoking status
variable is one. The regression coefficient (_b_) tells us the difference in
predicted attitude scores for a difference of one unit on the independent
variable. As a result, the unstandardized regression coefficient (_b_) tells us
the difference between the average (= predicted value) for smokers and the
average (= predicted value) for non-smokers.
* With equations:
* Smokers average: _attitude_ = _constant_ + _b_ * _status_ = _constant_ + _b_ * 1 = _constant_ + _b_

* Non-smokers average: _attitude_ = _constant_ + _b_ * _status_ = _constant_ + _b_ * 0 = _constant_
* Smokers - Non-smokers: _(constant + b)_ - _constant_ = _b_ [<img src="icons/2question.png" width=161px align="right">](#question8.1.5)
```

<A name="answer8.1.6"></A>

```{block2, type='rmdanswer'}
Answer to Question 6. 

* The precise answer to this question obviously depends on the group means in your plot.
* In general, the interpretation focuses on differences between the mean score of the reference group and the mean scores of the other groups. In this example, we are talking about average attitude towards smoking for each group defined by their smoking status. The reference group is selected with the Select reference drop-down list.

![](figures/S8_1Q6.png)

* The unstandardized regression coefficient (_b_) tells us how much larger (positive coefficient) or smaller (negative coefficient) the mean score of a group is in comparison to the reference group. In the above figure, the regression coefficient for smokers versus non-smokers (_b_ = 4.39) is equal to the average of the non-reference group (smokers: mean = 1.40) minus the average of the reference group (non-smokers: mean = -2.99).
* The associated _p_ value tells us how uncertain we are that there truly is a mean difference in the population. It tests the null hypothesis that the two groups have the same mean score on the dependent variable in the population.
[<img src="icons/2question.png" width=161px align="right">](#question8.1.6)
```

<A name="answer8.1.7"></A>

```{block2, type='rmdanswer'}
Answer to Question 7. 

* In the initial state of this figure, non-smokers are the reference group. The _p_ values, then, are associated with differences between on the one hand non-smokers and on the other hand people who stopped smoking or are still smoking. The comparison between the latter two is not included.
* It is hazardous to derive the _p_ value of the difference between two non-reference groups from the _p_ values of the differences between these groups and the reference group. Remember that a _p_ value depends on both effect size (difference between group means) and on the standard error. The latter depends, among other things, on the sample size per group. All of these aspects may vary across groups, so we cannot guess the _p_ value of the mean difference between two groups.
* The solution is to re-estimate the regression model with one of the groups
that you want to compare as reference group. If former smokers or smokers are the reference group, you obtain a _p_ value for the difference between former smokers and smokers. Select one of these groups in the drop-down list and there you go. [<img src="icons/2question.png" width=161px align="right">](#question8.1.7)
```

<A name="answer8.1.8"></A>

```{block2, type='rmdanswer'}
Answer to Question 8. 

There is not one right way of choosing a reference group; think about
arguments.

1. Substantive interest: Does your research focus on one particular group? If
so, use this group as the reference group, so it is included in all
comparisons. If, for example, the research is meant to support an anti-smoking
campaign, the group of current smokers is probably of central interest. Make
them your reference group.
2. If you expect a particular order in the group means, the group that you
expect to be in the middle is a good choice as reference group. If, for
example, you expect that attitude towards smoking is more positive for smokers
than for former smokers, and the latter are more positive than people who never
smoked, the former smokers are expected to be in the middle. If we use them as
reference group, we can test if they are more positive than people who never
smoked, and more negative than people who are currently smoking. If both differences are statistically significant, the difference between the highest scoring group and the lowest scoring group must also be statistically significant.
3. If two groups have relatively similar means in comparison to the third
group, you may be interested to know if the relatively small difference is
statistically significant. In this case, one of the two groups that have
similar means is a good reference group. [<img src="icons/2question.png" width=161px align="right">](#question8.1.8)
```

<A name="answer8.1.9"></A>

```{block2, type='rmdanswer'}
Answer to Question 9. 

* A red line depicts the residual or prediction error: The difference between
the actual dependent variable score (attitude) for a respondent (dot) and the
dependent variable score predicted by the regression line (blue) for the
independent variable score (exposure) of the respondent. [<img src="icons/2question.png" width=161px align="right">](#question8.1.9)
```

<A name="answer8.1.10"></A>

```{block2, type='rmdanswer'}
Answer to Question 10. 

* The residuals represented by the red lines in the scatterplot are counted in
the histogram.
* Drag the mouse pointer around one or more dots in the scatterplot while
pressing the left mouse button to select one or more observations (dots). You
will see where they are featured in the histogram (blue). The dot that is
furthest below the regression line is counted in the left-most bar of the
histogram. [<img src="icons/2question.png" width=161px align="right">](#question8.1.10)
```

<A name="answer8.1.11"></A>

```{block2, type='rmdanswer'}
Answer to Question 11. 

* No, sometimes the distribution is clearly skewed (asymmetrical). The top of the histogram is to the right or to the left of the center of the normal distribution and the right tail is shorter and fatter than the left tail or the other way around. [<img src="icons/2question.png" width=161px align="right">](#question8.1.11)
```

<A name="answer8.1.12"></A>

```{block2, type='rmdanswer'}
Answer to Question 12. 

* The blue regression line in the top graph represents the predicted values on
the attitude variable from exposure scores. The predicted attitude value for
the left-most observation is the value at which the blue line intersects with
the vertical red line dropping down from that observation.
* The left-most observation is the highest predicted value because the
regression line slopes down to the right. The highest predicted value is at
the far right of the residuals by predicted attitude graph (bottom graph)
because the predicted values are on the horizontal axis of this graph. The
left-most observation in the top graph, then, must correspond with the
right-most observation in the bottom graph.
* Note that this is not always true. If the regression slope is positive, that
is, the regression line goes up from left to right, the left-most observation
in the top graph has the lowest predicted value, so it corresponds to the
left-most observation in the residuals plot. [<img src="icons/2question.png" width=161px align="right">](#question8.1.12)
```

<A name="answer8.1.13"></A>

```{block2, type='rmdanswer'}
Answer to Question 13. 

![](figures/S8_1Q13.png)

* If the association between two variables is U-shaped, the regression line underestimates the attitude for observations with low exposure, overestimates the attitude for medium values of exposure, and underestimates the attitude for high exposure values. As a result, the residuals have a marked pattern from left to right: a set of positive residuals, followed by a set of negative residuals, followed by a set of positive residuals.
* The same phenomenon occurs for a curved association.
* In contrast, a linear association yields residuals without a clear pattern. At all levels of exposure and, hence, at all predicted levels of attitude, we may encounter both positive and negative residuals. In the residuals by predicted values plot, we have positive and negative residuals everywhere. We may underestimate as well as overestimate the dependent variable everywhere. [<img src="icons/2question.png" width=161px align="right">](#question8.1.13)
```

<A name="answer8.1.14"></A>

```{block2, type='rmdanswer'}
Answer to Question 14. 

![](figures/S8_1Q14.png)

* The residuals are larger for higher values of exposure. Residuals are prediction errors, so larger residuals mean worse predictions.
* In the top graph, the residuals tend to become larger from left to right: The residuals are larger for larger exposure scores (horizontal axis in the top graph).
* In this particular example, larger exposure scores predict more negative ('lower') attitudes towards smoking (vertical axis in the top graph). We can predict high attitude levels better (smaller residuals) than low attitude levels (larger residuals).
* Attitudes are on the horizontal axis in the bottom graph, so the residuals tend to become smaller if we go from left (low attitude scores) to right (high attitude scores).
* The regression model seems to predict attitude better for participants with low exposure scores than for participants with high exposure scores. [<img src="icons/2question.png" width=161px align="right">](#question8.1.14)
```

<A name="answer8.1.15"></A>

```{block2, type='rmdanswer'}
Answer to Question 15. 

* The pattern reverses. Now, residuals are larger for low values of exposure
or, equivalently in this model with a negative slope, for higher predicted
values of attitude.
* In this situation, we are better at predicting low attitude levels than high
attitude levels. Note that we prefer to predict all attitude levels equally
well. [<img src="icons/2question.png" width=161px align="right">](#question8.1.15)
```

<A name="answer8.1.16"></A>

```{block2, type='rmdanswer'}
Answer to Question 16. 

![](figures/S8_1Q16.png)

* If the slider is positioned at or around zero, the residuals are more or less equal for low, medium, or high predicted values of attitude.
Here, we can predict all levels of attitude equally well or equally badly.
* This is best seen in the bottom graph: The vertical spread of observations is more or less the same at the left, middle, and right of the graph. The vertical diameter of the dot cloud is more or less equal from left to right.
* This is how we like the plot to look. [<img src="icons/2question.png" width=161px align="right">](#question8.1.16)
```

## Regression Analysis in SPSS {#SPSS-regression}

### Instructions

```{r SPSSregsimple, echo=FALSE, out.width="640px", fig.pos='H', fig.align='center', fig.cap="(ref:regsimpleSPSS)", dev="png", screenshot.opts = list(delay = 5)}
knitr::include_url("https://www.youtube.com/embed/XrxlCOi6SgE", height = "360px")
# Goal: asymmetric association as prediction
# Example: consumers.sav, brand awareness by advertisement exposure
# Technique: regression with confidence intervals
# SPSS menu: regression>linear with CI under Statistics.
# Paste & Run.
# Interpret output: R2, F test, predictive effect strength (b*) and change (b) with 95% confidence interval.
# Check assumptions: in chapter on moderation with regression analysis?
```

------------------------------------------------------------------------

```{r SPSSregdummy2, echo=FALSE, out.width="640px", fig.pos='H', fig.align='center', fig.cap="(ref:regdummy2SPSS)", dev="png", screenshot.opts = list(delay = 5)}
knitr::include_url("https://www.youtube.com/embed/c2b4dtlPS54", height = "360px")
# Creating dummy variables in SPSS.
# Goal: Understand creating dummy variables.
# Example: smokers.sav, respondent's smoking status ( 3 categories).
# SPSS menu: Transform > Create Dummy Variables or Trasform > Recode into Different Variables.
# Inspect results: new variables, coded 0/1.
```

------------------------------------------------------------------------

```{r SPSSregdummy, echo=FALSE, out.width="640px", fig.pos='H', fig.align='center', fig.cap="(ref:regdummySPSS)", dev="png", screenshot.opts = list(delay = 5)}
knitr::include_url("https://www.youtube.com/embed/Vs26zuwAZdk", height = "360px")
# Using dummy variables in a regression model in SPSS.
# Example: smokers.sav, predict the attitude towards smoking from exposure to an anti-smoking campaign and respondent's smoking status (dummies).
# SPSS menu: Transform > Create Dummy Variables
# Remember: Leave one dummy variable out.
# Interpret results: unstandardized regression coefficient as average difference with reference category. Don't interpret the standardized regression coefficient.
```

------------------------------------------------------------------------

```{r SPSSregassumpt, echo=FALSE, out.width="640px", fig.pos='H', fig.align='center', fig.cap="(ref:regassumptSPSS)", dev="png", screenshot.opts = list(delay = 5)}
knitr::include_url("https://www.youtube.com/embed/hx2qdaVhlaM", height = "360px")
# Goal: Inspecting residuals.  (see Chapter 4 on hypothesis testing for video about regression basics.)
# Example: smokers.sav, predict the attitude towards smoking from exposure to an anti-smoking campaign and respondent's smoking status.
# Technique: regression analysis
# SPSS menu: linear regression, add plots
# Interpret output = check assumptions: Chart Editor of zresid * zpred plot: add reference line at 0 and perhaps at +2 and -2 to inspect shape of residual distribution.
```

### Exercises

<A name="question8.2.1"></A>

```{block2, type='rmdquestion'}
1. Use the data in [allsmokers.sav](http://82.196.4.233:3838/data/allsmokers.sav) to predict the attitude towards smoking from exposure to an anti-smoking campaign. Check the assumptions and interpret the results. [<img src="icons/2answer.png" width=115px align="right">](#answer8.2.1)
```

<A name="question8.2.2"></A>

```{block2, type='rmdquestion'}
2. Add smoking status (variable _status3_), and contact with smokers as predictors to the regression model of Exercise 1. Check the assumptions and interpret the results. [<img src="icons/2answer.png" width=115px align="right">](#answer8.2.2)
```

<A name="question8.2.3"></A>

```{block2, type='rmdquestion'}
3. The data set [allchildren.sav](http://82.196.4.233:3838/data/allchildren.sav) contains information about media literacy of children and parental supervision of their media use. Are the two related? Check the assumptions and interpret the results. [<img src="icons/2answer.png" width=115px align="right">](#answer8.2.3)
```

<A name="question8.2.4"></A>

```{block2, type='rmdquestion'}
4. How well can we predict brand awareness with ad exposure? Use [allconsumers.sav](http://82.196.4.233:3838/data/allconsumers.sav) to answer this question. [<img src="icons/2answer.png" width=115px align="right">](#answer8.2.4)
```

### Answers {.unnumbered}

<A name="answer8.2.1"></A>

```{block2, type='rmdanswer'}
Answer to Exercise 1. 

SPSS syntax:  
  
\* Check data.  
FREQUENCIES VARIABLES=exposure attitude  
  /ORDER=ANALYSIS.  
\* Simple regression analysis with assumption checks.  
REGRESSION  
  /MISSING LISTWISE  
  /STATISTICS COEFF OUTS CI(95) R ANOVA  
  /CRITERIA=PIN(.05) POUT(.10)  
  /NOORIGIN   
  /DEPENDENT attitude  
  /METHOD=ENTER exposure  
  /SCATTERPLOT=(\*ZRESID ,\*ZPRED)  
  /RESIDUALS HISTOGRAM(ZRESID).  
  
Check data:  
  
There are no impossible values on the two variables.  
  
Check assumptions:  

![](figures/S8_2Q1.png)  

* The distribution of the residuals is (left) skewed rather than normal. This is not good and we had better report this.

![](figures/S8_2Q1a.png)  

* The residuals are nicely centered around zero at all predicted levels of the dependent variable, so the association seems to be linear.
* The residuals are evenly spread around zero at all predicted levels.
Perhaps, the spread is slightly smaller at high predicted values. In all,
however, prediction accuracy seems to be more or less the same at all predicted levels (homoscedasticity).
  
Interpret the results:  
  
* Campaign exposure predicts attitude towards smoking among adults reasonably
well, *R*^2^ = .32, *F* (1, 310) = 143.05, *p* < .001.
* One additional unit of exposure decreases the predicted attitude by 0.32 to
0.44 points, *t* = -11.96, *p* < .001, 95% CI [-0.44; -0.32]. This is a moderate to strong effect (*b\** = -.56). [<img src="icons/2question.png" width=161px align="right">](#question8.2.1)
```

<A name="answer8.2.2"></A>

```{block2, type='rmdanswer'}
Answer to Exercise 2. 

SPSS syntax:  
  
\* Check data.  
FREQUENCIES VARIABLES=exposure status3 contact attitude  
  /ORDER=ANALYSIS.  

\* Create dummy variables for status3.  
\* With Transform > Create Dummy Variables.  
\* ENSURE THAT MEASUREMENT LEVEL IS SET TO NOMINAL OR ORDINAL.  
\* Define Variable Properties.  
\*status3.  
VARIABLE LEVEL  status3(ORDINAL).  
EXECUTE.  
SPSSINC CREATE DUMMIES VARIABLE=status3   
ROOTNAME1=status   
/OPTIONS ORDER=A USEVALUELABELS=YES USEML=YES OMITFIRST=NO.  

\* If your SPSS version does not have this command, use Recode.  
RECODE status3 (1=1) (ELSE=0) INTO status_2.  
VARIABLE LABELS  status_2 'Former smoker'.  
EXECUTE.  
RECODE status3 (2=1) (ELSE=0) INTO status_3.  
VARIABLE LABELS  status_3 'Smoker'.  
EXECUTE.  

\* Multiple regression analysis with assumption checks.  
REGRESSION  
  /MISSING LISTWISE  
  /STATISTICS COEFF OUTS CI(95) R ANOVA  
  /CRITERIA=PIN(.05) POUT(.10)  
  /NOORIGIN   
  /DEPENDENT attitude  
  /METHOD=ENTER exposure contact status_2 status_3  
  /SCATTERPLOT=(\*ZRESID ,\*ZPRED)  
  /RESIDUALS HISTOGRAM(ZRESID).  
  
Check data:  
  
All values on the variables seem to be valid.  
  
Check assumptions:  
  
![](figures/S8_2Q2.png)  

* The residuals look quite like a normal distribution. They are much more symmetrical than in Exercise 1. Note that a new regression model (we added predictors) may solve problems with the assumptions.

![](figures/S8_2Q2a.png)  

* The residuals by predicted values plot suggests that the effects can be linear. At all predicted levels, the residuals are both above and below zero, so the average residual is close to zero. 
* However, the spread of the residuals is larger at higher predicted
levels (right) than low predicted levels (left). More negative attitudes are predicted better than more positive attitudes. The  new regression model solves the linearity problem (above), but it creates a problem with homoscedasticity.
  
Interpret the results:  
  
* Include a table with regression coefficients, so we do not have to report all t
test results in our interpretation.

<div style="font-size: 0.8em">
| | _b_ |	SE | _b\*_ | _t_ | _p_ | 95\%CI |
|:------------------|----:|----:|----:|----:|----:|:--------:|
| Constant | 1.11 | 0.24 | | 4.52 | <.001 | [0.62, 1.59] |
| Exposure to anti-smoking campaign| -0.30 |0.02 | -0.45 | -13.31 | <.001 | [-0.35, -0.26] |
| Contact with smokers | 0.20 | 0.03 | 0.21 | 6.06 | <.001 | [0.14, 0.27] |
| Former smoker | -2.86 | 0.16 | -0.60 | -17.94 | <.001 | [-3.17, -2.54] |
| Smoker | -0.50 | 0.15 | -0.11 | -3.37 | .001 | [-0.79, -0.21] |
</div>
  
* The regression model predicts about seventy per cent of the variation in
attitude towards smoking, which is very much for a social scientific model, *R*^2^ = .69, *F* (4, 307) = 171.98, *p* < .001.
* Exposure to the anti-smoking campaign predicts a more negative attitude
towards smoking. We are 95% confident that an additional unit of exposure (on
a ten-point scale) decreases the predicted attitude by 0.26 to 0.35 points
(also on a ten-point scale) in the population.
* Contact with smokers is associated with a slightly more positive attitude in the population. An additional contact makes the predicted attitude 0.14 to 0.27 points more positive (with 95% confidence). Of the two, exposure (_b\*_ = -0.45) is a better predictor than contact with smokers (_b\*_ = 0.21).
* Former smokers are on average much (-2.86 points, 95% CI [-3.17, -2.54]) more negative about smoking than non-smokers. Smokers' attitude is on average only 0.50 points (95% CI [-0.79, -0.21]) below the attitude of non-smokers.
* The residuals suggest that the assumptions for using the theoretical
approximation of the sampling distributions may not have been met. [<img src="icons/2question.png" width=161px align="right">](#question8.2.2)
```

<A name="answer8.2.3"></A>

```{block2, type='rmdanswer'}
Answer to Exercise 3. 

* The analysis method that you choose depends on your substantive decision on the direction of the association.
* If you assume no direction, a correlation coefficient is the most appropriate choice. If you think one variable may depend on another, a (simple) regression model is the best choice.
  
SPSS syntax:  
  
\* Check data.  
FREQUENCIES VARIABLES=medliter supervision  
  /ORDER=ANALYSIS.  
\* Set supervision 25 to missing.  
\* Define Variable Properties.  
\*supervision.  
MISSING VALUES supervision(25.00).  
EXECUTE.  
\* Undirected: correlation (linear?).  
\* Check scatterplot.  
GRAPH  
  /SCATTERPLOT(BIVAR)=supervision WITH medliter  
  /MISSING=LISTWISE.  
\* Correlations.  
CORRELATIONS  
  /VARIABLES=medliter supervision  
  /PRINT=TWOTAIL NOSIG  
  /MISSING=PAIRWISE.  
\* Simple regression: media literacy dependent.  
REGRESSION  
  /MISSING LISTWISE  
  /STATISTICS COEFF OUTS CI(95) R ANOVA  
  /CRITERIA=PIN(.05) POUT(.10)  
  /NOORIGIN   
  /DEPENDENT medliter  
  /METHOD=ENTER supervision  
  /SCATTERPLOT=(\*ZRESID ,\*ZPRED)  
  /RESIDUALS HISTOGRAM(ZRESID).  
\* Simple regression: parental supervision dependent.  
REGRESSION  
  /MISSING LISTWISE  
  /STATISTICS COEFF OUTS CI(95) R ANOVA  
  /CRITERIA=PIN(.05) POUT(.10)  
  /NOORIGIN   
  /DEPENDENT supervision  
  /METHOD=ENTER medliter  
  /SCATTERPLOT=(\*ZRESID ,\*ZPRED)  
  /RESIDUALS HISTOGRAM(ZRESID).  
  
Check data:  
  
Score '25' for parental supervision cannot be right because the scale runs to
10. Define this score as a missing value.
  
Check assumptions:  
  
* In the regression models, the residuals are quite normally distributed and
nicely grouped around zero at all levels of the predicted scores.
* The variation of residuals is more or less the same at different levels of
the predicted scores if we use parental supervision as the dependent variable. 
* With media literacy as dependent variable, the spread of residuals seems to be slightly larger for average predicted media literacy scores (in the middle) than for low (left) or high (right) predicted scores. 

![](figures/S8_2Q3.png)

Interpret the results:  
  
Parental supervision and child media literacy are moderately correlated (*r* = .42, *p* < .001). 

If we use media literacy as the dependent variable, more supervision predicts more media literacy, *t* = 9.94, *p* < .001, 95% CI [0.37; 0.55]. One additional unit of parental supervision predicts 0.46 additional units of media literacy. 

If we use parental supervision as the dependent variable, on the other hand, one additional unit of media literacy predicts 0.38 additional units of parental supervision, _t_ = 9.94, *p* < .001, 95% CI [0.30; 0.45].
  
Note that the _t_ test on the regression coefficient is the same in a simple
regression model if you use supervision or media literacy as dependent variable. You may also note that the correlation coefficient equals _R_ and _b\*_ in both regression models in a simple regression model. [<img src="icons/2question.png" width=161px align="right">](#question8.2.3)
```

<A name="answer8.2.4"></A>

```{block2, type='rmdanswer'}
Answer to Exercise 4. 

SPSS syntax:  
  
\* Check data.  
FREQUENCIES VARIABLES=ad_expo brand_aw  
  /ORDER=ANALYSIS.  
\* Simple regression.  
REGRESSION  
  /MISSING LISTWISE  
  /STATISTICS COEFF OUTS CI(95) R ANOVA  
  /CRITERIA=PIN(.05) POUT(.10)  
  /NOORIGIN   
  /DEPENDENT brand_aw  
  /METHOD=ENTER ad_expo  
  /SCATTERPLOT=(\*ZRESID ,\*ZPRED)  
  /RESIDUALS HISTOGRAM(ZRESID).  
  
Check data:  
  
There are no impossible values.  
  
Check assumptions:  
  
* We need at least twenty observations (cases) for each predictor in the
regression model. Our model contains only one predictor, so the 520 cases in
our data set suffice for using the theoretical approximation (_F_ and _t_
distribution) here.
* The residuals are reasonably normal and quite equally distributed for all
predicted values.

![](figures/S8_2Q4.png)
  
Interpret the results:  
  
Ad exposure predicts about one fifth of the variation in brand awareness
scores, *R*^2^ = .19, *F* (1, 518) = 124.20, *p* < .001.
The predictive effect of exposure to brand advertisements is moderately strong
(*b\** = 0.44). An additional unit of exposure increases the predicted brand
awareness with 0.42 points on average, but the increase may vary between 0.35 and 0.50 in the population with 95% confidence, *t* = 11.15, *p* < .001, 95% CI [0.35; 0.50]. [<img src="icons/2question.png" width=161px align="right">](#question8.2.4)
```

## Different Lines for Different Groups {#categoricalmoderator}

What if the effect of campaign exposure on attitude towards smoking may be different in different contexts, e.g., for people who smoke themselves and people who do not smoke? Perhaps, the campaign is more effective among smokers than among non-smokers or the other way around. If so, the effect of campaign exposure is moderated by the smoking status of the participants.

In a conceptual diagram (Figure \@ref(fig:moderator-concept2)), moderation is represented by an arrow pointing at another arrow. The moderator (smoking status) changes the relation between the predictor (campaign exposure) and the dependent variable (attitude towards smoking).

```{r moderator-concept2, echo=FALSE, fig.pos='H', fig.align='center', fig.cap="Conceptual diagram of moderation.", fig.asp=0.3}
# Create coordinates for the variable names.
variables <- data.frame(x = c(0.35, 0.5, 0.65), 
                        y = c(.1, .3, .1),
                        hjust = c(1, 0.5, 0),
                        label = c("Predictor", "Moderator", "Dependent variable"))
ggplot(variables, aes(x, y)) + 
  geom_segment(aes(x = x[1], y = y[1], xend = x[3], yend = y[1]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  geom_segment(aes(x = x[2], y = y[2], xend = x[2], yend = y[1]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) +
  geom_label(aes(label=label, hjust = hjust)) + 
  coord_cartesian(xlim = c(0.3, 0.8), ylim = c(0, 0.4)) +
  theme_void()
# Cleanup.
rm(variables)
```

We used a similar diagram to express moderation in two-way analysis of variance (Section \@ref(moderationanova)). But now at least one of our independent variables is numeric, for example, the number of times the respondent has been exposed to the campaign.

Analysis of variance (ANOVA) investigates the effects of categorical variables on a numerical dependent variable. It cannot handle numerical independent variables. Although there are ways to include numerical independent variables in analysis of variance, for example, analysis of covariance (ANCOVA), we use regression analysis if we have a numerical dependent variable and at least one numerical independent variable.

```{block2, type='rmdimportant'}
Use regression analysis if you have a numerical dependent variable and at least one numerical independent variable.
```

In the current section, we discuss regression models with a numerical predictor and a categorical moderator. The next chapter (Chapter \@ref(moderationcont)) presents regression models in which both the predictor and the moderator are numerical.

### A dichotomous moderator and numerical predictor

```{r dichotomous-moderator, fig.pos='H', fig.align='center', fig.cap="Is the effect of exposure on attitude moderated by smoking status?", echo=FALSE, screenshot.opts = list(delay = 5), dev="png", out.width="420px"}
# Goal: Sensitize the student to the notion that moderation in a regression
# model means different slopes for different groups.
# In a graph with attitude as Y axis and exposure as X axis, generate two regression lines, one for smokers and one for non-smokers. 
# Plot the regression lines in a scatterplot. 
# Show the regression equation for each line (preferably in the plot). 
# Systematically vary the slopes (the same, one more negative than the other,
# opposite signs) and the constant difference (positive, nearly zero,
# negative).
# Add a Generate New button to replace the regression lines by a new pair of lines.
knitr::include_app("http://82.196.4.233:3838/apps/dichotomous-moderator/", height="350px")
```

<A name="question8.3.1"></A>

```{block2, type='rmdquestion'}
1. Is the effect of exposure on attitude moderated by the smoking status of respondents (smokers versus non-smokers) in Figure \@ref(fig:dichotomous-moderator)? Motivate your answer. Press the *Take new sample* button to practice some more with recognizing moderation. [<img src="icons/2answer.png" width=115px align="right">](#answer8.3.1)
```

In Section \@ref(regression-equation), we have analyzed the predictive effects of exposure to an anti-smoking campaign and smoking status on a person's attitude towards smoking. We have found a negative effect for exposure and a positive effect for smoking. More exposure predicts a more negative attitude whereas smokers have on average a more positive attitude towards smoking than non-smokers.

Our current question is: Does exposure to the campaign have the same effect for smokers and non-smokers? We want to compare an effect (exposure on attitude) for different contexts (smokers versus non-smokers), so our current question involves moderation. Is the effect of exposure on attitude moderated by smoking status?

Our moderator (smoker vs. non-smoker) is a dichotomous variable but our predictor (exposure) is numerical, so we cannot use analysis of variance. Instead, we use regression analysis, which allows numerical predictors.

In the context of a regression model, moderation means **different slopes for different groups**. The slope of the regression line is the regression coefficient, which expresses the effect of the predictor on the dependent variable. If we have different effects in different contexts (moderation), we must have different regression coefficients for different groups.

### Interaction variable {#interaction-variable}

How do we obtain different regression coefficients and lines for smokers and non-smokers? The statistical trick is quite easy: Include an additional predictor in the model that is the product of the predictor (exposure) and the moderator (smoking status). This new predictor is the *interaction variable*. The regression coefficient of the interaction variable is called the *interaction effect*.

```{r interaction-var, fig.pos='H', fig.align='center', fig.cap="How does an interaction variable create different regression lines for different groups?", echo=FALSE, out.width="775px", screenshot.opts = list(delay = 5), dev="png"}
# Goal: Intuitive understanding of the effect of an interaction variable.
# Generate a dataset with 30 observations for the regression model y = 3 -
# 0.5x_1 + 1.5x_2 + 0.3x_1*x_2 with x_1 in the range [0, 10] and x_2 a dummy (0
# or 1) with a random uniform component to each parameter in the range [-.1,
# .1].
# In a scatterplot of attitude (Y) versus exposure (X), display the regression
# line (fat, grey) for the equation with x_2 = 0, labelled with the regression
# equation without the interaction variable. Display smokers and non-smokers
# with different colours/shapes.
# Add a select list labeled 'Add product of exposure and smoking status' with
# the values '--', '0 - Non-smokers', and '1 - Smokers'. Selection of a value
# adds the corresponding regression line to the plot with the category name and
# regression equation. (The line for non-smokers is parallel to the fat gray
# line.)
# Clicking/hovering over the newly created line shows the slope as a sum of the
# conditional and interaction effect, e.g., "Slope: 0.5 * exposure + 0.3 * 1 *
# exposure".

knitr::include_app("http://82.196.4.233:3838/apps/interaction-var/", height="440px")

# 1. In Figure \@ref(fig:interaction-var-effect), does the fat grey line represent the effect of exposure on attitude in a simple regression model, a multiple regression model, or both?
# 
# 2. Select an option under "Add product" and explain what the newly created regression line means.
# 
# 3. Select the other option. Explain why the two regression lines that you created have different slopes.
```

<A name="question8.3.2"></A>

```{block2, type='rmdquestion'}
2. Change the slider position to make the regression line for smokers fit the dot cloud representing smokers. The grey line is your target. 
    - What happens if you move the slider? 
    - Can you explain when you have to move the slider to the right and when you have to move it to the left? [<img src="icons/2answer.png" width=115px align="right">](#answer8.3.2)
```

<A name="question8.3.3"></A>

```{block2, type='rmdquestion'}
3. The regression line for non-smokers always has the correct slope to fit the dot cloud representing non-smokers. Draw some new samples to verify this. 
    - What is the slope of the regression line for non-smokers? In other words, what is the effect (regression coefficient) of exposure on attitude for non-smokers? Tip: Replace the variable *smoker* in the regression equation by zero and simplify the equation. [<img src="icons/2answer.png" width=115px align="right">](#answer8.3.3)
```

The interaction variable must be included together with the original predictor and moderator variables, see Equation \@ref(eq:intvar). This is also visible in the statistical diagram (Figure \@ref(fig:moderator-statistical)) for moderation in a regression model.

```{=tex}
\begin{equation}
\small
\begin{split}
  attitude = &\ constant + b_1*exposure + b_2*smoker + b_3*exposure*smoker 
\end{split}
(\#eq:intvar) 
\normalsize
\end{equation}
```
```{r moderator-statistical, fig.pos='H', fig.align='center', fig.cap="Statistical diagram of moderation.", echo=FALSE, fig.asp=0.4}
library(ggplot2)
# Create coordinates for the variable names.
variables <- data.frame(x = c(rep.int(0.3, times = 3), 0.7), 
                        y = c(.4, .25, .1, .25),
                        label = c("Exposure", "Smoker", "Exposure*Smoker", "Attitude"))
# Add coordinates for arrow endpoint.
x_diff <- 0.04
variables$xend <- variables$x[4] - x_diff #fixed translation to the left
variables$yend <- variables$y[4] + x_diff * (variables$y - variables$y[4]) / (variables$x[4] - variables$x)
rm(x_diff)
ggplot(variables, aes(x, y)) + 
  geom_segment(aes(x = x[1], y = y[1], xend = xend[1], yend = yend[1]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  geom_segment(aes(x = x[2], y = y[2], xend = xend[2], yend = yend[2]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  geom_segment(aes(x = x[3], y = y[3], xend = xend[3], yend = yend[3]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  # geom_segment(aes(x = x[4], y = y[4], xend = xend[4], yend = yend[4]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  geom_label(aes(label=label)) + 
  coord_cartesian(xlim = c(0.2, 0.8), ylim = c(0.1, 0.5)) +
  theme_void()
```

The smoking status variable is coded 1 for smokers and 0 for non-smokers. For clarity, we name this variable *smoker* with score 1 for Yes and score 0 for No. We have two different regression equations, one for each group on the dichotomous predictor *smoker*. Just plug in the two possible values (1 and 0) for this variable. For non-smokers [Equation \@ref(eq:intvarnonsmoker)], the interaction variable drops from the model because multiplying by zero yields zero. For non-smokers, our reference group, $b_1$ represents the effect of exposure on attitude. It is called the *simple slope* of exposure for non-smokers.

```{=tex}
\begin{equation}
\small
\begin{split}
  attitude = &\ constant + b_1*exposure + b_2*smoker + b_3*exposure*smoker \\
  attitude_{non-smokers} = &\ constant + b_1*exposure + b_2*0 + b_3*exposure*0 \\
  attitude_{non-smokers} = &\ constant + b_1*exposure
\end{split}
(\#eq:intvarnonsmoker)
\normalsize
\end{equation}
```
In contrast, the interaction variable remains in the model for smokers [Equation \@ref(eq:intvarsmoker)], who score 1 on smoking status. Note what happens with the coefficient of the exposure effect if we rearrange the terms a little: The exposure effect equals the effect for the reference group of non-smokers ($b_1$) plus the effect of the interaction variable ($b_3$). The simple slope for smokers, then, is ($b_1 + b_3$).

```{=tex}
\begin{equation}
\small
\begin{split}
  attitude = &\ constant + b_1*exposure + b_2*smoker + b_3*exposure*smoker \\
  attitude_{smokers} = &\ constant + b_1*exposure + b_2*1 + b_3*exposure*1 \\
  attitude_{smokers} = &\ constant + b_1*exposure + b_3*exposure + b_2 \\
  attitude_{smokers} = &\ constant + (b_1 + b_3)*exposure + b_2 
\end{split}
(\#eq:intvarsmoker) 
\normalsize
\end{equation}
```
The interaction variable changes the slope of the effect of exposure on attitude. More specifically, the regression coefficient of the interaction variable ($b_3$) shows the difference between the simple slope of the exposure effect for smokers ($b_1+b_3$) and the simple slope for non-smokers ($b_1$). 

Let us assume that the unstandardized regression coefficient of the interaction effect is -0.3. This means that the effect of exposure on attitude is more strongly negative (or less positive) for smokers than for non-smokers. One additional unit of exposure decreases the predicted attitude for smokers by 0.3 **more** than for non-smokers.

### Conditional effects, not main effects {#conditional-effects}

It is very important to note that the effects of exposure and smoking status in a model with exposure by smoking status interaction are **not** main effects as in analysis of variance. As we have seen in the preceding section [Equation \@ref(eq:intvarnonsmoker)], the regression coefficient $b_1$ for exposure expresses the effect of exposure for non-smokers. It is a *conditional effect*, namely the effect for non-smokers only. Non-smokers are the *reference group* because they score zero on the moderator (*smoker*). This is quite different from a main effect in analysis of variance, which is an average effect over all groups.

In a similar way, the regression coefficient $b_2$ for smoking status expresses the effect for persons who score zero on the exposure predictor. Simply plug in the value 0 for exposure in the regression equation [Eq. \@ref(eq:simplestatus)].

```{=tex}
\begin{equation}
\small
\begin{split}
  attitude = &\ constant + b_1*exposure + b_2*smoker + b_3*exposure*smoker\\
  attitude_{no-expo} = &\ constant + b_1*0 + b_2*smoker + b_3*0*smoker \\
  attitude_{no-expo} = &\ constant + b_2*smoker
\end{split}
(\#eq:simplestatus) 
\normalsize
\end{equation}
```
Smoking status is a dichotomy, so its regression coefficient ($b_2$) tells us the average difference in attitude between smokers and non-smokers. Due to the inclusion of the interaction variable, it now tells us the difference in average attitude between smokers and non-smokers **who have zero exposure to the anti-smoking campaign**. Note again that this is a conditional effect, not a main effect.

### Interpretation and statistical inference {#interactioninterpretation}

```{r dich-moderator-output, echo=FALSE, message=FALSE, warning=FALSE}
# Table of regression coefficients for exposure moderated by status. Similar to SPSS output (with correct standardized coefficients).
# Create effect sizes.
smokers <- haven::read_spss("data/smokers.sav")
model_1 <- lm(attitude ~ exposure*status2, data = smokers)
# Table with results in SPSS style.
results <- coef(summary(model_1))
# Confidence intervals
ci <- confint.lm(model_1)
results <- cbind(results, ci)
# Correctly standardized coefficients.
attach(smokers)
z_exposure <- (exposure - mean(exposure))/sd(exposure)
z_status2 <- (status2 - mean(status2))/sd(status2)
z_expostatus2 <- z_exposure * z_status2
z_attitude <- (attitude - mean(attitude))/sd(attitude)
model_2 <- lm(z_attitude ~ z_exposure + z_status2 + z_expostatus2)
results_2 <- coef(summary(model_2))
results <- cbind(results[, 1:2], results_2[, 1], results[, 3:6])
results[1, 3] <- NA
# Adjust parameter names
attributes(results)$dimnames[[1]][1] <- "(Constant)"
attributes(results)$dimnames[[1]][2] <- "Exposure"
attributes(results)$dimnames[[1]][3] <- "Status (smoker = 1, non-smoker = 0)"
attributes(results)$dimnames[[1]][4] <- "Exposure*Status (smoker)"
# Table.
options(knitr.kable.NA = '')
knitr::kable(results, digits = 3, booktabs = TRUE,
             caption = "Predicting attitude towards smoking: regression analysis results.",
             col.names = c("B", "Std. Error", "Beta", "t", "Sig.", "Lower Bound", "Upper Bound")) %>%
  kable_styling(font_size = 12, full_width = F,
                latex_options = c("scale_down", "HOLD_position")) %>%
  column_spec(1, color = "white",
              background = c("#FDAE61", "#2B83BA", "#ABDDA4", "#D7191C")) %>%
  column_spec(2, color = c("#FDAE61", "#2B83BA", "#ABDDA4", "#D7191C"))
# Helper function for displaying results within the text.
source("report_n.R")
#Cleanup (partial).
rm(smokers, model_1, ci, results_2, model_2, z_attitude, z_status2, z_expostatus2, z_exposure)
```

In Table \@ref(tab:dich-moderator-output), the effect of exposure on attitude depends on the value of smoking status because the model includes an interaction effect of exposure with smoking status (red). Non-smokers are the reference group on the smoking status dummy variable because they are coded 0. Therefore, the regression coefficient for exposure (blue) gives us the effect of exposure on smoking attitude **for non-smokers**. 
If you want to check this, plug in 0 for the smoking status variable in the regression equation that we may construct from Table \@ref(tab:dich-moderator-output). 

```{=tex}
\begin{equation}
\small
\begin{split}
  attitude = &\ \color{#FDAE61}{constant} + \color{#2B83BA}{b_1*exposure} + \color{#ABDDA4}{b_2*status} + \color{#D7191C}{b_3*exposure*status}\\
  attitude_{non-smoker} = &\ \color{#FDAE61}{0.900} + \color{#2B83BA}{-0.162*exposure} + \color{#ABDDA4}{1.980*0} + \color{#D7191C}{-0.327*exposure*0}\\
  attitude_{non-smoker} = &\ \color{#FDAE61}{0.900} + \color{#2B83BA}{-0.162*exposure} + \color{#ABDDA4}{0} + \color{#D7191C}{0}\\
  attitude_{non-smoker} = &\ \color{#FDAE61}{0.900} + \color{#2B83BA}{-0.162*exposure}
\end{split}
(\#eq:nonsmokers) 
\normalsize
\end{equation}
```

Multiplying by zero yields zero, so for non-smokers, the resulting effect of exposure on attitude is `r report_n(results[2, 1], digits = 2)`. An additional unit of exposure predicts a smoking attitude among non-smokers that is `r report_n(abs(results[2, 1]), digits = 2)` points more negative. More exposure to the campaign goes together with a more negative attitude towards smoking for non-smokers. The *p* value for this effect tests the null hypothesis that the effect is zero in the population. If the exposure effect is statistically significant, we reject this null hypothesis.

Smokers are coded 1 on the (smoking) status variable and non-smokers are coded 0, so the regression coefficient for the interaction variable tells us that the slope of the exposure effect is `r report_n(abs(results[4, 1]), digits = 2)` lower for smokers than for non-smokers. The estimated slope of the exposure effect is `r report_n(results[2, 1], digits = 2)` for non-smokers. We can add the regression coefficient of the interaction variable to obtain the estimated slope for smokers, which is `r report_n(results[2, 1] + results[4, 1], digits = 2)`.

If you want to check this, plug in 1 for the smoking status variable in the regression equation. Add the two effects of exposure in the equation to obtain the effect of exposure on attitude for smokers.

```{=tex}
\begin{equation}
\small
\begin{split}
  attitude = &\ \color{#FDAE61}{constant} + \color{#2B83BA}{b_1*exposure} + \color{#ABDDA4}{b_2*status} + \color{#D7191C}{b_3*exposure*status}\\
  attitude_{smoker} = &\ \color{#FDAE61}{0.900} + \color{#2B83BA}{-0.162*exposure} + \color{#ABDDA4}{1.980*1} + \color{#D7191C}{-0.327*exposure*1}\\
  attitude_{smoker} = &\ \color{#FDAE61}{0.900} + \color{#2B83BA}{-0.162*exposure} + \color{#ABDDA4}{1.980} + \color{#D7191C}{-0.327*exposure}\\
  attitude_{smoker} = &\ \color{#FDAE61}{0.900} + \color{#2B83BA}{-0.162*exposure} + \color{#D7191C}{-0.327*exposure} + \color{#ABDDA4}{1.980}\\
  attitude_{smoker} = &\ \color{#FDAE61}{0.900} + (\color{#2B83BA}{-0.162} + \color{#D7191C}{-0.327})*exposure + \color{#ABDDA4}{1.980}\\
  attitude_{smoker} = &\ \color{#FDAE61}{0.900} + -0.489*exposure + \color{#ABDDA4}{1.980}
\end{split}
(\#eq:smokers) 
\normalsize
\end{equation}
```

Now we can compare the slopes (regression coefficients) for the two groups, which gives good insight into the nature of moderation in this example. The effect of exposure on attitude is more strongly negative for smokers (`r report_n(results[2, 1] + results[4, 1], digits = 2)`) than for non-smokers (`r report_n(results[2, 1], digits = 2)`).

The interaction variable is treated as an ordinary predictor in the estimation process, so it receives a confidence interval and a *p* value. The null hypothesis for the *p* value is that the interaction effect is zero in the population. In other words, the effect of exposure on attitude is hypothesized to be the same for smokers and non-smokers in the population; no moderation is expected in the population.

We know the confidence intervals and *p* values of the exposure effect for non-smokers (the regression coefficient for exposure) and for the difference between their exposure effect and the exposure effect for smokers (the regression coefficient for the interaction effect). We do not know, however, the confidence interval and statistical significance of the exposure effect for smokers. We cannot add confidence intervals or *p* values, so we do not know if the effect of exposure for smokers is significantly different from zero in the population.

If you want to know the confidence interval or *p* value of the exposure effect for smokers, you have to rerun the regression analysis using a different dummy variable for the moderator. You should create a dichotomous variable that assigns 0 to smokers and 1 to non-smokers, and an interaction variable created with this dichotomy. The regression coefficient of the exposure effect now expresses the effect for smokers because smokers are the reference group on the new dummy variable. The associated *p* value and confidence interval apply to the exposure effect for smokers.

Interaction variables are used just like ordinary predictors, so the general assumptions of regression analysis apply. See Section \@ref(regr-inference) for a description of the assumptions and checks.

In a similar way, the effect of smoking status on attitude is conditional on exposure because smoking status and exposure are included in the interaction variable. The regression coefficient for status tells us the difference between smokers and non-smokers who have 0 exposure. So, without exposure to the campaign, smokers are on average `r report_n(results[3, 1], digits = 2)` more positive towards smoking than non-smokers. The *p* value tests the null hypothesis that the difference is zero for people without exposure (exposure = 0) to the anti-smoking campaign.

Let us conclude the interpretation with a warning. The standardized regression coefficients that SPSS reports for interaction effects or effects of predictors that are involved in interaction effects **must not be used**. They are calculated in the wrong way if the regression model includes an interaction variable. As a result, they are misleading.

```{r dich-moderator-cleanup, echo=FALSE}
#Cleanup.
rm(report_n, results)
```

### A categorical moderator

What if we have three or more groups on our moderator? For example, smoking status measured with three categories: (1) never smoked, (2) formerly smoked, (3) currently smoking? Does the effect of exposure on attitude vary between non-smokers, participants who stopped smoking, and those who are still smoking?

```{r categorical-moderator, fig.pos='H', fig.align='center', fig.cap="When do we have moderation with a categorical moderator?", echo=FALSE, screenshot.opts = list(delay = 5), dev="png", out.width="420px"}
# Goal: Sensitize the student to the notion that moderation in a regression
# model means different slopes for different groups.
# In a graph with attitude as Y axis and exposure as X axis, generate two regression lines, one for smokers and one for non-smokers. 
# Plot the regression lines in a scatterplot. 
# Show the regression equation for each line (preferably in the plot). 
# Systematically vary the slopes (the same, one more negative than the other,
# opposite signs) and the constant difference (positive, nearly zero,
# negative).
# Add a Generate New button to replace the regression lines by a new pair of lines.
knitr::include_app("http://82.196.4.233:3838/apps/categorical-moderator/", height="350px")
```

<A name="question8.3.4"></A>

```{block2, type='rmdquestion'}
4. If we have exposure effects within three groups, as in Figure \@ref(fig:categorical-moderator), when do we have an interaction effect (moderation) and when do we not have an interaction effect? Motivate your answer. Press the **Take new sample** button to practice recognizing moderation. [<img src="icons/2answer.png" width=115px align="right">](#answer8.3.4)
```

In Section \@ref(categorical-predictor), we learned that we must create dummy variables for all but one groups of a categorical predictor in a regression model. This is what we have to do also for a categorical moderator. If the effect of a predictor, such as exposure, is moderated by a categorical variable, we have to create an interaction variable for each dummy variable in the equation. To create the interaction variables, we multiply the predictor by each of the dummy variables.

```{r categorical-moderator-fig, echo=FALSE, fig.pos='H', fig.align='center', fig.cap="Statistical diagram with a moderator consisting of three groups. Non-smokers are the reference group", fig.asp=0.4}
library(ggplot2)
# Create coordinates for the variable names.
variables <- data.frame(x = c(rep.int(0.3, times = 5), 0.7), 
                        y = c(.5, .4, .3, .2, .1, .3),
                        label = c("Exposure", "Former smoker", "Smoker", "Exposure*Former", "Exposure*Smoker", "Attitude"))
# Add coordinates for arrow endpoint.
x_diff <- 0.04
variables$xend <- variables$x[6] - x_diff #fixed translation to the left
variables$yend <- variables$y[6] + x_diff * (variables$y - variables$y[6]) / (variables$x[6] - variables$x)
rm(x_diff)
ggplot(variables, aes(x, y)) + 
  geom_segment(aes(x = x[1], y = y[1], xend = xend[1], yend = yend[1]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  geom_segment(aes(x = x[2], y = y[2], xend = xend[2], yend = yend[2]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  geom_segment(aes(x = x[3], y = y[3], xend = xend[3], yend = yend[3]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  geom_segment(aes(x = x[4], y = y[4], xend = xend[4], yend = yend[4]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  geom_segment(aes(x = x[5], y = y[5], xend = xend[5], yend = yend[5]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  geom_label(aes(label=label)) + 
  coord_cartesian(xlim = c(0.2, 0.8), ylim = c(0.1, 0.5)) +
  theme_void()
```

In the end, we have an interaction variable for all groups but one on the categorical moderator. Figure \@ref(fig:categorical-moderator-fig) shows the statistical diagram. Estimation of the model yields point estimates (regression coefficients), confidence intervals, and *p* values for all independent variables (Table \@ref(tab:cat-moderator-results)).

```{r cat-moderator-results, echo=FALSE, message=FALSE, warning=FALSE}
# Table of regression coefficients for exposure moderated by status3. Similar to SPSS output (without correct standardized coefficients).
# Create effect sizes.
smokers <- haven::read_spss("data/smokers.sav")
model_1 <- lm(attitude ~ exposure*factor(status3, levels = c(0,1,2)), data = smokers)
# Table with results in SPSS style.
results <- coef(summary(model_1))
# Adjust parameter names
attributes(results)$dimnames[[1]][1] <- "(Constant)"
attributes(results)$dimnames[[1]][2] <- "Exposure"
attributes(results)$dimnames[[1]][3] <- "Former smoker"
attributes(results)$dimnames[[1]][4] <- "Smoker"
attributes(results)$dimnames[[1]][5] <- "Exposure*Former smoker"
attributes(results)$dimnames[[1]][6] <- "Exposure*Smoker"
# Confidence intervals
ci <- confint.lm(model_1)
results <- cbind(results, ci)
# Set column names.
attributes(results)$dimnames[[2]] <- c("B", "Std. Error", "t", "Sig.", "Lower Bound", "Upper Bound")
# Table.
options(knitr.kable.NA = '')
knitr::kable(results, digits = 3, booktabs = TRUE,
             caption = "Predicting attitude towards smoking for three smoking status groups: regression analysis results.") %>%
  kable_styling(font_size = 12, full_width = F,
                latex_options = c("scale_down", "HOLD_position"))
# Cleanup.
rm(smokers, model_1, ci, results)
```

Remember that the effects of predictors that are included in interactions are conditional effects: effects for the reference group or reference value on the other variable involved in the interaction. Non-smokers are the reference group for participant's smoking status. The *p* value for the *exposure* predictor tests the hypothesis that the exposure effect for non-smokers is zero in the population.

For the two dummy variables *Former smoker* and *Smoker*, the null hypothesis is tested that they have the same average attitude in the population as the non-smokers (reference group) if they are not exposed to the anti-smoking campaign. Participants who are not exposed to the campaign (zero exposure) are the reference group here.

Interaction predictors show effect differences. In Table \@ref(tab:cat-moderator-results), the interaction predictors test the null hypotheses that the effect of exposure on attitude is equal for former smokers and non-smokers (*Exposure\*Former smoker*) or for smokers and non-smokers (*Exposure\*Smoker*) in the population.

If we would like to know whether the exposure effect for former smokers is significantly different from zero, we have to rerun the regression model using former smokers as reference group. This new model would also tell us whether the exposure effect for former smokers is significantly different from the exposure effect for people who are still smoking.

### Common support {#commonsupportdichotomous}

In a regression model with moderation, we have to interpret the effect of a predictor involved in an interaction at a particular value of the moderator (Section \@ref(conditional-effects)). The estimated effect at a particular value of the moderator can only be trusted if there are quite some observations at or near this value of the moderator. In addition, these observations should cover the full range of values on the predictor. After all, the effect that we estimate must tell us whether high values on the predictor go together with higher (or lower) values on the dependent variable than low values on the predictor.

For example, we need quite some observations for smokers to estimate the conditional effect of exposure on attitude for smokers. If there are hardly any smokers in our sample, we cannot estimate the effect of exposure on attitude for them in a reliable way. Even if we have quite some observations for smokers but all smokers have low exposure, we cannot say much about the effect of exposure on attitude for them. If we cannot say much about the effect within this group, we cannot say much about the difference between this effect and effects for other groups. In short, the moderation model is problematic in this situation.

```{r common-support, fig.pos='H', fig.align='center', fig.cap="How well do the observations cover the predictor within each category of smoking status?", echo=FALSE, screenshot.opts = list(delay = 5), dev="png", out.width="775px"}
# Goal: Sensitize students to the problem of lacking support for conditional
# effects by inspecting the coverage of the predictor for each moderator group.
# Generate a sample for smokers, for former smokers, and for non-smokers each of
# size 30. Give one or two randomly selected groups exposure values in the
# entire range [0, 10], but the remaining group(s) a restricted exposure range
# of 4 to 6 or 1 to 3 score points.
# Randomly assign a neutral, slightly negative, or moderately negative effect of
# exposure on attitude to the group. Display the three groups in a scatterplot
# (attitude by exposure) with different dot colours and their regression lines
# (coloured and labeled).
# Directly below the scatterplot, add a histogram of exposure, showing coverage.
# Allow the user to select groups 'All' (initial value), 'Smokers', 'Former
# smokers', or 'Non-smokers'. On selection, display the appropriate regression
# line and observations in the scatterplot and show their exposure scores in the
# histogram. Note that the scale of the histogram and the x axisof the
# scatterplot must be fixed to [0, 10].
# Add a Generate New button to generate a new dataset.
knitr::include_app("http://82.196.4.233:3838/apps/common-support/", height="305px")
```

<A name="question8.3.5"></A>

```{block2, type='rmdquestion'}
5. What does the histogram represent at the right of Figure \@ref(fig:common-support)? [<img src="icons/2answer.png" width=115px align="right">](#answer8.3.5)
```

<A name="question8.3.6"></A>

```{block2, type='rmdquestion'}
6. Are the exposure values nicely spread for each smoke status group? Inspect each smoke status group separately with the "Choose group" option. [<img src="icons/2answer.png" width=115px align="right">](#answer8.3.6)
```

<A name="question8.3.7"></A>

```{block2, type='rmdquestion'}
7. What is the problem if exposure scores are not nicely spread over the same range for all smoke status groups? [<img src="icons/2answer.png" width=115px align="right">](#answer8.3.7)
```

The variation of predictor scores for a particular value of the moderator is called *common support* [@RefWorks:3838]. If common support for predictors involved in moderation is poor, we should hesitate to draw conclusions from the estimated effects. Guidelines for good common support are hard to give. Common support is usually acceptable if there are observations over the entire range of the predictor.

It is recommended to check the number of observations per value of the moderator. For a categorical moderator, such as smoking status, a scatter plot of the dependent variable (vertical axis) by predictor (horizontal axis) with dots coloured according to the moderator category may do the job. The left panel of Figure \@ref(fig:common-support) shows an example. Check that there are observations for more or less all values of the predictor in each color. If the scatter plot is hard to read, create a histogram of predictor values grouped by moderator categories, as in the right panel of Figure \@ref(fig:common-support).

### Visualizing moderation and covariates

A plot with different regression lines for different categories of the moderator is a very useful way of presenting your results. We can, however, only plot a regression line if we have a single independent variable. After all, we only have one horizontal (X) axis in a plot to display a predictor.

In a moderation model, we have at least two independent variables: the predictor and the moderator. For example, exposure is our predictor because our interest focuses on the effect of campaign exposure on attitude. Participant's smoking status is the moderator because we expect different exposure effects for participants with different smoking statuses. We may even have additional independent variables for which we want to control (more on this in Chapter \@ref(confounder)), for example, a participant's contacts with smokers. Let us call these additional independent variables *covariates*.

```{block2, type='rmdimportant'}
A _predictor_ is the independent variable that is currently central to our analysis.

A _moderator_ is an independent variable for which we expect different effects of the predictor.

A _covariate_ is an independent variable that is currently not central to our analysis.
```

Note that the distinction between predictor, moderator, and covariate is temporary. As soon as we focus on another variable, that variable becomes the predictor and the other predictors become moderators or covariates. The distinction between predictor, moderator, and covariate is just terminology to show on which variable we focus.

```{r moderator-covar-statistical, fig.pos='H', fig.align='center', fig.cap="Statistical diagram of moderation with contact as covariate.", echo=FALSE, fig.asp=0.4}
library(ggplot2)
# Create coordinates for the variable names.
variables <- data.frame(x = c(rep.int(0.3, times = 4), 0.7), 
                        y = c(.4, .3, .2, .1, .25),
                        label = c("Exposure", "Smoker", "Exposure*Smoker", "Contact", "Attitude"))
# Add coordinates for arrow endpoint.
x_diff <- 0.04
variables$xend <- variables$x[5] - x_diff #fixed translation to the left
variables$yend <- variables$y[5] + x_diff * (variables$y - variables$y[5]) / (variables$x[5] - variables$x)
rm(x_diff)
ggplot(variables, aes(x, y)) + 
  geom_segment(aes(x = x[1], y = y[1], xend = xend[1], yend = yend[1]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  geom_segment(aes(x = x[2], y = y[2], xend = xend[2], yend = yend[2]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  geom_segment(aes(x = x[3], y = y[3], xend = xend[3], yend = yend[3]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  geom_segment(aes(x = x[4], y = y[4], xend = xend[4], yend = yend[4]), arrow = arrow(length = unit(0.04, "npc"), type = "closed")) + 
  geom_label(aes(label=label)) + 
  coord_cartesian(xlim = c(0.2, 0.8), ylim = c(0.1, 0.5)) +
  theme_void()
```

Figure \@ref(fig:moderator-covar-statistical) shows the statistical diagram of a moderation model with contact as covariate and Table \@ref(tab:dich-moderator-cov-output) summarizes the estimated effects. How can we get rid of the moderator and covariate(s), so exposure is left as the only independent variable and we can plot regression lines for exposure effects?

```{r dich-moderator-cov-output, echo=FALSE, message=FALSE, warning=FALSE}
# Table of regression coefficients for exposure moderated by status. Similar to SPSS output (with correct standardized coefficients).
# Create effect sizes.
smokers <- haven::read_spss("data/smokers.sav")
model_1 <- lm(attitude ~ exposure*status2 + contact, data = smokers)
# Table with results in SPSS style.
results <- coef(summary(model_1))
# Confidence intervals
ci <- confint.lm(model_1)
results <- cbind(results, ci)
# Correctly standardized coefficients.
attach(smokers)
z_exposure <- (exposure - mean(exposure))/sd(exposure)
z_status2 <- (status2 - mean(status2))/sd(status2)
z_expostatus2 <- z_exposure * z_status2
z_attitude <- (attitude - mean(attitude))/sd(attitude)
z_contact <- (contact - mean(contact))/sd(contact)
model_2 <- lm(z_attitude ~ z_exposure + z_status2 + z_contact + z_expostatus2)
results_2 <- coef(summary(model_2))
results <- cbind(results[, 1:2], results_2[, 1], results[, 3:6])
results[1, 3] <- NA
# Move effect of contact to the last row of the results table.
results <- rbind(results[1:3,], results[5,], results[4,])
# Adjust parameter names
attributes(results)$dimnames[[1]][1] <- "(Constant)"
attributes(results)$dimnames[[1]][2] <- "Exposure"
attributes(results)$dimnames[[1]][3] <- "Smoker (0 = no, 1 = yes))"
attributes(results)$dimnames[[1]][4] <- "Exposure*Smoker"
attributes(results)$dimnames[[1]][5] <- "Contact"
# Table.
options(knitr.kable.NA = '')
knitr::kable(results, digits = 3, booktabs = TRUE,
             caption = "Predicting attitude towards smoking: regression analysis results with contact as covariate.",
             col.names = c("B", "Std. Error", "Beta", "t", "Sig.", "Lower Bound", "Upper Bound")) %>%
  kable_styling(font_size = 12, full_width = F,
                latex_options = c("scale_down", "HOLD_position")) %>%
  column_spec(1, color = "white",
              background = c("#FDAE61", "#2B83BA", "#ABDDA4", "#D7191C", "brown")) %>%
  column_spec(2, color = c("#FDAE61", "#2B83BA", "#ABDDA4", "#D7191C", "brown"))
# Helper function for displaying results within the text.
source("report_n.R")
# Store average covariate value for a later plot.
contact_avg = mean(contact)
#Cleanup (partial, smokers and results are saved for inline use).
rm(model_1, ci, results_2, model_2, z_attitude, z_status2, z_expostatus2, z_exposure, z_contact)
```

As a first step, use the estimated values of the regression coefficients in the SPSS output (Table \@ref(tab:dich-moderator-cov-output)) to create a regression equation [Eq. \@ref(eq:intvarcov1)]. Just start at the top of the table and write down the regression coefficients (*B*) and the independent variable names.

```{=tex}
\begin{equation}
\small
\begin{split}
  attitude = &\ \color{#FDAE61}{`r report_n(results[1,1], 3)`} + \color{#2B83BA}{`r report_n(results[2,1], 3)`*exposure} + \color{#ABDDA4}{`r report_n(results[3,1], 3)`*smoker} + \color{#D7191C}{`r report_n(results[4,1], 3)`*exposure*smoker}\\
  &+ \color{brown}{`r report_n(results[5,1], 3)`*contact}
\end{split}
(\#eq:intvarcov1) 
\normalsize
\end{equation}
```

As a second step, choose an interesting value for every independent variable in the equation except the predictor. If we want to have the regression line for smokers, choose 1 for the *smoker* variable. For a numerical covariate such as *contact*, it is recommended to choose the mean. Average contact with smokers happens to be <span style="color:Brown;">`r report_n(mean(smokers$contact),3)`</span> in our example. Now replace the independent variables in the equation by the selected values and simplify the equation [Eq. \@ref(eq:intvarcov2)].

```{=tex}
\begin{equation}
\small
\begin{split}
  attitude = &\ \color{#FDAE61}{`r report_n(results[1,1], 3)`} + \color{#2B83BA}{`r report_n(results[2,1], 3)`*exposure} + \color{#ABDDA4}{`r report_n(results[3,1], 3)`*smoker} + \color{#D7191C}{`r report_n(results[4,1], 3)`*exposure*smoker}\\
  &+ \color{brown}{`r report_n(results[5,1], 3)`*contact}\\
  attitude = &\ \color{#FDAE61}{`r report_n(results[1,1], 3)`} + \color{#2B83BA}{`r report_n(results[2,1], 3)`*exposure} + \color{#ABDDA4}{`r report_n(results[3,1], 3)`*1} + \color{#D7191C}{`r report_n(results[4,1], 3)`*exposure*1} + \color{brown}{`r report_n(results[5,1], 3)`*`r report_n(round(mean(smokers$contact),3),3)`}\\
  attitude = &\ \color{#FDAE61}{`r report_n(results[1,1], 3)`} + \color{#2B83BA}{`r report_n(results[2,1], 3)`*exposure} + \color{#ABDDA4}{`r report_n(results[3,1], 3)`} + \color{#D7191C}{`r report_n(results[4,1], 3)`*exposure} + \color{brown}{`r report_n(results[5,1] * round(mean(smokers$contact),3),3)`}\\
  attitude = &\ (\color{#FDAE61}{`r report_n(results[1,1], 3)`} + \color{#ABDDA4}{`r report_n(results[3,1], 3)`}+ \color{brown}{`r report_n(results[5,1] * round(mean(smokers$contact),3),3)`}) + (\color{#2B83BA}{`r report_n(results[2,1], 3)`*exposure} + \color{#D7191C}{`r report_n(results[4,1], 3)`*exposure})\\
  attitude = &\ `r report_n(round(results[1,1],3) + round(results[3,1],3) + (round(results[5,1],3) * round(mean(smokers$contact),3)),3)` + `r report_n(round(results[2,1],3) + round(results[4,1],3), 3)`*exposure
\end{split}
(\#eq:intvarcov2) 
\normalsize
\end{equation}
```

The terms with *smoker* and *contact* disappear from the equation, so *exposure* is the only independent variable that remains in the equation. Now, we can draw the simple regression line predicting attitude from exposure for smokers using this equation. Note that this is the regression line for people with average contact with smokers.

Repeat these steps but plug in the score 0 for the *smoker* predictor to obtain the simple regression line for non-smokers. Figure \@ref(fig:dich-moderator-cov-plot) shows the two regression lines and their equations. The effect of exposure on attitude is more strongly negative for smokers than for non-smokers.

```{r dich-moderator-cov-plot, echo=FALSE, out.width="600px", fig.pos='H', fig.align='center', fig.cap="The effects of exposure on attitude for non-smokers and smokers. Both smokers and non-smokers are assumed to have average contact with smokers."}
#Create and show the plot.
ggplot(smokers, aes(x = exposure, y = attitude, 
                    colour = factor(status2,
                                    labels = c("Non-smoker", "Smoker")))) +
  geom_point() +
  geom_vline(xintercept = 0) +
  geom_abline(aes(
      intercept = results[1,1] + results[5,1]*mean(contact), 
      slope = results[2,1],
                colour = "Non-smoker"
      ),
      show.legend = F
    ) +
  geom_text(aes(x = 0.1, y = -4.9,
                label = paste0(
                  "Non-smokers: attitude = ",
                  report_n(round(results[1,1], 3) + round(results[5,1], 3)*round(mean(contact), 3),3),
                  " + ",
                  report_n(results[2,1],3),
                  " * exposure"
                ),
                hjust = 0,
                vjust = 0,
                colour = "Non-smoker"
                ),
      show.legend = F
            ) +
  geom_abline(aes(
      intercept = results[1,1] + results[3,1] + results[5,1]*mean(contact), 
      slope = results[2,1] + results[4,1],
      colour = "Smoker"),
    show.legend = F
    ) +
  geom_text(aes(x = 9.9, y = 4.9,
                label = paste0(
                  "Smokers: attitude = ",
                  report_n(round(results[1,1], 3) + round(results[3,1], 3) + round(results[5,1], 3)*round(mean(contact), 3),3),
                  " + ",
                  report_n(round(results[2,1], 3) + round(results[4,1], 3),3),
                  " * exposure"
                ),
                hjust = 1,
                vjust = 1,
                colour = "Smoker"
                ),
      show.legend = F
            ) +
  scale_colour_manual(
    name = "Smoking status",
    values = c("Non-smoker" = unname(brewercolors["Blue"]),
               "Smoker" = unname(brewercolors["Red"]))
    ) +
  scale_x_continuous(
    name = "Exposure",
    limits = c(0,10)
  ) +
  scale_y_continuous(
    name = "Attitude",
    limits = c(-5, 5),
    breaks = c(
      0,
      round(round(results[1,1], 3) + round(results[5,1], 3)*round(mean(smokers$contact), 3),3),
      round(round(results[1,1], 3) + round(results[3,1], 3) + round(results[5,1], 3)*round(mean(smokers$contact), 3),3)
    )
  ) +
  theme_general() +
  theme(legend.position = "bottom")

```

Instead of writing out the full equations, you can also obtain the constant and the simple slope by copying the table of regression coefficients and adding columns (Table \@ref(tab:cat-moderator-results2)):

1.  Add a column indicating which value you plug in for the moderator variable(s) and any covariates. In this example, *smoker* is the moderator variable, which appears both by itself and as part of the interaction variable. The variable *contact* is a covariate here; we use its mean as plug-in value.

2.  Add a column to calculate the constant. Use all variables that do *not* include the predictor. For each of these variables, multiply the regression coefficient by the plug-in value (if any). Sum the results to obtain the constant.

3.  Add a column to calculate the simple slope. Only use the variables that include the predictor, which is *exposure* in this example. For each of these variables, multiply the regression coefficient by the plug-in value (if any). Sum the results to obtain the simple slope.

```{r cat-moderator-results2, echo=FALSE, message=FALSE, warning=FALSE}
# Table of regression coefficients for exposure moderated by status3, similar to SPSS output.
# Additional columns for calculating the intercept and simpe slope of exposure for non-smokers.
# Create effect sizes.
smokers <- haven::read_spss("data/smokers.sav")
model_1 <- lm(attitude ~ exposure*status2 + contact, data = smokers)
# Table with unstandardizd regression coefficients.
results <- data.frame(B = c(report_n(model_1$coefficients,3), NA),
                      #non-smokers
                      Plugin0 = c("", "", "Smoker = 0", paste0("Mean contact = ", report_n(mean(smokers$contact), 3)), "Smoker = 0", ""),
                       Slope0 = c("", as.character(report_n(model_1$coefficients[2],3)), "", "", paste0(report_n(model_1$coefficients[5],3), " * 0"), paste0(report_n(model_1$coefficients[2],3), footnote_marker_alphabet(1))),
                      Constant0 = c(as.character(report_n(model_1$coefficients[1],3)), "", paste0(as.character(report_n(model_1$coefficients[3],3)), " * 0"), paste0(as.character(report_n(model_1$coefficients[4],3)), " * ", report_n(mean(smokers$contact), 3)), "", paste0(report_n(round(model_1$coefficients[1],3) + (round(model_1$coefficients[4],3) * round(mean(smokers$contact), 3)), 3), footnote_marker_alphabet(1))),
                      #smokers
                      Plugin1 = c("", "", "Smoker = 1", paste0("Mean contact = ", report_n(mean(smokers$contact), 3)), "Smoker = 1", ""),
                       Slope1 = c("", as.character(report_n(model_1$coefficients[2],3)), "", "", paste0(report_n(model_1$coefficients[5],3), " * 1"), paste0(report_n(round(model_1$coefficients[2],3) + round(model_1$coefficients[5],3), 3), footnote_marker_alphabet(2))),
                      Constant1 = c(as.character(report_n(model_1$coefficients[1],3)), "", paste0(as.character(report_n(model_1$coefficients[3],3)), " * 1"), paste0(as.character(report_n(model_1$coefficients[4],3)), " * ", report_n(mean(smokers$contact), 3)), "", paste0(report_n(round(model_1$coefficients[1],3) + round(model_1$coefficients[3],3) + (round(model_1$coefficients[4],3) * round(mean(smokers$contact), 3)), 3), footnote_marker_alphabet(2))),
                      row.names = c("(Constant)", "Exposure", "Smoker", "Contact",
                                     "Exposure*Smoker", "Result (sum)"),
                      check.names = FALSE)
# Table.
options(knitr.kable.NA = '')
results %>% select(B, Plugin0, Constant0, Slope0, Plugin1, Constant1, Slope1) %>%
  knitr::kable(digits = 3, booktabs = TRUE,
             caption = "Calculation of the constant and simple slope for non-smokers and smokers at average contact.",
             col.names = c("B", rep(c("1. Plugin", "2. Constant", "3. Slope"), 2)),
             align = "rlrrlrr",
             escape = F) %>%
  kable_styling(font_size = 12, full_width = F,
                latex_options = c("scale_down", "HOLD_position")) %>%
  add_header_above(c(" " = 2, "Non-smokers" = 3, "Smokers" = 3)) %>%
  footnote(alphabet = c(paste0("Equation for non-smokers: y = ", report_n(round(model_1$coefficients[1],3) + (round(model_1$coefficients[4],3) * round(mean(smokers$contact), 3)), 3) , " + ", report_n(model_1$coefficients[2],3), " * Exposure."), paste0("Equation for smokers: y = ", report_n(round(model_1$coefficients[1],3) + round(model_1$coefficients[3],3) + (round(model_1$coefficients[4],3) * round(mean(smokers$contact), 3)), 3), " + ", report_n(round(model_1$coefficients[2],3) + round(model_1$coefficients[5],3), 3), " * Exposure.")))
# Cleanup.
rm(smokers, model_1, results)
```

The calculated slope and constant constitute the simple regression equation of the effect of the predictor for the selected values of the moderator and covariates (see the footnotes to Table \@ref(tab:cat-moderator-results2)). You can interpret the slopes, which are unstandardized regression coefficients ($b$), or use the equations to draw regression lines.

### Answers {.unnumbered}

<A name="answer8.3.1"></A>

```{block2, type='rmdanswer'}
Answer to Question 1. 

* Remember: The slope of the regression line represents the effect (the
regression coefficient) of the predictor. Parallel lines mean equal slopes.
* The answer depends on the sample drawn:
  + If the regression lines for the two smoking status groups are more or less parallel, the (predictive) effect of exposure on attitude is more or less the same for both groups. In this case, there is no moderation.
  + If the regression lines for the two smoking status groups are clearly not parallel, there is moderation. [<img src="icons/2question.png" width=161px align="right">](#question8.3.1)
```

<A name="answer8.3.2"></A>

```{block2, type='rmdanswer'}
Answer to Question 2. 

* When you move the slider, the regression line for smokers rotates. In mathematical terms, the slope of the regression line changes.
* If you move the slider to the right, the regression coefficient of the interaction variable (_exposure\*smoker_) becomes (more) positive.
* What does that mean? According to the general interpretation of a regression coefficient, an additional unit of _exposure\*smoker_ has a (more) positive contribution to the predicted attitude.
* We are only dealing with smokers here, so dummy variable _smoker_ is 1. The interaction variable _exposure\*smoker_ reduces to _exposure_\*1 = _exposure_. The regression coefficient of the interaction variable represents the **additional effect of exposure for smokers**.
* As a result, the slope of the regression line for smokers becomes more positive (rotates counterclockwise) if the regression coefficient for the interaction variable becomes more positive.
* In the same way, moving the slider to the left makes the slope of the regression line for smokers more negative (rotating it clockwise). [<img src="icons/2question.png" width=161px align="right">](#question8.3.2)
```

<A name="answer8.3.3"></A>

```{block2, type='rmdanswer'}
Answer to Question 3. 

* _Smoker_ is a dummy variable with smokers coded '1' and non-smokers coded '0' as shown in the legend of the figure. If we replace the _smoker_ variable in the equation by zero, we get the predicted attitudes for non-smokers.
* For example, assume that the equation is as follows:

> _attitude_ = 0.78 + -0.54\*_exposure_ + 2.85\*_smoker_ + 0.23\*_exposure_\*_smoker_

* If we replace _smoker_ by zero in the equation, we obtain:
  
> _attitude_ = 0.78 + -0.54\*_exposure_ + 2.85\*0 + 0.23\*_exposure_\*0

* This can be simplified to:

> _attitude_ = 0.78 + -0.54\*_exposure_ + 0 + 0
> _attitude_ = 0.78 + -0.54\*_exposure_

* The slope or effect of exposure on attitude for non-smokers is simply the regression coefficient for the _smoker_ variable!
  
Note:

* The effect of a variable like _exposure_ that is also included in an interaction variable is **not** a general effect of this variable like a main effect in analysis of variance. It does **not** represent the average effect of exposure for all participants.
* Instead, the effect of _exposure_ is the effect for one particular group of participants, the non-smokers in this example. This is called a *conditional effect*. 
* More generally, the effect of _exposure_ is the effect for the participants who score zero on the other variable included in the interaction (_smoker_ in this example). These participants are called the *reference group*. [<img src="icons/2question.png" width=161px align="right">](#question8.3.3)
```

<A name="answer8.3.4"></A>

```{block2, type='rmdanswer'}
Answer to Question 4. 

* If the regression lines for all three smoking status groups are parallel, the (predictive) effect of exposure on attitude is the same for all groups. There is no moderation.

![](figures/S8_3Q4.png)

* In all other situations, there is moderation. There is an interaction
effect, for example, if two lines are parallel but the third line is not
parallel to the other two lines. [<img src="icons/2question.png" width=161px align="right">](#question8.3.4)
```

<A name="answer8.3.5"></A>

```{block2, type='rmdanswer'}
Answer to Question 5. 

* The histogram shows the counts of cases with particular exposure values.
* The counts are subdivided (coloured) by their score on the moderator variable
(smoking status). [<img src="icons/2question.png" width=161px align="right">](#question8.3.5)
```

<A name="answer8.3.6"></A>

```{block2, type='rmdanswer'}
Answer to Question 6. 

* The answer depends on the sample that was drawn.
* For one moderator (smoking status) group, the predictor (exposure) may have scores for the entire range, that is, from 0 to 10. But scores may also be available only for part of this range, for example, from 1 to 6 or from 4 to 8. [<img src="icons/2question.png" width=161px align="right">](#question8.3.6)
```

<A name="answer8.3.7"></A>

```{block2, type='rmdanswer'}
Answer to Question 7. 

* We may erroneously believe that the moderation effect applies to the entire
range of scores. For example, we may conclude that there is a negative effect
for smokers versus a positive effect for non-smokers. However, the effect for
smokers may be based on a range of exposure values that is different from and
perhaps hardly overlaps with the range of exposure values for non-smokers.
In this case, the effect difference may be due to the level of predictor scores
instead of the moderator.
* In extreme cases, we may have only very few observations for a moderator
group, for example, only a handful of smokers. In this situation, the
regression effect for this group is not to be trusted. [<img src="icons/2question.png" width=161px align="right">](#question8.3.7)
```

## A Dichotomous or Categorical Moderator in SPSS {#catmodSPSS}

### Instructions

```{r SPSSregpred, echo=FALSE, out.width="640px", fig.pos='H', fig.align='center', fig.cap="(ref:regpredSPSS)", dev="png", screenshot.opts = list(delay = 5)}
knitr::include_url("https://www.youtube.com/embed/5ORuOV4obUU", height = "360px")
# Goal: Creating categorical by numerical interaction predictors (2 methods).
# Example: smokers.sav, predict the attitude towards smoking from exposure moderated by smoking status (variable status2 and status3), use contact with smokers as a covariate.
# SPSS menu: Transform > Compute Variable for status2 (already 0/1 variable) ; Transform > Create Dummy Variables for status3 (3 categories)
# Interpret output: none.

# Create interaction predictors (also for cont*cont interaction variables in addition to interactions with dummies) and dummies for main effects in one go: 
#   - ensure that categorical variables are marked as Nominal or Ordinal in Variable View
#   - command Transform > Create Dummy Variables ; select (numerical) predictor and (categorical) moderator under 'Create Dummy Variables for:'
#   - under Create main-effect dummies (option checked by default) specify a short name for both variables, separated with a comma ; the name of the numerical variable is irrelevant but must be specified
#   - ensure that the option _Do not create dummies for scale variables values_ is selected under Measurement Level Usage
#   - select the _Create dummies for all two-way interactions_ option under Two-Way Interactions and give a short name, e.g., interact
#   - Note: this procedure can also be used to create a numerical by numerical interaction variable
```

```{r SPSSregcatmod, echo=FALSE, out.width="640px", fig.pos='H', fig.align='center', fig.cap="(ref:regcatmodSPSS)", dev="png", screenshot.opts = list(delay = 5)}
knitr::include_url("https://www.youtube.com/embed/9Yf2rHgoBt8", height = "360px")
# Goal: Estimating categorical by numerical moderation with regression in SPSS.
# Example: smokers.sav, predict the attitude towards smoking from exposure moderated by smoking status (variable _status2_), use contact with smokers as a covariate.
# SPSS menu: linear regression, include descriptives for means and standard deviations of covariates for making reference lines.
# Interpret output: use unstandardized regression coefficients because they show the average difference in slope (effect size) ; do not use the standardized regression coefficients that SPSS reports: they are calculated in the wrong way if the regression model includes an interaction variable. As a result, they are meaningless ; better interpret the regression lines in a scatterplot, see another video
# Check assumptions: See other video.
```

```{r SPSSregmodlines, echo=FALSE, out.width="640px", fig.pos='H', fig.align='center', fig.cap="(ref:regmodlinesSPSS)", dev="png", screenshot.opts = list(delay = 5)}
knitr::include_url("https://www.youtube.com/embed/5KgpEjFzHiQ", height = "360px")
# Goal: Representing moderation by regression lines in a scatterplot in SPSS.
# Example: smokers.sav, predict the attitude towards smoking from exposure moderated by smoking status (variable _status2_), use contact with smokers as a covariate.
# Technique: linear regression
# SPSS menu: 
# Interpret output: 

# * Visualize categorical moderator with reference lines in scatterplot:
#   - {skip} No covariates: Graphs > Regression Variable Plots: dependent variable on Y, predictor on X, categorical moderator in Color by, Options>Scatterplot Fit Lines > Linear and Grouping > Fit line for each categorical colour group
#   - With covariates: 
#     - create scatterplot with Graphs > Legacy Dialogs > Scatter/Dot > Simple Scatter
#     - select dependent variable under Y Axis: and (numerical) predictor under X Axis: 
#     - select (original) categorical moderator under Set Markers by: (colours the observations according to moderator value, useful for inspecting common support)
#     - Paste & Run 
#     - display the mean values of covariates with Analyze > Descriptive Statistics > Fequencies ; select Statistics > Mean
#     - in SPSS Output, double-click the scatterplot to open it in the Chart Editor 
#     - add reference line (Options > Reference Line from Equation) ; in the Properties window under the Reference Line tab, add the regression equation for the first group in the moderator variable: use x for the predictor displayed on the X axis and use the category value (0/1) for dummies and the average values for numerical covariates
#     - repeat for other categories of the moderator variable
#     - change type (or colour) of the line in the Properties window under the Lines tab
#     - if you like, add label to lines describing the moderator group: Options > Text Box
#     - close the Chart Editor
```

```{r SPSSregSupport1, echo=FALSE, out.width="640px", fig.pos='H', fig.align='center', fig.cap="(ref:regSupport1SPSS)", dev="png", screenshot.opts = list(delay = 5)}
knitr::include_url("https://www.youtube.com/embed/kxX4rqsyWQQ", height = "360px")
# Goal: Check common support for a predictor at different moderator values in SPSS.
# Example: smokers.sav, predict the attitude towards smoking from exposure moderated by smoking status (variable _status2_), use contact with smokers as a covariate.
# Technique: 
# SPSS menu: 
# Interpret output: 
# Check assumptions: 
#
# * Check common support: 
#   - make histograms of the predictor panelled by the categorical grouping variable ; check that there are observations for more or less all values of the predictor (on the X axis) 
```

### Exercises

<A name="question8.4.1"></A>

```{block2, type='rmdquestion'}
1. Use the data in [allsmokers.sav](http://82.196.4.233:3838/data/allsmokers.sav) to predict the attitude towards smoking from exposure moderated by smoking status (variable _status2_). Use contact with smokers as a covariate. Check the assumptions for regression analysis and interpret the results. [<img src="icons/2answer.png" width=115px align="right">](#answer8.4.1)
```

<A name="question8.4.2"></A>

```{block2, type='rmdquestion'}
2. Visualize the moderated effects of exposure on attitude (Exercise 1). Create a scatter plot with two regression lines and use the average value for the covariate (contact with smokers). Colour the regression lines and the dots (respondents) according to their smoking status category. Interpret how smoking status moderates the effect of exposure on attitude. [<img src="icons/2answer.png" width=115px align="right">](#answer8.4.2)
```

<A name="question8.4.3"></A>

```{block2, type='rmdquestion'}
3. Check common support of the predictor (exposure) in all groups of the moderator (smoking status). Could you also check common support with the scatter plot you made for Exercise 2? [<img src="icons/2answer.png" width=115px align="right">](#answer8.4.3)
```

<A name="question8.4.4"></A>

```{block2, type='rmdquestion'}
4. Repeat the analyses of Exercises 1 through 3 but use smoking status with three categories (_status3_). [<img src="icons/2answer.png" width=115px align="right">](#answer8.4.4)
```

### Answers {.unnumbered}

<A name="answer8.4.1"></A>

```{block2, type='rmdanswer'}
Answer to Exercise 1. 

SPSS syntax:  
  
\* Check data.  
FREQUENCIES VARIABLES=exposure status2 contact attitude  
  /ORDER=ANALYSIS.  
\* Compute interaction variable.  
COMPUTE expo_status=exposure \* status2.  
VARIABLE LABELS  expo_status 'Interaction exposure \* smoker'.  
EXECUTE.  
\* Multiple regression.  
REGRESSION  
  /MISSING LISTWISE  
  /STATISTICS COEFF OUTS CI(95) R ANOVA  
  /CRITERIA=PIN(.05) POUT(.10)  
  /NOORIGIN   
  /DEPENDENT attitude  
  /METHOD=ENTER exposure status2 expo_status contact  
  /SCATTERPLOT=(\*ZRESID ,\*ZPRED)  
  /RESIDUALS HISTOGRAM(ZRESID).  
  
Check data:  
  
All values seem to be valid.  
  
Check assumptions:  

![](figures/S8_4Q1.png)

* The residuals are skewed (long tail to the left).
* The residuals seem to average to zero at most levels of the predicted
scores, so a linear model seems to fit.
* However, the lower attitude values are predicted worse (more variation) than
the highest levels. The assumption of homoscedasticity seems to be violated.
  
Interpret the results:  

* Summarize the tests on the regression coefficients in a table: 

<div style="font-size: 0.8em">
| | _b_ |	SE | _b\*_ | _t_ | _p_ | 95\%CI |
|:------------|----:|----:|----:|----:|----:|:--------:|
| Constant | 0.13 | 0.35 |  | 0.36 | .716 | [-0.56, 0.81] |
| Exposure to anti-smoking campaign | -0.28 | 0.04 | -0.41 | -7.67 | < .001 | [-0.35, -0.21] |
| Smokings status | 1.23 | 0.41 | 0.27 | 3.01 | .003 | [0.43, 2.03] |
| Exposure * Smoker | -0.20 | 0.07 | -0.26 | -2.79 | .006 | [-0.34, -0.06] |
| Contact with smokers | 0.23 | 0.05 | 0.23 | 4.73 | < .001 | [.13, .32] |
</div>
  
* We can predict smoking attitude for about 38 per cent with the regression
model, *R*^2^ = .38, *F* (4, 307) = 47.72, *p* < .001.
* The predictive effect of exposure to the anti-smoking campaign on smoking
attitude for non-smokers is negative and significantly different from zero, *b* = -0.28, *t* = -7.67, *p* < .001, 95% CI [-0.35, -0.21]. More exposure tends to yield a more negative attitude among non-smokers.
* For smokers, the predictive effect of campaign exposure on smoking attitude
is more strongly negative. The moderation of the exposure effect by smoking
status is negative, ranging from -0.06 to -0.34 in the population (with 95%
confidence), and it is statistically significant, *b* = -0.20, *t* = -2.79, *p* = .006, 95% CI [-0.34, -0.06].
* More contact with smokers is associated with a more positive attitude
towards smoking rather than a negative attitude in the population. The predictive effect is weak (_b\*_ = 0.23) and significantly different from zero, so we may assume a positive effect in the population, *b* = 0.23, *t* = 4.73, *p* < .001, 95% CI [0.13, 0.32].
* Smokers have a more positive attitude than non-smokers if they are not
exposed to the campaign; an attitude that is on average circa 1.2 (0.4 to 2.0) points more positive. This difference is statistically significant, so we are sufficiently confident that the effect is positive in the population, *b* = 1.23, *t* = 3.01, *p* = .003, 95% CI [0.43, 2.03].
  
Remember: 

* In the presence of an interaction effect, the effect of a predictor is the
effect for the reference group or value on the moderator. It is NOT an overall
or average effect (main effect) as in analysis of variance.
* Standardized regression coefficients reported by SPSS are not correct for
interaction effects or effects of predictors that are involved in interaction
effects. They can only be used for predictors that are not involved in
interaction effects. Contact with smokers is not involved in an interaction in this model, so we can interpret the standardized regression coefficient for this effect. [<img src="icons/2question.png" width=161px align="right">](#question8.4.1)
```

<A name="answer8.4.2"></A>

```{block2, type='rmdanswer'}
Answer to Exercise 2. 

* First of all, you must write out the regression equations for different
values of the moderator. Plug in the estimated values of the regression
coefficients and the means of covariates (here: average contact with smokers).

\* SPSS syntax to get the average value of contact with smokers:

FREQUENCIES VARIABLES=contact  
 /FORMAT=NOTABLE  
 /STATISTICS=MEAN  
 /ORDER=ANALYSIS.  

* Copy the equation from the SPSS output:  

> attitude = constant + -.278\*exposure + 1.225\*status + -.197\*exposure\*status + .225\*contact  

* Plug in the average value of the covariate contact (4.995): 

> attitude = .127 + -.278\*exposure + 1.225\*status + -.197\*exposure\*status + .225\*4.995  

* Calculate the product for the covariate:

> attitude = .127 + -.278\*exposure + 1.225\*status + -.197\*exposure\*status + 1.124  

* And add the result for the covariate to the intercept (constant):

> attitude = 1.251 + -.278\*exposure + 1.225\*status + -.197\*exposure\*status

* Use the resulting equation to create a simple regression equation for each
category of the moderator. Remember that a simple regression equation contains
only one independent variable.

* Replace status by 0 for non-smokers:  
  
> attitude = 1.251 + -.278\*exposure + 1.225\*0 + -.197\*exposure\*0  
  
> attitude = 1.251 + -.278\*exposure  
  
* Replace status by 1 for smokers:
  
> attitude = 1.251 + -.278\*exposure + 1.225\*1 + -.197\*exposure\*1  
  
> attitude = 1.251 + 1.225 + -.278\*exposure + -.197\*exposure  
  
> attitude = 2.476 + (-.278 + -.197)\*exposure  
  
> attitude = 2.476 + -.475\*exposure  
  
* Finally, create a scatterplot of attitude by exposure, colouring the dots by
smoking status. Use the calculated two equations in the SPSS Chart Editor to
create two lines. Use the icon "Add a reference line from Equation" for each
line. Enter the equation using x instead of exposure as the predictor. Colour
the lines with the colours of the dots in the scatterplot.
  
SPSS syntax:  
  
\* Scatterplot with dots coloured by smoking status.  
GRAPH  
  /SCATTERPLOT(BIVAR)=exposure WITH attitude BY status2  
  /MISSING=LISTWISE.  
  
![](figures/S8_4Q2.png)

Check data:  
  
See Exercise 1.  
  
Check assumptions:  
  
See Exercise 1.  
  
Interpret the results:  
  
* For smokers, the predictive effect of campaign exposure on smoking attitude
is more strongly negative (*b* = -0.47) than for non-smokers (*b* = -0.28). An additional unit of exposure decreases the predicted attitude by 0.47 for smokers, whereas the predicted decrease is only 0.28 for non-smokers. [<img src="icons/2question.png" width=161px align="right">](#question8.4.2)
```

<A name="answer8.4.3"></A>

```{block2, type='rmdanswer'}
Answer to Exercise 3. 

SPSS syntax:  
  
\* Histogram of predictor (exposure) for each smoking status.  
GRAPH  
  /HISTOGRAM=exposure  
  /PANEL ROWVAR=status2 ROWOP=CROSS.  
  
![](figures/S8_4Q3.png)

Interpret the results:  
  
* Even for smokers, the much smaller group, we have exposure scores over
the entire range. We have good coverage both for smokers and
non-smokers. Do not mind the gap in scores around 6: we have plenty of
observations around 5 and 7.
  
* We could have seen this result in the scatterplot because we had red and
blue dots across the entire width of the plot. [<img src="icons/2question.png" width=161px align="right">](#question8.4.3)
```

<A name="answer8.4.4"></A>

```{block2, type='rmdanswer'}
Answer to Exercise 4. 

SPSS syntax:  
  
\* Check data.  
FREQUENCIES VARIABLES=exposure status3 contact attitude  
  /ORDER=ANALYSIS.  
\* Create dummies and interaction variables.  

\* With Create Dummy Variables.  
\* ENSURE THAT MEASUREMENT LEVEL IS SET TO NOMINAL OR ORDINAL FOR A CATEGORICAL VARIABLE.  
\* SELECT THE PREDICTOR AND MODERATOR FOR CREATING DUMMY VARIABLES.  
\* ADD A ROOTNAME FOR BOTH PREDICTOR AND MODERATOR.  
\* CHECK THE BOX FOR TWO-WAY INTERACTIONS AND ADD A ROOTNAME.  

\* Define Variable Properties.  
\*status3.  
VARIABLE LEVEL  status3(ORDINAL).  
EXECUTE.  
SPSSINC CREATE DUMMIES VARIABLE=exposure status3   
ROOTNAME1=exposure, status ROOTNAME2=expo_status   
/OPTIONS ORDER=A USEVALUELABELS=YES USEML=YES OMITFIRST=NO.  

\* With Recode.  
RECODE status3 (1=1) (ELSE=0) INTO status_3.  
VARIABLE LABELS  status_3 'Former smoker'.  
EXECUTE.  
RECODE status3 (2=1) (ELSE=0) INTO status_4.  
VARIABLE LABELS  status_4 'Smoker'.  
EXECUTE.  
\* Interaction variables (same name as those given by Create Dummy Variables).  
COMPUTE expo_status_2_2=exposure \* status_3.  
VARIABLE LABELS  expo_status_2_2 'expo \* formersmoker'.  
EXECUTE.  
COMPUTE expo_status_2_3=exposure \* status_4.  
VARIABLE LABELS  expo_status_2_3 'expo \* smoker'.  
EXECUTE.  

\* Multiple regression.  
\* Statistic Descriptives is added to get the means that we need  
\* to plug into the regression equation in the moderation plot.  
REGRESSION  
  /DESCRIPTIVES MEAN STDDEV CORR SIG N  
  /MISSING LISTWISE  
  /STATISTICS COEFF OUTS CI(95) R ANOVA  
  /CRITERIA=PIN(.05) POUT(.10)  
  /NOORIGIN   
  /DEPENDENT attitude  
  /METHOD=ENTER exposure status_3 status_4 expo_status_2_2 expo_status_2_3 contact  
  /SCATTERPLOT=(\*ZRESID ,\*ZPRED)  
  /RESIDUALS HISTOGRAM(ZRESID).  
\* Scatterplot with dots coloured by smoking status.  
GRAPH  
  /SCATTERPLOT(BIVAR)=exposure WITH attitude BY status3  
  /MISSING=LISTWISE.  
\* Histogram of predictor (exposure) for each smoking status.  
GRAPH  
  /HISTOGRAM=exposure  
  /PANEL ROWVAR=status3 ROWOP=CROSS.  
  
* Write out the regression equations for all three groups of the moderator.
Plug in the estimated values of the constant and the regression coefficients.
  
> attitude = 0.623 + -0.183\*exposure + -1.554\*former + 0.989\*smoker + 
  -0.259\*exposure\*former + -0.302\*exposure\*smoker + 0.186\*contact  
  
Plug in the means of covariates (contact: 4.995, see Exercise 2) and add to constant:  
    
> attitude = 0.623 + -0.183\*exposure + -1.554\*former + 0.989\*smoker + 
  -0.259\*exposure\*former + -0.302\*exposure\*smoker + 0.186\*4.995  
  
> attitude = 1.552 + -0.183\*exposure + -1.554\*former + 0.989\*smoker + 
  -0.259\*exposure\*former + -0.302\*exposure\*smoker  
  
Non-smokers (former = 0, smoker = 0):  
  
> attitude = 1.552 + -0.183\*exposure + -1.554\*0 + 0.989\*0 + -0.259\*exposure\*0 + 
  -0.302\*exposure\*0  
  
> attitude = 1.552 + -0.183\*exposure  
  
Former smokers (former = 1, smoker = 0):  
  
> attitude = 1.552 + -0.183\*exposure + -1.554\*1 + 0.989\*0 + -0.259\*exposure\*1 + 
  -0.302\*exposure\*0  
  
> attitude = 1.552 + -1.554 + -0.183\*exposure + -0.259\*exposure  
  
> attitude = -0.002 + (-0.183 + -0.259)\*exposure  
  
> attitude = -0.002 + -0.442\*exposure  
  
Smokers (former = 0, smoker = 1):  
  
> attitude = 1.552 + -0.183\*exposure + -1.554\*0 + 0.989\*1 + -0.259\*exposure\*0 + -0.302\*exposure\*1  
  
> attitude = 1.552 + 0.989 + (-0.183 + -0.302)\*exposure  
  
> attitude = 2.541 + -0.485\*exposure  
  
* Create a scatterplot of attitude by exposure, colouring the dots by smoking
status.
* Use the calculated three equations in the SPSS Chart Editor to create three
lines. Use the icon "Add a reference line from Equation" for each line. Enter
the the equation using x instead of exposure as the predictor. Colour the
lines with the colours of the dots in the scatterplot.
  
Check data:  
  
All values seem to be valid.  
  
Check assumptions:  
  
![](figures/S8_4Q4a.png)
  
* The residuals seem to be skewed a little bit.  
* The residuals by predicted values plot gives no reason to doubt the
linearity of the model but the problem of predicting higher values less
accurately than lower values seems to be worse than in Exercise 1. We should
warn the reader that the assumptions seem to be violated.
  
Interpret the results:  
  
* Again, summarize the tests on the regression coefficients in a table. 

<div style="font-size: 0.8em">
| | _b_ |	SE | _b\*_ | _t_ | _p_ | 95\%CI |
|:------------|----:|----:|----:|----:|----:|:--------:|
| Constant | 0.62 | 0.24 |  | 2.63 | .009 | [0.16, 1.09] |
| Exposure to anti-smoking campaign | -0.18 | 0.03 | -0.27 | -6.74 | < .001 | [-0.24, -0.13] |
| Former smoker | -1.55 | 0.30 | -0.32 | -5.11 | < .001 | [-2.15, -0.96] |
| Smoker | 0.99 | 0.28 | 0.22 | 3.57 | < .001 | [0.44, 1.53] |
| Exposure * Former smoker | -0.26 | 0.05 | -0.33 | -5.00 | < .001 | [-0.36, -0.16]
| Exposure * Smoker | -0.30 | 0.05 | -0.40 | -6.23 | < .001 | [-0.40, -0.21] |
| Contact with smokers | 0.19 | 0.03 | 0.19 | 5.91 | < .001 | [0.12, 0.25] |
</div>

* With three smoking status groups, we can predict attitude towards smoking
much better (*R*^2^ = 0.74) than with the two groups in Exercise 1 (*R*^2^ = .38).
  
* There is an important difference between non-smokers and former smokers
(they were lumped together in the preceding exercises). On average, former
smokers have a substantially more negative attitude than non-smokers, ranging from 0.9 to 2.15 points lower on a scale from -5 to +5) if they are not exposed to the campaign.
  
* In addition, exposure has a stronger negative predictive effect on smoking
attitude among former smokers than among non-smokers (the interaction effect
of _Exposure_ with _Former smoker_ has a negative unstandardized regression
coefficient; remember that non-smokers are the reference group). Exposure also
has a stronger negative effect on attitude among smokers than among
non-smokers (again, a negative unstandardized regression coefficient for the
interaction effect). For short, exposure to the campaign has less impact on
attitude towards smoking for non-smokers than for former smokers or smokers.
  
![](figures/S8_4Q4b.png)
  
* The coverage of exposure is good for non-smokers and smokers but former
smokers with high exposure are rare. [<img src="icons/2question.png" width=161px align="right">](#question8.4.4)
```

## Test Your Understanding

Figure \@ref(fig:regression-predict2) shows how much respondents were exposed to an anti-smoking campaign (horizontal axis) and their attitudes towards smoking, ranging from negative (-5) to positive (5, vertical axis). Attitude towards smoking is predicted from exposure as well as from the respondent's smoking status and daily contacts with smokers as specified by the (red) multiple regression equation. The blue line represents the (blue) simple regression equation, which predicts attitude from exposure for the selected value of smoking status and contact.

```{r regression-predict2, fig.pos='H', fig.align='center', fig.cap="How do predictions based on exposure depend on values of smoking status and smoker contact?", echo=FALSE, out.width="775px", screenshot.opts = list(delay = 5), dev="png"}
# Goal: sensitize student to the notion that prediction in a multiple regression
# model requires selecting one independent variable and fixed values for the other
# independent variables. Without moderation, the fixed values only change the vertical
# position of the line (that is, the constant) but not the slope.
# Generate a data set with attitude towards smoking as Y and exposure as X with a negative (b = -0.6) more or less linear relation and a regression line for the currently selected values of the covariates (smoking status, initial value is 0) and contact (initial value is 3)): attitude = 0.25 - 0.6 * exposure + 0.50 * smoker +  0.15 * contact
# Display the scatterplot.
# Add the line of a simple regression of attitude on exposure for the generated data in grey.
# Add the multiple regression line.
# Display the multiple regression equation: (see above). 'smoker' is a selection box with two options: 0 = non-smoker, 1 = smoker. 'contact' is followed by a numerical input that accepts values between 0 and 10.
# A change to the smoker selection or contact value triggers the app to redraw the regression line for attitude by exposure for these values of smoking status and (smoker)contact.
knitr::include_app("http://82.196.4.233:3838/apps/regression-predict2/", height="375px")
```

<A name="question8.5.1"></A>

```{block2, type='rmdquestion'}
1. Do smokers or non-smokers have a more positive attitude towards smoking according to Figure \@ref(fig:regression-predict2)? [<img src="icons/2answer.png" width=115px align="right">](#answer8.5.1)
```

<A name="question8.5.2"></A>

```{block2, type='rmdquestion'}
2. What is the predicted attitude for smokers who are not exposed to the campaign and who have 4 contacts with smokers? Use the equation in Figure \@ref(fig:regression-predict2) and see if you can easily check your answer in this figure. [<img src="icons/2answer.png" width=115px align="right">](#answer8.5.2)
```

<A name="question8.5.3"></A>

```{block2, type='rmdquestion'}
3. Do people with more contacts with smokers have a more positive or a more negative attitude towards smoking? Does this apply to all people or only to people with a particular smoking status or a particular level of exposure? [<img src="icons/2answer.png" width=115px align="right">](#answer8.5.3)
```

```{r exampleresults, echo=FALSE, eval=TRUE}
data.frame(
  par = c("Constant", "Exposure to anti-smoking campaign", "Former smoker", "Smoker", "Exposure \\* Former smoker", "Exposure \\* Smoker", "Contact with smokers"),
  b = c(0.55, -0.137, -1.139, 1.223, -0.402, -0.304, 0.171),
  SE = c(0.466, 0.047, 0.52, 0.498, 0.107, 0.093, 0.059),
  Beta = c(NA, -0.242, -0.269, 0.302, -0.442, -0.394, 0.202),
  t = c(1.181, -2.888, -2.188, 2.456, -3.739, -3.262, 2.91),
  p = c("0.241", "0.005", "0.032", "0.016", "<.001", "0.002", "0.005"),
  CI = c("[-0.377, 1.477]", "[-0.231, -0.042]", "[-2.174, -0.103]", "[.232, 2.214]", "[-0.616, -0.188]", "[-0.490, -0.119]", "[.054, .287]")
) %>%
  kable(align = "lrrrrrc", col.names = c("", "b", "SE", "b\\*", "t", "p", "95\\%CI"), caption = "Regression model predicting attitude towards smoking.", booktabs = TRUE) %>%
  kable_styling(font_size = 12, full_width = F,
                latex_options = c("scale_down", "HOLD_position"))
```

<A name="question8.5.4"></A>

```{block2, type='rmdquestion'}
4. Why is the third category _Non-smoker_ not included as a predictor in Table \@ref(tab:exampleresults)? [<img src="icons/2answer.png" width=115px align="right">](#answer8.5.4)
```

<A name="question8.5.5"></A>

```{block2, type='rmdquestion'}
5. Can you tell from this table whether the effect of exposure for smokers is statistically significantly stronger than the effect of exposure for non-smokers? And is it significantly stronger than the effect of exposure for former smokers?. [<img src="icons/2answer.png" width=115px align="right">](#answer8.5.5)
```

<A name="question8.5.6"></A>

```{block2, type='rmdquestion'}
6. Calculate the regression equation for the effect of exposure on attitude for former smokers who have two contacts with smokers using the results summarized in Table \@ref(tab:exampleresults). [<img src="icons/2answer.png" width=115px align="right">](#answer8.5.6)
```

```{r supportexamples, echo=FALSE, eval=TRUE, out.width="640px", out.height="478px", fig.pos='H', fig.align='center', fig.cap="Four examples of common support."}
include_graphics("figures/Q8.5.7.png")
```

<A name="question8.5.7"></A>

```{block2, type='rmdquestion'}
7. In which panel of Figure \@ref(fig:supportexamples) is common support best? In which panel is it worst? Motivate your answer. [<img src="icons/2answer.png" width=115px align="right">](#answer8.5.7)
```

<A name="question8.5.8"></A>

```{block2, type='rmdquestion'}
8. What would you report about common support based on the histogram in the bottom-left panel of Figure \@ref(fig:supportexamples)? [<img src="icons/2answer.png" width=115px align="right">](#answer8.5.8)
```

### Answers {.unnumbered}

```{block2, type='rmdanswer', echo=!ch8}
Answers to the Test Your Understanding questions will be shown in the web book when the last tutor group has discussed this chapter.
```

<A name="answer8.5.1"></A>

```{block2, type='rmdanswer', echo=ch8}
Answer to Question 1. 

* If you change smoking status from non-smoker to smoker, the regression line shifts up. This means that the predicted values of attitude increase. Smokers tend to have a more positive attitude towards smoking than non-smokers.
* The (red) multiple regression equation tells you the same story. The unstandardized regression coefficient for smoking status is 0.5. This is the average difference in attitude between smokers (coded '1') and non-smokers (coded '0'). In other words, the attitude of smokers is on average 0.5 higher (more positive towards smoking) than the attitude of non-smokers. [<img src="icons/2question.png" width=161px align="right">](#question8.5.1)
```

<A name="answer8.5.2"></A>

```{block2, type='rmdanswer', echo=ch8}
Answer to Question 2. 

* Plug in 1 for smoking status, 0 for exposure, and 4 for contact in the equation and simplify the equation:

> attitude = 0.65 - 0.6 \* exposure + 0.5 \* smoker + 0.15 \* contact
> attitude = 0.65 - 0.6 \* 0 + 0.5 \* 1 + 0.15 \* 4  
> attitude = 0.65 - 0 + 0.5 + 0.6  
> attitude = 1.75  

* The predicted value for smokers who are not exposed to the campaign and who have 4 contacts with smokers is 1.75. 
* If you select the right values in the figure, the (blue) simple regression equation will read _attitude_ = 1.75 - 0.6 \* _exposure_. If exposure is zero, the predicted attitude equals the constant, which is 1.75. [<img src="icons/2question.png" width=161px align="right">](#question8.5.2)
```

<A name="answer8.5.3"></A>

```{block2, type='rmdanswer', echo=ch8}
Answer to Question 3. 

* The regression coefficient for contact with smokers is 0.15, which is positive. More contacts, then, have higher predicted attitudes, that is, a more positive attitude towards smoking.
* In this model, contact with smokers has a positive effect regardless of a person's exposure and smoking status because the effect of contact is not moderated by exposure or smoking status. There is no interaction variable in the equation. [<img src="icons/2question.png" width=161px align="right">](#question8.5.3)
```

<A name="answer8.5.4"></A>

```{block2, type='rmdanswer', echo=ch8}
Answer to Question 4. 

* We have to leave one category of a categorical (or dichotomous) predictor out of the regression model. This is the reference category or group.
* The left-out category is perfectly predicted by the dummy variables for the included categories. After all, the non-smokers are the people scoring "No" (zero) on the _former smoker_ and _smoker_ variables. If an independent variable can be perfectly predicted by other independent variables, the model is multicollinear and it cannot be estimated.  [<img src="icons/2question.png" width=161px align="right">](#question8.5.4)
```

<A name="answer8.5.5"></A>

```{block2, type='rmdanswer', echo=ch8}
Answer to Question 5. 

* Non-smokers are the reference group (see Question 4), so the interaction effects represent the differences with non-smokers.
* The regression coefficient -0.304 for _Exposure \* Smoker_ is the difference between the effect of exposure for smokers and the effect of exposure for non-smokers (the reference group). This coefficient is negative, so the effect of exposure is more strongly negative for smokers than for non-smokers. The difference is statistically significant, _t_ = -3.62, _p_ = .002, 95% CI [-0.49, -0.12].
* If we want to test the difference between the effects of exposure for smokers and former smokers, one of these two categories must be left out as reference group. We cannot obtain the test results from Table \@ref(tab:exampleresults). We need a new regression model for this. [<img src="icons/2question.png" width=161px align="right">](#question8.5.5)
```

<A name="answer8.5.6"></A>

```{block2, type='rmdanswer', echo=ch8}
Answer to Question 6. 

* We only need the unstandardized regression coeficients (_b_) from the table.
* Create an equation with the unstandardized regression coeficients: 

> attitude = .550 + -.137 \* exposure + -1.139 \* former + 1.223 \* smoker + -.402 \* exposure \* former + -.304 \* exposure \* smoker + .171 \* contact

* To select former smokers, replace the variable _former_ by 1 (represents "Yes") and replace the variabe _smoker_ by 0 (represents "No"):

> attitude = .550 + -.137 \* exposure + -1.139 \* 1 + 1.223 \* 0 + -.402 \* exposure \* 1 + -.304 \* exposure \* 0 + .171 \* contact

* Note that multiplication with 0 yields 0, so we can drop these terms and multiplication by 1 does not change the result, so we can drop the ones:

> attitude = .550 + -.137 \* exposure + -1.139 + -.402 \* exposure + .171 \* contact

* Replace the variable _contact_ by 2, so the equation applies to people with two contacts:

> attitude = .550 + -.137 \* exposure + -1.139 + -.402 \* exposure + .171 \* 2

* Now, simplify the equation. Join the two exposure effects and join all other elements into a new constant (intercept). 

> attitude = -0.247 + -0.539 \* exposure

* This is the regression equation we are looking for. Former smokers having two contacts with smokers and who are not exposed to the campaign, have a predicted attitude of -0.25. An additional unit of exposure decreases the predicted attitude by 0.54 on a scale from -5 to +5. [<img src="icons/2question.png" width=161px align="right">](#question8.5.6)
```

<A name="answer8.5.7"></A>

```{block2, type='rmdanswer', echo=ch8}
Answer to Question 7. 

* Common support is best in bottom-right panel because there are observations across the entire range of exposure scores for all three smoking groups.
* Common support is worst in the upper-right and bottom-left panels because they contain one group with exposure scores confined to a narrow range. [<img src="icons/2question.png" width=161px align="right">](#question8.5.7)
```

<A name="answer8.5.8"></A>

```{block2, type='rmdanswer', echo=ch8}
Answer to Question 8. 

* You should report the range for which all three groups have exposure scores. This would be the range from approximately 2.5 to 7.5. Our conclusions must be restricted to this range of exposure scores. For lower or higher exposure scores, we do not know the exposure effect or how it is moderated by smoking status.
* In addition, you may (but do not have to) note that we have observed highest exposure scores only for smokers. [<img src="icons/2question.png" width=161px align="right">](#question8.5.8)
```

## Take-Home Points

-   We use regression analysis if our dependent variable is numeric and we have at least one numeric independent (predictor) variable.

-   We use dummy variables to include a categorical variable as a predictor in a regression model. We need a dummy (1/0) variable for each category on the categorical variable except for one category, which represents the reference group.

-   We use an *F* test to test the null hypothesis that the regression model does not help to predict the dependent variable in the population. We use a *t* test to test the null hypothesis that a regression coefficient is zero in the population.

-   In a regression model, moderation means that there are different slopes (effects of the predictor) for different groups or contexts (moderator).

-   Interaction variables represent moderation in a regression model.

-   An interaction variable is the product of the predictor and moderator. If a categorical moderator is represented by one or more dummy variables, we need an interaction variable for each of the moderator's dummy variables.

-   Statistical inference for an interaction variable is exactly the same as for "ordinary" regression predictors.

-   The effect of the predictor in a model with an interaction variable does *not* represent a main or average effect. It is a conditional effect: The effect for cases that score zero on the moderator.

-   To interpret moderation, describe the effects (slopes, unstandardized regression coefficients) and visualize the regression lines for different groups.

-   Warn the reader if the predictor scores are not nicely distributed for all groups or levels (no common support).

-   Don't use the standardized regression coefficients (Beta) for interaction variables, variables included in interactions, or for dummy variables in SPSS.