The general linear model: categorical predictors

Dummy coding

We can include categorical predictors using dummy coding (there are other forms of coding two, for example, contrast coding, which we’ll cover in the next tutorial). One source of confusion is that SPSS has different menu structures when the goal of the linear model is to compare means. This can create the impression that when you fit a model to compare means it is a different model to the one you fit when looking at predicting an outcome from continuous variables.

The model

As revision from the lecture/chapter, the model we’re fitting is:

\[ \text{Happiness}_i = b_0 + b_1\text{Long}_i+ b_2\text{Short}_i + ε_i \] In which the variables Long and Short are the dummy variables named dummy1 and dummy2 in the data file. The first variable (Long) represents the difference between the 30-minute group and the control group, whereas the second variable (Short) represents the difference between the 15-minute group and the control group.

Fitting the model using the Regression menu (optional)

Let’s first fit the model using the Regression menu. This isn’t what you’d normally do, but it is instructive for showing how what happens ‘under the hood’ is the same when you use the Regression menu to when you use the One-Way ANOVA menu. The different menus are simply different ‘wrappers’ or ‘packaging’ for the same statistical procedure. This video shows how to fit the model using the Regression menu:

Output

Output 1

Fitting the model

The following video shows how to fit the model using SPSS Statistics.

Descriptive information

The first part of the output (Output 2) shows the descriptive statistics for each group. I have also plotted the means and 95% confidence intervals in Figure 3. The line that joins the means seems to indicate a linear trend in that, as the dose of puppy therapy increases, so does the mean level of happiness.

Output 2

Figure 3

The F-statistic

Output 3 shows the main ANOVA summary table. The table is divided into between-group effects (effects due to the model – the experimental effect) and within-group effects (this is the unsystematic variation in the data).

Output 3

Output 4 shows the Welch and the Brown-Forsythe F-statistics. If you’re interested in how these values are calculated then look at my SPSS book (note that the error degrees of freedom have been adjusted and you should remember this when you report the values). Based on whether the observed p is less than 0.05, the Welch F yields a non-significant result (p = 0.054) whereas Brown-Forsythe is significant (p = 0.026). This is confusing, but only if you like to imbue the 0.05 threshold with magical powers and engage in black-and-white thinking of the sort that people who use p-values so often do.

Output 4

Cohen’s d

A useful measure of effect size is Cohen’s d, which is the difference between two means divided by some estimate of the standard deviation of those means:

\[ \hat{d} = \frac{\bar{X_1}-\bar{X_2}}{s} \]

I have put a hat on the d to remind us that we’re really interested in the effect size in the population, but because we can’t measure that directly, we estimate it from the samples (The hat means ‘estimate of’). By dividing by the standard deviation we are expressing the difference in means in standard deviation units (a bit like a z–score). The standard deviation is a measure of ‘error’ or ‘noise’ in the data, so d is effectively a signal-to-noise ratio. However, if we’re using two means, then there will be a standard deviation associated with each of them so which one should we use? There are three choices:

1. If one of the group is a control group it makes sense to use that groups standard deviation to compute d (the argument being that the experimental manipulation might affect the standard deviation of the experimental group, so the control group SD is a ‘purer’ measure of natural variation in scores)
2. Sometimes we assume that group variances (and therefore standard deviations) are equal (homogeneity of variance) and if they are we can pick a standard deviation from either of the groups because it won’t matter.
3. We use what’s known as a ‘pooled estimate’, which is the weighted average of the two group variances. This is given by the following equation:

\[ s_p = \sqrt{\frac{(N_1-1) s_1^2+(N_2-1) s_2^2}{N_1+N_2-2}} \]

Say we wanted to estimate d for the difference between the 30-minute group and the control, Output 1 shows us the means, sample size and standard deviation for the groups:

• Control: M = 2.2, N = 5, s = 1.304, \(s^2\) = 1.7
• 15-mins: M = 3.2, N = 5, s = 1.304, \(s^2\) = 1.7
• 30-mins: M = 5.0, N = 5, s = 1.581, \(s^2\) = 2.5

We have a logical control group so d is simply:

\[ \hat{d}_\text{30 mins vs control} = \frac{5.0-2.2}{1.304} = 2.15 \]

Calculate d for the comparison between the 15-minute group and control, and the 30-minute group compared to the 15-minute group.

Solution

For the comparison between the 15-minute group and control, we can again use the control group standard deviation:

\[ \hat{d}_\text{15 mins vs control} = \frac{3.2-2.2}{1.304} = 0.77 \] For the comparison between the 15- and 30-minute groups, there is no obvious control so we could use the pooled estimate:

\[ \begin{aligned} \ s_p &= \sqrt{\frac{(N_1-1) s_1^2+(N_2-1) s_2^2}{N_1+N_2-2}} \\ \ &= \sqrt{\frac{(5-1)2.5+(5-1)1.7}{5+5-2}} \\ \ &= \sqrt{\frac{16.8}{8}} \\ \ &= \sqrt{2.1} \\ \ &= 1.449 \end{aligned} \] which gives a d of:

\[ \hat{d}_\text{30 mins vs 15 mins} = \frac{5.0-3.2}{1.45} = 1.24 \]

There goes my hero … watch him as he goes (to hospital)

Let’s look at a second example from Field (2017) (Smart Alex task). Children wearing superhero costumes are more likely to harm themselves because of the unrealistic impression of invincibility that these costumes could create. For example, children have reported to hospital with severe injuries because of trying ‘to initiate flight without having planned for landing strategies’ (Davies et al. 2007). I can relate to the imagined power that a costume bestows upon you; indeed, I have been known to dress up as Fisher by donning a beard and glasses and trailing a goat around on a lead in the hope that it might make me more knowledgeable about statistics. Imagine we had data about the severity of injury (on a scale from 0, no injury, to 100, death) for children reporting to the accident and emergency department at hospitals, and information on which superhero costume they were wearing (hero): Superman ( = 1), Spiderman (= 2), the Hulk (= 3) or a teenage mutant ninja turtle (= 4). Fit a model with planned contrasts to test the hypothesis that different costumes give rise to more severe injuries.

To get you into the mood for hulk-related data analysis, here is a photo of my wife and I on the Hulk roller-coaster in Florida on our honeymoon (we also went on the Spiderman ride, which was incredible). You may well resort to similar facial expressions during this example:

Figure 4: My wife regretting marrying me

There are detailed answers to this task on the companion website of my SPSS textbook.

Cohen’s d

Say we wanted to estimate d for the effect of Superman costumes compared to Ninja Turtle costumes. The means, sample size and standard deviation for these two groups are (these should be in your SPSS output):

• Superman: M = 60.33, N = 6, s = 17.85, \(s^2\) = 318.62
• Ninja Turtle: M = 26.25, N = 8, s = 8.16, \(s^2\) = 66.50

Neither group is a natural control (you would need a ‘no costume’ condition really), but if we decided that Ninja Turtle (for some reason) was a control (perhaps because Turtles don’t fly but supermen do) then d is simply:

\[ \hat{d}_\text{superman vs ninja} = \frac{60.33-26.25}{8.16} = 4.18 \]

In other words, the mean injury severity for people wearing superman costumes is 4 standard deviations greater than for those wearing Ninja Turtle costumes. This is a pretty huge effect. Let’s have a look at using the pooled estimate.

\[ \begin{aligned} \ s_p &= \sqrt{\frac{(N_1-1) s_1^2+(N_2-1) s_2^2}{N_1+N_2-2}} \\ \ &= \sqrt{\frac{(6-1)318.62+(8-1)66.50}{6+8-2}} \\ \ &= \sqrt{\frac{2058.6}{12}} \\ \ &= 13.10 \end{aligned} \]

When the group standard deviations are different, this pooled estimate can be useful; however, it changes the meaning of d because we’re now comparing the difference between means against all of the background ‘noise’ in the measure, not just the noise that you would expect to find in normal circumstances. Using this estimate of the standard deviation we get:

\[ \hat{d}_\text{superman vs ninja} = \frac{60.33-26.25}{13.10} = 2.60 \]

Notice that d is smaller now; the injury severity for Superman costumes is about 2 standard deviations greater than for Ninja Turtle Costumes (which is still pretty big). Compute Cohen’s d for the effect of Superman costumes on injury severity compared to Hulk and Spiderman costumes. Try using both the standard deviation of the control (the non-Superman costume) and also the pooled estimate.

Solution