The analysis methods we have studied so far assume that the observations are independent. This assumption is often wrong, and it is intentionally violated in some experimental designs to increase the sensitivity of the tests. In this section we will exercise with a well known procedure repeated-measures ANOVA for analyzing (experimental) data where same subjects are measured more than once (hence, observations are not independent).
We will use two (again hypothetical) new data sets. In the ﬁrst data set we assume that we investigate whether newborns distinguish their mother’s native language from another language. We recruit 30 newborns, and when we ﬁnd them awake during their ﬁrst day in life, we let them listen to two comparable short stories, one in their mother’s language and another in a foreign language. While they are listening to these stories, they are equipped with a paciﬁer through which we can measure their sucking rate. Crucially, each infant is tested on both conditions (the order of languages are randomized). Since our hypothetical newborns never fall asleep, start crying or spit out the paciﬁer in the middle of the story, we have 60 measurements from 30 infants. You can load the data set using,
Notice that this time the ﬁle extension is diﬀerent, and we used load() function. This data ﬁle is in R’s native binary format. The advantage is that you retain all the information in the original data (including the variable name). The disadvantage is in its portability. The CSV ﬁles we used earlier can be read virtually in any environment by many applications, while the .rda ﬁles can only be read by R (but you do not need to worry about the platform or the operating system incompatibilities). The reason we wrapped the load() within a print is because of the fact that load() silently loads the variables in the data ﬁle. If you print() the return value, you will know which variables are loaded. Otherwise, you need to inspect your R environment to ﬁgure out the changes introduced by load(). If all went ﬁne, you should have a new data frame named newborn. Note again that the data is fake, but the method, high-amplitude sucking paradigm described here without many details is a well-known technique for studying infants. For a real study of the sort described here see Nazzi et al. [1998].
The second data set we will use comes from another hypothetical language acquisition study. This time we are interested in children who are raised bilingually, where one of the languages they speak is ‘home only’ and the other language is also used in their school. The data set can be found at http://coltekin.net/cagri/R/data/bilingual.txt as a ‘tab-separated ﬁle’.
You can use read.delim() for reading this ﬁle. Note that generic function for reading ‘tabular ﬁles’ is read.table(). $\u22b3$
Optionally you can repeat Exercise 5.1, but supply the read.delim() (or read.table()) command with a colClasses option (see help on these functions for detailed use of this option). $\u22b3$
As we did with independent-measures ANOVA, we will start with the simplest case: when we have only two conditions. If we only have two conditions measured repeatedly over a sample of individuals (people, animals, countries, hospitals, …) the we typically use a paired t test. Our data set newborns provides a textbook case application of paired t test. But for the sake of comparison, we will ﬁrst take a detour, and revisit the independent samples t test.
Check whether t-test assumptions (except independence) are violated or not. $\u22b3$
In R, The paired t test can be performed using the same function, t.test(). All you need to do is pass the argument paired=TRUE.
Do you also need to check whether variances across the groups are approximately equal? $\u22b3$
Repeated measures ANOVA can be performed in R using a few diﬀerent ways. In this tutorial, we will exercise with the function aov() that comes with the base R installation (‘stats’ package). aov() can handle only standard cases—no violation of the assumptions, no missing data— and only displays minimal information—no eﬀect sizes. For more complex designs, one can use utilities found in additional packages or libraries, such as Anova() (note the capital ‘A’) from the car package and ezANOVA() (read ‘easy ANOVA’) from the package ez. We will conclude the section with an example run on ezANOVA(). However, we note that if your design is not ideal for repeated measures ANOVA you should probably use the ‘multi-level’ or ‘mixed-eﬀect’ linear regression that we will see later in this tutorial.
As before, we will start with the simplest case. Remember that when we have two groups, the independent-measures ANOVA is equivalent to two-samples independent measures t test. Similarly, when we have only two groups, the repeated-measures ANOVA gives you the same results as the paired t test. Here is how we do a repeated-measures ANOVA using aov() on the data set newborn.
> m <- aov(rate ~ language + Error(participant/language),
data=newborn)
> summary(m)
Error: participant
Df Sum Sq Mean Sq F value Pr(>F)
Residuals 29 5792 199.7
Error: participant:language
Df Sum Sq Mean Sq F value Pr(>F)
language 1 306.9 306.95 28.24 1.06e-05 ⋆⋆⋆
Residuals 29 315.2 10.87
Not surprisingly, the p-value matches to the p-value found in Exercise 5.4. As in earlier ANOVA models, we speciﬁed a model predicting the rate from the language using the formula notation. Crucial diﬀerence is in the speciﬁcation of the Error() term. This term speciﬁes the replication in the experiment design. In this case, we tell aov() that the response corresponding to every level of language is measured for each participant. The speciﬁcation of the error term could be confusing at ﬁrst sight. Within the Error() term, the part before the slash ‘/’ speciﬁes the ‘case’ or ‘subject’ variable. The part after the slash speciﬁes the ‘within subject’ variable(s). Note that we used summary() instead of summary.aov(), since the default summary method for an aov() object is ANOVA-like summary.
Without the Error() term, the function aov() is equivalent to the lm() function we used for independent measures ANOVA.
Extending the repeated measures ANOVA in Exercise 5.8 to include more predictors is easy. We just add all predictors to the formula notation as we do for the factorial ANOVA. You only need to be careful to include all the ‘within-subject’ variables in the error term. For example, for two-way within-subject factorial ANOVA with interaction term where age and language are the predictors, the error term becomes Error(subj/(language⋆age)).
Often, we want to include predictors that are not or cannot be replicated in a repeated measures design. Such a variable in our bilingual data set is gender, which is a good example of a variable that can hardly be measured within-subjects. In this cases we use so-called mixed-design ANOVA analysis. There is nothing interesting specifying a mixed-design ANOVA in R. We just add the between-subject variable(s) to the model formula, but exclude it from the error term.
The above exercises exemplify a variety ANOVA designs that can be ﬁt using aov(). However, aov()
In such cases a few packages in R provide solutions that are similar to other statistical software. We will only present an example using ezANOVA() from the package ez. However, when things are not perfectly balanced, and neat, the repeated-measures ANOVA becomes diﬃcult to interpret. One reasonable course of action when ANOVA design becomes too complicated, or assumptions are violated at some level is to switch to so-called mixed-eﬀect models which also oﬀer some other beneﬁts. We will discuss mixed-eﬀect linear models later in this tutorial.
Listing 8 repeats Exercise 5.11 using ezANOVA(). The listing is slightly edited for clarity (you still need to exercise your skills in reading scientiﬁc notation).
The ﬁrst thing to note in Listing 8 is the command library() on line 1. This includes the functions deﬁned in library ez. There are a large number of (mostly free/open-source) libraries for R for various (statistical) tasks. In this tutorial, we try to stick to the bare-bones, but you should check existing libraries for the tasks that are not possible or diﬃcult to do with the basic packages of R. The central repository for all R packages is at http://cran.r-_project.org/. If the package you need is not installed on your computer, you can install any package from CRAN with the command install.packages().
Returning to the ANOVA results Listing 8, the main ﬁndings should be the same as what you found with aov() in Exercise 5.11. The additional information presented here include the eﬀect size on the column labeled with ges (generalized ${\eta}^{2}$); the results of Mauchly’s Test; and in case we could not maintain the sphericity assumption two common corrections used in the literature: Greenhouse-Geisser $\mathit{\epsilon}$ (GGe), and Huynh-Feldt $\mathit{\epsilon}$ (HFe) and their corresponding p-values. Furthermore, ezANOVA() includes options for cases when the data is not balanced (so called type I, type II and type III sums of squares—aov() only calculates type I sums of squares.). However, you should better read and understand these before using it in a real analysis.
We will stop the discussion of repeated measures ANOVA here, but we will revisit some of the concepts and exercises when we discuss the mixed-eﬀect models.