A Cross-Cultural Study of a Circumplex Model of Affect

In the quest to understand how people conceptualize emotional feelings, one approach has been to seek the dimensions by which they perceive the similarities and differences among feelings. A circumplex model of affect represents this set of mutual relations by placing feeling-related concepts in a circular order in a space formed by two bipolar dimensions: pleasure-displeasure and arousalsleepiness. This article offers evidence that the circumplex structure, rather than being somehow dependent on the English-speaking student population in which it was originally obtained, occurs in different languages and cultures. In Study 1, Estonian, Greek, and Polish Ss judged the similarity between feelings described by 28 words in their respective native languages. In Study 2, Greek and Chinese Ss judged the similarity between feelings conveyed by 10 facial expressions. In all cases, multidimensional scaling of pairwise similarity scores yielded the circular order and underlying dimensions predicted by the circumplex.

ize emotional feelings has been to seek the dimensions by which they perceive the similarities and differences among feelings.
Figure 1 provides a closer look at the Cartesian space formed from the pleasure and arousal axes.The configuration shown is the outcome of a multidimensional scaling of 28 words chosen to place emotions in the broader context of feelings or affect (Russell, 1980).The 28 words include what have been called basic emotions (happy, angry, afraid, sad), as well as less prototypical emotions (gloomy, bored) and affective states that may or may not be emotions (sleepy, serene).Multidimensional scaling provided a pictorial representation of this domain by placing those feelings judged as more similar closer together, and those feelings judged as more dissimilar farther apart.The 28 words fell throughout the space in a meaningful pattern.Thus, delighted fell opposite sad, and aroused opposite sleepy.Rather than clustering about the axes, the words spread around the perimeter of the space, in a roughly circular order, with different regions corresponding to different combinations of pleasure  Psychology, 1980, 39, p. 1168. Copyright 1980 by the American Psychological Association.Adapted by permission.)Lancee, & Polivy, 1983;Plutchik, 1980;Russell, 1980;Schlosberg, 1952;Watson & Tellegen, 1985).The present study did not seek to explore the advantages or disadvantages of these various models.Rather, we took one perspective on this topic: that summarized in the idea of a circumplex, which emphasizes the two bipolar dimensions and the circular ordering seen in Figure 1.Our study was motivated by the question of whether these two dimensions and the circular ordering are unique to our language and culture, or instead are generalizable to other languages and cultures.Cross-cultural evidence on this question is scarce.Studies of the semantic differential technique hint that the two hypothesized dimensions may be pancultural.This is the common interpretation of Osgood, May, and Miron's (1975) study in which 22 culturally and linguistically different groups produced the same three dimensions of affective meaning: evaluation, activity, and potency.The problem is that although there is an obvious similarity of evaluation to pleasure-displeasure, and of activity to arousal-sleepiness, they might not be equivalent in all or even most contexts.For example, to say that a meal is good may be to say that it affords pleasure, but to say that a person's action is good may be to say that it accords with moral principles.Evaluating truth higher than falsity is different than evaluating right higher than wrong, and neither evaluation need be related to pleasure.On the other hand, when Block (1957) used the semantic differential technique to explore the meaning of emotions specifically, he obtained evidence of two dimensions, pleasure and arousal.Moreover, he found highly similar results in a replication in Norwegian.
Factor analysis of subjects' self-reported mood provides evidence that a structure of affect similar to that seen in Figure 1 can be obtained in Swedish (Sjoberg et al., 1979) and Japanese (Watson, Clark, & Tellegen, 1984).
Evidence from multidimensional scaling has been less consistent, and, indeed, has been used to argue against an arousal dimension (Lutz, 1982).Studies of German (Gehm & Scherer, 1988), Swedish (Lundberg & Devine, 1975), Hebrew (Fillenbaum & Rapoport, 1971), Japanese (Yoshida, Kinase, Kurokawa, & Yashiro, 1970), and Ifalukian (Lutz, 1982) all yielded a pleasure-displeasure dimension.But of these, only Lundberg and Devine's study of Swedish produced a dimension interpretable as arousal.What is more problematic is that even the plea-sure-displeasure dimension could be questioned: Nothing in the various studies can rule out the skeptic's objection that dimensions with the same name need not be the same.A study of Chinese, Japanese, Croatian, and Oujarati found both pleasure-displeasure and arousal-sleepiness, and established their equivalence across the various languages (Russell, 1983); but all subjects were residents of an English-speaking region of Canada.
Each of the studies cited in the preceding paragraph can be faulted on its solution to one or more of three issues: (a) which cultural-linguistic groups are studied, (b) how affect is sampled in each group, and (c) how the purported equivalences or differences are verified.The present study was designed with these three issues in mind.We offer two multidimensional scaling studies that address these issues in the following way.
1.The goal was to include within the study groups that are as different and that have as little contact as practical.Four cultural-linguistic groups were chosen, two of which spoke a non-Indo-European language.In Study 1, the subjects were native speakers of (a) Polish, (b) Greek, or (c) Estonian (a Finno-Ugric language), all of whom lived in their respective countries.Also, the Greeks lived on a relatively remote island, were um'lingual, and most lacked a college education.In Study 2, subjects were unilingual (a) native speakers of Cantonese Chinese (a Sino-Tibetan language) who resided in Canada, (b) native speakers of Cantonese Chinese who resided in Hong Kong, and (c) native speakers of Greek who resided in a remote village in Greece.
2. The problem of sampling affect is difficult conceptually.All cross-cultural multidimensional scaling studies reported to date have sampled affect by sampling the lexicon.The domain of terms in English for feeling and emotion is not a sharply denned class (Fehr & Russell, 1984), and there is therefore no rigorous way to sample from that class.Just what is the domain of such terms in Estonian, Polish, Greek, Chinese, and any other language is even less clear, and sampling from them is even more problematic.Various strategies for dealing with this problem could be offered, and the one chosen for this study has advantages and disadvantages.We attempted to solve this problem by sampling affect in two quite different ways.In Study 1, we presented subjects with feeling-denoting words.The 28 words used in the Russell (1980) study were translated into Polish, Greek, and Estonian.For Study 2, we presented subjects with 10 facial expressions, 6 of which were from Ekman and Friesen's (1976) Pictures of Facial Affect.
3. Verification of the interpretation of a multidimensional scaling solution and of cross-cultural similarities and differences has been especially neglected.In the present study, we began with a specific hypothesis, the circumplex model of affect, and built on studies of that model.In each cultural group, the same multidimensional scaling procedure was used, and the resulting dimensions were compared through canonical correlational analyses with each other and with the original results obtained with English-speaking subjects.In addition, the dimensions were compared with direct ratings on the hypothesized pleasure and arousal dimensions.

Method
The same procedure was followed in each culture.The 28 English emotion-related words from the Russell (1980) study, which had yielded the results of Figure 1, were translated into the target language.Selection of the final translation for each word (Table 1) was based on translations and back-translations provided by bilingual speakers, plus a bilanguage dictionary.The emphasis, however, was on finding commonly used words in the native language rather than precise equivalents.
The study was carried out entirely in the target language.Native speakers were given the 28 words, written on separate cards.The subject indicated the degree of similarity between each pair of feelings by carrying out a multiple-category sort.Although this measure of similarity is rough, it is simple to use, even for subjects unfamiliar with experimental procedures.The average pairwise similarity matrix was then analyzed with a multidimensional scaling procedure, and the three solutions were compared quantitatively.
Estonian.Estonian-speakers were 50 (22 male, 28 female) high school and university students.The data were gathered in 1979 in Tallinn, the capital of the Estonian Soviet Socialist Republic (E.S.S.R.).Subjects ranged in age from 17 to 45 years old (M =21).All were native speakers of Estonian, although most had studied Russian, Finnish, English, or German in school.Translation was provided by five bilingual Readers of English Langauge and Literature at Tartu State University, E.S.S.R.
Greek.Greek-speakers were 30 (15 male, 15 female) residents of Patitiri, a small Greek village of about 2,000 inhabitants on the island of Alonnisos.The data were gathered in 1984 in Patitiri.Subjects ranged in age from 18 to 48 years old (M = 29) and, in education, from Grade 7 to college diploma (mode was Grade 12 education).None spoke any other language fluently, although most knew a few common words or phrases in one or more other European language.Translation was provided by three bilingual academicians who had spent many years living in English-speaking countries and had been educated at English-speaking universities.
Polish.Polish-speakers were 50 (25 male, 25 female) students in various disciplines at the University of Warsaw.The data were gathered in 1980 in Warsaw.Translation of the 28 English words was provided by several assistant professors in the Department of British and American Literature in Poznan together with a psychologist who is bilingual in English and Polish.
Procedure.Each subject was given a deck of 28 cards, with one emotion term on each, and asked to sort the cards into 4, 7, 10, and 13 groups on separate trials.Order of trials was random.Instructions were in the subject's native language and simply requested that more similar feelings be grouped together.
The similarity of each of all possible pairs of words was assessed for a subject by the number of trials in which the pair was placed in the same group, with the score for each trial (each sort) weighted by the number of alternatives available in that sort.For example, a score of 13 was given to a pair of words placed in the same group during the trial in which the subject sorted into 13 groups.In addition, a score of 1 was added to each pair, for they presumably would have been placed in the same group in a degenerate sort into one group.Thus, minimum similarity was 1; maximum possible similarity was 1 + 4 + 7 + 10 + 13 = 35, which would have occurred had the subject placed the pair in the same group on all trials.This sorting procedure was used because it is an easy and quick task for subjects and has been shown to yield a similarity measure that is a rough but unbiased approximation to that yielded by a paired comparison procedure (Ward, 1977).A final composite similarity matrix for each language was formed by taking the mean entry across subjects for each corresponding cell of the pairwise similarity matrix.

Results
The final similarity matrix for each language was analyzed by the Guttman-Lingoes (Lingoes, 1965(Lingoes, ,1973))SSA-1 multidimensional scaling procedure.This procedure provides a visual them in a geometric space (Euclidean space was used here) of specified dimensionality.Greater similarity between two words is represented by their closeness in the space.
The proper number of dimensions in the solution can be estimated by considering "stress," which is a measure of how poorly the solution accounts for the original data (stress for a perfect fit would be 0).Generally, dimensions are added until stress is low enough and can no longer be improved substantially by adding more dimensions.For all three matrixes, the onedimensional solutions yielded stress values that were high (range, .269-.274).Adding a second dimension substantially improved stress (range, .084-.106).In each case, a weak "elbow" appeared at this point, meaning that the addition of further dimensions failed to improve stress substantially and that a two-dimensional solution was justified.(Stress generally improves slightly with each additional dimension merely by fitting error variance.)We also examined three-and four-dimensional The appearance of similarity between the solutions was confirmed by canonical correlational analyses.Table 2 shows average redundancy indexes for each pair of solutions.A redundancy index can be compared with r 2 , the proportion of vari-ance in one variable accounted for by another.A redundancy index is the proportion of variance in one set of variables accounted for by another set.Unlike r 2 , however, redundancy is not symmetric.The variance in A accounted for by B need not equal the variance in B accounted for by A. We therefore report the average of the two redundancy indexes.For example, the Polish solution accounted for 90.0% of the variance in the English solution; the English solution accounted for 90.2% of the variance in the Polish solution; and the average of these two, 90.1, is the figure shown in Table 2.
The bottom row of Table 2 shows the results of a similar analysis using direct ratings by Canadian college students of stimulus words in English on the dimensions of pleasure and arousal (for details of the rating procedure, see Russell, 1980).
It is tempting to examine Figure 2 for differences between the various solutions.For example, Greek seems to put the cluster relaxed, calm, and so forth closer to sleepy than to happy, whereas Polish and Estonian do the opposite.Although such differences may exist, the measure of similarity used here was a rough one, and according to Table 2, all but a fraction of the variance is common to all the solutions.Further and more focused research would be required to find out whether the differences seen are reliable.

Discussion
Certain limitations of this study must be borne in mind.The present study leaves open the question of the degree to which the set of words obtained in each target language was representative of the domain of feeling-denoting words in that language.•SERENE

• AT EASE
Figure2.Two-dimensional scaling solutions for 28 words.
The present data therefore cannot speak to the question of how salient the pleasure and arousal dimensions are in each language or culture, or what other dimensions are involved in the understanding of emotion.The 28 words with which we began are representative of English, in that when other investigators have chosen other samples of affect terms, pleasure and arousal are the salient dimensions (Averill, 1975;Block, 1957;Bush, 1973;Daly et aL, 1983;Osgood et at, 1975).But whether a similar result occurs in other language communities is an empirical question.Rather than pursue the problem of further sampling of the domain of words, in Study 2 we chose to sample the affect domain in another manner entirely, sampling those feelings expressible via facial expression.
What the present results did show was this: The circumplex with its two underlying dimensions is available to Greek, Polish, and Estonian speakers.This result, together with similar results from Chinese, Japanese, Gujarati, and Croatian speakers, is evidence for the generalizability of the circumplex, however salient that structure may turn out to be.Native speakers of each language, given a sample of 28 commonly used words for affect in their own language, judged the similarities and differences among the feelings denoted in a way that revealed that compari-

Note.
A canonical correlational analysis found in each case that both canonical correlations were significant at alpha equal to .01.Redundancy index is the proportion of variance accounted for in one multidimensional set (criterion) by another (predictor).Another redundancy index results when the roles of criterion and predictor are exchanged.In these analyses, the two indexes were close enough that an average between the two represents them well.
sons were being made in terms of degree of pleasure and degree of arousal.Moreover, individual words did not reflect just one of these dimensions; they did not cluster at one or the other axis, but simultaneously reflected both dimensions operating independently.
Study 2: Facial Expressions of Emotion

Method
The method was identical to that of Study 1, except as noted.
Chinese (Vancouver).Subjects were 30 (11 male, 19 female) Chinesespeaking residents of Vancouver, Canada, ranging in age from 16 to 70 years old (M = 45).The data were gathered in 1984 in Vancouver.Although living in an English-speaking environment, these subjects were elderly residents or recent immigrants who had little or no knowledge of English.
Chinese (Hong Kong).Subjects were 11 (4 male, 7 female) residents of Hong Kong, ranging in age from 25 to 79 years old (M = 46).The data were gathered in 1985 in Hong Kong.
Greek.Subjects were 30 (15 male, 15 female) residents of Patitiri, the village described in the previous study.They ranged in age from 17 to 55 years old (M = 28).The data were gathered in 1987 in Patitiri.One subject was fluent in Italian; none spoke English.
Facial expressions.The stimuli for this study were those studied by Russell and Bullock (1986b).Ten photographs had been selected that represented as broad a range of feelings as possible.To control one possible source of variation, all were photographs of White adult women.Six of the photographs were from Ekman and Friesen's (1976) collection of prototypical expressions of what they consider basic emotions: one each for happiness, surprise, fear, anger, disgust, and sadness.Another four expressions were added from James A. Russell's collection: one each for excitement, calm, sleepiness, and boredom (these were the feelings the actresses had been asked to portray).Although we use these and Ekman and Friesen's labels in referring to the expressions, the labels play no role in the study; the subjects were not given a label for any expression.

Results
The final similarity matrix for each group of subjects was analyzed by the Guttman-Lingoes (Lingoes, 1965(Lingoes, , 1973) ) SSA-1 multidimensional scaling procedure.For all three matrixes, the one-dimensional solutions yielded stress values that were high (range, .190-.320).Adding a second dimension substantially improved stress (range, .022-.051).In each case, an elbow appeared at this point, meaning that the addition of further dimensions failed to improve stress substantially and that a twodimensional solution was justified.The appearance of similarity was again confirmed through canonical correlational analyses.Each multidimensional scaling solution was also compared with pleasure and arousal ratings of the 10 photographs made by English-speaking college students (for details of the rating procedure, see Russell & Bullock, 1985).In all cases, a high proportion (between 90% and 98%) of the variance was accounted for (see Table 3).Indeed, these redundancies were greater than those achieved in Study I, perhaps because here no error variance had been introduced through translation.

Concluding Remarks
Relatively few cross-cultural studies of emotion are available (e.g., Averill, Opton, & Lazarus, 1969;Ekman, 1972;Heelas & Lock, 1981;Izard, 1971;Marsella, 1981;Niit, 1980;Triandis & Lambert, 1958) in spite of a clear need.As Triandis and Lambert observed, Many propositions in the literature of psychology are stated in such a general form that replication with human subjects from very diverse cultural backgrounds is directly relevant.In the present state of our understanding, such replication is probably required whenever a proposition reflects experimental operations dependent upon the more subtle uses of perceptual or language responses.If the proposition is upheld by the test, then the possibility that the relation depends on rare cultural or social factors is rendered less likely.(1958, p. 321) One such proposition is the circumplex model of affect.The different structures of affect reported in studies carried out in different languages-German (Gehm & Scherer, 1988), Swedish (Lundberg & Devine, 1975), Hebrew (Fillenbaum & Rapoport, 1971), Japanese ffeshida et al., 1970), and Ifalukian (Lutz, 1982)-appear to say that a different structure occurs in each language.
The present results demonstrated that the circular ordering and underlying dimensions seen in studies of North American speakers of English emerged from Chinese living in North America or Asia, from Poles and Estonians living in Eastern Europe, and from Greeks living on a relatively isolated island in southern Europe.These subjects also differed in age, education, and social background.Two of the languages the subjects spoke were Indo-European, and two were non-Indo-European.
In one study, emotion was coded in language; in the other, emotion was coded in facial expression.In short, any of the factors of culture, cultural contact, linguistic similarity, medium of expression, and demographic characteristics of subjects could have been, but in fact were not, influences on the basic dimensions in terms of which feelings were compared in this study.An interesting pattern therefore appears.The same results are obtained when the same method is applied in different Ian- guages and cultures.Different results are obtained when different methods are applied in different languages and cultures.By different methods, we mean that the studies by Gehm and Scherer (1988), Lundberg and Devine (1975), Yoshida et al. (1970), Lutz (1982), and Fillenbaum and Rapoport (1971) differed from each other in the way in which affect was sampled, in data analytic decisions, and in how the results were verified.
Conclusions regarding culture and the conceptualization of emotion require that similar methods be applied in all cultures being compared.
One thing in common to the various results reported here was the use of multidimensional scaling, and skeptics might therefore point to multidimensional scaling as the cause of the impressive convergence seen here.Two findings argue against this line of reasoning.First, convergence occurs with other methods, such as factor analysis of mood ratings (Sjoberg et al., 1979;Watson et al., 1984) and of semantic differential ratings (Block, 1957;Osgood et al., 1975).Second, multidimensional scaling need not, and in fact often does not, produce a structure like those seen in Figures 1, 2, and 3.This can be seen in the studies (Fillenbaum & Rapoport, 1971;Gehm & Scherer, 1988;Lundberg & Devine, 1975;Lutz, 1982;Yoshida etal., 1970)  Finally, we wish to avoid several possible misunderstandings.
Results such as those seen in Figures 1, 2, and 3 can be accounted for by somewhat different abstract models.Therefore, let us say that something containing, or something very like, a circumplex model of affect best accounts for our results.In particular, we do not claim that pleasure and arousal are the only dimensions of affect.We cannot yet say that the circumplex is pancultural; too small a sample of human cultures has been tested.The ethnographic work of Lutz (1982)

Figure 1 .
Figure 1.A circumplex representation of feelings; shown is a two-dimensional scaling solution of 28 English words.(From "A Circumplex Model of Aflect" by James A. Russell, Journal of Personality and Social Psychology, 1980, 39, p. 1168.Copyright 1980 by the American Psychological Association.Adapted by permission.) solutions, but failed to find interpretable dimensions beyond the 1st two.Separate two-dimensional SSA-1 solutions are shown in Figure 2. The first (horizontal) dimension contrasts unpleasant states with pleasant ones.The second (vertical) dimension contrasts low with high arousal states.Moreover, terms did not cluster about the axes.Instead, each solution yielded a roughly circular ordering.
Separate two-dimensional SSA-1 solutions are shown in Figure 3.Each solution looked roughly as expected.The first (horizontal) dimension contrasts unpleasant states with pleasant ones.The second (vertical) dimension contrasts low with high arousal states.The 10 photographs, rather than being clustered next to the axes, were spread around the space.
on the Ifalukians and ofGerber (1975) on the Samoans suggests that cultures of the South Pacific may not compare feelings in terms of arousal.Ifalukians, Samoans, and other peoples of the South Pacific should be at the top of the list of targets for cross-cultural studies of affect.Nevertheless, our evidence did demonstrate the generality of the circumplex across a broad range of cultures.It therefore speaks against any theory that puts culture as the sole determinant of how humans conceptualize emotion.Of course, the role of culture is not an either-or matter.Our results do not rule out culture-specific aspects of the human conceptualization of emotion.

Table 1
Translations for 28 Emotion-Related Terms

Table 2
Average Redundancy Indexes for Study 1

Table3
in Average Redundancy Indexes for Study 2