Reasoning, gesture production,and intelligence

Linguistics 49–1 (2011), 105–134 0024–3949/11/0049–0105 DOI 10.1515/LING.2011.003 © Walter de Gruyter Show your hands — Are you really clever? Reasoning, gesture production, and intelligence* UTA SASSENBERG, MANJA FOTH, ISABELL WARTENBURGER, AND ELKE VAN DER MEER Abstract This study investigates the relationship of reasoning and gesture production in individuals differing in fluid and crystallized intelligence. It combines mea- sures of speed and accuracy of processing geometric analogies with analyses of spontaneous hand gestures that accompanied young adults’ subsequent ex- planations of how they solved the geometric analogy task. Individuals with superior fluid intelligence processed the analogies more efficiently than par- ticipants with average fluid intelligence. Additionally, they accompanied their subsequent explanations with more gestures expressing movement in non- egocentric perspective. Furthermore, gesturing (but not speaking) about the most relevant aspect of the task was related to higher fluid intelligence. Within the gestures-as-simulated action framework, the results suggest that i ndividuals with superior fluid intelligence engage more in mental simulation during vi- sual imagery than those with average fluid intelligence. The findings stress the relationship between gesture production and general cognition, such as fluid intelligence, rather than its relationship to language. The role of gesture pro- duction in thinking and learning processes is discussed. 1. Introduction The aim of this study is to characterize the relationship of reasoning and ges- ture production and their interaction with intelligence. We focus here on ges- tures produced by the hands that represent semantic content, often called rep- resentational gestures. For convenience, we will refer to them as “gestures”. When people engage in conversation, gestures that accompany speech are part of the communication system (Kendon 2005; McNeill 1992, 2005) and they reflect thinking processes (Beattie 2003; Garber and Goldin-Meadow 2002; Goldin-Meadow 2003; Emmorey and Casey 2001). Moreover, recent evidence suggests that gesturing plays a causal role in facilitating reasoning and learning Brought to you by | The Chinese University of Hong Kong Authenticated | 137.189.85.75 Download Date | 8/5/13 5:48 AM 106 U. Sassenberg et al. (Broaders et al. 2007; Chu and Kita 2008; Goldin-Meadow et al. 2009; Wagner Cook et al. 2008). The present paper reviews evidence for the gesture-speech relationship and the gesture-thinking relationship. We argue for a relationship between reasoning, gesture, and intelligence, and we provide new empirical evidence for it. The findings are discussed within the context of the gestures- as-simulated action framework (Hostetter and Alibali 2008). 1.1. Gesture, speech, and thinking The production of gesture and speech is tightly linked. Gestures are temporally and semantically coordinated with speech (Kendon 2005; McNeill 1992, 2005). They are produced while speaking rather than listening (e.g., Saucier and Elias 2001), and the most meaningful part of a gesture, the stroke phase, is synchro- nized with the co-expressive part of speech (McNeill 1992). The development of gesture and speech is also related. For example, children use gesture-word combinations before producing corresponding constructions in speech alone (Özcalıskan and Goldin-Meadow 2005; Iverson and Goldin-Meadow 2005). Furthermore, difficulties in speech are accompanied with adjusted gesture be- havior. Speech disfluencies change the temporal execution of gestures (Sey- feddinipur 2006), gestures are held during stuttering (Mayberry et al. 1998), and verbal deficits in aphasic patients are correlated with gestural deficits (e.g., Duffy 1981). Several theories of gesture production assume that gestures strictly depend on the communicative situation. They are suggested to be primarily produced to communicate (McNeill 1992, 2005; de Ruiter 1998, 2000) or to facilitate speaking (Krauss et al. 2000). However, Kita and Özyürek (2003; cf. also Kita 2000) propose that gestures can be influenced by linguistic properties of the accompanying spoken utterance but are not determined by them. This view can also account for gestures that are not produced for communicative or speaking purposes (for evidence of gestures that do not accompany speech, cf. Chu and Kita 2008; Kessell and Tversky 2005). Also in line with a broader perspective on gesture production is the gestures- as-simulated action framework (Hostetter and Alibali 2008) that will serve as the theoretical background for this paper. This theory views gesture production as not necessarily intended to communicate or to facilitate speech production although gestures usually accompany speech. According to this theory, gesture and speech both are based on the same underlying system of thinking. More specifically, they are based on mental simulation or simulated action in mental imagery (Barsalou 1999; Glenberg 1997; Glenberg and Kaschak 2002). Usu- ally, when a person engages in mental imagery a simulated action is planned Brought to you by | The Chinese University of Hong Kong Authenticated | 137.189.85.75 Download Date | 8/5/13 5:48 AM Reasoning, gesture production, and intelligence 107 but not executed. However, if the activation is sufficiently strong, it can spread from the planning to production stage, and result in an observable movement — a gesture. In other words, if the simulation is very intensive, an action is performed. According to Hostetter and Alibali, the necessary strength of acti- vation for a gesture to be produced is determined by several factors. One such factor is the individual’s neural architecture, for example, the connection strengths between premotor planning and motor production areas that develop due to genetics and experience. Another is the speaker’s gesture threshold, which in turn is assumed to depend, for example, on the gesturer’s level of cognitive effort and beliefs about the current social situation and the use of gestures. For example, if speakers think that gesturing is impolite or that it expresses an inability to verbalize their thoughts, the threshold increases to inhibit gestures. Finally, the gestures-as-simulated action framework con- tends that gesture production is enhanced by the simultaneous involvement of the complex motor demand of speech production. Hence, gestures often ac- company speech. However, is the influence from thinking to gesturing only unidirectional? Recent evidence suggests that producing gestures also affects thinking and learning. Wagner and her colleagues (Goldin-Meadow et al. 2001; Wag- ner et al. 2004) showed that gesturing facilitates memory performance. When participants explained math problems while trying to keep in mind verbal or spatial stimuli, their memory performance increased when they were allowed to gesture compared to when gesturing was prohibited. Also, when participants used gestures spontaneously without any instruction re- garding gesturing, their performance was better in trials in which they ges- tured compared to trials in which they did not. The findings suggest that gesturing affects working memory, that is “the collection of mental processes that permit information to be held temporarily in an accessible state, in the service of some mental task” (Cowan 2005: 77; cf. also Baddeley 1995; Con- way et al. 2005). Gesturing while counting also helps both, children and adults, to keep track and to coordinate the items to be counted and the corresponding number words or functional roles (Alibali and DiRusso 1999; Carlson et al. 2007). Other studies reported that gesturing enhances learning in children (B roaders et al. 2007; Goldin-Meadow et al. 2009; Wagner Cook et al. 2008). Broaders and her colleagues instructed children to gesture while they were learning how to solve a new math problem. As a result, the children tended to indicate new correct strategies in their gestures. Most important, these children learned bet- ter compared to children who were not told to gesture. The results demonstrate that children who have some implicit knowledge about how to solve a problem express it in their gestures, and that this expression in turn facilitates their problem solving process. Brought to you by | The Chinese University of Hong Kong Authenticated | 137.189.85.75 Download Date | 8/5/13 5:48 AM 108 U. Sassenberg et al. Finally, Chu and Kita (2008) showed that gesturing affects adults’ develop- ment of strategies in a mental rotation task. Over the duration of the e xperiment, participants’ gestures developed from first-person pantomimes that expressed how the participants would rotate the objects to gestures that expressed the rotations of the objects themselves (in Chu and Kita’s terms hand-object inter- action gestures and object movement gestures, respectively). According to the authors, this process of deagentivization is important in the development to- wards an efficient strategy in mental rotation tasks. Generally speaking, movement can be expressed in gesture from the first- person perspective or in a more abstracted way showing how something moves. The first is equivalent to character viewpoint and the second sometimes — but not necessarily — corresponds to observer viewpoint (cf. McNeill 1992). Lausberg (2007; cf. also Lausberg et al. 2007) suggests the following terminol- ogy to categorize gestures expressing movements into pantomimes and kineto- graphs. Gesturers expressing their own actions produce pantomimes. They use their hands as if doing the actions themselves. In contrast, when describing how something moves kinetographs depict how it moves (as opposed to how the gesturers move it). The gestures-as-simulated action framework (Hostetter and Alibali 2008: 504) makes predictions about the type of simulation and mental imagery the gesturer is engaged in and about the viewpoint of a gesture where “. . . character-viewpoint gestures are produced as a result of simulated motor imagery (. . .) [and] observer-viewpoint gestures result from simulated visual imagery.” According to the authors, motor imagery always involves simulated action, whereas visual imagery can involve simulated perception and/or simulated action. Especially when engaged in visual imagery of mental transformation, action is likely to be simulated. We will explore the influence of simulated action in motor imagery versus visual imagery in this study and categorize gestures into (character viewpoint) pantomimes that are assumed to result from mental simulation in motor imagery and kinetographs that are sometimes produced in observer viewpoint and that are assumed to result from mental simulation in visual imagery when describing mental transformations. As we have discussed, gestures not only reveal what gesturers are thinking (Goldin-Meadow 2003) but they also influence cognitive processes, such as memory, learning, and reasoning. There are considerable individual differ- ences concerning the frequency of gesture production as studies investigating gesture frequencies report that some participants did not produce any gestures while the rest of them did to different extents (e.g., Melinger and Kita 2007). Although to our knowledge no study has specifically investigated this, some people are assumed to generally produce gestures more frequently than others. Individuals who gesture habitually might be better learners, especially when integrating new information and solving new problems. It follows then that people who habitually gesture more might be better trained for reasoning than Brought to you by | The Chinese University of Hong Kong Authenticated | 137.189.85.75 Download Date | 8/5/13 5:48 AM Reasoning, gesture production, and intelligence 109 people who do not engage in gesturing so much. It is thus an important ques- tion whether individuals with different levels of cognitive ability engage in gesturing to different extents. 1.2. Individual differences and gesture production In terms of individual differences, studies investigated the relationship of spa- tial skills and gesture production (Ehrlich et al. 2006; Hostetter and Alibali 2007). In these studies, spatial skills were measured with tasks involving men- tal transformations that are also central operations in many general reasoning tasks. Gestures are produced predominantly while speaking about spatial con- cepts (Alibali 2005). Thus, individual differences in spatial reasoning are likely to be related to differences in gesture behavior. Ehrlich and her colleagues (Ehrlich et al. 2006) investigated 5-year-old chil- dren’s spatial reasoning performance on a mental transformation task. The children were also asked to explain their strategies after each trial. They talked about several strategies that they also expressed in their gestures. However, they did not always express the same strategy simultaneously in speech and in gesture, making so-called gesture-speech mismatches (Goldin-Meadow 2003). Expressing movement in gestures was uniquely related to correct performance (with or without the same strategy in speech). Thus, children with better spatial skills expressed movement more often in their gestures than children who per- formed less well on the spatial task. Hostetter and Alibali (2007) asked participants to describe a short cartoon video and to describe how to wrap a package. In addition, their spatial skills were assessed with a mental transformation task. Participants with superior spatial skills produced more gestures in the descriptions compared to those with average or low spatial skills. Although the authors did not describe the content of gestures produced in detail, we can assume that a substantial part of the gestures was also expressing movements related to the actions of the char- acters in the video and involved in wrapping a package. In summary, previous evidence suggests that spatial skills in mental trans- formation tasks are positively related to gestures expressing movement. Ac- cording to the gestures-as-simulated action framework (Hostetter and Alibali 2008), many reasoning and problem-solving activities involve the engagement in simulation of perceptions and actions that result in gestures. It is still unclear whether these findings translate to more general cognitive abilities that might be related to mental transformations, such as general fluid and crystallized in- telligence (Horn and Cattell 1966). Fluid intelligence refers to the ability to solve new problems efficiently. I ndividuals with high fluid intelligence are assumed to focus on the central Brought to you by | The Chinese University of Hong Kong Authenticated | 137.189.85.75 Download Date | 8/5/13 5:48 AM 110 U. Sassenberg et al. information and on a limited set of task-relevant cognitive operations (e.g., V ernon 1983). Furthermore, there is a positive relationship between fluid intel- ligence and executive processes of working memory (Engle et al. 1999). Ex- ecutive processes comprise the setting of intentions and planning, selection of relevant information and inhibition of irrelevant information, the generation of strategies, and monitoring. These executive processes, that also play a role in spatial reasoning, are critical components in analogical reasoning. Hence, psy- chometric tests of fluid intelligence usually include analogical reasoning tasks (e.g., Raven Advanced Progressive Matrices, RAPM; Raven 1958). Crystallized intelligence refers to the ability to accumulate, store, and re- trieve knowledge, such as facts and general rules (Horn and Cattell 1966). Therefore, crystallized intelligence might also play a crucial role in reasoning. For example, preexisting knowledge on specific strategies or global rules for solving problems could support performance. How could the use of gestures affect fluid and crystallized intelligence? The development of fluid and crystallized intelligence could benefit from gestur- ing about strategies. Gesturing could help exploring new strategies or consoli- date them within the gesturer’s repertoire. Furthermore, the load on working memory could be decreased by externalizing some of the information that has to be processed. Gesturing could also assist in focusing attention on relevant information or in learning how to access relevant knowledge. However, the relationship between reasoning performance and gestures in individuals dif- fering with respect to fluid and crystallized intelligence is yet unclear. To ex- plore this relationship, we first investigated performance (response times and error rates) in individuals solving a prototypical reasoning task, namely judg- ing geometric analogies. We expected participants with superior fluid and crys- tallized intelligence to outperform those with average intelligence. More spe- cifically, we expected fluid intelligence to predict performance better than crystallized intelligence because fluid intelligence is assumed to be more cen- tral for analogical reasoning (cf. van der Meer et al. 2010). Second, we as- sessed gesture frequencies and gesture types (expressing movement vs. not expressing movement, pantomimes and kinetographs) while individuals re- ported what they experienced to be relevant in solving the geometric analogy task. Based on the literature reviewed above, we expected that gestures ex- pressing movement are produced more often by participants with superior compared to average fluid and crystallized intelligence. We also explored whether this difference could be characterized more specifically. We therefore further distinguished between gestures expressing movement from an egocen- tric perspective ( pantomimes) that are assumed to result from mental simula- tion in motor imagery and those from a non-egocentric, more abstracted, per- spective (kinetographs) that are assumed to result from mental simulation in visual imagery. Brought to you by | The Chinese University of Hong Kong Authenticated | 137.189.85.75 Download Date | 8/5/13 5:48 AM Reasoning, gesture production, and intelligence 111 2. Method 2.1. Participants This study was part of another research project (van der Meer et al. 2010). A subset of fifty-one high school students contributed to the gesture analyses (40 males and 11 females; age [M ± SD]: 16.5 ± 0.5). Because of technical prob- lems, we had to exclude the behavioral data sets of the geometric analogy task from three participants. All participants were right-handed (Oldfield 1971), na- tive speakers of German, and attended the 11th grade of one of three Berlin schools specialized in mathematics and natural sciences. They were paid for their participation. The students and their parents gave written consent before the investigation according to the Declaration of Helsinki of 1964 (World Medical Organization 1996). Three months prior to the experiment, all participants were screened for their fluid intelligence by administering the RAPM (Heller et al. 1998; Raven 1958) and for their crystallized intelligence by administering the subpart ver- bal knowledge of the Intelligenz-Struktur-Test 2000 R (I-S-T, Amthauer et al. 2001). Each participant was assigned to one of two groups based on their RAPM scores and also one of two groups based on their I-S-T scores. The cut- off between the two groups was one standard deviation (15) above the norm (100). This means that for both types of intelligence, participants were as- signed to the superior group if their scores were 115 or above and they were assigned to the average group if their scores were below 115. No participant had scores more than one standard deviation below the norm (i.e., 85). First, four fe