Language Assessments as Shibboleths, A Poststructuralist Perspective

Applied Linguistics 2012: 33/5: 564–581 � Oxford University Press 2012 doi:10.1093/applin/ams052 Advance Access published on 2 November 2012 Language Assessments as Shibboleths: A Poststructuralist Perspective TIM McNAMARA School of Languages and Linguistics, The University of Melbourne, Victoria 3010, Australia E-mail: tfmcna@unimelb.edu.au The biblical story of the shibboleth is widely cited in language testing as emblematic of the social and political function of language tests. But the mean- ing of the shibboleth has also been explored within poststructuralism, specific- ally within Derrida’s discussion of the dilemmas of identity in the work of the German Jewish poet Paul Celan. Derrida discusses language itself as shibboleth, and emphasizes the ambiguity and indeterminacy of the linguistic sign, its ‘un- decidability’. This article considers the implications of Derrida’s argument for the area of language testing in order to see what his interpretation of the shibboleth might mean for understanding its practices, typically framed as they are within a modernist paradigm. Examples are drawn from various areas of language assess- ment, including the use of language analysis in the determination of the claims of asylum seekers. What implications for understanding such assessment prac- tices does a poststructuralist perspective offer? INTRODUCTION In this article, I wish to present a poststructuralist perspective on the practice of language testing. Language testing for a long time saw itself as having a purely practical focus, the construction and validation of useable language tests. Like engineering, it yielded products which deserved to be judged in terms of their practicality, usefulness, and robustness. This enabled language testing to avoid consideration of itself as a social practice. But the social function and meaning of tests has moved more and more into focus, both inside and outside language testing. For example, in a famous passage from Discipline and Punish (Foucault 1977/1975), Michel Foucault identified tests as playing a defining role in modern social organization. Tests and examinations are a principal mode of surveillance in modern societies: The examination combines the techniques of an observing hier- archy and those of a normalizing judgement. It is a normalizing gaze, a surveillance that makes it possible to qualify, to classify and to punish. It establishes over individuals a visibility through which one differentiates and judges them. That is why, in all the mechanisms of discipline, the examination is highly ritualized. In it are combined the ceremony of power and the form of the at CSU Fullerton on D ecem ber 13, 2012 http://applij.oxfordjournals.org/ D ow nloaded from experiment, the deployment of force and the establishment of truth. At the heart of the procedures of discipline, it manifests the subjection of those who are perceived as objects and the objectifi- cation of those who are subjected. (Foucault 1977/1975: 184–185) Within language testing itself, there has been an increasing realization of its character as a social practice, partly provoked by the argument within Messick’s theory of validity (Messick 1989) that all test constructs embody social values, and test validation includes the need to render these explicit and defensible. The exploration of values in language tests, led by the move- ment known as Critical Language Testing (Shohamy 2001, 2006), is continu- ing to develop (Chapelle 2012; McNamara 2012). Poststructuralist thought, as the quote from Foucault suggests, is potentially a current and relevant source to draw on in conceptualizing what is at issue. A site where language testing and poststructuralism comes together expli- citly, if metaphorically, is in the notion of the shibboleth, a topic addressed in both fields. The use of the way a person speaks as an indication of their identity as friend or foe is the subject of a famous passage in the Bible. According to the Biblical story (Judges 12: 4–6), the word shibboleth—a Hebrew word meaning ‘an ear of wheat’, or perhaps ‘a stream’—functioned, as a recognizable dialect feature, as a way of distinguishing friend from foe in a military conflict between speakers of two related dialects. Being identified as a friend through the pronunciation of this word gave protection; identifi- cation as foe spelled death. Although the interpretation of the significance of the shibboleth in the two fields has been different in focus, this article ex- plores the relevance for language testing of the discussion of language as shibboleth in poststructuralism. In what follows I consider first the discussion of the shibboleth within lan- guage testing, and then introduce the discussion of language, and language as shibboleth, in the work of the philosopher Jacques Derrida. Derrida’s under- standing of the social character of language and the vulnerability of the indi- vidual in the face of language is based on, but not restricted to his own experience of social and political violence, themes he pursues in an essay on the 20th century German-Jewish poet Paul Celan, an argument I present in the first part of the article. I go on to ask what implications Derrida’s discussion of the shibboleth might have for language testing. I argue that test scores can be thought of as texts or signs, subject to multiple interpretations, and that validation in language testing involves engaging with this indeterminacy. I consider the responsibilities of language testers in the light of this. THE SHIBBOLETH AND LANGUAGE TESTING The notion of the shibboleth has frequently been cited by writers on language testing (Lado 1949; Spolsky 1995; Davies 1997a; McNamara 2005) as a meta- phor for the social and political functions of tests. Most earlier research on T. MCNAMARA 565 at CSU Fullerton on D ecem ber 13, 2012 http://applij.oxfordjournals.org/ D ow nloaded from language testing, rooted in cognitive and psychometric frameworks, treated tests asocially, failing to acknowledge the social role language tests play, and avoiding issues to do with the question of values in language testing. Discussions of the shibboleth within language testing draw attention to the potential of language tests to be used unjustly, for example within contexts of immigration and citizenship (Extra et al. 2009; Hogan-Brun et al. 2009; Shohamy and McNamara 2009). As a result of this realization, there has been debate on the ethics of language testing and the responsibility of language testers (Davies 1997b, 2004; Kunnan 2004, 2010; Xi 2010). Much of this dis- cussion is designed to make the question of the ethics of test use a manageable issue for test developers. It has focused on procedural fairness, and on how test validation can guarantee the meaningfulness of test scores in terms of a given construct. Where the broader question of what social values underpin test constructs has been addressed, to the extent that it has at all, writers have often simply begged the question, evident in simple exhortations such as the following: ‘A test ought to bring about good in society; that is it should not be harmful or detrimental to society’ (Kunnan 2004: 33). Similarly, Bachman and Palmer (2010) have proposed that among the claims within an assessment use argument (an argument justifying the use of a language test) there will be the following: ‘The consequences of using an assessment and of the decisions that are made are beneficial to all stakeholders’ (p. 105) and ‘The decisions that are made on the basis of assessment-based interpretations take into consider- ation existing educational and societal values and relevant legal require- ments’ (p. 111); see also the related treatment in Xi (2010). These approaches simplify the issue of the highly contested values dimensions of tests. Even where a philosophical basis for considering the values supporting ethical language testing practice has been explored, the philosophical reflec- tion is open to criticism, for example on the grounds that its reliance on the early moral philosophy of Rawls (1971) has not acknowledged recent critiques such as those of Sen (2009) or Sandel (2009), and that it remains relatively narrow in its focus (Davies 2010; McNamara and Ryan 2011). In general, the field has found it easier to consider the consequences of tests than, with the notable exception of Shohamy (2001, 2006), to engage with the question of values in tests. LANGUAGE AS SHIBBOLETH IN POSTSTRUCTURALISM What does poststructuralism say about the shibboleth? It is the basis for a discussion of justice and injustice in language, all the more potent as language is the very ground of personal and social existence. Shibboleth is the title of a poem by Paul Celan, and is the title of an essay (Derrida 2005a/1986) in which Derrida uses a discussion of Celan’s poetry to explore a number of complex themes, including the nature of poetic expression, time as a social con- struction, memory and the Holocaust, the possibility of translation, and the question of identity, including Jewish identity, among others. Of particular 566 LANGUAGE ASSESSMENTS AS SHIBBOLETHS at CSU Fullerton on D ecem ber 13, 2012 http://applij.oxfordjournals.org/ D ow nloaded from relevance here is his discussion of the nature of language as a shared medium and as a site of power. The fact that the means through which individual experience is articulated pre-exists the speaker, is shared, is public, cannot be owned, and is not in the control of the speaker means that the individual faces the task of making unique experiences intelligible in the words of others; the hearer similarly wrestles with the impersonal, shared medium to develop an interpretation of what has been so encoded. In Derrida’s words, language is always in ‘the ear of the other’ (Derrida 1988/1982). Language thus simultaneously permits and frustrates the expression of indi- vidual thought and experience. The individual’s vulnerability in the face of language moreover does not lie in the purely personal realm. The medium itself is saturated with cultural and social discourses, which may work against understanding. When what is being communicated is an experience of suf- fering as a result of social or political values embedded within a culture and its language, then those same social and political values may make it harder for the hearer to hear the experience that is being communicated. The very means by which the individual exists as a social being, language, is also potentially the means by which individual existence is potentially denied, either figuratively, or literally. This is particularly, and painfully, true of communication in the mother tongue. Derrida’s awareness of the potential of language to deny voice to its subjects is rooted in, but not limited to, his own biography (McNamara 2010). He grew up in colonial Algiers in the 1930s as (in his own words) ‘a little black and very Arab Jew’ (Bennington and Derrida 1993/1991: 58) but speaking only French, and neither Arabic nor the traditional Jewish family language, Ladino, a result of linguistic assimilation under colonialism. The ancient indigenous Jewish communities of Algeria were given legal recognition as French citizens in 1870, leading to rapid linguistic and cultural assimilation. In 1940, however, during the period of Vichy rule, this legal recognition was withdrawn and Jews were now ‘recognized’ in very different terms, as a race enemy. Derrida like other Jewish students and teachers was refused access to education in the state school system (Derrida and Roudinesco 2004/2001); his family’s livelihood, security, and ultimate survival were threatened. The resulting paradoxical re- lation to French, the language in which this violence was perpetrated and experienced, is expressed in the following terms: ‘I only have one language; it is not mine.’ I am monolingual. My monolingualism . . . feels like one to me, and I remain in it and inhabit it. It inhabits me. . . . This monolingualism is me . . . I would not be myself outside it. It constitutes me . . . Yet it will never be mine, this language . . . . (Derrida 1998/1996: 1–2) Yet there is no question of simply rejecting one’s mother tongue, the language of the Other: it remains the medium in which one lives, to which one is passionately attached, not only symbolically but also practically. The relationship of the in- dividual to language for Derrida is one of desire. He thus scandalously claims that T. MCNAMARA 567 at CSU Fullerton on D ecem ber 13, 2012 http://applij.oxfordjournals.org/ D ow nloaded from he both ‘suffer[s] and take[s] pleasure in’1 his paradoxical relationship to French (Derrida 1998/1996: 2). Language is simultaneously the bearer of a set of poten- tially inimical discourses and the medium within which our own meaning must be inscribed. French for him is ‘a forbidding as well as a forbidden speech’ (Derrida 1998/1996: 32). How can such a language be a medium of communi- cation for its subjects without it being an experience of a kind of violation? It is only by ‘deforming, reforming, and transforming it’ (Derrida 1998/1996: 33) that its potential for exclusion can be resisted. Derrida’s discussion of Paul Celan identifies similar themes. Although he ex- plicitly disclaims any direct parallel between his own biography and the situ- ation of Celan (‘One cannot, for a thousand all too obvious reasons, compare my experience, my history, or my relationship with the French language to Celan’s experience and history and to his experience of the German language’ (Derrida 2005b: 101)), there are nevertheless clearly significant parallels. Paul Celan was born in 1920 within the German-speaking Jewish minority in Czernovitz, Romania. In 1942, his parents were deported to labour camps in the Ukraine, where his father died of typhus; his mother was shot in the winter of that year as unfit for work. Celan worked in labour camps in Romania until liberation in 1944. He lived most of the rest of his life in Paris until his suicide in 1970. He became the best-known German poet of his generation. His poems are notable for their struggle to express in the language of the perpetrators of the violence, his own language, the violence he had witnessed and been subject to. His best-known poem, Death Fugue (Todesfuge) deals with the theme of the Holocaust directly, and is famous for its refrain: ‘Der Tod ist ein Meister aus Deutschland’—‘Death is a master from Germany’. More typical of Celan’s poetry is an extremely indirect, even cryptic, style, as if the voice that would witness has itself been silenced, and can only express itself with extreme diffi- culty in the language in which the violence has been carried out. Here is a brief example, the opening of a short poem:2 CHYMISCH Schweigen, wie Gold gekocht, in verkohlten Ha¨nden Große, graue, wie alles Verlorene nahe Schwestergestalt: Alle die Namen, alle die mit- verbrannten Namen. Soviel zu segnende Asche. Soviel gewonnenes Land u¨ber den leichten, so leichten Seelen- ringen CHYMICAL3 Silence, like gold, cooked, in coaled hands Great, grey, sisterly shape near like all that is lost: All the names, all those names burnt with the rest. So much ash to be blessed. So much land won above the light, so light rings of souls 568 LANGUAGE ASSESSMENTS AS SHIBBOLETHS at CSU Fullerton on D ecem ber 13, 2012 http://applij.oxfordjournals.org/ D ow nloaded from Derrida speaks of Celan’s relationship with German and compares it with his own relationship to French: Celan’s language is itself a bodily struggle with the German lan- guage, which he deforms, transforms, which he assaults, and which he incises. He wrestles with the body of the German lan- guage. In my own modest way, I do likewise in French. (Derrida 2005b: 168–169) However, by contextualizing Derrida’s struggle with French (and Celan’s with German) within the biography of each writer, there is a risk of suggesting that Derrida’s argument is primarily of biographical or historical interest. But Derrida argues for its general applicability: I knew that what I was saying in The Monolingualism of the Other was valid to a certain extent for my individual case, to wit, a generation of Algerian Jews before the Independence. But it also had the value of a universal exemplarity, even for those who are not in such his- torically strange and dramatic situations as . . . mine. I would ven- ture to claim that the analysis is valid even for someone whose experience of his own mother tongue is sedentary, peaceful, and without any historical drama. (Derrida 2005b: 101) For Derrida, the meanings that language carries are subject to dispute: . . . Each [language] is caught up in its own civil war. There is a hand-to-hand, bodily struggle ‘within’ every national language. (Derrida 2005b: 169) Among speakers of a language, there will be a struggle for meaning, conducted not as it were from outside any particular language, but within language itself, for all speakers of all languages. The individual’s relationship to language in general is therefore paradoxical and contradictory, and inescapable. This dilemma of the protection language simultaneously affords and denies4 is expressed by Derrida in the metaphor of the shibboleth. The word ‘shibbo- leth’, with in this context the extended meaning of a political or ideological slogan, features in Celan’s poem of that name:5 SCHIBBOLETH SHIBBOLETH [. . .] Herz: gib dich auch hier zu erkennen, hier, in der Mitte des Marktes. Ruf’s, das Schibboleth, hinaus in die Fremde der Heimat: Februar. No pasara´n. [. . .] Heart: make yourself known even here, here, in the midst of the market. Call it out, the shibboleth, into the foreign land of the homeland: February. No pasara´n. The references here are to a rallying cry of the anti-Fascists in the Siege of Madrid in 1936 during the Spanish Civil War (No pasara´n),6 and to the brief T. MCNAMARA 569 at CSU Fullerton on D ecem ber 13, 2012 http://applij.oxfordjournals.org/ D ow nloaded from civil war at the end of the period of socialist government known as ‘Red Vienna’ in February 1934 (Februar), prefiguring as they did the struggle against the violence of Nazism. The (possibly apocryphal) use of Shibboleth expres- sions in various contexts of self-defence against Nazi infiltrators during the Second World War (McNamara and Roever 2006) many would see as a legit- imate use of this boundary-marking function of language. Thus, on the one hand, shibboleths may be a means of inclusion and protection: ‘A password, a. . .word transmitted like. . .a handclasp, a rallying cipher, a sign of member- ship and a political watchword’ (Derrida 2005a/1986: 27). But this protective function can also be its opposite: The value of the shibboleth may always, and tragically, be inverted. Tragically because the inversion sometimes overtakes the initiative of subjects, the goodwill of men, their mastery of language and politics. Watchword or password in a struggle against oppression, exclusion, fascism and racism, it may also corrupt its differential value, making of it a discriminatory limit, the grillwork of policing, of normalization, and of methodical subjugation. (Derrida 2005a/ 1986: 30) (Note that the final sentence of this quotation is an explicit reference to the function of tests in the text of Foucault cited at the beginning of this article.) Derrida elaborates on the paradox of the shibboleth, its ‘double edge’,7 using as an example not a linguistic sign of belonging, but a physical signifier, the mark of circumcision as signifying Jewish identity:8 The mark