Cloze Procedure as a Variety of Pragmatic Language Testing

1 REVIEW OF THE TYPES OF TESTS

1. Discrete and Integrative Tests

Tests have come to be an accepted component of instructional programs throughout the world. The main reasons for testing are: a) to measure individual progress; b) to assess the degree of success of teaching and learning and to identify areas of weakness and difficulty; c) to measure students probable performance; d) to sort new students into teaching groups so that they are approximately the same level as others when they start. Tests are powerful educational tools that serve at least four functions. First, tests help you weight up students and assess whether they are learning what you are expecting them to learn. Second, well-designed tests serve to motivate and help students structure their academic efforts. Crooks, McKeachie, and Wergin report that students study in ways that reflect how they think they will be tested. If they expect an exam focused on facts, they will memorize details; if they expect a test that will require problem solving or integrating knowledge, they will work toward understanding and applying information. Third, tests can help you understand how successfully you are presenting the material. Finally, tests can reinforce learning by providing students with indicators of what topics or skills they have not yet mastered and should concentrate on [1, p.5]. Despite these benefits, testing is also emotionally charged and anxiety producing. Before writing a test it is vital to think about what it is you want to test and what its purpose is. First of all it would be useful to define some terms used in language testing. Language tests have been divided into four categories based on their use: a) language aptitude tests are used to predict probable success or failure in certain kinds of language study. In other word this kind of testing predicts how a student will perform on a course. b) language achievement tests are used to determine how effective teaching has been, or how much of what has been taught and learned. c) diagnostic tests point out areas in which a student requires additional concentrated teaching and study. To put it more simply such tests highlight the strong and weak points that a learner may have in a particular area. d) language proficiency tests indicate whether or not an individual is proficient enough in a language to perform certain tasks or undertake certain training programs in the target language. In that case a proficiency test is one that measures a candidate's overall ability in a language [2, p. 27]. John B. Carroll (1961) has divided proficiency tests into two types: a) discrete- point tests. b) integrative tests [3, p. 31]. He first proposed the distinction between discrete point and integrative language tests. Though these two are not always different for practical purposes there are two basic differences. a) the theoretical bases of the two approaches contrast markedly; b) predictions concerning the effects and relative validity of the two procedures differ. Discrete tests take language skills apart and attempt to test the knowledge of language one bit at a time. Each test item is aimed at one and only one element of a particular component of a grammar, such as phonology, syntax, or vocabulary. The basic tenet of the discrete-point approach involved each point of language being tested separately. That is to say discrete point tests touch on those areas that linguists include in linguistic competence: vocabulary knowledge, recognition of correct grammatical structure, sound discrimination, etc. – in other words, the mechanics of language [4, p. 20]. We can declare that discrete point test is a common test used by the teachers in our schools. Having studied a grammar topic or new vocabulary, having practiced it a great deal, the teacher basically gives a test based on the covered material. This test usually includes the items that were studied and will never display anything else from a far different field. The same will concern the language skills; if the teacher’s aim is to check reading skills; the other skills will be neglected. Integrative tests in contrast to discrete-point tests are intended to tap the total communicative abilities of second language learners- i.e. those factors crucial to how a language is used for communicative purpose [5, p. 45]. Integrative tests put the skills back together and attempt to assess learner’s capacity to use many bits all at the same time and possibly while exercising several presumed components of a grammatical system and perhaps more than one of the skills or aspects of skills. Alderson poses that, by and large, most teachers prefer using integrative testing to discrete point type [6, p.219]. He explains the fact that basically the teachers either have no enough of spare time to check a certain split item being tested or the purpose of the test is only considered to view the whole material. Some well-known integrative tests are the cloze test, dictation, listening and reading comprehension, and oral interviews. Examples of discrete-point tests are grammar, vocabulary, and auditory discrimination tasks. It should be noted that labeling a given grammar or vocabulary test as discrete-point does not mean that all such tests are necessarily discrete-point in nature. For example, one could develop a grammar test which is highly integrative. The shift from discrete-point to integrative tests is not yet fully accomplished; so, they still tend to compete as separate avenues of testing. And, at the same time, another trend in the development of language proficiency testing has emerged. It was soon realized that not all integrative tests were assessing the communicative abilities of the students, nor were all of them as true to life as had been assumed. Consequently, in line with the pragmatic approach to teaching, Oller introduced the pragmatic testing approach. He defines it more precisely: any procedure or task that causes the learner to process sequences of elements in a language that conform to the normal constrains of that language, and which required the learner to relate sequences of linguistic elements via pragmatic mapping to extra-linguistic context [7, p87] . In other words, he believes that pragmatic tests should meet two requirements: first, they must require context, i.e., the meaning requirement; and second, they must require the processing to take place “under temporal constraints”, i.e., the time requirement. Oller concludes that integrative tests are a much broader class of tests which are usually pragmatic, but pragmatic tests as a subclass of integrative tests are always integrative.

1.2 The Notion of Pragmatic Test

Nowadays we have many definition of the term “pragmatics”. For example:
David Crystal supposed that “pragmatics studies the factors that govern our choice of language in social interaction and the effects of our choice on others” [8, p.231]. George Keith thought that “Pragmatics is all about the meanings between the lexis and the grammar and the phonology...Meanings are implied and the rules being followed are unspoken, unwritten ones” [9, p.32]. We know without asking that there are no babies are for sale - that what is for sale are items used for babies. Pragmatics allows us to investigate how this “meaning beyond the words” can be understood without ambiguity. The extra meaning is there, not because of the semantic aspects of the words themselves, but because we share certain contextual knowledge with the writer or speaker of the text. Steve Campsall suggested the following definition: “pragmatics is an important area of study for your course. A simplified way of thinking about pragmatics is to recognise, for example, that language needs to be kept interesting - a speaker or writer does not want to bore a listener or reader, for example, by being over-long or tedious. So, humans strive to find linguistic means to make a text, perhaps, shorter, more interesting, more relevant, more purposeful or more personal. Pragmatics allows this ” [10, p.141]. Pragmatics is a systematic way with the help of it you can explain the language use in context. Its aim is to explain aspects of meaning which cannot be found in the obvious sense of words or structures, as explained by semantics. As a field of language study, pragmatics is fairly new. Its origins lie in philosophy of language and the American philosophical school of pragmatism. As a discipline within language science, its roots belong to the work of (Herbert) Paul Grice on conversational implicature and the cooperative principle, and on the work of Stephen Levinson, Penelope Brown and Geoff Leech on politeness. To sum up, we may say that the area of language that embraces the functional use of language in social contexts is called pragmatics. The main categories of pragmatics include [11, p.7]: a) speech act theory. We use language all the time to make things happen. We ask someone to pass the salt or marry us - not, usually at the same time. We order a pizza or make a dental appointment. Speech acts include asking for a glass of beer, promising to drink the beer, threatening to drink more beer, ordering someone else to drink some beer, and so on. Some special people can do extraordinary things with words, like baptizing a baby, declaring war, awarding a penalty kick to Arsenal FC or sentencing a convict. Linguists have called these things “speech acts” - and developed a theory (called, unsurprisingly, “speech act theory”) to explain how they work [12, p.85]. b) felicity conditions. These are conditions necessary to the success of a speech act. They take their name from a Latin root - “felix” or “happy”. They are conditions needed for success or achievement of a performative. c) conversational implicature is a message that is not found in the plain sense of the sentence. The speaker implies it. d) the cooperative principle is the way in which people try to make conversations work e) conversational maxims that in ordinary conversation, speakers and hearers share a cooperative principle. Speakers shape their utterances to be understood by hearers. The principle can be explained by four underlying rules or maxims. Relevance speakers' contributions should relate clearly to the purpose of the exchange [13, p. 41]. f) politeness is the form of behaviour that establish and maintain comity. That is the ability of participants in a social interaction to engage in interaction in an atmosphere of relative harmony [14, p.36]. g) phatic tokens These are ways of showing status by orienting comments to oneself, to the other, or to the general or prevailing situation h) deixis concerns the ways in which languages encode...features of the context of utterance ... and thus also concerns ways in which the interpretation of utterances depends on the analysis of that context of utterance [15, p. 123]. The main aspects of language studied in pragmatics include: a) deixis: meaning 'pointing to' something. In verbal communication however, deixis in its narrow sense refers to the contextual meaning of pronouns, and in its broad sense, what the speaker means by a particular utterance in a given speech context. b) presupposition: referring to the logical meaning of a sentence or meanings logically associated with or entailed by a sentence. c) performative: implying that by each utterance a speaker not only says something but also does certain things: giving information, stating a fact or hinting an attitude. The study of performatives led to the hypothesis of Speech Act Theory that holds that a speech event embodies three acts: a locutionary act, an illocutionary act and a perlocutionary act. d) implicature: referring to an indirect or implicit meaning of an utterance derived from context that is not present from its conventional use [16, p. 65]. Pragmatics refers to the underpinnings of conversation: how something is said, the intentions of the speaker, the relationship between the participants, and the cultural expectations of the exchange. It is, by its nature, a complicated and elusive part of communication. Pragmatics is concerned with the relationship between linguistic contexts and extralinguistic contexts. Oller distinguished two major kinds of context [17, p. 124]. First, there is physical stuff of language which is organized into more or less linear arrangement of verbal elements skillfully and intricately interrelated with a sequence of rather precisely timed changes in tone of voice, facial expression, body posture, and so on. To call attention to the fact that human beings even the latter so-called “paralinguistic” devices of communication are an integral part of language use, we may refer to the verbal and gestural aspects of language in use as constituting linguistic context. According to these terms linguistic context and verbal context may be used interchangeably. A second major type of context has to do with the world, outside of language, as it is perceived by language users in relation to themselves and valued other persons or group [18, p. 204]. We will use the term extralinguistic context to refer to states of affairs constituted by things, events, people, ideas, relationships, feelings, perceptions, memories, and so forth. The objective aspect of extra linguistic context, the world of existing things, may be distinguished from the subjective aspect of extra linguistic context, the world of self - concept and inter-personal relationships. Neither linguistic nor extralinguistic contexts are simple in themselves, but what complicates matters still further and makes meaningful communication possible is that there are systematic correspondences between linguistic contexts and extralinguistic ones. That is, sequences of linguistic elements in normal uses of language are not haphazard in their relation to people, things, attitudes, etc., but are systematically related to states of affairs outside language. Thus we may say that linguistic contexts are pragmatically mapped onto extralinguistic contexts. Nevertheless not all researchers single out pragmatics as a separate field of language. A traditional criticism has been that pragmatics does not have a clear-cut focus, and in early studies there was a tendency to assort those topics without a clear status in linguistics to pragmatics. Thus pragmatics was associated with the metaphor of ‘a garbage can' [19, p.252]. Other complaints were that, unlike grammar which resorts to rules, the vague and fuzzy principles in pragmatics are not adequate in telling people what to choose in face of a range of possible meanings for one single utterance in context. An extreme criticism represented by Marshal was that pragmatics is not eligible as an independent field of learning since meaning is already dealt with in semantics. However, there is a consensus view that pragmatics as a separate study is more than necessary because it handles those meanings that semantics overlooks. This view has been reflected both in practice at large and in meaning in interaction: an introduction to pragmatics by Thomas. Thus in spite of the criticisms, the impact of pragmatics has been colossal and multifaceted. The study of speech acts, for instance, provided illuminating explanation into sociolinguistic conduct. The findings of the cooperative principle and politeness principle also provided insights into person-to-person interactions. The choice of different linguistic means for a communicative act and the various interpretations for the same speech act elucidate human mentality in the relevance principle which contributes to the study of communication in particular and cognition in general. Implications of pragmatic studies are also evident in language teaching practices. Deixis, for instance, is important in the teaching of reading. Speech acts are often helpful for improving translation and writing. Pragmatic principles are also finding their way into the study of literary works as well as language teaching classrooms. “Pragmatics is about how people communicate information about facts and feelings to other people, or how they merely express themselves and their feelings through the use of language for no particular audience, except possibly an omniscient God” [20, p. 407]. Quite often we know much more than what we actually express in words. We also leave a lot of it unsaid and we depend on the receiver to fill in what is unsaid and interpret our message. “In normal use of language, no matter what level of language or mode of processing we think of, it is always possible to predict partially what will come next in any given sequence of elements. The elements may be sounds, syllables, words, phrases, sentences, paragraphs, or larger units of discourse. The mode of processing may be listening, speaking, reading, writing, or thinking, or some combination of these. In the meaningful use of language, some sort of pragmatic expectancy grammar must function in all cases”. The assessment of pragmatic language skills is, necessarily, a difficult and complex challenge for speech-language pathologists. Because of the nature of pragmatics, it is almost impossible to construct a standardized test that accurately captures the essence of social communication. Past attempts at doing so have resulted in tasks that actually assess underlying linguistic or cognitive skills rather than true social communicative functioning. At this time, there are no reliable and valid standardized tests of pragmatics for the preschool population. However, the assessment of pragmatic development is necessary to understanding a child’s competence in language use. In that the development of pragmatics in preschool and school age children has been well documented by research, that research can be used as a guide in assessing the pragmatic language of children. The above discussions indicate that pragmatic tests like dictation and cloze can be used effectively to test language proficiency. These two tests are valid and reliable measures of language use. They are practical because it is not very difficult to prepare the material, administer the test, score the test and interpret the results. They also have diagnostic value, and error analysis based on the results could offer suggestions for further instructional programmes.

1. Factive and Emotive Aspects of Language Use

John W Oller Jr. discusses pragmatic tests and criticizes the discrete point tests at length and offers an alternative in the form of pragmatic tests and suggests ways of constructing, administering and scoring pragmatic tests. To understand his concepts it is necessary to get a clear picture of his views on language proficiency, expectancy grammar and pragmatics. According to Oller there are two aspects of language use:
a) factive use – when language is used to convey information about people, things, events, ideas and states of affairs. It is coded primarily in distinctly verbal sequences.
b) emoitive use – when language is used to convey our attitude about the factual information we want to convey. It’s coded primarily in gestures, tone of voice, facial expression, and the like. Whereas verbal sequences consist of a finite set of distinctive sounds, syllables, words, idioms, and collocations, and generally of discrete and countable sequences of elements, the emotive coding devices are typically non-discrete and are more or less continuously veriable. In fact, Watzlawick, contend that emotive messages concerning such abstract aspects of interpersonal realities are probably much more important to the success of communicative exchanges than the factively coded messages themselves. Every time we use language, we use both the aspects of language. It is quite possible for people to agree on the factual information conveyed but differ on the attitude towards those facts.

2. Expectancy Grammar

The term expectancy grammar calls attention to the peculiarity sequential organization of language in actual use. Natural language is perhaps the best known example of the complex organization of elements into sequences and classes, and sequences of classes which are composed of other sequences of classes and so forth. The term pragmatic expectancy grammar further calls attention to the fact that the sequences of classes of elements and hierarchies of them which constitute a language are available to the language user in real life situations because they are somehow indexed with reference to their appropriateness to extralinguistic contexts [21, p.165]. According to Oller the notion of an expectancy grammar characterizes the psychologically real system that governs the use of a language in an individual who knows that language. The characteristic of such an expectancy system helps in two ways: a) to explain why certain kinds of language tests apparently work as well as they do; b) to device other effective testing procedures that take account of these salient characteristics of functional language proficiency. What is very interesting for a theory of pragmatic expectancy grammar is that in normal communication, easy of expressing attitudes are nearly perfectly coordinated with ways of expressing factive information. The demonstrated coordination of mechanisms that usually code factive information and devices that usually code emotive information shows that the anticipatory planning of the speaker and the expectations of the listener must be in close harmony in normal communication. Moreover, from the fact that they are so synchronized we may infer something of the complexity of the advance planning and hypothesizing that normal internalized grammatical system must enable language users to accomplish. Static grammatical devices which do not incorporate an element of real time would seem hard put to explain some of the empirical facts which demand explanation. Some sort of expectancy grammar, or a system incorporating temporal constraints on linguistic contexts seems to be required. A valid language test should press the learners’ internalized expectancy system into action and must further challenge its limits of efficient functioning in order to discriminate among degrees of efficiency. According to Oller, a language test to be valid should meet the pragmatic naturalness criteria. A test is said to meet the pragmatic naturalness criteria when it invokes and challenges the efficiency of the learners’ expectancy grammar by causing him to process temporal sequences in the language that can conform to normal contextual constraints and by requiring him to understand the systematic correspondences of linguistic and extralinguistic contexts.

3. Pragmatic Test Types

There are two closely interrelated criteria of construct validity, which need to be imposed on language tests to find out if they meet the pragmatic naturalness criteria. The test must cause the learner to processes – either produce or comprehend or possibly to comprehend, store and recall, or some other combination – temporal sequences of elements in the language that conform to normal contextual constraints which may be linguistic constraints or extra linguistic constraints. The test must require the learner to understand the pragmatic interrelationship of linguistic contexts and extralinguistic contexts. Some tests which meet the pragmatic naturalness criteria are as follows: a) dictation requires examinees to write verbal sequences of material as spoken by an examiner or played back from a recording b) cloze is the sort of test that is constructed by deleting every fifth, sixth word from a passage of prose. c) combined cloze and dictation – A cloze passage is prepared deleting single words, parts of words or sequences of words and given to the learner in the written form. The same passage, without any deletions, is heard by the learner either live or on tape. The learner is asked to complete the cloze passage with the help of what he hears. d) oral cloze procedure – A cloze passage with deletions is dictated for the learners to copy down and complete the deleted portions with appropriate words. e) dictation with interfering noise – Taped verbal material is super imposed with `white noise`. If the linguistic context under the noise is fully meaningful and subject to the normal extralinguistic constraints, this procedure qualifies as a pragmatic testing technique. For example, if it is a telephone conversation with one party calling from a market place, the noise superimposed will have sounds of the market place. f) paraphrase recognition – Learners are asked to read a sentence, paragraph or longer passage and then select from several alternatives the one, which best represent the central meaning or idea of the given piece. g) question answering – Four answers are given for each question. Learners listen to the question orally and select the best answer. There may be a short dialogue followed by a question. Four or five answers are given and learners select the best answer. This type of test is used in TOEFL examinations. h) bilingual syntax measure – A picture or cartoon is shown and learners are asked some questions based on the picture. Burt, Dulay and Hernandez made learners answer questions based on colourful cartoon style pictures. i) oral interview – Where learners are asked questions as would happen in a normal interview for a job or admission to a course. j) composition or essay writing k) narration is one of the techniques sometimes used successfully to elicit relatively spontaneous speech samples. l) translation – which is similar to the one taking place in a real life context [11, p. 45]. Having discussed the nature and types of pragmatic tests in general, it would be useful to consider cloze in more detail because it meets the criteria like validity, reliability and practicality, being easy to construct, administer and score.

2. Theoretical Basis of the Cloze Test Procedure

Cloze procedure first appeared in the work of Wilson Taylor, who studied the effectiveness of cloze as in instrument for assessing the relative readability of written materials for school children in the United States. Research then turned to the utility of cloze as a test reading proficiency among native speakers. During the sixties, studies were also done on the value of cloze as a measure of overall ESL proficiency. As noted in Brown, this literature on cloze as a measure of overall ESL proficiency has been far from consistent [22, p. 109]. The results of such studies, especially the reliability and validity results, have varied greatly both within and among studies. Readability is a property of the text itself and predicts the education level typically needed for people to read the content without undue difficulty. Comprehension or validity is a combined property of the text and a specific user segment and indicates whether this target audience actually understands the material's meaning. Cloze procedure is a technique in which words are deleted from a passage according to a word-count formula or various other criteria. The passage is presented to students, who insert words as they read to complete and construct meaning from the text. Cloze tests require the ability to understand context and vocabulary in order to identify the correct words or type of words that belong in the deleted passages of a text [23, p. 414]. The cloze test is a common empirical comprehension test. This task requires students to build an internal representation of the text, to put the words together in a meaningful way, so that they will be able to interpolate what words belong in the blanks. Cloze is considered an integrative rather than a discrete point test because it draws at once on the overall grammatical, semantic and rhetorical knowledge of the language. To reconstruct the textual message, learners have to understand key ideas and perceive inter – within a stretch of continuous discourse and they have to produce, rather than simply recognize an appropriate word for each blank. The focus of the task involved is more communicative than formal in nature and it is therefore considered to reflect a person’s ability to function in the language. According to Moller, both cloze procedure and paragraph writing are integrative in nature. They require the learner to draw upon several language skills simultaneously and involve complex processing of language while the focus is on content. Both require the production of language rather than mere recognition of correct items, although writing may be considered to include the communicative dimension more directly [24, p. 21]. There is some disagreement among researchers about the suitability of cloze to test high – order language abilities. According to Alderson cloze provides a measure of core linguistic skills of a relatively low order [25, p. 193]. According to a number of other researchers like, Chihara, Oller, Weaver and Chavez, Bachman and Brown, cloze procedures can test not only lower – order linguistic skills, but also higher – level ability involving discourse constraints across sentences. Oller gives detailed instruction for the selection of cloze passage and for scoring procedure.

1.3.1 The Application of Cloze Procedure

Cloze procedure has been used in English teaching for decades since 1953. It was firstly used as an instrument for assessing the readability of written materials for school children in the United [26, p. 79]. Then it was used in teaching for different purposes. There are some major purposes for using cloze procedure in English teaching. Cloze procedure can measure the difficulty of a text. As a student, I understand only too well the problems caused by the difficulty of some expository materials. Rye explains that if the language is too difficult, the task will lead to frustration and an increasing lack of confidence about reading. Therefore, it is necessary for teachers “… to assess the difficulty of the material which they expect their pupils to read”. Cloze procedure has turned out to be a subtle instrument measuring readability. It is used to assess the difficulty of textbooks and other reading materials. As Rye claims: cloze procedure can measure the difficulty of a text, not in terms of word length or familiarity, or of sentence length, but in terms of a particular individual’s understanding of, and response to, the language structure of the text. Cloze procedure measures a personal response to linguistic variables [27, p. 83]. Rye and Kilfoil and Van der Walt point out how particular cloze scores have been adopted and used as criteria for establishing the readability and comprehension level of texts. The classification of readability level was derived from Betts’s ‘reading comprehension level’, which has been used by many researchers, e.g. Harris and Sipay, Bormuth, Rankin et al. cited in Rye and Legenza and Elijah . Betts used three levels to classify readers: a) independent level is the level at which a student reads the text with excellent comprehension and fluency without instructional interventions. Vacca, Grove have described the independent reading level as a recreational reading level because students are very motivated by and interested in materials at this level. It seems logical that interest and motivation would be very high since students can and do read these materials unassisted and successfully b) instructional level is the level at which the student can improve her reading skills with appropriate instruction; therefore, it is critical that a teacher know this information. This is the cutting edge of instruction where the material is challenging to a student but not out of reach. Knowing the instructional reading levels of a group of students, a teacher can successfully group children for differentiated instruction c) frustration level is the level at which a child may be unable to decode and comprehend the majority of the text. This material is simply too demanding to be useful instructionally. Finally, although assessment such as the cloze procedure does provide the teacher with reading levels for a group of children, it is important to remember that reading levels are not static and change constantly. Therefore, caution is urged when grouping children for instruction. Grouping should always be flexible [28, p. 351]! If learners get 90 percent in the multiple-choice cloze test – the text is considered suitable for learners at the Independent Level and would not require much teacher guidance; between 75 and 89 per cent – the text is considered suitable for learners at the Instruction Level and requires some teacher guidance; and less than 75 percent – the text is considered to be at the Frustration Level and would be too difficult without teacher guidance. Cloze procedure is not only valuable in providing information as to how readable the particular texts are for the students who are going to have to use them, but also invaluable as a means of comparing different texts. Cloze procedure is also used for testing purposes. It seems that a wide range of skills like vocabulary, grammar, structure, and reading skills are involved in the process of completing a cloze procedure. Many researchers indicate that cloze procedure is a good test of overall English language proficiency. For example, Ahluwalia claims that cloze procedure is an integrative, global measure of language competence. She explains that cloze tests measure the grammar of expectancy underlying the skills of thinking, understanding, speaking, reading and writing [29, p. 81]. For Cohen cloze procedure measures global language competence consisting of linguistic knowledge, textual knowledge, and knowledge of the world. As it calls on testees to use knowledge such as vocabulary, grammar, sentence construction, text structure, cohesion and the reader’s prior background knowledge. Askes regards cloze procedure as one of the integrative tests (global tests) that integrate language components into a total language event, which requires an integrated performance from the learner in a meaningful context [30, p. 36]. Thus for Steinman, a cloze can replace the sections on test structure, written expression, vocabulary, and reading comprehension [31, p. 291]. Researchers such as Rashid conclude that cloze tests are reliable for measuring the language proficiency of ESL students. Cloze tests are also important parts of foreign language tests. International scholars recognize cloze procedures as a very efficient means of measuring integrative English language competence. The ‘Integrative language competence’ involves the skills of thinking, speaking, reading, understanding and writing. English teachers accept the effectiveness of cloze procedure as an instrument testing integrative English language competence almost without question, which might explain why cloze procedure is so popularly used in various English exams, both formal and informal [32, p. 68]. However, Ahluwalia and Steinman acknowledge that many researchers criticize cloze procedures for merely involving the ability to utilize redundancy in a passage. That is, they view cloze procedure as sentence-bound, requiring readers to look only at the immediate sentence to figure out the word. Urquhart and Weir question the reliability and validity of cloze as a device for testing global comprehension of a text, since cloze procedures delete words, rather than phrases or clauses, which do not usually carry textual cohesion and discourse coherence and thus focus readers’ attention on individual words to the detriment of global understanding [33, p. 315]. The third use of cloze procedure in English teaching is as a teaching instrument to help improve learners’ language ability. Many researchers and teachers have used it successfully in improving learners’ language ability. Helfeldt, Henk and Fotos mention cloze procedure as ‘passage-completion’ technique, which is an informal instrument to determine learners’ instructional reading level. Knowing the actual reading level of the learners, teachers can adjust and give guidance to the learners more properly [34, p.216]. Lombard describes the use of cloze tests in her English second language classes for junior and senior students. She illustrates how cloze tests help solve reading problems of learners and increase their confidence when they receive immediate and satisfactory feedback [35, p. 21]. Legenza and Elijah claim that cloze is effective as a teaching technique especially when teaching is based on an error profile. For example, one such method is using teacher-developed cloze exercises to remedy specific error types, e.g. deleting only one part of speech at a time, which students then have to insert so that the sentences are semantically correct. Steinman describes her use of cloze procedure as a teaching instrument for students to practise using context clues as a reading strategy and to encourage vocabulary improvement in teaching. In her teaching, the methods of teacher-made cloze texts, rational deletion and accepted word scoring were combined with the negotiation of comprehension of the text and discussion of the word choice after the completion of cloze texts. Cloze exercises are widely used by many teachers in English classes, but they seem to be less effective than the teachers expected. They expect cloze exercises to improve their students’ language ability, but the repeated cloze exercises fail to improve this significantly. This situation may be the result of the teachers’ inappropriate use of cloze procedure. Because of the teachers’ inappropriate use of cloze procedure, students regarded cloze exercises as practice for cloze tests in examinations, rather than exercises that related to their practical reading and writing abilities. Cloze procedure is acknowledged by many as a means of assessing reading ability. Rye confirms the usefulness of cloze procedure in measuring reading ability while acknowledging its weaknesses as a reading test. Many researchers recognize that cloze procedure correlates with other reading tests. Neville claims that if the readers of the cloze texts are so confident in their reading that they can restore the exact words used by the author (or synonyms), they must be competent readers [36, p. 26] . Cloze tests had been popularly used in some standardized reading tests to test reading ability in the 1970s. Rye lists these standardized cloze tests which are published in the United Kingdom, such as GAPADOL Reading Comprehension, Wide-span Reading Test, and the London Reading Test. Scholars claims that the processes of cloze procedure reflect learners’ success in reading. Reading requires learners to utilize clues available in language, and cloze procedure requires a similar ability [37, p. 42]. Some researchers emphasize that cloze procedures measure reading comprehension. Helfeldt, Henk and Fotos claim that the traditional cloze test has enjoyed twenty-five years of use as a measure of reading comprehension ability. Research has shown that cloze performances are closely related to the performances on other reading comprehension tests, such as multiple-choice questions and simple answer questions. Rye cites the result of Taylor’s experiment that there is an eighty-five percent overlap between the cloze test and the comprehension test. From this point of views, cloze tests can be considered reliable in testing reading comprehension. Gunning also claims that cloze procedure can be used as a comprehension test (assessing comprehension). As he puts it, “If you have not comprehended what you’ve read, you have no basis for filling in the blanks” [38, p. 344]. Because the words are omitted, the readers are forced to pay close attention to meaning. Neville contends, “If we can restore correctly the words omitted from a passage, we must have a good grasp of its comprehension”. Yamashita claims that the result of his study supported the claims of some other researchers that cloze procedure can be used to assess reading comprehension [39, p. 285]. However, as Urquhart and Weir point out, some researchers doubt the validity of cloze procedure as a device for testing global comprehension of a text, because it appears to focus a reader’s attention on local words and sentences rather than on a piece of connected discourse. Cloze procedure is also used to assess reading strategies used by readers or to develop learners’ reading strategies. Because some words are deleted, cloze forces readers to be more aware of the meaning and calls for the use of reading skills like scanning and searching that are often neglected in second language reading. Ahluwalia considers the process of taking a cloze test as involving more active reading rather than passive reading and readers should be more conscious of their reading strategies during this process. As Brown points out, cloze tests are based on contextualized written language; they need readers’ conscious and effective use of reading strategies. Rye seems to support the above opinions that the normal flow of the reading process is interrupted in cloze procedure; readers have to use reading strategies more flexibly. Fyfe and Mitchell, mention their use of cloze exercises in the formative assessment of reading strategies in secondary schools. The cloze exercises were used as the assessment task, which they introduced into the natural reading task [40, p. 9]. Gunning concludes that cloze is an essential teaching technique that fosters reading for meaning and use of context, especially effective for students who fail to read for meaning. To fill in the gaps correctly, students must have a good grasp of the meaning of the text as a whole. Processes involved include the use of reading strategies such as previewing (read the whole text before starting the cloze test), predicting (activate background knowledge to make a guess about the information contained in the text), using context clues about the deleted word in the surrounding context, using language knowledge to select appropriate lexical items and proper grammar forms for the gap. More particularly, Rye points out that cloze procedure develops scanning and search reading abilities. He cites the authors of the manual for the Edinburgh Reading Test who argue that when facing a cloze passage, students will employ the reading skills of searching for information from the surrounding syntactic and semantic circumstance, re-reading the whole passage (not necessarily following the order of the author’s sequence), looking for clues that their memory tells them are present, and looking for material that may provide a basis for a guess. Rye argues that the reader must use evidence derived from the context and scan for unspecified information. Cloze tests are valid and reliable measures of how far a reader is able to predict the language sequence of a text. Rye declares: When parts of the main content of the text are removed, namely when certain nouns and verbs are deleted, the resulting test can be used to assess the reader’s understanding of the content of the text. Although all cloze tests inevitably assess an understanding of content, the removal of only certain content words frees the reader from having to concentrate on syntax and puts a greater emphasis on the actors and their states, processes and actions. His declaration seems to be supported by many researchers. For example, Yamashita argues that cloze can be used to measure readers’ higher-order skills relating to cohesion and coherence of a text. Harrison and Gilliland, cited in Greene, claim that cloze procedure measures readers’ grasp of ‘between sentence meanings and relationships’ and their ability ‘to use a variety of contextual interrelationships’ [41, p. 247]. Greene claims the importance of ‘seeing the connections’ would seem to make the cloze task particularly appropriate for evaluating the coherence of a student’s comprehension [42, p. 82].

1.3.2 Selecting Materials for the Task

The cloze procedure is probably appropriate to just about any text. True, some may be more difficult than others, but it has been demonstrated that for some purposes the level of difficulty of the task does not greatly affect the spread of scores that will be produced. This does not mean that a teacher or other educator would ever want to use a cloze test without some sense of the level of skill of the students for whom the test was intended, but it does not mean that fairly wide latitude can be tolerated by the technique. If the purpose is to assess the readability of a certain book for children at a certain grade level, then the sampling technique that would be most appropriate would be one that selected portions of the text most representative of the entire book. According to Klare and Stolurow, the technique that best assures representativeness is a random one. For instance, if the text is long, say more than a few thousand words, samples may be taken at intervals by some arithmetic procedure. If a book is divided into chapters, it might make sense to take at least one sample from each chapter, or one from each of ten equal portions of textual material. Following the general guidelines suggested earlier, portions of running text long enough to produce tests of at least fifty blanks each should be constructed over each sample, and at least twenty-five subjects should be tested on each cloze passage [43, p. 77]. If the purpose is to assess overall language proficiency of students in a foreign language classroom, the most appropriately gauged material would depend on the level of the students in the class. However, unless the material used in the teaching of the language is quite idiomatic and natural, it would probably be best to select a passage from a book intended for a comparable level of native speakers of the language in question. For example, if the students are studying English as a foreign language, it may be well be that reading material intended for fifth grade geography students would be usable for cloze material [44, p. 61]. Some of the things to be avoided are texts that involve topics that are intrinsically disturbing or so emotionally charged that they would distract the attention of the students from the main problem set by the test – filling in the blanks. Passages that require esoteric or technical knowledge not generally available or which is available only to some of the students should also be avoided. Texts that contain arguments or statements that some of the students may strongly disagree with(e.g. texts that state strong pro or con views on controversial issues such as capital punishment, abortion, religion, politics) should also probably be avoided. Passages that do not contain enough words of running text to provide a sufficient number of blanks (about 50) should probably be set aside in favor of longer texts. Of course, there may be special circumstances where any or all of the foregoing suggestions should be disregarded. For instance, it would make no sense to avoid using a short text when it is that same short text that has been studied. It would not make sense to avoid a religious topic when the class is a course on religion, and so forth [45, p. 96].

1.3.3 Scoring Procedure

The test will usually provide some meaningful discrimination even if it is poorly judged in difficulty level. The scoring technique, of course, may greatly affect the difficulty level of the test. A test that is too difficult by the exact word technique may not be too difficult at all if a more lenient scoring technique is used. The most common method of scoring is the exact word method. The rationale behind the exact word scoring method goes back to Taylor. He reasoned that the ability of a reader to fill in the very word used by writer would be a suitable index of the degree of correspondence between the language system employed by the writer and that employed by the reader. However, it can be argued that requiring the reader to replace the very word used by the writer is too stringent a requirement. It certainly places a severe limit on the range of possible responses. In fact, it would keep the creative reader from supplying an even better word in some cases. For instance, “As a matter of fact, his _____ started before he even left home”. The correct word by the exact word criterion is ‘problems’. Other alternatives which seem equally good, or possibly better by some standards, such as ‘difficulties’, ‘troubles’, ‘trials’, or ‘worries’, would have to be marked wrong – just as wrong, in fact, as such clearly off the wall fill-ins as ‘methods’, ‘learning’, or worse yet ‘of’, ‘be’, or ‘before’. Many writers would have preferred to use some word other than ‘problems’ to avoid a repetition in two consecutive sentences. Nonetheless, in spite of the fact that it does seem counter-intuitive to give more credit for the very word the author used at a particular point in a text than for some equally or possibly even more appropriate word, this whole line of argument is easily countered by the fact that the exact word method of scoring cloze tests correlates so strongly with all of the other proposed methods. However, if one is interested in specific performances of examinees on particular items, much information may be lost by the exact word technique. It would seem that some of the other methods have some instructional advantages and there is evidence to show that they are somewhat more strongly correlated with other proficiency criteria than the exact word method is. Another method is scoring for contextual appropriateness. The main guideline of this method is to count any word fully fits the total surrounding context as correct. It is simply to say, count the exact word or any synonym for it as correct. The notion of synonym makes sense in relation to some nouns, verbs, adjectives, and adverbs, but all words have their synonyms. For example, the word ‘even’ in the sentence ‘He couldn’t even arrive on time’ can be replaced on the words such as ‘just’, ‘quite’, but they are not synonyms of ‘even’. Thus, the general guidelines of this method is, first, see if the word in a given blank is the exact word, so count it correct; second, if it is not the exact word, check if it fits the immediately surrounding context, any local constraints in the same sentence or surrounding phrases; third, see if it is consistent with all of the preceding and subsequent text (this includes previous and subsequent responses in other blanks as filled in by the examinees). If the response passes all of these checks, score it as correct. Of course some responses seem to be more appropriate than others. In general, it seems that responses which violate local constrains are more serious errors than responses that violate longer range constraints. In fact, several schemes for weighting responses for their degree of conformity to contextual constraints have been suggested and in fact used. There are two kinds of weighting systems. The first is based on distinction between types of errors in relation to some linguistic analysis of the text, and the second is based on the performance of a group of native speakers on the same items. In the latter case, the frequency of occurrence of a given word in response to a given cloze item by a specified reference group(e.g., native speakers) is used as a basis for assigning a weighted score to responses to each item made by a test group(e.g., non-native). The first method of weighting scores is based on an analysis (whether implicit or explicit) of the text. Categories of errors are distinguished along the following lines. The most serious is the sort that violates the strongest and most obvious contextual constraints. The least serious is the sort that violates the weakest and least obvious contextual constraints. If a response violates constraints in its immediately surrounding context, it will usually violate longer range constraints as well, but this is not always the case. It is possible, for instance, for a response to fit the sense of the text but to violate same local constraint – e.g., the use of a plural form where a mass noun has been deleted, as in ‘the peoples of the world’ instead of the ‘people’. If groups of people were the intended signification, the first form would be correct, but if all the people of the world in the collective sense were intended, the second form would be more correct. In either case, the sense of the text is partly preserved even if the wrong morphological form is selected. A scale of degrees of appropriateness can be exemplified best with reference to a particular cloze item in a particular context. All of the foregoing systems of scoring tests for adult foreign students studying English as a second language were investigated by Oller. The result showed all of the methods to be either equivalent or slight inferior to a method that counted categories a and b worth 1 point and all the rest worth nothing – i.e., for scoring purposes, differentially weighting degrees of appropriateness had no significant effect on the overall variance produced. Not only were all of the methods investigated highly correlated with each other, but all were about equally correlated with scores produced by other language tests. The contextual appropriate method yielded slightly better results than the exact word method, however. Thus it is probably safe to conclude that complex weighting systems do not afford much if any improvement over the simpler methods of scoring. One other system of weighting responses deserves discussion. It is the technique referred to as ‘clozentropy’ by D. K. Darnell. No doubt Darnell’s system is the most complex one yet proposed as a basis for scoring cloze tests, but it has a great deal of intuitive appeal. Moreover, if computer time and programming expertise are available, the technique is feasible – at least for research purposes. The reason for discussing it here is not to recommend it as a classroom procedure, but rather to show that it is roughly equivalent to simpler methods that are more appropriate for classroom use [46, p. 36]. Darnell was interested in testing the English skill of non-native speakers – in particular, foreign students at the University of Colorado at Boulder. Because of the negative results reported by Carroll, Carton, and Wilds, Darnell shied away from the exact word scoring method, and opted to develop a whole new system based on the responses of native speakers. He administered a cloze test to native speakers (200 of them) on the assumption that the responses given by such a group would be a reasonable criterion against which to judge the adequacy of responses given by non-native foreign students to the same test items. He reasoned that the responses given more frequently by native speakers should be assigned a heavier weighting than less frequently responses. Without going into any details, Darnell showed that his method of scoring by the clozentropy technique generated surprisingly high correlations with the Test of English as a Foreign Language. The only difficulty was that Darnell’s method required about as much work as the preparation of an objectively scorable test of some other type. In short, the virtue of the ease of preparing a cloze test was swallowed up in the difficulty of scoring it. Fortunately, Pike showed that there was a very high correlation between Darnell’s scoring technique and an exact word scoring technique. Both scoring methods produced strong correlations with the TOEFL [47, p. 214]. This latter result has been further substantiated by Irvine et al. It follows that if both exact and contextually appropriate methods are substantially correlated with each other, and if exact and contextually appropriate scores as well as exact and clozentropy scores are strongly correlated with TOEFL, it follows that exact, contextually appropriate, and clozentropy scores are all strongly correlated with each other [48, p. 248]. From all of the foregoing, we may conclude that except for special research purposes there is probably little to be gained by using a complex weighting scale for degrees of appropriateness in scoring cloze tests. Either an exact word or contextually appropriate method should be used. The choice between them should be governed by the purpose to which the scores are to be put. If the main intent is to use only the overall score, probably the exact word method is best. If the intent is to look at specific performances on particular test items or in particular student protocols, probably the contextually appropriate scoring should be used.

1.3.4 Types of Cloze Tests

. The standard cloze test is defined as an appropriate passage of around 250 words with every 7th word deleted. According to the psycholinguistic classification of the test forms from the test taker perspective, the standard cloze test would be categorized as comprehension/production type test in written mode. Since research on cloze tests has demonstrated highly acceptable psychometric characteristics, i.e., reliability and validity, scholars have conducted research to develop different versions of the cloze procedure. Through one line of research, scholars attempted to develop varieties of cloze on the basis of the psycholinguistic processes going on in the minds of the test takers. Instead of having comprehension and production activities that are required by the standard cloze, they tried to construct cloze tests which required only comprehension. Thus, the production part, i.e., finding out a suitable word for each blank and writing it, was eliminated. Removing the production part from the requirements of a cloze test led to the development of multiple-choice type cloze tests. Through another line of research, testers concentrated on the modality of the cloze test, i.e., the mode of language in which cloze is presented. Contrary to the standard cloze in which the language is in written form, scholars developed oral cloze tests. Through the third line of research, the test developers investigated the possible varieties due to the content, form, and the number of blanks in the cloze test [49, p. 225]. Each of these lines of research led to the development of varieties of cloze to be discussed below: a) comprehension type cloze tests. The most common type of cloze test that aims at measuring students’ comprehension ability is the multiple-choice type test. All the characteristics of the standard cloze test are applied to the multiple-choice type as well. The difference, however, lies in the fact that in multiple-choice cloze, students are not required to produce a set of words to complete the passage. Rather, they are provided with three, four, or five choices for each blank and are required to select the most appropriate word from among the given choices. The following is an example of a multiple-choice cloze test based on the sample passage: Education has two goals. First, it (provides/gives/offers) means so that we may earn (our/the/some) living. Some of these tools are (basic/main/major) tools, such as reading or writing. (There/Some/Others) are technical skills, such as (accounting/management/engineering) and data processing. Still others are (all/highly/more) professional skills peculiar, for example, to speaking/medicine/reading), to science, and to teaching. Secondly, (it/education/they) provides experience so that we can (know/learn/understand) how to live. Education provides (background/knowledge/science) of ideas for using the past. (It/Education/Science) provides a sense of values for (forming/knowing/meeting) the future. The advantage of the multiple-choice cloze over the standard cloze is that the task of comprehension is easier than that of production. Therefore, a multiple-choice cloze would be easier, and thus, students’ will obtain scores that are higher than on an open-ended form [50, p. 291]. Of course, in a norm-referenced situation, the increase at the level of the scores would not influence the ranking of the students, i.e., all testees will score higher on the multiple-choice form than they will on the open-ended form. In a criterion-referenced situation, however, where there exists a predetermined criterion for the students to meet, low scores would hurt those at the borderline. This shortcoming can be overcome by developing certain adjustment formulas to equate the scores obtained from multiple choice and open-ended forms. The disadvantage of multiple-choice cloze test is the difficulty of its development. To construct a multiple-choice cloze test, the open-ended form should be administered to a group of testees similar to the target group in order to elicit the distractors. Through this procedure, from among the wrong responses produced by the testees, the most frequent ones are selected to form the distractors for each blank. Of course, pretesting adds to the burden of the test developers. However, the ease of scoring, the extent of the objectivity, and the degree of practicality of the multiple-choice cloze tests make using such tests worthwhile. b) oral type cloze tests. Another major variation of cloze tests relates to the instrumentality. That is, standard cloze and multiple-choice cloze tests are both in written form. The oral cloze test, however, is given in oral form. To administer an oral cloze, the text is read to the students with pauses at the blanks. The students are required to write a word, which they think is appropriate to be written after the pause. Of course, this type of test puts a heavy load on the short-term memory of the test takers. Research on cloze test shows high correlations between oral cloze and other forms of cloze tests indicating that oral cloze is as valid and reliable as other types of cloze tests. This implies that the oral cloze test is as good a measure of language ability as the written cloze is. However, it might have some limitations regarding its practicality. That is why oral cloze did not gain popularity among the testers.
Each of the three major types of cloze tests, i.e., standard, multiple-choice, and oral has many varieties which can be used both for teaching and testing purposes. Below is a brief description of these varieties that can be applied to each type: 1) fill in the blank. This form of the cloze test is adapted from the traditional fill-in-the-blank type activities. The difference is that traditional type activities were often sentence based, whereas a cloze activity involves a passage with every nth word deleted. The deleted words are listed and the testees are required to fill in the blanks with the appropriate words from the list (Appendix A). In this form, the testees should read the text and understand the context. Then based on the context, they should select and write the appropriate words in each blank. 2) deleting a particular class of words. In the standard cloze procedure, when a value is assigned to n, the deletion procedure follows through the text. That is every nth word is deleted. This method, in which the value of n is fixed and constant, is called the fixed-ratio deletion. Sometimes, the testers, specially the teachers, are interested in deleting a particular set of words to check the students' ability in using these words. In such cases, the value of n cannot be fixed because the blank may correspond to a word that should not be deleted. Therefore, teachers use random ratio deletion procedure. That is, n is not fixed and it is determined by the test developer prefers to delete a particular word. In this version, which is most useful for instructional purposes, a particular group of words is deleted. For instance, the teacher might be interested to see the extent to which the students have command over the use of prepositions, articles, verb forms, adverbs, adjectives, etc. Thus, instead of deleting every 7th word, or whatever it might be, one or two of the above-mentioned classes of words are deleted. In such cloze activities, known as cloze tasks rather than cloze tests, certain useful diagnostic pieces of information can be obtained. The passage given as the example will have the following form when all the propositions are deleted (Appendix B). Although this type of cloze does not usually serve as a testing device, it eliminates one of the major shortcomings attributed to cloze tasks that they do not provide diagnostic information. Utilizing this version would enable the teachers to have a quite reasonable idea of the students’ strengths and weaknesses on a predetermined element of language. Through the findings from these tasks, teachers can plan remedial work to alleviate the weak points of the students in different areas.
In educational settings, it is sometimes necessary to check the students’ ability to provide more than a single word for a given blank. This necessity is particularly felt in grammatical constructions, especially in tenses. For example, in most tense constructions, a modal such as will, would, etc. and auxiliary verbs such as “be” or
“have” are present. Considering the various forms of these modals and auxiliary verbs, cloze tests can be constructed to check the students’ ability using the chunk of language rather than just a word. 3) catered technique. In some versions of cloze, to facilitate the testees’ task, some pieces of information related to the deleted words are given to provide a cue. Depending on the kind and extent of cueing, certain varieties have been developed. In its simplest form, it resembles a spelling test known as the letter cloze. In this version, one or more letters of the word in the blank are left out and the testees are required to fill in the blanks with the missing letters. Consider the example passage in which two letters are deleted from each word. Education has two purposes. First, education provides tools so that w__ can earn a living. Some of t__es__ tools are basic tools, such as r__ad__ng and writing. Others are technical skills s__c__ as typing, accounting, and data processing. S__m__ others are highly professional skills peculiar, f__ __ example, to medicine, to law, and t__ teaching. Secondly, education provides experience so t__ __t we can learn how to live. E__ provides a background of ideas for u__in__ the past. It provides a sense of values for meeting the future. This form can be made more difficult by reducing the cue. In its most difficult form, only the initial letter of each word is provided and the rest of the word is to be provided by the testees. In this form, the example passage would have the following form: Education has two purposes. First, education provides tools so that w___ can earn a living. Some of t___ tools are basic tools, such as r___ and writing. Others are technical skills, s___ as typing, accounting, and data processing. S___ others are highly professional skills peculiar, f___example, to medicine, to law, and t___ teaching. Secondly, education provides experience so t___ we can learn how to live. I___ provides a background of ideas for u___ the past. It provides a sense of values for meeting the future. This version is often used to check and improve the spelling ability of the students. These varieties can be utilized at different levels with varying degrees of difficulties. 4) random deletion procedure. Another version of cloze that is also based on the rate of the deletion of the words is called random deletion procedure. In this version, there is neither fixed ratio deletion nor deleting a particular class of words in the mind. Words are deleted on the basis of pure randomization. To construct such a test, all the words in the passage are numbered and every number is written on a piece of paper. These pieces of papers that carry the numbers corresponding to the words in the passage are located in a hat. Then the numbers are drawn out of the hat one by one. The word that corresponds to every number is deleted from the passage. Although random deletion procedure conforms to the principles of randomization in research and removes some of the criticisms made on the selection of an arbitrary number for n, it has a drawback. Since words are deleted on the basis of the randomly drawn numbers, there is a possibility of having two or more adjacent words deleted. This would complicate the task of the test takers. Of course, this problem can be alleviated by drawing the numbers and deleting the words providing that no adjacent words would be deleted. 5) the cloze-elide test. A different variety of cloze test is known as the cloze-elide test. In this type, instead of deleting a set of words from the passage, a certain number of words are added to the text. The testees are required to read the text and identify the extra or redundant words and cross them out. In fact, in the cloze-elide test, the processing of the text is somewhat opposite to that of the standard cloze test. In the cloze test, the testees should read the text and add some words; whereas in the cloze-elide test, the testees should read the text and delete some words. Since the cloze-elide test is different from the varieties of the cloze test mentioned here, certain points should be taken into account in developing and using a cloze-elide test. The first point is the selection of the passage. In selecting the passage, all the rules and regulations regarding the difficulty and the length, discussed before, should be observed. That is, the selected passage should have appropriate length and difficulty in respect to the expected group of testees. The second point is determining the locations where the extra words should be inserted. The most appropriate method of inserting the words in the text is the random insertion procedure. That is, the words in the passage should be numbered. Then the numbers should be randomly selected and the redundant word should be inserted after or before the word that corresponds to the randomly selected number. The third point is the selection of the words to be inserted in the passage. Again, a good procedure is selecting certain words from the dictionary on a random basis. Of course, some scholars believe that the inserted word should be similar to the adjacent words. However, the difference between inserting words selected randomly and the words inserted logically needs empirical evidence. In the sample passage presented before, there are 12 blanks. Therefore, twelve words should be selected randomly. Following a random insertion procedure, the passage would look like the following. Education has two goals. First, it provides very means so that we may earn the our living. Some of these tools are actual main tools, such as reading or writing. Others may are technical skills, such as typing, quite accounting, and data processing. Still others are would highly professional skills peculiar, for example, to such medicine, to science, and to teaching. Secondly, it is provides experience so that we can learn the how to live. Education provides knowledge that of ideas for using the past. It after provides a sense of values for the meeting the future. In this passage, underlined words are inserted. They are redundant and the examinees should detect them and cross them out. Of course, they are underlined here to inform the reader. Otherwise, in the original test, the inserted words are written in exactly the same form as the other words in the text. In scoring the cloze-elide test, the words that are detected and crossed out correctly will be given credit. Sometimes, testees cross out the words that are not redundant and are parts of the text. In such cases, the testees are penalized and one point is deducted for any error. The cloze-elide test as another variety of cloze is an interesting cognitive exercise and it can be used both as a test and as a class activity.

1.3.5 Modification of Cloze Test

The C-test was developed 10 years ago by Christine Klein- Braley and Ulrich Raatz in Duisburg as a modification of the cloze test. Since then this language-testing instrument has gained wide popularity because of its high reliability, sufficient validity, and remarkable practicality. Various studies have revealed that C-tests are integrative and highly reliable and valid measures of general language proficiency [51, p. 134]. The C-test is an integrative testing instrument that measures overall language competence, very much like the cloze test. It consists of four to six short, preferably authentic, text in the target language, to which ‘the rule of two’ has been applied: the second half of every second word has been deleted, beginning with the second word of the second sentence; the first and last sentences are left intact. If a word has an odd number of letters, the “bigger” part is omitted, for example “proud” becomes “pr__”. One letter words, such as “I”, are ignored in the counting. The students’ task is to restore the missing parts. In a typical C-test there are 100 gaps- that is, missing parts. Only entirely correct restorations are accepted. By way of illustration, here is a short C-test: “One cool autumn evening, Bob L., a young professional, returned home from a trip to the supermarket to find his computer gone. Gone! All so__ of cr__ thoughts ra__ through h__mind: H__ it be__ stolen? H__ it be__ kidnapped? H__ searched h__ house f__ a cl__ until h__ noticed a sm__ piece o__ printout pa__ stuck un__ a maga__ on h__ refrigerator do__. His he__ sank a__he re__ this sim__ message: CAN’T CONTINUE, FILE CLOSED, BYE.” The C-test appears to have some advantages over the cloze test: a) as students are confronted with a variety of short passages, a better sampling of content areas is possible. Also, a person with special knowledge in a certain field does not have an unfair advantage; b) by “damaging” every second word, we can obtain a more representative sample of all the different language elements in the text than in the cloze, where normally only every fifth or sixth word is left out; c) many more items can be included in much shorter texts, making the test less time-consuming for the students than the cloze; d) the general language proficiency is measured more accurately; e) unlike the cloze test, scoring is easy and objective, as there is only one acceptable solution in most cases; f) as a rule, students actually like doing C-test, whereas the cloze test is one of the most frustrating test types for learners; g) students can easily design C-test for each other, which could be made into a game or competition. To sum up, the C-test is an excellent testing method, as it provides a good and quick assessment of general language competence. It is also useful: a) to select and place students in appropriate groups; b) to assess their achievement at the end-of-term exams by selecting several typical passages from the term’s materials; c) to test certain grammar areas (e.g., tenses or word formation) by including texts that contain several examples of the structures in question; d) to check home reading or homework by taking passages from the texts the students had to work on; e) to measure the specialized knowledge of ESP groups by choosing suitable texts from their particular field of specialization.

Conclusion on Part 1

From the educational perspective, language testing is central to language teaching. It not only influences the goals for language teaching, but also monitors success in achieving the pre-determined goals. Furthermore, language testing supplies a methodology for experiment and investigation in both language teaching and language learning. In this part we have tried to clarify classification of the tests, to explain the term ‘pragmatics’ and ‘pragmatic tests’, to explain that not all tests have pragmatic naturalness. We also gave classification of cloze tests and may conclude that cloze tests are marvelously adaptive because: a) cloze tests are based on contextualized written language; b) it is not difficult to get people to try taking a cloze test because of the human need to fill gaps (closure) which appears to be almost a compulsion among students; c) students must predict in a manner similar to how they must predict in the reading process; d) in a sense, cloze challenges students with a semi-random selection of language items; e) students will then only correctly answer those items at their level of ability; f) cloze tests administered to students of different ability levels will automatically be testing different things because only those items that at least some of the students can answer will discriminate. It is hoped that language teachers will begin experimenting with cloze testing as a viable option to the traditional tests which are normally administered in schools. So, teachers should remember that cloze test is a pragmatic task that develops different students’ reading, writing, comprehension skills, knowing of vocabulary, grammar, spelling.

2 PRACTICAL USAGE OF CLOZE TEST PROCEDURE IN PRAGMATIC LANGUAGE TESTING

2.1 Cloze Test as Pragmatic Tasks

Cloze procedure is a technique in which words are deleted from a passage according to a word-count formula or various other criteria. The passage is presented to students, who insert words as they read to complete and construct meaning from the text. This procedure can be used as a diagnostic reading assessment technique. The cloze procedure is a multifunctional strategy, which can be used for reading diagnosis, reading comprehension instruction, and improving different skills. The concept of closure refers to the human tendency to fill in a missing word in a piece of text or a familiar language pattern. In addition, every classroom teacher has all the skills needed in order to create, administer, use and interpret this strategy making it very teacher-friendly [52, p. 239]. Diagnostically, it is an effective way to determine how a reader is utilizing the context of a sentence or paragraph to get the meaning of the context. In addition, once the diagnostic function of a cloze test is done, the same text can be used instructionally making this a truly multipurpose strategy. Instructionally, the cloze procedure develops inferential comprehension. Students focus their attention on clues in the text, and relate this information to prior knowledge. The cloze procedure forces the reader to activate background knowledge, look for context clues and make reasonable inference. This process requires the reader to make word level predictions but attend to the text as a whole. The cloze procedure requires that students use the background knowledge of a subject, understand basic word-order relationships (syntax), understand word and sentence meaning in context (semantics), and apply decoding strategies in order to read with comprehension. The cloze results give the teacher information about text difficulty, students’ reading level, and students’ prior knowledge and language competence. This knowledge can be used in planning instruction and in selecting reading materials. Cloze procedure has many purposes, which are divided into two types, according to instructional and diagnostic points of view [53, p. 21]. Instructionally, the cloze procedure may be used: a) to develop students’ self-monitor for meaning while reading; b) to develop students’ critical and analytic reading skills; c) to develop students’ awareness of their reading process; d) to facilitate student self-awareness of their reading process; e) to develop students’ ability to activate their prior knowledge of a subject; f) to develop students’ metacognitive skills. Diagnostically, the cloze procedure may be used to: a) to determine the extent to which students are monitoring for meaning while reading; b) to identify students’ knowledge and understanding of the reading process; c) to determine which cueing systems readers effectively employ to construct meaning from print; d) to assess the extent of students’ vocabularies; e) to assess students’ use of prior knowledge of a subject. The multidimensional nature of the cloze procedure makes it applicable in all content areas and grade levels. For example, initial cloze activities could involve students in predicting and naming missing parts of objects, puzzles, pictures and sets. The teacher could use a modified cloze procedure for class and morning messages. As students develop their reading skills the cloze procedure can be used to introduce or conclude units of study in all subject areas. Finally, in addition to deletion of words, the cloze procedure can be modified as a post reading activity in that following the reading of a story or book, teachers can summarize the story deleting main events or ideas. Students read the story and supply the missing events and details. Therefore, the adaptability and applicability of the cloze procedure makes it an invaluable instructional and diagnostic strategy [54, p. 245]. If used appropriately, cloze procedure can be used to help learners improve their reading competence. According to Rye, many researchers have proven the reliability of standardized cloze tests in this regard. He argues that teacher-made cloze tests can also be validly and reliably used in some situations. He suggests that teachers should use cloze procedure to construct their own reading tests. These need not necessarily replace professionally developed tests, but “… there are situations where teacher-made cloze tests could be adequately used”. Rye reports that cloze exercises produced significant gains in reading ability. He cites Rankin’s report, which explains that there is no distinct different score between Rankin’s own cloze tests and standardized reading tests on the same material. That is, teacher-made cloze tests measure the same abilities measured by conventional tests [55, p. 237]. It is worth pointing out that only the proper use of cloze procedure can improve the reading ability of the learners. Rye concludes that cloze is effective when the instruction is carefully sequenced in length and difficulty, and adjusted to the reading abilities of the students. As Steinman puts it, when preparing a cloze, we choose to delete those items that force the readers to read beyond the sentence. We should provide a cloze training for our students, during which we emphasize the value of reading the whole text before focusing in on the deletions [56, p. 61]. Therefore each teacher should make his/her purpose clear for using cloze. For example, the teacher can gain an understanding of particular problem areas in learners’ reading ability by analyzing the errors learners made in the cloze passage. Rye argues that performances on the cloze procedure reflect the learner’s ability to use the language context. A cloze exercise is especially valuable for students who neglect context clues. Rye argues that cloze procedure also helps in developing the ability to infer implicit meaning from a cloze passage when using group discussion activities. “All through the discussion a thoughtful awareness of the context has underpinned that discussion”. In such activities cloze procedure makes students read carefully and reason together meaningfully about their choice of word to fill the deletion. This helps them to think about the context beyond the immediate linguistic segment. The teacher can help the learning process to continue after the group discussion by giving credit for appropriate, as well as correct responses, and by explaining possible reasons behind the author’s choice of words [57, p. 159]. The teacher should also know the advantages and weaknesses of cloze procedure. Rye claims that cloze procedure has the advantage that it does not require a teacher to ‘sit in’ on the whole reading process, but just to check the learners’ progress at his/her convenience. However, working on short passages would produce frustration when students are unfamiliar with cloze procedures. Therefore, some instructions are necessary for students before a cloze exercise starts. For example, Gunning advises, “…give them (students) some tips for completing it (cloze exercise)”. Rye holds that “Students should have the chance to become acquainted with the technique (the cloze procedure)”. In addition, Rye argues that the cloze passage should be related to learners’ interests or expected areas of interest to avoid increasing the difficulties [58, p. 121]. The incorrect use of cloze procedure fails to help improve reading abilities. Rye summarizes such improper use, such as exposing children to cloze procedure for comparatively short experimental periods; giving the pupils no training in how to use context clues; and asking the subjects to fill in cloze passages without involving them in any group discussion or feedback from the teacher. Because the less competent reader finds cloze disproportionately difficult, Fyfe and Mitchel suggest that cloze exercises could be used more appropriately in a mixed ability class if cloze formats were modified and different kinds of cloze exercises were set, based on the same passage [59, p.221]. The cloze procedure used by teachers shows many of the problems listed above. Some of the teachers gave short periods (e.g. a month, several weeks) for students to practise cloze exercises just before the examinations. Some gave students a cloze exercise to finish without giving proper examples and an explanation of the skills to be employed in this process. Some just asked students to practise by filling in different cloze passages repeatedly and gave only the correct answers rather than explaining them or allowing for group discussion. These inappropriate uses of cloze procedure do not help to improve students’ language ability but increase their frustration with cloze tests. In preparing the test the following procedures are used: a) select a self-contained passage of a length appropriate for the grade level of the students being assesses; b) leave the first and last sentences and all punctuation intact; c) select the words for omission using a word-count formula, such as every fifth word or other criteria. In traditional cloze passage construction, every nth word – whatever it may be – is deleted. But in updated versions, the teacher may choose to delete specific words in students’ knowledge of the topic or their abilities to use semantic cues, thus you may delete content words that carry meaning, such as nouns, main verbs, adjectives and adverbs. To assess students’ use of syntactic cues, delete some conjunctions, prepositions and auxiliary words. When preparing the final draft of the passage, make all blanks of equal length to avoid including visual clues about the length of omitted words; d) students should be advised to read the entire passage before they fill in the blanks; e) teacher should encourage the students to fill each blank if possible; f) teacher should suggest that students reread the completed passage; g) finally, although there is no time limit for this exercise, the student’s time of completion should be noted since it may shed some light on student reading behavior that may be significant in the overall analysis of reading skills and abilities. As with any instructional activity, the teacher must model the completing cloze passages. The cognitive coaching model is a recommended approach. Cognitive coaching and modeling helps students understand the teacher’s reading process public. Students see how the teacher uses language patterns, syntactic structures and word recognition skills when reading. Students see the teacher use prior knowledge of the topic to read for meaning and to predict content and ‘think through’ reading tasks. This process of thinking through or ‘thinking-aloud’ shows students how a reader comprehends text [60, p. 29].

2.2 Description of the Experiment

The purpose of this experiment is to show that the cloze procedure is a necessary technique in studying foreign languages, to examine how a cloze procedure works for a group of secondary school students at a relatively high proficiency level, and for a group of students with relatively low proficiency, and to convince that the using of cloze procedure as a variety of pragmatic language testing helps to improve reading skills, to develop writing skills, to refine on comprehension skills, to use correctly grammar and vocabulary, to improve abilities of memory. Learning a language can be separated into two aspects: language usage and actual language use. Proficiency in language usage means knowing the linguistic rules of the language, and having the ability to produce grammatically correct sentences either in writing or speaking. Proficiency in language use means being able to use the language appropriately in different situations for effective communication. The participants of this research with high proficiency level were eight pupils from the eighth grade of lyceum ‘BEST’. We selected these pupils because their level of English knowledge confirm to that type of exercise. They were willing to do the cloze test, able to answer our questions and complete a questionnaire specially developed for the purpose. We chose standard or fixed-ratio type of cloze test where the every fifth word is deleted. First of all, I have prepared an interview. The interview is one way of investigating a group’s attitudes and opinions. I used semi-structured interviews, because I needed the students’ views and opinions and I wanted them to express freely what they wanted to say about the topic. Nunan states, “in a semi-structured interview, the interviewer has a general idea of where he/she wants the interview to go, and what should come of it, but does not enter the interview with a list of predetermined questions”. In other words, the interviewer determines the topics and issues rather than the order of the questions of a semi-structured interview. Semi-structured interviews can produce rich information because of their flexibility. The interviews were conducted before the cloze test [61, p. 186]. I designed and composed the main questions for the interviews. Then I translated the questions into Russian, because I believe using Russian could reduce the interviewees’ anxiety and avoid obstructing the flow of the interview due to the interviewees’ limited English proficiency. They could obviously express themselves with greater ease and fluency in their first language. Before conducting the interviews, I explained the purpose of my study to the interviewees. I also explained clearly to them that the information I got from them would be used for academic purposes only. Then I asked the interviewees for permission to take notes of their answers. Once I had their permission, I conducted the interviews, taking detailed notes of their answers, which were translated into English afterwards. I used the semi-structured interviews to find out the students’ attitudes, views and opinions of cloze tests and how their teachers assisted them with the cloze procedures. Seven sets of interview questions were asked in the interviews, but the order of the questions and further questions depended on the interviewees’ responses. The interview questions were the following: a) Have you had any experience of cloze tests or cloze exercises (standard cloze test, multiple-choice test, C-test)? b) How often did you have a cloze exercise or a cloze test (approximately)? c) How do you feel about cloze tests or cloze exercises? d) What abilities listed below do you think are tested by cloze procedures: reading comprehension, grammar skills, vocabulary skills, or all of them? e) What did you usually do during cloze exercises: completed the deletions individually and silently without any feedback or discussed with a partner or in a group? f) What did you usually do after cloze exercises? g) Did you have any problems with cloze tests? What were the problems? The interviewees were interviewed individually. They were also informed that they were welcome to add more relevant information during the interview process. Lastly, I organized the notes from the interviews. The second part of my research was the cloze test. The text for the test was selected from the student’s book by O. V.Afanasyeva and I. V. Mikheeva for the eighth grade for the test. The text was about Canada and consisted of seven separate parts. This whole text was big for cloze test so we took several sentences from each part and made a 250 word text (Appendix C). Fifty words were missed and the pupils were asked to fill in the missing words. We chose this text because pupils had learned it previously and it is the one with which pupils were most familiar. This text was used to help us assess the pupils’ performances with this type of cloze procedure. The respondents were not allowed to use any kinds of dictionaries and references, and the time for them to finish the test was limited to twenty minutes. We chose quiet places such as a classroom or library for the respondents to complete the test. Before they commenced with the test, we gave them a brief introduction to the cloze test: “In the following passage some of the words have been left out and you should write down missing words”. After that we gave them some instructions: a) read over the entire passage; b) try to understand what it is about; c) try to fill in each blank. It takes exactly one word to fill in each blank; d) try to guess if you are not sure of the word that has been left out; e) do not use any dictionaries or references during the process. I also told them that they had exactly twenty minutes to complete the test. These directions, of course, would have to be simplified for children in the early grades, and probably they should be given orally and with an example on the blackboard. However, they are quite adaptable to almost any cloze test procedure for most any purpose. In addition, if the participants have a low level of English knowledge, the test instructions can be presented in their native language. The last part of my research was the questionnaire. The questionnaire was submitted to the student respondents after the completion of the cloze test. I used a questionnaire to find out what strategies can facilitate general language proficiency. I designed and composed the questionnaire in advance, and presented it to the respondents immediately after the completion of the cloze test. The students were told that there was no time limit in this process, and that they could use either English or Russian, or even code-mix these two languages. I allowed the interviewees to choose the preferred language(s) for them to use, because it helped them to express themselves fluently, reducing the obstructions caused by having to translate between the two languages. The students could consult me at any moment during this process if they had any problems in understanding the questions. I translated their responses into English afterwards. To determine how the students processed the cloze procedure, the 16 questions in the questionnaire were designed to reflect their reading processes during the cloze test they had completed (Appendix D). The students’ answers to the rest of the questions were likely to reveal what strategies they used to complete the test. Each part of this research was necessary because we wanted to find out what problems the students actually had in dealing with the cloze procedure. Let us now turn to the interpretation of the data. The participating students’ responses in the interviews, the students’ performances in the cloze test and their self- reports on the cloze procedure are presented first. As was mentioned above the objective of the interviews was to find out the students’ experiences, attitudes, views and opinions on cloze tests. According to the investigation the responses fall into two major themes: the students’ experiences with the cloze procedure and their attitudes or opinions towards cloze tests. As respects to the students’ experiences with the cloze procedure all eight students gave an affirmative reply that they had had an experience with the cloze test or exercise, generally it had been a multiple-choice test, but they had no idea that is called cloze procedure. A half of the students said that they had a cloze exercise/test on a weekly basis only during an English lessons, two students out of the eight indicated that they had a cloze exercise/test on a daily basis, explaining that a cloze procedure is an obligatory part for entering university, another two students indicated that they had a cloze exercise/test on a monthly basis because they are sportsmen and don’t have enough time for studies. Concerning their activity during the cloze exercises, all eight students indicated that they usually finished cloze exercises individually and quietly without a chance to have discussions with teachers or other students. However, one student said that he tried to discuss possible answers with other students during cloze exercises, even though discussions during exercises were not encouraged or allowed by teachers. After finishing cloze exercise students indicated that they received correct answers and some simple explanations, nevertheless student mentioned that they were given the chance to provide reasons for their answers and that their teacher usually asked them for their reasons for choosing the answers and had discussions with them before he gave final explanations. To sum up briefly this paragraph, students have only a general idea about cloze procedure because the students had cloze tests or cloze exercises not frequently. What is more their teacher used only one type of cloze procedure namely multiple-choice and didn’t explain the purpose of its task. As a result students are not experienced enough to use cloze procedure independently without teacher’s help and interaction. However, students received clear or detailed introductions to cloze from their teachers the first time they did cloze exercises, so they had some skills in dealing with cloze test. Surprisingly, there was only one way in which teachers expected students to complete cloze exercises; that is, by completing the exercises individually and quietly without any chance of discussion with others. All eight students indicated that they did cloze exercises in this way and the only one tried to discuss possible answers with other students during cloze exercises, even though discussions during exercises were not encouraged or allowed by teachers. The data also indicates that most of the students received correct answers and explanations from their teachers after finishing cloze exercises. They could find out whether their skills and understanding of the cloze text was correct or not from their teachers’ instructions. Also students had opportunities to provide reasons for their answers. In summary, students didn’t have a plenty of experience with cloze tests or exercises. As a result, students rarely analyzed their skills in the cloze procedures. This paragraph is about students’ attitudes and opinions towards cloze tests. Six students out of eight indicated that they thought cloze test was difficult and that they did not like cloze exercises and cloze tests; only two students indicated that they thought cloze was easy and that they enjoyed doing cloze exercises and tests because it was interesting, they have read this text previously, and that they always have practice at home. Responses to the question concerning the abilities were the following: five students among the eight declared that cloze procedures tested all of the three abilities listed above; three students declared that cloze procedures tested only one of the abilities. In addition, the students mentioned that some other knowledge was also required to complete cloze procedures. Three students said that cloze procedures tested their knowledge of phrases, such as the relationships between verbs and prepositions; one student said that cloze procedure also tested background knowledge; one student said that it tested one’s ability to use synonyms; and one student said that it required familiarity with the English language. Of course, all students had problems with cloze test, they are as follows: Seven students said that they had problems in comprehending the cloze passages because the number of blanks in the text hampered their reading of the text; two students indicated that they had problems in using grammar skills in such tests; six students indicated that they had problems in using vocabulary skills, and one student indicated that the problem she had was due to unfamiliarity with English. In brief, students’ attitudes and opinions towards cloze test are negative to some extent, many of them regarded these procedures as painful, and did not want to repeat them. This situation was further confirmed by the finding that the majority of students, that is, six out of the eight students, did not like cloze tests or exercises because it takes much time, and it doesn’t have any hint, and it requires great effort to concentrate and recall the text and they seldom did well in such exercises. Nevertheless, the students’ dislike of cloze test did not hamper their due recognition of cloze. Many of the students regarded cloze as a measure of integrative English language competence, which tested students’ vocabulary ability, grammatical ability, comprehension ability, reading ability, writing ability, etc. The biggest problem the students identified with cloze tests or exercises was comprehension of the cloze text. The students said that the blanks in cloze texts hampered their understanding of the texts. They were unable to get a coherent understanding of an incomplete text. The second problem for the students was the vocabulary problem. The students mentioned that the main difficulty they had in this regard was synonyms. The third problem for some students was grammar. The students were bothered that they had to consider many factors such as time, tense, aspect, concord, etc. The students also considered unfamiliarity with English as a problem. Thus, analyzing the participating students’ responses in the interviews we came to the conclusion that students have only a general notion about the cloze procedure, it is not a frequently used task in our schools, that is why, cloze test seems a very difficult task for a great part of students but they agree that this procedure is a good task for developing many skills which are necessary for English language study. As for the cloze test on the text about Canada from the student’s book by O. V.Afanasyeva and I. V. Mikheeva for the 8th grade it looked as this: The name Canada is _______ to come from the _______ word Kanata, meaning ‘village’ _______ ‘community’. This fact alone _______ that the native population _______ the country consisted mostly _______ Indian tribes. Now Canada _______ the world’s second largest _______ .The capital of the _______ is Ottawa. Its currency _______ the Canadian dollar, which _______ divided into 100 cents. _______ country’s most important symbol _______ the maple leaf. The _______ southern areas of Canada _______ a moderate climate when … the north the climate _______ characterized by very long, _______ winters with average temperatures _______ below freezing and cool _______ months. Canada has a _______ of over 30 million. _______ French were the first _______ settle in the country _______ large numbers. People of _______ origin came to stay _______ mainly after 1763. As _______ result Canada has a _______ cultural heritage and both_______ and English are official _______ . Canada has 10 provinces_______ 3 territories, each with _______ own capital city. The _______ is rich in natural _______: gas, oil, coal, gold _______ so on. It is _______ of the world’s leaders _______ mineral exports. Its soils, _______ are especially rich in _______ three prairie provinces, are _______ utilized and make Canada _______ of the world’s largest _______ of agricultural products. Forests _______ about half of Canada. _______ are the basis for _______ important industries as lumbering _______ paper production. Canada is _______ constitutional monarchy. The Queen _______ the head of state, _______ is represented in Canada _______ the governor-general. Let us now introduce the results of this test. All eight students finished the cloze test, which included twenty five blanks. The objective of this test was to find out whether students are really weak in writing cloze tests. Students’ performances on the cloze test were assessed by calculating the number of correct answers. As we mentioned in our theoretical part of our present diploma paper there are different methods of scoring procedures. The well-known method is the exact word method. We used it in scoring the results when the missing words in the text didn’t have any synonyms. First of all they were prepositions, auxiliary verbs, and articles for example: or, of, is, the, to, in, a, its, and, etc.; and some words which don’t have synonyms according to the context of the text, such as: Indian, British, French, country, summer, language, etc. For other words we used contextual appropriateness method which includes not only the exact word but some of the possible alternatives of this word as a correct answer. We used some general guidelines: a) see if the word in a given blank is the exact word, so count it correct; b) if it is not the exact word, check if it fits the immediately surrounding context, any local constraints in the same sentence or surrounding phrases; c) if it is consistent with all of the preceding and subsequent text (this includes previous and subsequent responses in other blanks as filled in by the examinees); d) if the response passes all of these checks, score it as correct. It should be mentioned that each method has its own advantages and disadvantages. The advantage of exact-word method is that it is objective and easy to apply. That is, no scorer judgment is needed in order to score the test. However, it has the disadvantage of lowering the students’ scores. On the other hand, contextual appropriateness method increases the students’ scores but it is somehow subjective and difficult to apply. The problem is that determining the words which might be acceptable for a particular blank is both subjective and beyond the ability of most EFL teachers. In other words, most EFL teachers would not feel comfortable with deciding on all appropriate and acceptable words for a deletion. The results here show the percentage of correct answers. Two students got the lowest score of 30% and two students got the highest score of 85%. There is no high correlation between students because all of them have almost equal knowledge of English (Table 1).

Table 1 - The students’ performances on the cloze test Students’ Number Score percentage Student 1 85 % Student 2 60 % Student 3 70 % Student 4 30 % Student 5 50 % Student 6 85 % Student 7 75 % Student 8 30 %

After reading the results, I handed out this test again and gave students time to look at their results and try to find out the mistakes. Then I gave them correct answers but I also gave them the chance to provide reasons for their answers and asked them for their reasons for choosing the answers and had discussions with them before giving final explanations. As we may see from the table above no one got 100 %. Only two students got 85 % it means that they filled in almost twenty two words correct. Thus, these two students have a good memory and they remembered the text about Canada or they have very high level of English. We cannot say that students who get a low marks have a low level of English they were absent when their class read about Canada. Anyway they coped with this task better in comparison with students with low proficiency, where the 30 % is a good percent of performance. Also all eight students who finished the cloze test responded to the questionnaire. The questionnaire consisted of forty questions. The students’ responses are organized around three main themes: the type of information that students used to finish each item of the cloze test, reading, writing and comprehension skills used by students during the process of completing the cloze and skills normally used by the students during the lesson. Question 1 dealt with this theme. By asking what the students were thinking in the initial fifty questions, I wanted to gain an understanding of how they used their reading strategies, writing skills and vocabulary knowledge when completing each blank space of the cloze text The objective of these questions was to find out how different levels of information helped students to finish the cloze. For the basis of this classification we took Yamashita’s classification of information type: a) clause level: the students use information provided by a clause in which an item appears; b) sentence level: the students use information provided by a larger context than the clause in which an item appears, but within the sentence; c) text level: the students use information provided by a larger context than the sentence in which an item appears, but a context from within the text: 1) adjacent context: the students use information provided by a sentence that immediately precedes or follows the sentence in which an item appears. 2) wider context: the students use information provided by a context, which is more than a sentence apart from the sentence in which an item appears. d) extra textual: the students use information not provided by the text, which includes such mental resources as the students’ background knowledge, beliefs, and images; e) guessing: the students’ guesses; f) missing: the student does not/cannot say anything about his/her cognitive processes. The following table shows the six information sources used correctly by the eight students in my sample (Table 2):

Table 2 - Information categories used in the cloze test Categories Percentage Clause 90 % Sentence 50 % Text 60 % Extra-textual 70 % Guessing 5 % Missing 2 %

The results showed that 70% of students used extra-textual information type because they have read this text previously, 90% use information provided by a clause in which an item appears, in other words clause level, 5% guessed and 50% used information provided by a larger context than the clause in which an item appears, but within the sentence, and surprisingly only 2 % missed the words. Data from the students’ questionnaire responses shows that, the students seldom used two information sources but just used a single source. As shown in table clause-level information was the source most frequently used by the students. However, their use of clause-level information was not very successful. The students often tried to guess the answers. The students did not use text-level information frequently. Despite this, text-level information was the source of information most successfully used by them. In many cases, the students could not identify the source of information they used for the answers, so they just guessed. In short, to complete the cloze test, the students preferred to use a single source of lower level information, or they merely guessed. The less successful the student was at using different levels of information, the lower the score was. Questions 51 to 65 dealt with this theme. The students’ answers to the rest of the questions were likely to reveal what strategies they used to complete the test. The objective of these questions was to find out whether these reading and writing skills were used by students when they were reading the cloze text. If the student gave an affirmative answer to the question, it meant that she/he used the reading or writing skills. The results concerning the reading skills are shown in table 3:

Table 3 - Reading skills used by students in reading the cloze text Reading skills Number of students(8) 1. Work on the blanks directly before reading the whole text. 6 2. Read fast for the gist of the text by skipping the blanks before focusing on them. 4 3. Scan the text constantly when finding answers to the blanks. 7 4. Stop and reread the confusing sections. 5 5. Ask yourself questions about what the text was about after reading the whole text. 5 6. Ask yourself what you know about the topic of the passage. 1 7. Ask yourself whether you understand the text or not. 6 8. Compare what you have just learned from the reading with what you already knew. 4 9. Read the text twice or more. 5 10. Find out the main ideas of the passage (main idea of every paragraph). 7 11. Skip the unknown words when you were reading the text. 7 12. Use context to try to figure out an unfamiliar word. 5 13. Try to use semantic or phonic analysis to figure out an unfamiliar word. 4 14. Review the paragraphs immediately preceding or following the one in the blank you are doing. 6 15. Review the words preceding or following the blank you are doing constantly. 4

The above table shows how many students used the particular reading skills during their reading of the cloze text. As we said previously students really performed poorly in the cloze test. On the other hand, the students’ responses to the questionnaire (shown in table 3) indicate that they used all of the fifteen reading skills. How is it that they obtained low scores on the cloze test if they used so many reading skills? As discussed in the literature review, comprehension of the cloze text is important for completing a cloze, because it is the basis for filling in the blanks. To gain a good grasp of the comprehension of a text, students have to use reading skills flexibly. When one compares the students’ responses with the teachers’ recommendation, one notes that the students even used the reading skills that teachers did not expect them to use. In the following section the students’ problems in using reading skills are discussed. Firstly, good comprehension of a text requires students’ efficient use of a reading pace when they are reading. Questions 1 to 4, Question 11 and Question 14 and 15 examined whether students used different reading paces during the process or not. Teachers agreed that students should not have worked on the blanks directly without reading the whole text. However, a majority of the students, that is, six students out of the eight, worked on the blanks directly before reading the text. Further investigation showed that almost all the students worked on the blanks directly when they got the paper. In other words, they filled in the blanks before getting to know what the text was about. However, it is nearly impossible to complete a cloze successfully without knowing the gist of the passage, because students have no basis for filling in the blanks without comprehending what they have read. By not knowing the gist of the test passage, the students in my study did not have a good basis for retrieving the words in the blanks. It seems that they did not realize that this problem hampered their performance on the cloze test. They probably thought the information they picked up while they were working on the blanks was enough to help them to comprehend the text. However, they did not know that they were at risk of misinterpreting the writer’s original meaning and reading their own meanings into the text. Therefore the possibility for them to fill in the words correctly would have been very low. My investigation and the students’ responses to Question 1 made me doubt the students’ affirmation of Question 2. Six students claimed that they worked on the blanks directly without reading the whole text for answering Question 1, while four students said that they read the text fast to get the gist before working on the blanks for answering Question 2. There were two students who did not realize that they gave affirmative answers to two contradictory questions, or that the answers they gave contradicted each other. These students probably misinterpreted the questions. Understanding the gist of the text and thinking about a coherent understanding of the text can help students to complete the cloze deletions. While teachers believed that the students should read fast for the gist of the text before focusing on filling in the blanks, only two of the eight students agreed with this. In other words, only one fourth of students were concerned with a coherent understanding of the text before they worked on the blanks. At least they completed the cloze deletions based on an incomplete and fragmentary understanding of the text. To get a coherent understanding of the text, students have to read rapidly and fluently, so they need to skip unknown words in the text. While teachers expected students to use this skill, seven out of the eight students skipped the unknown words while they were reading the text. The reason for the students regarding this skill as very important and necessary was because they encountered a number of unknown words in the text. I knew this to be the truth, because some students asked me for the meanings of the words ‘community’, ‘tribes’, ‘maple leaf’, ‘lumbering’, and so on after they had finished the cloze test. It was clear that almost all the twenty students knew how to deal with unknown words in the text. Their understanding of the text was not hampered by the unknown words. There seemed to be a mismatch between the students’ claim in the interviews and their actual performance in the cloze tests. Seven of the students said in the interviews that one of the biggest problems in their reading of a cloze text was that their understanding of the text was hampered by the blanks in the cloze. If the students could read rapidly and fluently by skipping unknown words, they were supposed to get a global meaning of the text without being hampered by the blanks. There was no evidence to show that the existence of the unknown words was a problem for the students to understand the text. To get a good comprehension of the text, rereading was necessary. While all teachers felt the students should read the text twice or more, only five of the students read the text twice. The teachers obviously regarded this skill as more important than some of the students. It is possible that the five students who reread the text felt that their understanding of it was not sufficient enough for completing the blanks after only one reading. There could be three explanations why the remaining three students read the text only once. One is that they may have thought that they understood what the text was about after the first reading. However, it is hard to see a foreign language learner making sense of a fairly complex text, which has a number of blanks, after only one reading and without first reading the text rapidly to get the gist. The second explanation is that they may have been in a hurry to complete the blanks, so were not overly concerned about the content of the text, even though they did not understand the text completely. The third explanation could be that, as they knew that the result of this test did not count as part of their studies, they did not treat it as seriously as a formal test. When confusion or difficulties occur, slowing down the reading pace or even stopping and rereading the confusing section is effective for getting a better understanding of the text. Teachers indicated that stopping and rereading the confusing sections was necessary for comprehending the text, and seven of the students actually did so. Both the teachers and the students regarded this skill as necessary during the reading. Six of the students reviewed the paragraph(s) immediately preceding or following the one they were reading and four students reviewed the words preceding or following the ones they were reading. This was a skill teachers expected them to use. There was therefore a very close correlation between teacher expectation and student performance in this regard. To complete the deletions, the search for specific information needed to be more focused. Six of the students scanned the text constantly when finding answers to the blanks and teachers expected them to use this skill. The students clearly exceeded teacher expectation in regard to the use of this skill. In short, the students tended to adjust their pace according to specific parts of the text rather than to the whole text. Secondly, good comprehension of a text involves adequate reading comprehension skills. Questions 5 to 10 and Questions 12 to 14 examined whether the students used reading comprehension skills or not. Using pre-reading skills such as activating background knowledge and predicting something about the text can help prepare students for understanding the text. Five of the students predicted what would be in the passage when they read the first sentence. It seems that many of the students thought this skill was useful for them to understand the text, while the teachers did not feel the same. It means that this skill might not have been necessary for reading this specific text. As Carrell states, meaning does not reside in the text but is constructed out of the interaction between a reader’s background knowledge and the text. Understanding the text also requires the students to activate their background knowledge, to relate what they read in the text to what they already know. One of the students said he asked himself what he knew about the topic, an aspect only two of the teachers thought was important. However, the student who claimed that he had asked himself what he knew about the topic could not explain what part of his background knowledge he had activated. It seems that his use of background knowledge was not efficient or he could not access the appropriate existing knowledge. Four of the students tried to use semantic or phonic analysis to figure out an unfamiliar word, while not all teachers thought they should use this skill; thus many more students used this skill than the teachers expected. According to the students’ responses in the questionnaire, they mainly used this skill to deal with the word ‘agricultural’. The different opinions between the use of this skill by the students and the teachers’ expectations occurred because of the different opinions on the word ‘agricultural’. This word was easy and could be recognized at first glance by the teachers because they were more familiar with English than the students. However, it was not so easy for the students to recognize this word at merely a glance, so semantic analysis was necessary to be used. In total, many of the students used while-reading skills properly when they were reading the text. Post-reading skills help learners check and extend their understanding of the text. Five of the students asked themselves questions about the text after reading the whole text, while not all teachers thought they ought to do so. The students merely asked themselves questions, but they did not really answer the questions themselves. Or, even if they could not answer their own questions, they did not take further actions for improving their understanding of the text. Six of the students asked themselves whether they understood the text or not. This means that the students could use this skill properly just as the teachers expected. Four of the students compared what they had just learned from the reading with what they already knew, which two teachers expected them to do. As discussed above, the topic of the text and the examples given by the author were related to the students’ practical lives, so this skill must have been very useful for the students to understand the text. Many of the students used this skill even though not many teachers regarded it as very important. Seven of the students could find out the main ideas of the passage (main idea of every paragraph), and teachers regarded this skill as important. In short, most of the while-reading skills and post-reading skills were used successfully by the students, but the students did not really use pre-reading skills during the reading of the cloze text. Cloze procedure improves not only reading skills but also writing, comprehension skills, knowledge of vocabulary. The objective of this research was also to find out how different aspects of different skills helped students to complete the cloze test. The results are shown in table 4:

Table 4 – Aspects of skills used by students in performance of cloze test. Aspects of skills Number of student (8) Correct use of English grammar 6 Correct use of phrasal verbs 3 Spelling 7 Semantic analysis 4 Background knowledge 7

The above table shows how many students used the particular writing, comprehension skills and knowledge of vocabulary during their performance of the cloze text. As we said previously students really performed poorly in the cloze test. On the other hand, the students’ responses to the questionnaire indicate that they used all of the six different aspects of different skills. Of comprehension of the cloze text is important for completing a cloze test, because it is the basis for filling in the blanks but to gain good results for performance of the cloze, students have to use not only reading skills but all other skills flexibly. Thus, six students out of eight thought that use of English grammar correctly is a necessary skill for cloze procedure. This skill is very important for example for our cloze text, because most of missed words were auxiliary verbs which required an appropriate tense. For that reasons teachers often use this procedure for checking tenses, modal verbs and prepositions. Sometimes correct use of phrasal verbs is very important and necessary skill, some proficiency tests especially TOEFL have many phrasal verbs or set expressions which students should fill in the blanks. Only three students used this skill. There could be two explanations why not so many students used this skill. One is that they didn’t pay due attention to the existence of such set expressions and phrasal verbs in our cloze text. The second explanation could be that their level of English didn’t make it possible to notice them for the reason that they haven’t learned these phrasal verbs yet. Understanding the text requires the students to activate their background knowledge. The highest number of student, to be more precise seven, supposed that spelling is a necessary skill, as also background knowledge, because they are interrelated. Students always use these skills to do different exercises not only the cloze test. Four of the students tried to use semantic analysis for unknown words. This analysis helped them understand what part of sentence a word is and try to analyze which part of this sentence was missed according to the structure of the sentence. Thus, cloze test measures three types of knowledge: linguistic knowledge, textual knowledge, and knowledge of the world. In a standard cloze, almost all these types of knowledge are somehow tested. That is, the test takers should utilize their linguistic knowledge to read the text and understand it. They should use their textual knowledge to determine the logical relationship between the sentences and determine the references made in the text. Finally, they should use their knowledge of the world to map their linguistic and textual knowledge to the context of the text. However, different versions of the cloze test may focus on utilizing different types of knowledge. For instance, when a certain class of words such as articles or prepositions is deleted, the focus might be more on the linguistic knowledge rather than on the other types of knowledge. Further, when only cohesive ties, such as coordinators, or transitional expressions are deleted, textual knowledge might be tested more than other types of knowledge. Thus, depending on the focus of the course and the purpose of the test, the examiners would utilize one or the other version of the cloze test. The participants with low proficiency level were eight pupils from the eighth grade of ‘DARYN’ school. We selected these pupils because their level of English knowledge is not very high. However, they are willing to do the cloze test, available to answer to our questions and complete a prepared questionnaire. We chose standard or fixed-ratio type of cloze test where the every fifth word is deleted. First of all I proposed the same interview, the same text and questionnaire. Surprisingly, but the results of questionnaire were the same as the results of questionnaire for the grade with high proficiency level. Therefor we are not going to give clear explanations of these results again. For the interview I took the same questions: a) Have you had any experience of cloze tests or cloze exercises (standard cloze test, multiple-choice test, C-test)? b) How often did you have a cloze exercise or a cloze test (approximately)? c) How do you feel about cloze tests or cloze exercises? d) What abilities listed below do you think are tested by cloze procedures: reading comprehension, grammar skills, vocabulary skills, or all of them? e) What did you usually do during cloze exercises: completed the deletions individually and silently without any feedback or discussed with a partner or in a group? f) What did you usually do after cloze exercises? g) Did you have any problems with cloze tests? What were the problems? The interviewees were interviewed individually. They were also informed that they were welcome to add more relevant information during the interview process. Lastly, I organized the notes from the interviews. As respects to the students’ experiences with the cloze procedure only four students gave an affirmative reply that they had had an experience with the cloze test or exercise, generally it was a multiple-choice test, but they had no idea that is called cloze procedure. All eight students said that they had a cloze exercise/test on a weekly or even rare basis only during English lessons. They explained that this task is very difficult for them, cloze procedure is not necessary for learning English, because it only checks their knowledge, and they usually get a bad marks for this task. What is concerning their activity during the cloze exercises, all eight students indicated that they usually discussed possible answer with other students during the cloze procedure or they worked in groups in spite of their teacher require to finish cloze exercises individually and quietly without any discussions with teachers or other students. However, one student said that he tried to do this exercise quietly without any discussion of possible answers with other students during cloze exercises, but other classmates didn’t give the chance to do it, because they always ask a correct answer from that student. After finishing cloze exercise students indicated that they received correct answers and some simple explanations, nevertheless student mentioned that they were given the chance to provide reasons for their answers and that their teacher usually asked them for their reasons for choosing the answers, but they didn’t use this chance explaining that it was not interesting for them because that text wouldn’t be anymore that is why they didn’t see the necessity to discuss the correct answers. To sum up briefly this paragraph, students have a general idea about cloze procedure because they had cloze tests or cloze exercises not frequently. What is more teacher used only one type of cloze procedure namely multiple-choice and only for one purpose to check students’ knowledge of grammar. As a result students are not interested in performance of such kind of task and not experienced enough to use cloze procedure independently without teacher’s help and interaction. However, students received clear or detailed introductions to cloze from their teachers the first time they did cloze exercises, so they had some skills in dealing with cloze test. Concerning students’ attitudes and opinions towards cloze tests, seven students out of eight indicated that they thought cloze test was difficult and that they did not like cloze exercises and cloze tests; only one student indicated that he thought cloze was easy and that he enjoyed doing cloze exercises and tests because it was interesting, he has a good memory and he has read this text previously. Responses to the question concerning the abilities were the following: only two students among the eight declared that cloze procedures tested all of the three abilities listed above; six students declared that cloze procedures didn’t test any of the abilities, that tasks only checked the knowledge. In addition, the students mentioned that some other knowledge was also required to complete cloze procedures. Two students said that cloze procedures tested their knowledge of phrases, such as the relationships between verbs and prepositions; one student said that cloze procedure also tested background knowledge; one student said that it tested one’s ability to use synonyms; and one student said that it required familiarity with the English language. Of course, all students had problems with cloze test, they are following: seven students said that they had problems in comprehending the cloze passages because the number of blanks in the text hampered their reading of the text and they didn’t remember the text; eight students indicated that they had problems in using grammar skills in such tests; seven students indicated that they had problems in using vocabulary skills, and one student indicated that the problem she had was due to unfamiliarity with English. In brief, students’ attitudes and opinions towards cloze test are negative. All of them regarded these procedures as painful and ineffective, and did not want to repeat them. This situation was further confirmed by the finding that the majority of students, that is, seven out of the eight students, did not like cloze tests or exercises because it takes much time and power, and it doesn’t have any hint, and it requires great effort to concentrate and recall the text and they seldom did well in such exercises. Nevertheless, the students’ dislike of cloze test did not hamper their due recognition of cloze. Many of the students regarded cloze as a measure of integrative English language competence, which tested students’ vocabulary ability, grammatical ability, comprehension ability, reading ability, writing ability, etc. The biggest problem the students identified with cloze tests or exercises was comprehension of the cloze text. The students said that the blanks in cloze texts hampered their understanding of the texts. They were unable to get a coherent understanding of an incomplete text. The second problem for the students was the vocabulary problem. The students mentioned that the main difficulty they had in this regard was synonyms. The third problem for some students was grammar. The students were bothered that they had to consider many factors such as time, tense, aspect, concord, etc. The students also considered unfamiliarity with English as a problem (Figure 1).

Figure 1 – Comparison of the results of the interview Thus, analyzing the participating students’ responses in the interviews we came to the conclusion that students have a negative notion about the cloze procedure, it is not a frequently used task in our schools, that is why, cloze test seems very difficult, ineffective, unnecessary, and useless task for a great part of students. In a grade with low proficiency level only two students agree that this procedure is a good task for developing many skills which are necessary for English language study. The results of the interview in a grade with high and low proficiency level showed that students with high proficiency level are more interested in learning English language, they are better experienced in comparison with students with low proficiency level. However, all students have only a general notion about cloze procedure and they think that cloze procedure is only a multiple-choice task and that is all. Anyway, at their lessons a teacher tries to use various interesting task for developing general language proficiency but doesn’t explain the purpose of these tasks. For that reason students usually thought that the teacher only wanted to check their knowledge. Students with low proficiency level have more problems with cloze test; they think that this task is painful and ineffective. Also, only two students with low proficiency level think that cloze procedure develops reading, writing, comprehension skills. We also gave the same text about Canada to students with low proficiency level. All eight students finished the cloze test, which included twenty five blanks. The objective of this test was to find out whether students are really weak in writing cloze tests and to compare results with a grade with high performance level. Students’ performances on the cloze test were assessed by calculating the number of correct answers. We used the same two methods of scoring the results: exact-word and contextual appropriateness methods. For those students we also remembered that they should not be penalized for spelling errors if words are recognizable. Results of this scoring were striking. The results here show the percentage of correct answers. Two students got the lowest score of 10 % and only one student got the highest score of 60 %.

Table 5 - The students’ performances on the cloze test Students’ Number Percent of score Student 1 60 % Student 2 40 % Student 3 55 % Student 4 10 % Student 5 30 % Student 6 35 % Student 7 10 % Student 8 30 %

These results show the low quality of performance on this test. Two students got 10 %, it means that they wrote down only two correct words or they got more missing blanks. Only one student got 60 %, it means that he filled in only half of the missing words correctly. The reasons of that may be: a) disinterest in performance on this kind of task; b) lack of experience in doing such kind of test; c) lack of time on the lesson; d) low level of English language; e) lack of interest of the teacher in developing students’ writing, reading, and comprehension skills; f) students were not so responsible for the results because this task was only for experimental purpose. Comparison of two tables with results of students’ performances on the cloze test we made in diagram (Figure 2):

Figure 2 – Comparison of the results of students’ performance on cloze test

Diagram 2 shows the correlation between students with high and low proficiency level. As we may see from the diagram students with high performance level performed the test better than students with low proficiency level. It may be explained by some reasons. Students in lyceum ‘BEST’ have more hours of English lessons, they are more responsible, they are interested in education, and students are more disciplined and ambitious. Nevertheless, they didn’t show excellent results, because they are also not experienced enough and have only general notion about cloze procedure. Two teachers also participated in our experiment. There were particular criteria for selecting the teachers for our research. Firstly, they had to have at least two years of English teaching experience. Secondly, they should know something about cloze procedure. As we have mentioned above, not all teachers know the application of these tests, for that reason the cloze procedure is not commonly used in the process of studying foreign languages. I have prepared the questionnaire for teachers. I asked several teachers to recommend a number of strategies, which they believed students ought to use when reading cloze texts (Appendix E), a number of skills which are necessary to make the cloze test and to rank these strategies in order of importance. The questionnaire was used to find out what skills the teachers believed ought to be used by the students in reading and performance of the cloze text and which of the six skills were the most important ones that would enable the students to finish the cloze. The first part of the questionnaire dealt with this theme. Teachers were asked to select the skills they thought ought to be used by students from those listed below the question. The objective of this part was to find out what reading skills the teachers believed the students should use when reading the cloze text. The findings are illustrated below: a) stop and reread the confusing sections; b) read fast for the gist of the text by skipping the blanks before focusing on them; c) ask yourself whether you understand the text or not; d) read the text twice or more; e) find out the main ideas of the passage (main idea of every paragraph); f) use context to try to figure out an unfamiliar word; g) review the paragraphs immediately preceding or following the one featuring a particular blank; h) constantly review the words immediately preceding or following a particular blank. The second part of the questionnaire was aimed at finding out whether there were some other skills the teachers thought were necessary for students’ performance of the cloze test. The teachers were asked to write down the skills. They are: a) use grammatical rules; b) consider genre of the text, analyze text structure, context and language. c) use knowledge of collocation and combination relationships between words; d) picture in your mind the people, events and places that you are reading about; e) prior knowledge As shown above, the teachers recommended more skills than they thought were necessary for the students’ reading of the cloze text. One of the teachers recommended the skill ‘Use grammatical rules.’ As time, tense, aspect, etc. are expressed differently in English and in Russian, using grammatical rules can help the students understand the text better. By-turn, six of the students’ responses in the questionnaire revealed that they used grammatical rules to help them complete the blanks. One of the teachers also recommended the skill ‘Use knowledge of collocation and combination relationships between words.’ The students’ responses in the questionnaire revealed that they used this skill twenty-four times to deal with the blanks in the cloze. If the students were unfamiliar enough with the collocation and combination relationships between words, this skill was very useful for them to choose the most appropriate words. However, if the students had been familiar with these relationships between words, using this skill would have caused misunderstanding of local parts of the text and confusion in understanding the meaning of phrases, which were collocated or combined by a word with other different words. The students’ responses to the questionnaire indicated that they also used “knowledge of phrases” in their performance of the cloze text. Their use of this skill would have been more efficient in normal reading than in reading a cloze text, because they could have referred to a dictionary when confusions occurred. One of the teachers recommended the skill ‘Consider genre of the text, analyze text structure, context and language’ as well. Sometimes, considering what particular purpose a text is being composed for can help learners understand what information is conveyed and how linguistic patterns work in the text, because “… the writer is seen as having certain goals and intentions, certain relationships to his or her readers, and certain information to convey, and the forms of a text are resources used to accomplish these”. However, this skill must have been the one fewest students used in performance of the cloze text, as none of the students mentioned it in their responses to the questionnaire. When the students were doing the cloze test, their teachers seldom asked them to consider the genre of the text and to analyze the text structure; context and language were considered much more important. The students were not used to considering the genre of a text because they were not trained to do so. The third part of the questionnaire was to find out which skills the teachers felt were most important for the students to complete the cloze. They were asked to select six skills and rank them according to their importance: a) read fast for the gist of the text by skipping the blanks before focusing on them; b) find out the main ideas of the passage (main idea of every paragraph); c) use grammatical rules; d) read the text twice or more; e) ask yourself what you know about the topic of the passage (use prior knowledge related to the topic of this passage); f) use context to try to figure out an unfamiliar word. In my research I decided to compare the six important skills chosen by students and by teachers. My comparison of the six reading skills most popularly used by the students to deal with the cloze and those chosen by the teachers, revealed big differences between what students actually used and what teachers thought ought to be used (Table 6).

Table 6 - Comparison of six most important skills selected by the students and the teachers Students’ choice Teachers’ choice Find out the main ideas of the passage (main idea of every paragraph). Read fast for the gist of the text by skipping the blanks before focusing on them. Spelling Find out the main ideas of the passage (main idea of every paragraph). Work on the blanks directly before reading the whole text Use grammatical rules. Correct use of English grammar Read the text twice or more. Skip the unknown words when you were reading the text Ask yourself what you know about the topic of the passage (use prior knowledge related to the topic of this passage). Background knowledge Use context to try to figure out an unfamiliar word.

In summary, the students used different paces and various skills during the process of performance of the cloze test, but there were disadvantages of their use of these skills. They ignored the adjustment of reading pace for understanding of the whole text, overused some of the skills such as ‘spelling’ and ‘Work on the blanks directly before reading the whole text’, and insufficiently used some of the skills like ‘using prior knowledge’ or ‘correct use of English grammar’. The aim of this analysis is to see what difficulties the students had with the cloze test and to propose exercises for cloze procedure. Firstly, I found that many of the students really did not enjoy their experiences with cloze tests and exercise because they seldom did well at them and consequently felt frustrated with cloze procedures. As foreign language learners, the students faced problems with various aspects such as lexical and grammatical problems when they were reading a cloze text, and needed to know the skills for dealing with them. The quality of the first exposure to the cloze procedure was very important for the students. A proper introduction, which needed to explain what a cloze test or exercise was, the skills for dealing with it and how to use these skills laid an essential foundation for the students. The students’ confidence and performance on cloze tests did not necessarily relate to the duration or frequency or the time they had worked on cloze tests or exercises, but were closely related to the quality of their first exposure to it. The students, who had been given a proper introduction to the cloze procedure by their teachers, usually did better in it because they knew what skills could be used. The students who did not have proper introduction to the cloze procedure usually lacked confidence in doing such tests or exercises and did not like doing them, because they did not know how to deal with them. Many students were in this situation. They had a concept of what the cloze procedure entailed as they had been introduced to it by their teachers, but they rarely knew what skills to use to deal with a cloze text. Without adequate preparation, it was impossible for them to do well in the cloze procedures. The second finding from the data is that the students did not improve their skills with cloze exercises and tests. This situation can be related to the way in which the students did cloze exercises. All of the students always did cloze exercises silently and individually without discussion with others. Their teachers did not provide opportunities for them to think about their own use of reading skills during the cloze procedure. After the cloze exercises, the teachers seldom encouraged the students to think about whether their use of the skills was effective or not. The students expected the teachers to tell them what the right answers were and what skills should have been used after a cloze exercise. This dependence on the teacher meant that the students could never work out for themselves how they should do a cloze test or exercise. Also there were some problems in the use of reading skills by the students. The students failed to adjust their reading speed in order to get the gist of the cloze text, although they could manage this with other types of reading. It does not mean they did not know how to control their reading speed to gain a coherent meaning of the text. They could read rapidly and fluently by skipping unknown words and using context or semantic analysis to figure out difficult words. They just ignored the content of the text or they did not mind whether they understood the text or not before they concentrated on the blanks. They were concerned with parts of the text rather than understanding the whole text. It seems that the students did not first do a quick reading of the text before they worked on the blanks. For them, understanding the phrase closest to a blank was more important than the overall meaning of the text, and their incomplete understanding of the text hampered their performance in the cloze test. I found that although the students had mastered all the necessary skills, they did not use them flexibly. Using skills flexibly means selecting and using the skills that are most helpful for understanding the text. It was not necessary to use all the skills in performance on the cloze text, because not all of these skills were very useful for this text. The students also had problems with using some of the reading skills efficiently. For example, they did not activate their prior knowledge successfully although they had related background knowledge. Although they asked themselves questions about the text, they were unable to apply this skill effectively to improve their understanding of the text. As we mentioned before teachers used cloze procedure or exercises seldom therefor students have their own problems such as difficulties in comprehending the text, unfamiliarity with sentence structures, insufficient mastery of grammar and vocabulary, lack of knowledge of English, inadequate mastery of synonyms and prepositions and so on, it is extremely difficult for students to deal with such cloze exercises. The difficulty causes frustration and makes the students lose interest. Perhaps it is wrong to use the cloze procedure to help students improve their reading competence, or the teachers do not realize that more appropriate kinds of cloze can help students improve their reading competence. Also the aim of learning a language is to use it in communication. Reading is regarded as communication between readers and the writer through the text. However, I found out through this study that reading of cloze texts does not facilitate the students’ abilities in English communication; because students assume that the only function of a cloze text is the completion of the blanks instead of improving communication skills in English. Perhaps the students’ misunderstanding comes from the inappropriate use of cloze exercises by their teachers, who frequently use them to help students prepare for tests instead of using them as instruments that can help students improve their reading abilities. As the teacher creates cloze passages, he/she can vary the difficulty levels of the predictions by deleting different classes of words and their position with the text. Structure words such as conjunctions, articles, prepositions, and auxiliary verbs are easier to predict than context words such as nouns, main verbs, adjectives and adverbs. Also, the position of the deletion affects its difficulty. Generally, words in the middle of a sentence are easiest to predict. Next easiest to predict are words at the end of a sentence. And, the most difficult to predict are words at the beginning of a sentence. In general, the more context that is provided, the easier it is complete the cloze passage. For young children, it is recommended that there be at least ten words on each side of deletion. In summary, one must consider word class, position of the deletion, and the amount of context provided when constructing cloze passages based on real instructional materials such a content area textbooks and children’s literature.

2.3 Suggestion of Gradual Introduction of Cloze Procedure

According to results of comparison grades with high and low proficiency level, we decided to suggest some exercises for gradual introduction to the cloze procedure. For the beginners we suggest to use a multiple-choice type of cloze procedure. Today multiple-choice test is the most common and widely used assessment tool for the measurement of knowledge, ability and complex learning outcomes. A multiple-choice item usually consists of a stem which presents a problem situation, and several alternatives which provide possible solutions to the problem. The stem may be a question or an incomplete statement. The alternatives include the correct answer and several plausible answers called distracters. Students are not required to produce a set of words to complete the passage. Rather, they are provided with three, four, or five choices for each blank and are required to select the most appropriate word from among the given choices. This form of task should be based on the texts which students have read previously and contain no more than 15 deletions. Directions must be clear and simple. For example: Directions: There are 12 blanks in the following passage. For each blank there are four choices marked A), B), C) and D) on the paper. You should choose the ONE that best fits into the passage. Wise buying is a positive way in which you can make your money go further. The 1__ to go about purchasing an article or a service can actually 2__ you money or can add 3__ the cost. Take the 4__ example of a hairdryer. If you are buying a hairdryer, you might 5__ you are making the 6__ buy if you choose one 7__ look you like and which is also the cheapest 8__ price. But when you get it home you may find that it 9__ twice as long as a more expensive 10__ to dry your hair. The cost of the electricity plus the cost of your time could well 11__ your hairdryer the most expensive one of all. So what principles should you 12__ when you go out shopping? 1. A) form B) fashion C) way D) method 2. A) save B) preserve C)in D)similar 3. A) up B) to C) in D) on 4. A) easy B) single C) simple D) similar 5. A) convince B) accept C) examine D) think 6. A) proper B) best C) reasonable D) most 7. A) its B) which C) whose D) what 8. A) for B) with C) in D) on 9. A) spends B) takes C) lasts D) consumes 10. A) mode B) copy C) sample D)model 11. A) cause B) make C) leave D) prove 12. A) adopt B) lay C) stick D) adapt The advantage of the multiple-choice cloze over the standard cloze is that the task of comprehension is easier than that of production. At that stage teacher may delete different auxiliary verbs, prepositions and articles. Multiple choice tests do have disadvantages. One of these is ambiguity; failing to interpret information as the test maker intended can result in an "incorrect" response, even if the taker's response is potentially valid. Another disadvantage of multiple choice examinations is that a student who is incapable of answering a particular question can simply select a random answer and still have a chance of receiving a mark for it. During mastering of a multiple –choice tests teacher may enlarge an amount of deletion and delete some context words. In any case teacher must explain the reason of the selecting an appropriate word for each blank. After mastering of the first stage teacher may use the C-test which is a modification of cloze test. The C-test is an integrative testing instrument that measures overall language competence, very much like the cloze test. It consists of four to six short, preferably authentic, text in the target language, to which ‘the rule of two’ has been applied: the second half of every second word has been deleted, beginning with the second word of the second sentence; the first and last sentences are left intact. If a word has an odd number of letters, the “bigger” part is omitted. At this stage teacher may delete every nth word and enlarge a number of deletion to twenty five. However, a text chosen for C-test should be familiar or at least students should have background knowledge about the theme of the cloze text. As the example we suggest this text: Instruction: There are twenty five blanks. You may the beginning of a missed word. The task is to restore the missing parts. Fog is really a low near the ground. Fog a___ clouds a___ made o___ many lit____ drops o___ water. Th___ drops st___ in t___ air bec___ they a___ so sm___. You can___ see ea___ drop. B___ fog c___ make i___ hard t___ see ot___ thongs. I___ can b___ dangerous i___ you a___ driving, f___ example. Some___ where th___ is a lot of fog you cannot see the road. The advantage of this type is that various forms of C-tests measure different aspects of language proficiency, measure general and global language ability, or assess language ability in a different way. For this type of cloze test it is appropriate only exact-word method of scoring. In any case teacher must explain all mistakes done by students and give the clear explanation for all appearing questions. The C-test offers considerable advantages compared with the cloze test which are listed as follows: a) as students are confronted with a variety of short passages, a better sampling of content areas is possible. Also, a person with special knowledge in a certain field does not have an unfair advantage; b) by “damaging” every second word, we can obtain a more representative sample of all the different language elements in the text than in the cloze, where normally only every fifth or sixth word is left out; c) many more items can be included in much shorter texts, making the test less time-consuming for the students than the cloze; d) unlike the cloze test, scoring is easy and objective, as there is only one acceptable solution in most cases; e) as a rule, students actually like doing C-tests, whereas the cloze test is one of the most frustrating test types for learners. Finally, for high experienced and having a plenty of exposure of cloze test students we may suggest a standard cloze test with fixed-ratio or random-ratio deletion. Both of these types are effective because they include checking of different skills, such as reading, writing, grammar, spelling, vocabulary, etc. A fixed-ratio cloze deletes every nth word consistently, so that all classes and types of words have an equal chance of being deleted. A random cloze is the one in which a specific type of word is deleted according to a linguistic principle, such as nouns, verbs, adjectives, etc. In my opinion, if a teacher wants to check overall English language proficiency it is better use fixed-ratio type of cloze test as we used in our research. It gives the opportunity to involve in the process of completing a cloze procedure a wide range of skills like vocabulary, grammar, text structure, cohesion and the reader’s prior background knowledge. Education has two goals. First, it ____ means so that we may earn ____ living. Some of these tools are _____ tools, such as reading or writing. ____ are technical skills, such as _____ and data processing. Still others are ____ professional skills peculiar, for example, to____, to science, and to teaching. Secondly, ____ provides experience so that we can ____ how to live. Education provides ____ of ideas for using the past. ____ provides a sense of values for ____ the future. There are two advantages of fixed-ratio type of standard cloze test: testing global language proficiency, and helping learners to improve their language ability. Nevertheless, such kind of test has disadvantage it is very difficult to score because teacher should use different methods of scoring and always remember about the synonyms and compare them with correct answer. If the teacher’s purpose to check only one necessary skill such as grammar, spelling, vocabulary, in other words for instructional purposes, it is better to use random-ratio type of standard cloze test. For example: In World War II, while America was at war with Japan, more than 100,000 Japanese people living (1) _____ the Pacific coast were (2) _____ and put into special camps. During this same period many Japanese were forced to give up their jobs because fellow employees (3) _____ that anyone who was Japanese must be on the side of the Axis powers. Families were forced out of their homes because constant threats (4) _____ life unbearable. For some Japanese men and women, it (5) _____ even safe to be seen on the street because the colour of their was liable to arouse hostile feelings. As you see here teacher checks the tenses. Such kind of test is easy to construct and to score because teacher may use only exact-word method of scoring.

Conclusion on Part 2

To sum up, cloze procedure is a good test of overall English language proficiency. It is an integrative, global measure of language competence. Cloze tests measure the grammar of expectancy underlying the skills of thinking, understanding, speaking, reading and writing. Cloze procedure measures global language competence consisting of linguistic knowledge, textual knowledge, and knowledge of the world. As it calls on testees to use knowledge such as vocabulary, grammar, sentence construction, text structure, cohesion and the reader’s prior background knowledge. It is regarded as a very efficient test for measuring students’ integrative competence in English that integrate language components into a total language event, which requires an integrated performance from the learner in a meaningful context. English teachers accept the effectiveness of cloze procedure as an instrument testing integrative English language competence almost without question, which might explain why cloze procedure is so popularly used in various English exams, both formal and informal. It is also used as a teaching instrument to help students to improve their reading competence. With the varieties presented here, teachers, educators, and testers have access to a versatile measurement device that can be used for many purposes. The following are but a few suggestions on the applications of the cloze test. a) teachers can use the letter cloze test to improve students' spelling ability at the elementary levels without much burden on the students. The common procedure to improve the spelling ability of the learners is to have them copy the words or sentences for a few times. Copying, although sometimes useful, is often a mechanical activity that does not stimulate students’ minds, and thus, the outcome of learning is not satisfactory. Cloze, on the other hand, activates the learners’ minds and leads to a higher achievement; b) teachers at intermediate and advanced levels can use cloze as a cognitive activity. Of course, some of the students may resist doing the tasks because cloze type activities may seem nontraditional. However, with a little care and attention, cloze would change to an enjoyable problem solving activity in almost all classes and at different levels; c) educators can utilize varieties of cloze in the learners’ mother tongue. Research has shown that cloze is a useful technique in improving students’ command in their mother tongue as well; d) teachers can use varieties of oral cloze to improve students’ speaking and listening ability. Oral cloze is a useful technique to help students practice their production through cognitive activities. Our gradual introduction to the cloze procedure is an effort to explain that developing of cloze procedure is a necessary object for modernization of our Kazakhstan education. As our President says in his message to the people of Kazakhstan we must strive for the best especially in education and science. Thus if students want to study in a prestige universities of the world and to be international specialists they should be able to cope with all international tests such as TOEFL, IELTS and some other proficiency tests without problems, where the close exercises or tests are the common task. To sum up all above, cloze procedure is a necessary technique for developing general competence in English. Cloze procedure has many applications and in correct use by teachers this technique can give wonderful results. Today, cloze tests are widely used in some places and as part of some large-scale language tests (such as TOEFL, IELTS). Following our suggestions of gradual introduction of cloze procedure this task will be commonly used in school and students will do that exercises with pleasure and without any problems.
In conclusion, according to all above, we proved our hypothesis that using of cloze procedure as a variety of pragmatic language testing helps to improve reading skills, to develop writing skills, to refine on comprehension skills, to use correctly grammar and vocabulary, to improve abilities of memory.