Test Feasibility Survey PAKISTAN: Education Sector
Tahir Andrabi* Pomona College Jishnu Das The World Bank Asim Ijaz Khwaja Harvard University With Duriya Farooqi (Harvard University) and Tristan Zajonc (Pomona College) October 29, 2002
firstname.lastname@example.org, email@example.com, firstname.lastname@example.org. We would like to acknowledge additional help in instrument design provided by Anila Asghar.
Education in Pakistan I. Introduction
As a preliminary effort in the process of obtaining data on the quality of education provided by schools in Pakistan, a feasibility study was conducted during June and July of 2002 aimed at building the needed knowledge and capacities to conduct more extensive testing in the future. This document provides background information for the feasibility test instrument, the rationale behind the test design, a description of the test content, a formal validation of each individual test and notes on issues that arose during the test’s administration and preliminary data analysis. Thus, the document serves as a composite review and test validation base for future assessment work in Pakistan that can directly feed into the ongoing project on National Assessment.
The document is organized as follows. Section II summarizes previous assessments of primary education undertaken in Pakistan over the past two decades. These studies provide a contextualization for the current efforts. Section III details the justification behind conducting the present study. Section IV provides an overview of the assessment instrument, discussing both the test design and content. Section V covers the procedural and implementation issues faced during the administration of the test instrument and a diverse range of concerns related the interpretation of the results. This section includes a summary of the actions that were taken, or that should be taken in the future, to address these issues and concerns. Section VI presents a detailed assessment of the testing instrument using methods derived from Item-Response Theory to examine the validity of each question (henceforth item) as well as the precision of the test taken in it’s entirety. Finally, section VII outlines the kinds of results that can be studied with such a testing exercise. However, we strongly caution against the use of the results presented here for drawing any conclusions about learning outcomes in Pakistani schools given the non-random nature of the sample and the small sample size of 245 test takers.
In addition, the appendices provide the educational frameworks referenced for literacy and mathematics (appendix 1), the actual test instrument with its scoring key and preliminary analysis on its discrimination and potential problems (appendix 2) and results from the detailed itemresponse analysis carried out on each question (appendix 3).
Education in Pakistan
II. Previous Studies
Since 1984, at least 19 assessments of primary education in Pakistan have been conducted. These efforts gained momentum after the World Declaration on Education for All (EFA) in 1990 and again with the joint UNESCO-UNICEF global initiative for Monitoring Learning Achievement in 1996. The studies have been both national and provincial in scope and focused on various competencies/content areas. UNESCO (2001) provides an excellent summary of previous assessment work in Pakistan and the summaries below draw from this document.
A. Early Studies – 1980s
The first two major assessments of primary education in Pakistan were the World Bank’s Primary Education Project in 1984 and the BRIDGES project of the Harvard Institute of International Development in 1988-89. The Primary Education Project study compared Science and Mathematics achievement of 3,300 students of grades 4 and 5 in a representative sample of project and non-project schools in Punjab, Sindh and the...