Ei-iE

The short answer is no.

Last month, I participated in a conference titled "Learning from Learning Assessments: The Politics and Policies of Attaining Quality Education" which was organized by NORRAG and the Brookings Institute. At the conference I engaged the audience in a discussion about two competing models for international large-scale assessments in Africa: regional assessments and PISA for Development (PISA-D). The regional assessments were established in the mid-1990s with the support of the International Institute for Educational Planning (IIEP) and the international educational community: PASEC serves Francophone countries and SACMEQ serves countries in southern and eastern Africa. PISA-D is a new and enhanced PISA instrument that is, arguably, more relevant for the contexts found in middle- and low-income countries.

While, I am not an expert on Africa or education systems in Africa, I spent the last decade exploring the emergence of international large-scale assessments. The focus of much of my research has been researching the origins of the International Association for the Evaluation of Educational Achievement (IEA). My research has shown that there has been a significant shift in the purpose of these international assessments; from originally addressing distinct research questions to more recently international assessment for the sake of educational governance.

My argument is simple: Low-income countries should avoid PISA-D. This is especially true for African countries that have invested in the development of regional assessments. The rationale for this argument is six-fold.

First, PISA-D does not fit the educational reality in Africa. Both PISA and PISA-D survey a sample of 15-year-old students. In many African countries, however, the net enrollment in lower secondary schools is still low. The sample will not yield a meaningful picture of what 15-year-old students in a given country know or how they can apply their knowledge. PISA-D plan to address this problem by including out-of-school students. But, the value of this effort is unclear. Would out-of-school youth be able to engage with the PISA-D instrument?

Second, and related, various reports suggest that students in Africa have weak competences when entering lower secondary schools. Given that PISA-D is assessing the ways students apply knowledge, how would students with weak competences engage with the PISA-D instrument? Further, if PISA-D will be too difficult to students, the achievements scores will suffer from low variance and from the floor effect (overestimation). How useful would the data be for analysts and policy makers?

Third, because PISA-D is not curriculum-based or grade-based assessment, it offers little information about system variables, such as: teacher’s characteristics, pedagogy, and curriculum. Without these variables, analysts cannot examine how different system’s characteristics matter for student achievement. This limitation raises the question whether PISA-D can inform educational planning.

Fourth, research from high-income countries shows that PISA results have negative consequences for public discourse and opinion. Research in the United States and Israel, for example, demonstrates that exposure to PISA results and discourse decreases public’s confidence in national education system and reduces support for public spending on education. This is often because of scandalized media reporting and a lack of proper dissemination and interpretation of the results. Findings from the 2012/2013 Afrobarometer in 34 countries show that the general public is satisfied with the way governments address educational needs. The general public is more concerned with overcrowded classrooms and poor facilities than from poor teaching. Given that public confidence and endorsement is important, are we willing to take the risk that PISA-D results will change public opinion towards education in African countries?

Sixth, regional assessments are embedded in the local context and are flexible to change. These are real strengths that allow regional assessments to evolve overtime, together with the educational systems they serve. In contrast, PISA is much more institutionalized with a set cycle (every three years) and a focus on monitoring trends. It is unclear what would be the added value of using PISA-D over regional assessments?

My hope is that policy makers will further develop PASEC and SACMEQ and use their insights to shed light on educational policy questions at the country and regional levels.