“At least 250 million children are failing to learn the basics” (Education for All Global Monitoring Report (GMR) 2012). By putting a number to the problem, the GMR highlighted a critical message to leaders and citizens around the world. Whether these children are in school or not, they have not mastered basic reading and writing skills needed to thrive in today’s world.
With the new and ambitious Sustainable Development Goals (SDGs), we must go beyond the advocacy messages and estimates and develop the robust indicators needed to monitor the learning outcomes of youth globally.
Measuring learning is key to monitoring the SDGs
Business is booming in measuring learning, with new initiatives underway at international, regional and national levels. Research is flourishing and the media are eager to pounce on new results from an international survey.
While assessment results stir debate in every country, the opposition can be fierce with concerns raised about a country’s sovereignty over its education policies and the methodologies used to monitor the results.
With the upcoming endorsement of the SDGs, a strong voice is needed to inform the discussions about learning outcomes in a neutral and meaningful way. A forum is needed for the international community to reconcile different assessment approaches and develop a tool to produce cross-nationally comparable information that can be used and understood by a range of stakeholders – from policymakers to teachers, students and their families.
Cataloging learning assessments used around the world
In response, the UNESCO Institute for Statistics (UIS) is working with partners to develop a new approach to monitor learning outcomes globally.
As a first step, we have created the Catalogue of Learning Assessments, which is the only open data source to provide comparable information about assessments administered in primary and lower secondary education programs in countries around the world.
This project was initially funded by a GRA grant from the Global Partnership for Education.
For each assessment, the UIS works with countries to complete a standardized questionnaire about all aspects of the examination – from test design, coverage and sample to the funding source, production and use of the data.
The Catalogue is uniquely designed to help Member States make informed decisions about a new or existing assessment while learning from the experience of others. It also provides the foundation needed to develop statistical capacity-building initiatives at country and regional levels. This is a key priority for governments and donors, such as the Global Partnership for Education.
The Learning Outcomes Index
The UIS is using information from the Catalogue and related sources to develop the Learning Outcomes Index (LOI), which will address several key policy issues, such as countries’ readiness to monitor learning globally as well as the quality of existing assessment data and access to this information. The Catalogue currently contains information for about 30 countries.
Figure 1 presents a snapshot of the extent to which countries are ready or prepared for global monitoring. It is based on responses to four questions:
- Does the country have national assessments at the primary level of education?
- Does the country have national assessments at the secondary level of education?
- Did the country have a national assessment between 2012 and 2015?
- Did the country participate or is it participating in an international assessment?
Figure 1. Are countries ready for global monitoring? Initial results of the Learning Outcomes Index
For each question, negative responses amount to 0 and a positive response equals 1. For instance, if a country replies "yes" to the first and third questions, and "no" to the others then it will have a score of 2. A country that conducts annual assessments at both education levels and participates in international studies will have a value of 4.
This simple approach can yield practical insight. For example, many people might assume that higher-income countries would obtain higher values in the index. Yet as shown in Figure 2, there does not appear to be any strong correlation between a country’s income level and LOI value.
Figure 2. Learning Outcomes Index results compared to income level by country
Next steps: sharing findings to inform policies
This is just one example of the ways in which the Catalogue can be used to inform discussions about the global monitoring of learning outcomes.
Ultimately, the goal is not just to produce internationally-comparable data but to generate the knowledge that countries can use to improve the learning levels of all children while benefiting from the experience of others.
This will create new approaches and investment in statistical capacity building so that countries have a shared understanding of learning assessments and can make effective use of data for evidence-based policymaking.
By facilitating a transfer of knowledge, these initiatives can also help to ensure that countries take an active role in the development and ‘buy in’ of the tools and methodologies needed for international monitoring.
To this end, the UIS will continue to work with partners to develop a robust and neutral tool to monitor learning outcomes globally. We are currently preparing to launch a consultation with Member States to better understand their needs and priorities in this area and to evaluate possible options in terms of UIS products and services.
In particular, we will be seeking to gain the insight of the people actually providing the data (e.g. ministries of education and national statistical offices) used to produce internationally-comparable indicators in order to reinforce the entire process– from the survey instruments used to collect the data to the products designed to disseminate the results.
This consultation will help us improve our services while informing debate about the global monitoring of learning by highlighting the needs and expectations of countries, who are both the producers and users of the data.