This blog was originally published on UNESCO’s IIEP website

If a tree falls down in the forest but nobody is around to hear it, does it make a sound?

This thought experiment has been around for hundreds of years, but raises new questions for international efforts to improve learning outcomes. A fundamental question we need to stop and ask ourselves is: does a learning assessment matter if nobody understands or even reads the report?

Capacity building workshop

The Network for Education Quality and Monitoring in Asia-Pacific (NEQMAP) hosted by UNESCO Bangkok recently held a four-day capacity building workshop to tackle precisely this problem—the reporting and dissemination of large-scale learning assessments.

In collaboration with the Australian Council for Education Research (ACER) and with support from the Global Partnership for Education (GPE), it gathered about 30 government officials from 15 countries and other partners in the region to exchange knowledge and good practice on the critical steps in a learning assessment program.

Here are some of the key lessons learned from the workshop.

The challenge: assessment data is not used to inform policy

Large-scale assessments should be designed with the aim of obtaining data to help address policy issues and intended policy goals, ultimately improving learning. However, assessment data is often not used for policy formulation.

And there is even less evidence that large-scale assessment data has an impact on teaching and learning policies at the school and classroom level, as a recent study from the Asia-Pacific region shows.

Workshop participants discussed a range of barriers they have faced in using large-scale assessment data. Dissemination has often not been part of the planning process: there is often no strategy for how findings will be shared, there is a lack of funding for developing and disseminating reports, and assessments may be poorly timed in relation to the policy development cycle.

Other obstacles have to do with capacity problems: there is a lack of human resources with sufficient technical knowledge to conduct in-depth analysis, and a need exists to create targeted reports for different stakeholders, and to translate assessment findings into policy-relevant issues.

A low capacity among key stakeholders to understand and interpret the results also exists, and there is even perceived low technical quality of the assessment program itself, which hampers its legitimacy as a source of information for policy making.

Finally, reports may inappropriately focus too much on rankings and raw results, rather than giving clear information about the implications for teaching and learning that is relevant for specific audiences, such as parents, teachers, and decentralized education authorities.

Steps to a solution: strategic reporting and dissemination

The NEQMAP workshop offered numerous insights for reporting, disseminating, and translating assessment results into policy and practice.

Key ideas included:

• The assessment plan should include a clear reporting and dissemination strategy that outlines links between analysis and defined policy goals.

• Appropriate data analysis techniques should be used to analyze achievement and contextual data, and to interpret the associations between them.

• Prior to creating reports, it is necessary to identify which findings are of greatest interest to which stakeholder audiences.

• Use a variety of formats and approaches for reporting, and employ diverse dissemination avenues to cater for diverse audiences, including different types of both traditional and new media. Successful new media strategies include Facebook, Twitter and YouTube to share key report findings with a wider audience.

• Reporting and dissemination should occur throughout the assessment program, not just at the end.

Many of these ideas were new to the workshop participants. Most countries present had not considered their target audience when preparing their report, neither prepared more than one knowledge product to communicate findings to a range of stakeholders, nor developed a plan to disseminate the findings.

Sharing good practices

Good practices for reporting and dissemination throughout the region were also shared. The National Education Assessment System (NEAS) in Pakistan, for example, has a dissemination strategy that caters to diverse audiences.

In addition to publishing a technical report at national level, they publish several non-technical smaller reports for different audiences. Teachers received a one-page summary of “good news and bad news” from the assessment, and other popular products for office use, like calendars and notepads, were used to achieve widespread dissemination of short key findings from the assessment.

Similarly, the Annual Status of Education Report (ASER), the large-scale, citizen-led assessment in India, created different assessment reports for government audiences and a separate graphic-rich report with powerful messages for the public.

ASER also succeeds in disseminating its report findings online through rich content on its website as well as creative and concise messaging on its social media channels. 

With support from the facilitators and their peers, government officials developed a dissemination strategy for their large-scale assessment and returned home with an action plan for how they would implement what they learned in the workshop in the short, medium, and long-term.

The NEQMAP Secretariat at UNESCO Bangkok is now working with ACER on a series of topical case studies to provide innovative examples of how large-scale assessment results can be used to inform education policy and practice.

Building on previous work, this series will profile the efforts of specific assessment programs and countries to ensure that the results of assessments are disseminated and used. It is heartening that such examples exist, but we need to know more about them, and the case studies will help to share this knowledge for the benefit of all countries and stakeholders.

Moving towards a systems approach to measuring and improving learning

Assessment in and of itself will not improve learning. We must go further and invest in robust, nationally-owned learning assessment systems.

As part of this effort, the Global Partnership of Education is launching the Assessment for Learning (A4L) initiative early in 2017.

The A4L initiative aims to strengthen learning assessment systems by providing technical and financial assistance to build capacity at the country and regional level.

The initiative will build on existing diagnostic tools, support sector planning and analysis, as well as the work of regional assessment networks like NEQMAP. Investments in strengthening learning assessments will be linked to the national planning process and sustainability will be ensured through the integration with education sector plans.

Without adequate reporting and dissemination, assessment results will not reach key stakeholders who influence how our children learn. We cannot afford to let assessments be inconsequential, as the tree that falls silently because no one is there to witness it. Learning assessments play an essential role in the drive to improve education quality for all, and we must continue to support countries to put a system in place that enables results to be heard, understood, and acted upon.