Artificial intelligence (AI) has emerged as a promising opportunity to accelerate foundational learning, particularly for learners in low- and middle-income countries, but the evidence base on AI in education and shared understanding of what good evidence looks like remains thin.
Governments and funders are being asked to decide which digital tools to back with little more to go on than promises, product demonstrations and anecdotal reports of success.
In the introductory blog of this series, the Global Coalition for Foundational Learning set out three tactical priorities for accelerating foundational learning with EdTech and AI: strengthening the evidence base, investing in digital public goods and building a more coordinated ecosystem.
This blog takes up the first, strengthening the evidence base, and starts with a question that sounds simple but is not: what counts as robust evidence when we talk about AI in education?
The type of evidence we need
The global education sector largely agrees that the most rigorous method to generate evidence is a randomized controlled trial (RCT).
RCTs provide a strong research design for measuring causal impact, but they are costly and are best placed once an EdTech tool and its implementation design have sufficiently matured with the basics checked: the soundness of the pedagogy, the usability of and engagement with the tool by teachers and students, and whether it is safe and age-appropriate for the children it is meant to help.
An RCT on an untested product is premature. Even if methodologically robust, it cannot tell us what we need to know: whether the tool failed or whether it was never ready to test.
Robust evidence for an AI tool in education is contextually relevant, generated at every stage of the tool’s development and kept current.
The evidence base is built by evaluating a tool throughout its development lifecycle, using what each stage reveals to improve it and decide whether it’s ready to advance further for use in practice. This requires a sector to be clear and agree about what good evaluation looks like at each step of the development lifecycle and not just at the point of measuring impact that comes much later.
Leave a comment
Your email address will not be published. All fields are required.