Executive Summary
· The Springfield Centre is a mission-driven private research, training, and consultancy organisation involved for over 20 years in designing, advising, and implementing development projects, including in education.
· Our experience and the statistics demonstrate that while significant nominal gains have been made in access to education as a result of substantial additional support in recent years, gains in the quality of education have been far less impressive.
· The scale of the problem in education is far too large to be overcome by direct delivery of aid, accounting as it does for only 4% of the 4% that developing country governments spend on education.
· There is a more useful framework to structure the inquiry’s questions to look at the achievements and prioritisation of objectives for DFID’s aid to education. In terms of the type of impact sought, each of these categories needs to be explicitly recognised and the trade-offs between them assessed.
o Profile of beneficiaries – who the targets of intervention are.
o Depth of impact – how much we expect each of these people to benefit by.
o Longevity of impact – how long the impact will last after intervention has ended.
o Scale of impact – how many people will be impacted by intervention.
o Expediency of impact – how quickly impact will be achieved.
· The trade-offs between these types of impact are realised by the means through which these ends are achieved.
· These means should not be the prescriptive means posed by the inquiry: public or private, large or small, technology driven approach or changing perceptions in communities. What is needed is a pragmatic and analytical approach cognisant of local social, economic and political realities, seeking locally appropriate, sustainable solutions to achieve impact at scale.
· These general principles of aid are wholly relevant to education, and DFID has already attempted to pilot this alternative approach. However, it is difficult to do so in the context of a push of ever larger amounts of distortionary and temporary aid. If the true potential of a systemic approach is to be realised, then it has to be supported and given the space to overcome some of the failures seen in conventional aid delivery.
Introduction
1. The Springfield Centre is a mission driven private research, training, and consultancy organisation involved in designing, advising, and implementing development projects. We have worked for over 20 years, across more than 50 countries in a wide range of sectors with the common goal of improving the way development is done to improve outcomes for poor and disadvantaged people. This mission is borne of hundreds of years of collective development experience, characterised by short-term impacts for the select few, while at the same time undermining local systems of provision.
2. Our work in education has included research, design, and implementation, much of it with DFID. It has focused on providing pragmatic solutions to real and pressing problems, circumventing dogmatic or politicised narratives in order to truly assist those in need, both now and in the future. It is in this context, drawing on these experiences, that our submission is made.
Performance of developing country education systems
Access
3. Much has been made of the gains in access to education in recent years. However, access is not a well-defined phenomenon. Comparing enrolment in low and high income country averages (within which there is huge variation), where there has been far greater convergence than in attendance or achievement, in 2014, primary school-aged children were five times more likely to be out of school in developing countries than in high income countries, with the difference greater for girls. For attendance, at primary level there is a drop-out rate of 43% in low income countries compared with less than 5% in developed countries (UIS, 2015).
Quality
4. In terms of quality, which is notoriously difficult to measure, using a mixture of proxies with the implied caveats contained in the literature, inequality is stark, perhaps more so than access. For both primary and secondary schools, around 10% of students in low income countries have to repeat their grade every year compared with less than one percent for high income countries (UIS, 2015).
5. Adult literacy rates in low income countries average 61% compared with near total literacy in high income countries. Even for those 15-24 years old, who have benefited from improvements in education over the last three decades, almost one third are unable to read and write.
6. While quality metrics are notoriously problematic, some frequently used metrics include teacher training and pupil-teacher ratio. The pupil-teacher ratio in developing countries is three times that in developed countries at primary level and more than double at secondary level. At least 20% of teachers in low income countries have no training (UIS, 2015).
Performance of ODA in addressing education problems
7. ODA for education is significant, but not when compared to the task of educating well over 1bn young people. DAC countries spend 7.3% of total ODA on education[1]. Developing countries spend an average of 4% of their budgets on education but that still means that ODA spending on education as a proportion of total spending on education in developing countries is less than 4% (OECD, 2015).
8. Exacerbating the relative insignificance of aid money in respect of national budgets is the degree of fungibility of aid to education – so 4% additional budget for developing country governments is not likely to result in an increase in spending of 4% in this area (Feyzioglu et al., 1998; Van de Sijpe, 2012).
9. Given these constraints, it is perhaps not surprising that– as with the majority of aid spending over the past half century - the overall impact of aid to education has not been transformative, in that it has not caused a significant change in the nature or scale of the problem.
10. Fundamentally, if all of the money spent by governments on education is not resulting in a bridging of the substantial gap in outcomes, then an extra 4% budget is unlikely to change this; nor is an extra 8%, 50% or even 100%, and crucially if it were to result in improved outcomes then this additional commitment of ODA would have to be ongoing. If the link between spending on education and educational outcomes is assumed – although this is certainly not uncontested – then unless development actors want to increase spending on education exponentially and indefinitely, a different approach is needed.
11. This analysis is not particularly revelatory. The national audit office (NAO) concluded in 2010 that:
Educational quality and attainment…have remained at the very low levels prevailing at the start of DFID’s 2001 Education Strategy. DFID support has increased the scale of provision, but placed insufficient emphasis on quality and cost-effectiveness. DFID has only recently started to address this imbalance. The available evidence indicates that aided education systems remain inefficient, consuming scarce existing financial and human resources. There is considerable scope, within existing resources, to improve effectiveness, particularly through more cost effective delivery of teaching time and reduced pupil drop-out. (NAO, 2010, Bilateral Support to Primary Education, 8-9)
12. However, in the six years since this time, indications are that little has changed despite considerable further resources[2] having been directed towards education.
Are we asking the right questions?
13. The inquiry has been established with a broad range of questions about DFID’s contributions to education in developing countries, but chief amongst them is whether DFID is leaving no one behind in its endeavours in this area. This question sets an end point which is unrealistic and has not been achieved in any developed country, with far greater resources – both financial and institutional – at their disposal. The problem facing many developing countries is that they are leaving most people behind in relation to both the number of children excluded from education services and the quality of education provided.
14. The sub-questions for the inquiry are posed at different levels. Firstly, the impact that aid hopes to achieve and, secondly, the means by which it seeks to achieve impact. The nature of the impact can be understood best when a framework for rationalising donor spending is applied (Taylor, 2016). The dimensions of impact - scale, depth, profile, longevity, and expediency – can be seen as explanatory and in some cases competing priorities which need to be analysed openly and honestly if outcomes from aid to education are to be improved.
15. Some of the inquiry’s questions address the profile of beneficiaries. Are they the poorest of the poor, the unemployed, high-potential individuals, or disadvantaged groups according to disability, gender, indigeneity or humanitarian crises. These are the questions that need to be asked up front with clear priorities set. The nature of the failures which impact negatively on these different groups are different, and they determine the strategies that are needed to tackle them. DFID’s work to date has suffered a lack of clarity of purpose in this area, evidenced by the questions the enquiry is asking. Are we expecting DFID’s work in education to deliver all things to all people?
16. Other questions in the enquiry focus on scale – how do we get the 59m out-of-school children into school? This is the question that has dominated education policy over the last 15 years as reflected in the MDGs. However, the evidence above outlines the fact that the subsequent push towards scale has had significant negative impact on the depth of impact, i.e. the quality of the education provided[3], and also its longevity[4].
17. One question relates most closely to the objective of expedient impact, asking if DFID is dedicating enough funding to global education. The question is, if the huge increases but still relatively insignificant amounts of aid of the last decade have failed to significantly impact the target metrics of depth and longevity of impact, can more of the same deliver these results, and for how long?
18. Two questions within the inquiry address these dimensions of depth and longevity of impact more directly, asking how DFID can make aid more effective and whether it could do more to strengthen education systems.
19. The enquiry, then, addresses all of the different dimensions of impact but does not explicitly recognise the trade-offs between them. To increase funding to increase expediency of impact might be to sacrifice the development of strong education systems, undermining local capacities. This is reflective of DFID’s own mixture of priorities as it seeks to address issues of sustainability, while funding programmes such as Big Results Now in Education.
20. In terms of the means to achieve impact, then, the inquiry asks a number questions which summarise the issues affecting the efficacy of aid to education; confusing means with ends, allowing dogma and politics to override pragmatism and, ultimately, impact. The questions focus on who support should go to; public or private, multilateral or government, and how it should be structured, looking at scaling up best practice. The danger here is two-fold.
21. Firstly, in focusing on where support is directed, we lose sight of what really matters – quantity, quality, and price of education for our target groups. Discussing whether low-fee schools should be eligible for support is a question raised abstract of context and intended impact. The second and related danger is that education systems, problems, and ‘solutions’ are seen as uniform, befitting of a uniform policy response. Education systems, and indeed all systems, are comprised of functions and rules, which are performed by different players in different contexts; public, private, and third sector, small, medium, and large, each with institutional histories and reasons why those systems operate in the ways that they do. One size does not fit all when trying to increase enrolment, attendance, and attainment within these systems.
22. What is needed is a pragmatic and analytical approach cognisant of local social, economic and political realities, seeking locally appropriate, sustainable solutions to achieve impact at scale. Such an approach has been embraced by DFID in other sectors, but due to the domestic political and programming realities along with the power of dogma, the extent to which it has been adopted in education has been limited.
A systemic approach to impact on education
23. The making markets work for the poor approach has had over £1bn of DFID funds programmed against it, primarily in agriculture but also in urban development, WASH, health, and infrastructure. The approach is analysis based and seeks to utilise the incentives and capacities of local players to overcome systemic constraints. While this approach essentially represents the means, the ends – the dimensions of impact prioritised by the approach – can be applied to education[5] as follows:
24. Scale of impact: where traditional approaches might work with individual schools, local authorities or government departments, systemic approaches focus on leverage points. Before intervention begins, there is always a view of how, if the intervention is successful, it might lead to impact on a whole sector. That is not to say that this doesn’t mean working with an individual school, but benefit for that school and its pupils would not be seen as an end in itself. In seeking to change the education system, rather than the quality of a school, or learning outcomes or enrolment rates of a cohort of pupils, a systemic approach to education inherently sees scale of impact as one of its priorities.
25. Depth of impact: incentives are central to a systemic approach; by ensuring the desired change aligns with the incentives – financial, political, cultural etc – and capacities of individuals, groups, and institutions is how the approach sees change as being sustainable. The counterpoint to this, then, is the question of whether these drivers will be sufficient to deliver the desired depth of impact. Taking an example of the information function, the theory is that parents will use information to inform how they select schools and, once a choice is made, to put pressure on schools that their children attend, improve and maintain standards, as they are invested in the process. If these mechanisms are functioning well then the depth of impact has no theoretical limitations. However, in removing one constraint to the improved functioning of the system, others will become apparent of greater or lesser severity.
26. Profile of beneficiaries: One of the criticisms frequently levelled at systemic approaches is that they don’t reach certain groups such as the poorest or most marginalised members of society. By adopting a ‘facilitative approach’ which works through others, the ability of a programme to target certain groups is limited. This has been referred to in the literature as the ‘viability void’ (Hitchins et al., 2004). That said, programmes exhibit agency in who they work with which allows them to shape outcomes towards certain beneficiaries; the entire approach is predicated on making systems work better for the poor. Often, the more marginalised people are within society, the weaker the incentives are for actors within a system to respond to their needs. Therefore, as with the depth of impact, programmes must determine their priorities and recognise the potential trade off in focusing impact on certain beneficiaries versus other dynamics.
27. Longevity of impact: This is one of the primary dynamics considered in adopting a systemic approach. In adopting a systemic approach, impact should not only persist but continue to grow and adapt in the absence of programme involvement. So, for example, if attainment is increased through improved mobility of pupils to better schools based on exam performance as a result of better information availability on that performance, then the information transmission system continues to expand and adapt in the absence of programme involvement. This might be providing information on more aspects of performance, more information service providers entering the system, or government incorporating the information gathering mechanism into public schools. Along with scale, longevity of impact is the primary shaping dynamic for programmes adopting a systemic approach.
28. Expediency of impact: Traditional delivery of aid results in fast impact. Aid programmes can build schools, subsidise uniforms and books, and provide school feeding programmes, all of which should improve outcomes with rapid effect, albeit often to limited numbers of schools/pupils, and in ways that require ongoing funding if they are to continue. A systemic approach to intervention aims to facilitate changes in systems that deliver sustainable impact at scale. However, a systemic approach is dependent upon behaviour change and this is a far more lengthy process. Further, the scale of impact is dependent upon escalation by partners and emulation by others rather than replication of programme activities and so the rate at which impact occurs is beyond the direct control of the programme. Expediency, therefore, is explicitly sacrificed in favour of longevity and scale of impact in following a systemic approach.
What questions should we be asking under a systemic approach?
29. While, a systemic approach presents a clear view of the types of impact it seeks to achieve, the application of the approach is guided by its own set of deductive questions.
30. A systems approach first ask in what ways is the system currently failing to deliver the desired outcomes for a target group? In the context of education this might be that children aren’t achieving the desired learning outcomes because of the quality of the education process and/or not enough children attend school regularly.
31. Note that questions at this level are determined by the desired outcomes, rather than the mechanisms or structures to achieve them. Ideologically driven questions of, for example private or public are not relevant – what is relevant is finding out what is happening in the education system and why it isn’t working.
32. The DFID-funded DEEPEN programme in Lagos illustrates this process. DFID invested in an initial design phase for DEEPEN which enabled an assessment of the system in which poor children are educated. In Lagos, with only an estimated 3% of primary level children not attending school, and approximately 60% of primary level children attending private schools, the key problem is one of quality, not access.
33. The next level of investigation is to ask what are the constraints in the system that lead to this failure? Firstly, what functions, rules, and norms determine educational outcomes – teacher training, grading systems, supplies of inputs such as text books, physical infrastructure, school’s access to services such as finance etc? Which of these functions, rules, and norms are not performing as well as they could?
34. Having determined that a given function or set of functions are not performing well, a systemic approach asks, for each of these functions, what are the functions and rules that comprise this transaction and why are they not performing better? So, for example, the fee payment system is comprised of monitoring systems or book-keeping, access to finance, technology development for easier management and formal and informal rules regarding the practical and social consequences of non-payment. These together represent the underlying causes of underperformance.
35. Having completed this process, DEEPEN now works in four market systems – financial services, information, rules and standards, and supporting services. In assessing the reasons for the underlying causes of underperformance, a systemic approach asks who does, and who pays? Who is or isn’t performing a function to the desired level? Why aren’t they performing it better? What are their capacities and incentives for change?
36. These questions about capacities and incentives apply in finally intervening to improve the functioning of the system. Who will do, and who will pay? Again here there is no predisposition as to which type of actor – public, private, NGO, large, small, foreign, local – can play this role in a more effective system except to say that this shouldn’t be an aid programme – which cannot ‘do’ and ‘pay’ indefinitely.
37. In thinking about systems of provision rather than individual problems or individuals, we are obliged to think about scale, while in considering how we intervene to ensure that those providing functions within those systems have the requisite incentives and capacities to continue to do so, we inherently consider sustainability. In order to improve the effectiveness of DFID’s aid to education, a systemic approach helps to frame the right questions to avoid the pitfalls of a chequered past.
[1] [1] Total ODA to developing countries is approximately $100bn
[2] From 2010-2016, counting only programmes with a budget over £10m and excluding any direct institutional or bi-lateral budget support, funding explicitly directed to education by DFID amounted to approximately £5bn (Devtracker, 2016).
[3] Since 2003 the youth literacy rate of lower and lower-middle income countries has been static despite huge increases in enrolment (UIS, 2016).
[4] In 2004, expenditure by governments of low income countries on education accounted for 18% of total public spending, by 2012, with significant increases in ODA to education, this had fallen to 15% (World Bank, 2016).
[5] DFID’s first foray into the education sector under an M4P banner, is through the DEEPEN programme in Lagos which started towards the end of 2013. This is starting to yield key lessons for DFID’s wider approach to funding for education, although it is still too early to draw substantive lessons.
* The article was written by Ben Taylor as the Springfield Centre's submission to the International Development Committee's Parliamentary Inquiry of the same name.